2023-11-24T06:06:30.012 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T06:06:30.013 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --verbose --name rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555 --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} -- /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/orig.config.yaml 2023-11-24T06:06:30.060 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T06:06:30.155 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} email: null first_in_suite: false job_id: '7465555' kernel: flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-mgr-vol-misc sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3305 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-mgr-vol-misc suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGD20YXGiNkytCHcjhlA6pBSfrL5G0UeUJxAQF/SQdBLaGQ6PMVANaB6OXial3GG4ffe7UR6oBNGhVUHZMKliLk= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIt7kG84kgBObmDZ7+jmRlniMGqurplR3EL62tMOa+cq2ZU5Agt28ZQNwGdsUXFv+G1bQUPSmbQKlSBOCkcQT6M= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots - tasks.cephfs.test_volumes.TestSubvolumeSnapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_05:41:33 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T06:06:30.155 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa; will attempt to use it 2023-11-24T06:06:30.156 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks 2023-11-24T06:06:30.156 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T06:06:30.158 INFO:teuthology.task.internal:Checking packages... 2023-11-24T06:06:30.187 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T06:06:30.188 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T06:06:30.188 INFO:teuthology.packaging:ref: None 2023-11-24T06:06:30.188 INFO:teuthology.packaging:tag: None 2023-11-24T06:06:30.188 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T06:06:30.188 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:06:30.188 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T06:06:30.335 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T06:06:30.335 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T06:06:30.344 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T06:06:30.344 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T06:06:30.349 INFO:teuthology.task.internal:Saving configuration 2023-11-24T06:06:30.364 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T06:06:30.370 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:06:30.394 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:01:30.759339', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGD20YXGiNkytCHcjhlA6pBSfrL5G0UeUJxAQF/SQdBLaGQ6PMVANaB6OXial3GG4ffe7UR6oBNGhVUHZMKliLk='} 2023-11-24T06:06:30.417 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:01:30.758117', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIt7kG84kgBObmDZ7+jmRlniMGqurplR3EL62tMOa+cq2ZU5Agt28ZQNwGdsUXFv+G1bQUPSmbQKlSBOCkcQT6M='} 2023-11-24T06:06:30.417 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T06:06:30.423 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T06:06:30.424 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T06:06:30.424 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T06:06:30.491 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5784d4eb80>, signals=[15]) 2023-11-24T06:06:30.492 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T06:06:30.498 INFO:teuthology.task.internal:Opening connections... 2023-11-24T06:06:30.498 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:06:30.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:06:30.597 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:06:30.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:06:30.682 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T06:06:30.689 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-11-24T06:06:30.707 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-11-24T06:06:30.707 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-11-24T06:06:30.764 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2023-11-24T06:06:30.764 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2023-11-24T06:06:30.764 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T06:06:30.765 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T06:06:30.766 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-11-24T06:06:30.788 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2023-11-24T06:06:30.806 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2023-11-24T06:06:30.806 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2023-11-24T06:06:30.862 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T06:06:30.863 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T06:06:30.864 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2023-11-24T06:06:30.879 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T06:06:30.888 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T06:06:30.893 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T06:06:30.893 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2023-11-24T06:06:30.896 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2023-11-24T06:06:30.917 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T06:06:30.923 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T06:06:30.923 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2023-11-24T06:06:30.953 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2023-11-24T06:06:31.081 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T06:06:31.136 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T06:06:31.155 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T06:06:31.156 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, timeout 300 2023-11-24T06:06:31.156 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T06:06:31.156 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T06:06:31.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:06:31.173 DEBUG:teuthology.task.kernel:sha1 for mon.b is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:06:31.173 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T06:06:31.173 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2023-11-24T06:06:31.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:06:31.174 DEBUG:teuthology.task.kernel:sha1 for mon.a is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:06:31.175 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T06:06:31.175 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-11-24T06:06:31.230 INFO:teuthology.orchestra.run.smithi173.stdout:5.14.0-361.el9.x86_64 2023-11-24T06:06:31.231 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:06:31.231 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T06:06:31.231 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:06:31.235 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-361.el9.x86_64 2023-11-24T06:06:31.235 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:06:31.235 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T06:06:31.235 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:06:31.550 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum upgrade -y linux-firmware 2023-11-24T06:06:31.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum upgrade -y linux-firmware 2023-11-24T06:06:33.981 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-9 - AppStream 7.9 MB/s | 6.4 MB 00:00 2023-11-24T06:06:33.990 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 7.6 MB/s | 6.4 MB 00:00 2023-11-24T06:06:35.798 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 2.8 MB/s | 2.0 MB 00:00 2023-11-24T06:06:35.855 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-9 - BaseOS 2.4 MB/s | 2.0 MB 00:00 2023-11-24T06:06:36.564 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 7.9 MB 00:00 2023-11-24T06:06:38.590 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 22 MB/s | 18 MB 00:00 2023-11-24T06:06:43.670 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 7.9 MB 00:07 2023-11-24T06:06:44.427 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 5.8 MB 00:02 2023-11-24T06:06:46.075 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2023-11-24T06:06:48.654 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 20 MB 00:02 2023-11-24T06:06:50.637 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 18 MB 00:05 2023-11-24T06:06:53.726 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2023-11-24T06:06:56.221 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-11-24T06:06:56.230 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-11-24T06:06:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-11-24T06:06:56.230 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-11-24T06:06:56.230 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-11-24T06:06:56.230 INFO:teuthology.orchestra.run.smithi016.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T06:06:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:06:56.231 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-11-24T06:06:56.231 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-11-24T06:06:56.231 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 1 Package 2023-11-24T06:06:56.231 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:06:56.231 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 328 M 2023-11-24T06:06:56.231 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-11-24T06:06:56.349 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 5.8 MB 00:02 2023-11-24T06:06:58.171 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 15 kB 00:00 2023-11-24T06:07:01.244 INFO:teuthology.orchestra.run.smithi016.stdout:linux-firmware-20231030-141.el9.noarch.rpm 65 MB/s | 328 MB 00:05 2023-11-24T06:07:01.245 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:07:01.245 INFO:teuthology.orchestra.run.smithi016.stdout:Total 65 MB/s | 328 MB 00:05 2023-11-24T06:07:01.245 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-11-24T06:07:01.336 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-11-24T06:07:01.336 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-11-24T06:07:03.538 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-11-24T06:07:03.538 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-11-24T06:07:03.600 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 3.7 MB/s | 20 MB 00:05 2023-11-24T06:07:08.564 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 8.8 kB/s | 1.7 kB 00:00 2023-11-24T06:07:09.527 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-11-24T06:07:09.864 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T06:07:09.969 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:07:10.981 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2023-11-24T06:07:10.989 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2023-11-24T06:07:10.989 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2023-11-24T06:07:10.989 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2023-11-24T06:07:10.989 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2023-11-24T06:07:10.990 INFO:teuthology.orchestra.run.smithi173.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T06:07:10.990 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:07:10.990 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2023-11-24T06:07:10.990 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2023-11-24T06:07:10.990 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 1 Package 2023-11-24T06:07:10.990 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:07:10.990 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 328 M 2023-11-24T06:07:10.990 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2023-11-24T06:07:11.069 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:07:11.069 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T06:07:11.450 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:07:11.450 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:07:11.450 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-11-24T06:07:11.451 INFO:teuthology.orchestra.run.smithi016.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T06:07:11.451 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:07:11.451 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-11-24T06:07:11.690 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T06:07:11.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:07:12.075 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T06:07:12.075 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T06:07:12.078 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi016.front.sepia.ceph.com to complete... 2023-11-24T06:07:15.981 INFO:teuthology.orchestra.run.smithi173.stdout:linux-firmware-20231030-141.el9.noarch.rpm 66 MB/s | 328 MB 00:04 2023-11-24T06:07:15.982 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:07:15.983 INFO:teuthology.orchestra.run.smithi173.stdout:Total 66 MB/s | 328 MB 00:04 2023-11-24T06:07:15.983 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2023-11-24T06:07:16.066 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2023-11-24T06:07:16.067 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2023-11-24T06:07:18.188 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2023-11-24T06:07:18.188 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2023-11-24T06:07:23.796 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2023-11-24T06:07:24.239 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T06:07:24.329 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:07:25.300 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:07:25.300 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T06:07:25.697 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T06:07:25.697 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:07:25.697 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2023-11-24T06:07:25.697 INFO:teuthology.orchestra.run.smithi173.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T06:07:25.697 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:07:25.697 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2023-11-24T06:07:26.009 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T06:07:26.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T06:07:26.310 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T06:07:26.310 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T06:07:26.312 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi173.front.sepia.ceph.com to complete... 2023-11-24T06:09:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24 06:09:55 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T06:09:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:FINISHED --2023-11-24 06:09:55-- 2023-11-24T06:09:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:Total wall clock time: 2m 44s 2023-11-24T06:09:55.789 INFO:teuthology.orchestra.run.smithi016.stderr:Downloaded: 1 files, 306M in 2m 43s (1.87 MB/s) 2023-11-24T06:09:55.790 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T06:09:55.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T06:09:57.734 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying... ######################################## 2023-11-24T06:09:57.795 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing... ######################################## 2023-11-24T06:09:57.880 INFO:teuthology.orchestra.run.smithi016.stdout:Updating / installing... 2023-11-24T06:10:05.248 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T06:10:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T06:10:40.750 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T06:10:40.751 DEBUG:teuthology.orchestra.run.smithi016:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T06:10:40.817 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T06:10:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T06:10:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T06:10:40.820 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T06:10:40.821 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T06:10:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T06:10:40.823 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T06:10:40.824 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T06:10:40.825 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T06:10:40.826 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T06:10:40.827 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T06:10:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T06:10:40.829 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T06:10:40.830 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T06:10:40.831 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T06:10:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T06:10:40.833 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T06:10:40.834 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T06:10:40.835 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T06:10:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T06:10:40.837 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T06:10:40.838 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T06:10:40.839 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T06:10:40.840 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T06:10:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T06:10:40.842 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T06:10:40.843 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T06:10:40.844 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T06:10:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T06:10:40.846 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T06:10:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T06:10:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T06:10:40.849 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T06:10:40.850 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T06:10:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T06:10:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T06:10:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T06:10:40.854 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T06:10:40.855 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T06:10:40.856 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T06:10:40.857 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T06:10:40.858 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T06:10:40.859 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T06:10:40.860 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T06:10:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T06:10:40.862 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T06:10:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T06:10:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T06:10:40.865 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T06:10:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T06:10:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T06:10:40.868 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T06:10:40.869 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T06:10:40.870 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T06:10:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T06:10:40.873 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T06:10:40.874 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T06:10:40.875 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T06:10:40.876 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T06:10:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T06:10:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T06:10:40.879 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T06:10:40.880 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T06:10:40.881 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T06:10:40.882 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T06:10:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T06:10:40.884 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T06:10:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T06:10:40.886 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T06:10:40.887 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T06:10:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T06:10:40.889 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T06:10:40.890 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T06:10:40.891 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T06:10:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T06:10:40.893 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T06:10:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T06:10:40.895 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T06:10:40.896 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T06:10:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T06:10:40.898 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T06:10:40.899 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T06:10:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T06:10:40.901 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T06:10:40.902 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T06:10:40.903 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T06:10:40.904 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T06:10:40.905 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T06:10:40.906 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T06:10:40.907 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T06:10:40.908 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T06:10:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T06:10:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T06:10:40.911 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T06:10:40.912 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T06:10:40.913 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T06:10:40.914 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T06:10:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T06:10:40.916 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T06:10:40.917 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T06:10:40.918 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T06:10:40.919 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T06:10:40.920 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T06:10:40.921 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T06:10:40.922 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T06:10:40.923 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T06:10:40.924 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T06:10:40.925 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T06:10:40.926 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T06:10:40.927 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T06:10:40.928 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T06:10:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T06:10:40.930 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T06:10:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T06:10:40.932 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T06:10:40.933 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T06:10:40.934 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T06:10:40.935 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T06:10:40.936 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T06:10:40.937 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T06:10:40.938 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T06:10:40.939 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T06:10:40.940 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T06:10:40.941 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T06:10:40.942 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T06:10:40.943 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T06:10:40.944 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T06:10:40.945 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T06:10:40.946 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T06:10:40.947 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T06:10:40.948 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T06:10:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T06:10:40.950 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T06:10:40.951 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T06:10:40.952 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T06:10:40.953 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T06:10:40.954 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T06:10:40.955 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T06:10:40.956 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T06:10:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T06:10:40.958 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T06:10:40.959 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T06:10:40.960 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T06:10:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T06:10:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T06:10:40.963 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T06:10:40.964 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T06:10:40.965 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T06:10:40.966 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T06:10:40.967 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T06:10:40.968 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T06:10:40.969 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T06:10:40.970 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T06:10:40.971 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T06:10:40.971 DEBUG:teuthology.orchestra.run.smithi016:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T06:10:40.994 INFO:teuthology.orchestra.run.smithi016.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T06:10:40.994 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T06:10:40.994 INFO:teuthology.orchestra.run.smithi016.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:10:40.994 INFO:teuthology.orchestra.run.smithi016.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T06:10:40.995 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T06:10:40.996 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2023-11-24T06:10:40.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2023-11-24T06:10:41.080 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2023-11-24T06:10:41.080 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2023-11-24T06:10:41.080 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2023-11-24T06:10:41.081 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 68.el9 2023-11-24T06:10:41.081 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2023-11-24T06:10:41.081 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T06:10:41.081 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2023-11-24T06:10:41.081 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8264254 2023-11-24T06:10:41.081 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2023-11-24T06:10:41.081 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T06:10:41.082 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T06:10:41.082 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T06:10:41.082 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T06:10:41.082 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2023-11-24T06:10:41.082 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2023-11-24T06:10:41.082 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T06:10:41.083 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2023-11-24T06:10:41.083 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2023-11-24T06:10:41.083 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:10:41.083 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T06:10:41.083 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T06:10:41.084 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T06:10:41.084 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2023-11-24T06:10:41.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:10:41.084 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T06:10:41.085 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T06:10:41.086 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 6.6.0-g8247bbb5b470 2023-11-24T06:10:41.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T06:10:41.781 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2023-11-24T06:10:43.299 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T06:10:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:done 2023-11-24T06:10:43.340 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-11-24T06:10:43.355 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.PzdCJowvZN 2023-11-24T06:10:43.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PzdCJowvZN 2023-11-24T06:10:43.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.PzdCJowvZN 2023-11-24T06:10:43.574 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.PzdCJowvZN is 6KB 2023-11-24T06:10:43.622 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.PzdCJowvZN 2023-11-24T06:10:43.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T06:10:43.705 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T06:10:43.705 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T06:10:43.705 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T06:10:43.705 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T06:10:43.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T06:10:43.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2023-11-24T06:10:53.150 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24 06:10:53 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T06:10:53.150 INFO:teuthology.orchestra.run.smithi173.stderr:FINISHED --2023-11-24 06:10:53-- 2023-11-24T06:10:53.150 INFO:teuthology.orchestra.run.smithi173.stderr:Total wall clock time: 3m 27s 2023-11-24T06:10:53.150 INFO:teuthology.orchestra.run.smithi173.stderr:Downloaded: 1 files, 306M in 3m 26s (1.48 MB/s) 2023-11-24T06:10:53.151 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T06:10:53.151 DEBUG:teuthology.orchestra.run.smithi173:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T06:10:55.002 INFO:teuthology.orchestra.run.smithi173.stdout:Verifying... ######################################## 2023-11-24T06:10:55.139 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing... ######################################## 2023-11-24T06:10:55.220 INFO:teuthology.orchestra.run.smithi173.stdout:Updating / installing... 2023-11-24T06:11:02.355 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T06:11:13.863 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-24T06:11:13.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-24T06:11:13.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:11:16.378 INFO:teuthology.orchestra.run.smithi173.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T06:11:32.322 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-11-24T06:11:37.530 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi173.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T06:11:37.530 DEBUG:teuthology.orchestra.run.smithi173:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T06:11:37.619 INFO:teuthology.orchestra.run.smithi173.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T06:11:37.619 INFO:teuthology.orchestra.run.smithi173.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T06:11:37.619 INFO:teuthology.orchestra.run.smithi173.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T06:11:37.619 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T06:11:37.619 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T06:11:37.619 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T06:11:37.619 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T06:11:37.620 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T06:11:37.620 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T06:11:37.620 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T06:11:37.620 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T06:11:37.620 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T06:11:37.620 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T06:11:37.620 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T06:11:37.621 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T06:11:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T06:11:37.623 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T06:11:37.624 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T06:11:37.625 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T06:11:37.626 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T06:11:37.627 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T06:11:37.628 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T06:11:37.629 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T06:11:37.630 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T06:11:37.631 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T06:11:37.631 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T06:11:37.631 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T06:11:37.631 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T06:11:37.631 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T06:11:37.631 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T06:11:37.631 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T06:11:37.631 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T06:11:37.631 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T06:11:37.632 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T06:11:37.633 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T06:11:37.634 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T06:11:37.635 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T06:11:37.636 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T06:11:37.637 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T06:11:37.637 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T06:11:37.637 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T06:11:37.637 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T06:11:37.637 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T06:11:37.637 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T06:11:37.637 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T06:11:37.637 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T06:11:37.637 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T06:11:37.638 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T06:11:37.639 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T06:11:37.640 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T06:11:37.641 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T06:11:37.642 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T06:11:37.642 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T06:11:37.642 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T06:11:37.642 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T06:11:37.642 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T06:11:37.642 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T06:11:37.642 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T06:11:37.642 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T06:11:37.643 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T06:11:37.643 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T06:11:37.643 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T06:11:37.643 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T06:11:37.643 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T06:11:37.643 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T06:11:37.643 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T06:11:37.643 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T06:11:37.643 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T06:11:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T06:11:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T06:11:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T06:11:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T06:11:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T06:11:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T06:11:37.645 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T06:11:37.645 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T06:11:37.645 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T06:11:37.645 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T06:11:37.645 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T06:11:37.645 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T06:11:37.646 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T06:11:37.646 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T06:11:37.646 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T06:11:37.646 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T06:11:37.646 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T06:11:37.646 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T06:11:37.647 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T06:11:37.647 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T06:11:37.647 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T06:11:37.647 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T06:11:37.647 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T06:11:37.647 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T06:11:37.648 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T06:11:37.648 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T06:11:37.648 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T06:11:37.648 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T06:11:37.648 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T06:11:37.648 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T06:11:37.649 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T06:11:37.649 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T06:11:37.649 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T06:11:37.649 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T06:11:37.649 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T06:11:37.649 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T06:11:37.649 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T06:11:37.649 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T06:11:37.650 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T06:11:37.650 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T06:11:37.650 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T06:11:37.650 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T06:11:37.650 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T06:11:37.650 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T06:11:37.650 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T06:11:37.651 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T06:11:37.651 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T06:11:37.651 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T06:11:37.651 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T06:11:37.651 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T06:11:37.651 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T06:11:37.651 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T06:11:37.651 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T06:11:37.651 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T06:11:37.652 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T06:11:37.653 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T06:11:37.654 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T06:11:37.654 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T06:11:37.654 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T06:11:37.654 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T06:11:37.654 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T06:11:37.654 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T06:11:37.654 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T06:11:37.654 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T06:11:37.655 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T06:11:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T06:11:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T06:11:37.658 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T06:11:37.658 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T06:11:37.658 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T06:11:37.658 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T06:11:37.658 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T06:11:37.658 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T06:11:37.658 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T06:11:37.658 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T06:11:37.659 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T06:11:37.660 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T06:11:37.661 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T06:11:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T06:11:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T06:11:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T06:11:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T06:11:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T06:11:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T06:11:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T06:11:37.662 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T06:11:37.663 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T06:11:37.664 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T06:11:37.665 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T06:11:37.666 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T06:11:37.667 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T06:11:37.668 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T06:11:37.669 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T06:11:37.670 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T06:11:37.671 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T06:11:37.672 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T06:11:37.673 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T06:11:37.674 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T06:11:37.675 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T06:11:37.676 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T06:11:37.677 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T06:11:37.678 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T06:11:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T06:11:37.680 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T06:11:37.681 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T06:11:37.682 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T06:11:37.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T06:11:37.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T06:11:37.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T06:11:37.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T06:11:37.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T06:11:37.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T06:11:37.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T06:11:37.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T06:11:37.683 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T06:11:37.684 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T06:11:37.685 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T06:11:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T06:11:37.687 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T06:11:37.688 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T06:11:37.689 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T06:11:37.690 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T06:11:37.691 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T06:11:37.692 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T06:11:37.693 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T06:11:37.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T06:11:37.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T06:11:37.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T06:11:37.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T06:11:37.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T06:11:37.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T06:11:37.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T06:11:37.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T06:11:37.694 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T06:11:37.695 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T06:11:37.696 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T06:11:37.697 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T06:11:37.698 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T06:11:37.699 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T06:11:37.700 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T06:11:37.701 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T06:11:37.702 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T06:11:37.703 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T06:11:37.704 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T06:11:37.705 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T06:11:37.706 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T06:11:37.707 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T06:11:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T06:11:37.709 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T06:11:37.710 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T06:11:37.711 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T06:11:37.712 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T06:11:37.713 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T06:11:37.714 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T06:11:37.715 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T06:11:37.716 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T06:11:37.717 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T06:11:37.718 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T06:11:37.719 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T06:11:37.720 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T06:11:37.721 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T06:11:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T06:11:37.723 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T06:11:37.724 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T06:11:37.725 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T06:11:37.726 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T06:11:37.727 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T06:11:37.728 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T06:11:37.729 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T06:11:37.730 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T06:11:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T06:11:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T06:11:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T06:11:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T06:11:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T06:11:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T06:11:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T06:11:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T06:11:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T06:11:37.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T06:11:37.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T06:11:37.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T06:11:37.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T06:11:37.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T06:11:37.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T06:11:37.732 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T06:11:37.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T06:11:37.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T06:11:37.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T06:11:37.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T06:11:37.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T06:11:37.733 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T06:11:37.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T06:11:37.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T06:11:37.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T06:11:37.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T06:11:37.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T06:11:37.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T06:11:37.734 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T06:11:37.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T06:11:37.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T06:11:37.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T06:11:37.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T06:11:37.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T06:11:37.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T06:11:37.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T06:11:37.735 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T06:11:37.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T06:11:37.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T06:11:37.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T06:11:37.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T06:11:37.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T06:11:37.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T06:11:37.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T06:11:37.736 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T06:11:37.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T06:11:37.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T06:11:37.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T06:11:37.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T06:11:37.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T06:11:37.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T06:11:37.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T06:11:37.737 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T06:11:37.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T06:11:37.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T06:11:37.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T06:11:37.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T06:11:37.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T06:11:37.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T06:11:37.738 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T06:11:37.739 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T06:11:37.740 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T06:11:37.741 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T06:11:37.742 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T06:11:37.743 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T06:11:37.744 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T06:11:37.745 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T06:11:37.746 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T06:11:37.747 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T06:11:37.748 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T06:11:37.749 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T06:11:37.750 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T06:11:37.751 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T06:11:37.752 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T06:11:37.753 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T06:11:37.754 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T06:11:37.755 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T06:11:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T06:11:37.757 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T06:11:37.758 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T06:11:37.759 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T06:11:37.760 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T06:11:37.761 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T06:11:37.762 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T06:11:37.763 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T06:11:37.764 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T06:11:37.765 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T06:11:37.766 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T06:11:37.767 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T06:11:37.768 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T06:11:37.769 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T06:11:37.770 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T06:11:37.771 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T06:11:37.772 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T06:11:37.773 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T06:11:37.774 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T06:11:37.775 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T06:11:37.776 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T06:11:37.777 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T06:11:37.778 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T06:11:37.779 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T06:11:37.780 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T06:11:37.781 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T06:11:37.782 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T06:11:37.783 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T06:11:37.784 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T06:11:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T06:11:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T06:11:37.787 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T06:11:37.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T06:11:37.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T06:11:37.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T06:11:37.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T06:11:37.788 INFO:teuthology.orchestra.run.smithi173.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T06:11:37.788 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T06:11:37.789 DEBUG:teuthology.orchestra.run.smithi173:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T06:11:37.810 INFO:teuthology.orchestra.run.smithi173.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T06:11:37.810 INFO:teuthology.orchestra.run.smithi173.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T06:11:37.810 INFO:teuthology.orchestra.run.smithi173.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:37.810 INFO:teuthology.orchestra.run.smithi173.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:fi 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:fi 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T06:11:37.811 INFO:teuthology.orchestra.run.smithi173.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T06:11:37.812 INFO:teuthology.orchestra.run.smithi173.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T06:11:37.812 INFO:teuthology.orchestra.run.smithi173.stdout:fi 2023-11-24T06:11:37.813 DEBUG:teuthology.orchestra.run.smithi173:> sudo rpm -qi grub2-tools 2023-11-24T06:11:37.886 INFO:teuthology.orchestra.run.smithi173.stdout:Name : grub2-tools 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Epoch : 1 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Version : 2.06 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Release : 68.el9 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Architecture: x86_64 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Group : Unspecified 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Size : 8264254 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:License : GPLv3+ 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T06:11:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:Packager : builder@centos.org 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:Vendor : CentOS 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:Summary : Support tools for GRUB. 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:Description : 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:hardware devices. 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:11:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T06:11:37.889 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T06:11:37.889 INFO:teuthology.task.kernel:Updating grub on smithi173 to boot 6.6.0-g8247bbb5b470 2023-11-24T06:11:37.889 DEBUG:teuthology.orchestra.run.smithi173:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T06:11:38.467 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2023-11-24T06:11:39.877 INFO:teuthology.orchestra.run.smithi173.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T06:11:39.912 INFO:teuthology.orchestra.run.smithi173.stderr:done 2023-11-24T06:11:39.914 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2023-11-24T06:11:39.929 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.YEVOD747e7 2023-11-24T06:11:39.929 DEBUG:teuthology.orchestra.run.smithi173:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YEVOD747e7 2023-11-24T06:11:40.005 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.YEVOD747e7 2023-11-24T06:11:40.142 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.YEVOD747e7 is 6KB 2023-11-24T06:11:40.193 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.YEVOD747e7 2023-11-24T06:11:40.209 DEBUG:teuthology.orchestra.run.smithi173:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T06:11:40.279 INFO:teuthology.orchestra.run.smithi173.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T06:11:40.279 INFO:teuthology.orchestra.run.smithi173.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T06:11:40.280 INFO:teuthology.orchestra.run.smithi173.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T06:11:40.280 INFO:teuthology.orchestra.run.smithi173.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T06:11:40.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T06:11:40.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo shutdown -r now 2023-11-24T06:11:41.324 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-24T06:11:41.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:12:10.483 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-24T06:12:10.483 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-11-24T06:12:10.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:12:41.358 DEBUG:teuthology.orchestra.remote:timed out 2023-11-24T06:12:53.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-24T06:12:53.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:12:53.757 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-11-24T06:12:54.442 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-24T06:12:54.442 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-g8247bbb5b470"... 2023-11-24T06:12:54.442 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-11-24T06:12:54.500 INFO:teuthology.orchestra.run.smithi016.stdout:6.6.0-g8247bbb5b470 2023-11-24T06:12:54.500 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T06:12:54.500 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T06:12:54.500 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T06:12:55.501 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-24T06:12:55.501 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T06:12:55.651 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2023-11-24T06:12:55.704 DEBUG:teuthology.parallel:result is None 2023-11-24T06:13:10.486 DEBUG:teuthology.orchestra.remote:timed out 2023-11-24T06:13:19.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-11-24T06:13:19.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:13:27.220 DEBUG:teuthology.orchestra.run.smithi173:> true 2023-11-24T06:13:27.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-11-24T06:13:27.866 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-g8247bbb5b470"... 2023-11-24T06:13:27.866 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2023-11-24T06:13:27.905 INFO:teuthology.orchestra.run.smithi173.stdout:6.6.0-g8247bbb5b470 2023-11-24T06:13:27.905 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T06:13:27.905 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T06:13:27.906 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T06:13:28.906 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-24T06:13:28.907 DEBUG:teuthology.orchestra.run.smithi173:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T06:13:29.079 INFO:teuthology.orchestra.run.smithi173.stdout:ttyS1 2023-11-24T06:13:29.130 DEBUG:teuthology.parallel:result is None 2023-11-24T06:13:29.131 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T06:13:29.140 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T06:13:29.140 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T06:13:29.149 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T06:13:29.205 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T06:13:29.217 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T06:13:29.230 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T06:13:29.231 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T06:13:29.237 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T06:13:29.302 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T06:13:29.313 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T06:13:29.314 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T06:13:29.320 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T06:13:29.411 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:13:29.443 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:13:31.631 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:13:31.631 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:13:31.632 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T06:13:31.642 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T06:13:31.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T06:13:31.651 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T06:13:31.754 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T06:13:31.763 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T06:13:31.763 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T06:13:31.771 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T06:13:31.824 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:13:31.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:13:32.041 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:13:32.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:13:32.230 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:13:32.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T06:13:32.338 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:13:32.412 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:13:32.507 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:13:32.573 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:13:32.662 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:13:32.662 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T06:13:32.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2023-11-24T06:13:32.741 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2023-11-24T06:13:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:13:32.877 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:13:33.277 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T06:13:33.285 INFO:teuthology.task.internal:Starting timer... 2023-11-24T06:13:33.285 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T06:13:33.337 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T06:13:33.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2023-11-24T06:13:33.488 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2023-11-24T06:13:33.493 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2023-11-24T06:13:33.624 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2023-11-24T06:13:33.627 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T06:13:33.628 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2023-11-24T06:13:33.698 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2023-11-24T06:13:33.699 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2023-11-24T06:13:33.754 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2023-11-24T06:13:33.754 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive'} 2023-11-24T06:13:33.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:13:33.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:13:33.859 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:13:33.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:13:33.949 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T06:13:33.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2023-11-24T06:13:34.077 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2023-11-24T06:13:34.184 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T06:13:34.195 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-24T06:13:34.195 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T06:13:34.210 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T06:13:34.211 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T06:23:49.084 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2023-11-24T06:23:49.085 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-24T06:23:49.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:23:49.226 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-11-24T06:23:49.378 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-11-24T06:23:49.378 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-11-24T06:23:49.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:23:49.511 DEBUG:teuthology.orchestra.run.smithi173:> true 2023-11-24T06:23:49.667 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-11-24T06:23:49.667 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T06:23:49.680 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T06:23:49.680 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:23:49.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:23:49.690 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:23:49.690 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:23:49.796 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:23:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:23:49.850 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:23:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:23:49.935 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2023-11-24T06:23:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2023-11-24T06:23:49.980 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2023-11-24T06:23:50.003 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2023-11-24T06:23:50.035 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:23:50.062 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:23:50.086 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:23:50.115 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:23:50.207 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:23:50.215 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:23:50.215 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2023-11-24T06:23:50.215 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:23:50.215 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:23:50.215 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:23:50.215 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:23:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:23:50.250 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:23:50.250 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-11-24T06:23:50.251 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:23:50.251 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:23:50.251 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:23:50.251 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:23:50.252 INFO:teuthology.run_tasks:Running task install... 2023-11-24T06:23:50.261 DEBUG:teuthology.task.install:project ceph 2023-11-24T06:23:50.262 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T06:23:50.262 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T06:23:50.262 INFO:teuthology.task.install:Using flavor: default 2023-11-24T06:23:50.268 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T06:23:50.268 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T06:23:50.268 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:23:50.269 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:23:50.271 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:23:50.271 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:23:50.536 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T06:23:50.536 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:23:50.550 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T06:23:50.550 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:23:50.668 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:23:50.668 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:23:50.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:23:50.736 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:23:50.736 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:23:50.737 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:23:50.781 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:23:50.782 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:23:50.838 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:23:50.838 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:23:50.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:23:50.978 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:23:51.077 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:23:51.104 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:23:51.176 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2023-11-24T06:23:51.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2023-11-24T06:23:51.192 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2023-11-24T06:23:51.196 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2023-11-24T06:23:51.549 INFO:teuthology.orchestra.run.smithi016.stdout:73 files removed 2023-11-24T06:23:51.567 INFO:teuthology.orchestra.run.smithi173.stdout:73 files removed 2023-11-24T06:23:51.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:23:51.608 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:23:52.678 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 140 kB/s | 78 kB 00:00 2023-11-24T06:23:52.736 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 127 kB/s | 78 kB 00:00 2023-11-24T06:23:53.237 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 20 kB/s | 9.5 kB 00:00 2023-11-24T06:23:53.412 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 16 kB/s | 9.5 kB 00:00 2023-11-24T06:23:53.838 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2023-11-24T06:23:54.034 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2023-11-24T06:23:54.771 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 9.7 MB/s | 6.4 MB 00:00 2023-11-24T06:23:54.958 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-9 - AppStream 6.1 MB/s | 6.4 MB 00:01 2023-11-24T06:23:56.498 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2023-11-24T06:23:56.617 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.0 MB 00:00 2023-11-24T06:23:58.218 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 7.9 MB 00:01 2023-11-24T06:23:59.787 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 7.9 MB 00:02 2023-11-24T06:24:01.860 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 18 MB 00:02 2023-11-24T06:24:05.561 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 18 MB 00:04 2023-11-24T06:24:07.936 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 5.8 MB 00:02 2023-11-24T06:24:10.149 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 15 kB 00:00 2023-11-24T06:24:10.753 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 5.8 MB 00:01 2023-11-24T06:24:12.362 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 15 kB 00:00 2023-11-24T06:24:16.027 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 20 MB 00:04 2023-11-24T06:24:16.027 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 20 MB 00:02 2023-11-24T06:24:20.124 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2023-11-24T06:24:20.738 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2023-11-24T06:24:23.039 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:24:23.040 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:24:23.061 INFO:teuthology.orchestra.run.smithi173.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T06:24:23.062 INFO:teuthology.orchestra.run.smithi173.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T06:24:23.064 INFO:teuthology.orchestra.run.smithi173.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T06:24:23.065 INFO:teuthology.orchestra.run.smithi173.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T06:24:23.068 INFO:teuthology.orchestra.run.smithi173.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T06:24:23.071 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T06:24:23.072 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T06:24:23.151 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2023-11-24T06:24:23.166 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================================ 2023-11-24T06:24:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2023-11-24T06:24:23.166 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================================ 2023-11-24T06:24:23.166 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2023-11-24T06:24:23.166 INFO:teuthology.orchestra.run.smithi173.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T06:24:23.167 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T06:24:23.168 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T06:24:23.169 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T06:24:23.170 INFO:teuthology.orchestra.run.smithi173.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T06:24:23.171 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T06:24:23.172 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T06:24:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T06:24:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T06:24:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T06:24:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T06:24:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T06:24:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T06:24:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T06:24:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T06:24:23.173 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T06:24:23.174 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T06:24:23.175 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T06:24:23.176 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T06:24:23.177 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T06:24:23.178 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout:Installing weak dependencies: 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T06:24:23.179 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:24:23.180 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2023-11-24T06:24:23.180 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================================ 2023-11-24T06:24:23.180 INFO:teuthology.orchestra.run.smithi173.stdout:Install 132 Packages 2023-11-24T06:24:23.180 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 15 Packages 2023-11-24T06:24:23.180 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:24:23.180 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 204 M 2023-11-24T06:24:23.181 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2023-11-24T06:24:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:24:23.736 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:24:23.756 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T06:24:23.757 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T06:24:23.759 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T06:24:23.761 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T06:24:23.763 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T06:24:23.767 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T06:24:23.768 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T06:24:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-11-24T06:24:23.879 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-11-24T06:24:23.879 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-11-24T06:24:23.879 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-11-24T06:24:23.879 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-11-24T06:24:23.879 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T06:24:23.880 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T06:24:23.881 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T06:24:23.882 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T06:24:23.883 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T06:24:23.884 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T06:24:23.885 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T06:24:23.886 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T06:24:23.887 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T06:24:23.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T06:24:23.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T06:24:23.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T06:24:23.891 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T06:24:23.892 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T06:24:23.892 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T06:24:23.892 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2023-11-24T06:24:23.892 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T06:24:23.892 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:24:23.892 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-11-24T06:24:23.892 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-11-24T06:24:23.892 INFO:teuthology.orchestra.run.smithi016.stdout:Install 132 Packages 2023-11-24T06:24:23.892 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 15 Packages 2023-11-24T06:24:23.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:24:23.893 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 204 M 2023-11-24T06:24:23.894 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-11-24T06:24:24.150 INFO:teuthology.orchestra.run.smithi173.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 21 kB/s | 6.5 kB 00:00 2023-11-24T06:24:24.759 INFO:teuthology.orchestra.run.smithi173.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.4 MB/s | 859 kB 00:00 2023-11-24T06:24:24.926 INFO:teuthology.orchestra.run.smithi173.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 851 kB/s | 141 kB 00:00 2023-11-24T06:24:25.006 INFO:teuthology.orchestra.run.smithi016.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 21 kB/s | 6.5 kB 00:00 2023-11-24T06:24:25.373 INFO:teuthology.orchestra.run.smithi016.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 2.3 MB/s | 859 kB 00:00 2023-11-24T06:24:25.432 INFO:teuthology.orchestra.run.smithi016.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 2.4 MB/s | 141 kB 00:00 2023-11-24T06:24:25.484 INFO:teuthology.orchestra.run.smithi173.stdout:(4/147): ceph-common-18.0.0-7484.ga13ea5ff.el9. 12 MB/s | 19 MB 00:01 2023-11-24T06:24:25.609 INFO:teuthology.orchestra.run.smithi173.stdout:(5/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 3.0 MB/s | 5.2 MB 00:01 2023-11-24T06:24:25.710 INFO:teuthology.orchestra.run.smithi173.stdout:(6/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 6.5 MB/s | 1.5 MB 00:00 2023-11-24T06:24:25.857 INFO:teuthology.orchestra.run.smithi016.stdout:(4/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 4.9 MB/s | 2.1 MB 00:00 2023-11-24T06:24:25.918 INFO:teuthology.orchestra.run.smithi173.stdout:(7/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 2.1 MB/s | 2.1 MB 00:00 2023-11-24T06:24:26.274 INFO:teuthology.orchestra.run.smithi016.stdout:(5/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 3.5 MB/s | 1.5 MB 00:00 2023-11-24T06:24:26.441 INFO:teuthology.orchestra.run.smithi016.stdout:(6/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 3.0 MB/s | 5.2 MB 00:01 2023-11-24T06:24:26.635 INFO:teuthology.orchestra.run.smithi173.stdout:(8/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 4.3 MB/s | 4.4 MB 00:01 2023-11-24T06:24:26.769 INFO:teuthology.orchestra.run.smithi173.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 181 kB/s | 24 kB 00:00 2023-11-24T06:24:27.236 INFO:teuthology.orchestra.run.smithi173.stdout:(10/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 12 MB/s | 18 MB 00:01 2023-11-24T06:24:27.319 INFO:teuthology.orchestra.run.smithi173.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.6 MB/s | 217 kB 00:00 2023-11-24T06:24:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:(7/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 4.1 MB/s | 4.4 MB 00:01 2023-11-24T06:24:27.386 INFO:teuthology.orchestra.run.smithi173.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 474 kB/s | 31 kB 00:00 2023-11-24T06:24:27.570 INFO:teuthology.orchestra.run.smithi173.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 3.9 MB/s | 719 kB 00:00 2023-11-24T06:24:27.670 INFO:teuthology.orchestra.run.smithi173.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.6 MB/s | 163 kB 00:00 2023-11-24T06:24:27.770 INFO:teuthology.orchestra.run.smithi173.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.2 MB/s | 126 kB 00:00 2023-11-24T06:24:27.887 INFO:teuthology.orchestra.run.smithi173.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 4.0 MB/s | 476 kB 00:00 2023-11-24T06:24:28.463 INFO:teuthology.orchestra.run.smithi173.stdout:(17/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 8.3 MB/s | 4.8 MB 00:00 2023-11-24T06:24:28.555 INFO:teuthology.orchestra.run.smithi173.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 498 kB/s | 45 kB 00:00 2023-11-24T06:24:28.655 INFO:teuthology.orchestra.run.smithi173.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 1.3 MB/s | 129 kB 00:00 2023-11-24T06:24:28.747 INFO:teuthology.orchestra.run.smithi173.stdout:(20/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.7 MB/s | 162 kB 00:00 2023-11-24T06:24:28.830 INFO:teuthology.orchestra.run.smithi173.stdout:(21/147): python3-rados-18.0.0-7484.ga13ea5ff.e 3.8 MB/s | 319 kB 00:00 2023-11-24T06:24:28.922 INFO:teuthology.orchestra.run.smithi173.stdout:(22/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 3.2 MB/s | 298 kB 00:00 2023-11-24T06:24:28.966 INFO:teuthology.orchestra.run.smithi016.stdout:(8/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x86 7.0 MB/s | 18 MB 00:02 2023-11-24T06:24:29.014 INFO:teuthology.orchestra.run.smithi173.stdout:(23/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 1.1 MB/s | 99 kB 00:00 2023-11-24T06:24:29.018 INFO:teuthology.orchestra.run.smithi016.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 482 kB/s | 24 kB 00:00 2023-11-24T06:24:29.089 INFO:teuthology.orchestra.run.smithi173.stdout:(24/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.1 MB/s | 85 kB 00:00 2023-11-24T06:24:29.401 INFO:teuthology.orchestra.run.smithi016.stdout:(10/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 4.2 MB/s | 8.6 MB 00:02 2023-11-24T06:24:29.432 INFO:teuthology.orchestra.run.smithi173.stdout:(25/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 2.5 MB/s | 8.6 MB 00:03 2023-11-24T06:24:29.485 INFO:teuthology.orchestra.run.smithi016.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.6 MB/s | 217 kB 00:00 2023-11-24T06:24:29.532 INFO:teuthology.orchestra.run.smithi173.stdout:(26/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 6.8 MB/s | 3.0 MB 00:00 2023-11-24T06:24:29.535 INFO:teuthology.orchestra.run.smithi016.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 634 kB/s | 31 kB 00:00 2023-11-24T06:24:29.590 INFO:teuthology.orchestra.run.smithi173.stdout:(27/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 1.1 MB/s | 170 kB 00:00 2023-11-24T06:24:29.607 INFO:teuthology.orchestra.run.smithi173.stdout:(28/147): ceph-grafana-dashboards-18.0.0-7484.g 377 kB/s | 28 kB 00:00 2023-11-24T06:24:29.683 INFO:teuthology.orchestra.run.smithi173.stdout:(29/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.4 MB/s | 133 kB 00:00 2023-11-24T06:24:29.752 INFO:teuthology.orchestra.run.smithi016.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 3.3 MB/s | 719 kB 00:00 2023-11-24T06:24:29.828 INFO:teuthology.orchestra.run.smithi016.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 2.1 MB/s | 163 kB 00:00 2023-11-24T06:24:29.841 INFO:teuthology.orchestra.run.smithi173.stdout:(30/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 7.6 MB/s | 1.8 MB 00:00 2023-11-24T06:24:29.903 INFO:teuthology.orchestra.run.smithi016.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.7 MB/s | 126 kB 00:00 2023-11-24T06:24:29.950 INFO:teuthology.orchestra.run.smithi173.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7484.ga1 2.2 MB/s | 243 kB 00:00 2023-11-24T06:24:30.025 INFO:teuthology.orchestra.run.smithi173.stdout:(32/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 676 kB/s | 50 kB 00:00 2023-11-24T06:24:30.073 INFO:teuthology.orchestra.run.smithi016.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 2.8 MB/s | 476 kB 00:00 2023-11-24T06:24:30.084 INFO:teuthology.orchestra.run.smithi173.stdout:(33/147): ceph-prometheus-alerts-18.0.0-7484.ga 262 kB/s | 15 kB 00:00 2023-11-24T06:24:30.176 INFO:teuthology.orchestra.run.smithi173.stdout:(34/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 2.8 MB/s | 262 kB 00:00 2023-11-24T06:24:30.309 INFO:teuthology.orchestra.run.smithi173.stdout:(35/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 5.7 MB/s | 772 kB 00:00 2023-11-24T06:24:30.393 INFO:teuthology.orchestra.run.smithi173.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 221 kB/s | 18 kB 00:00 2023-11-24T06:24:30.660 INFO:teuthology.orchestra.run.smithi173.stdout:(37/147): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2023-11-24T06:24:30.769 INFO:teuthology.orchestra.run.smithi173.stdout:(38/147): bison-3.7.4-5.el9.x86_64.rpm 8.4 MB/s | 920 kB 00:00 2023-11-24T06:24:30.819 INFO:teuthology.orchestra.run.smithi173.stdout:(39/147): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2023-11-24T06:24:30.861 INFO:teuthology.orchestra.run.smithi173.stdout:(40/147): elfutils-libelf-devel-0.190-1.el9.x86 527 kB/s | 21 kB 00:00 2023-11-24T06:24:30.911 INFO:teuthology.orchestra.run.smithi173.stdout:(41/147): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 307 kB 00:00 2023-11-24T06:24:30.981 INFO:teuthology.orchestra.run.smithi173.stdout:(42/147): flexiblas-3.0.4-8.el9.x86_64.rpm 457 kB/s | 32 kB 00:00 2023-11-24T06:24:31.103 INFO:teuthology.orchestra.run.smithi173.stdout:(43/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 25 MB/s | 3.0 MB 00:00 2023-11-24T06:24:31.112 INFO:teuthology.orchestra.run.smithi016.stdout:(17/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 4.6 MB/s | 4.8 MB 00:01 2023-11-24T06:24:31.153 INFO:teuthology.orchestra.run.smithi173.stdout:(44/147): flexiblas-openblas-openmp-3.0.4-8.el9 350 kB/s | 17 kB 00:00 2023-11-24T06:24:31.196 INFO:teuthology.orchestra.run.smithi016.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 549 kB/s | 45 kB 00:00 2023-11-24T06:24:31.212 INFO:teuthology.orchestra.run.smithi173.stdout:(45/147): jq-1.6-16.el9.x86_64.rpm 3.2 MB/s | 186 kB 00:00 2023-11-24T06:24:31.254 INFO:teuthology.orchestra.run.smithi016.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 2.2 MB/s | 129 kB 00:00 2023-11-24T06:24:31.254 INFO:teuthology.orchestra.run.smithi173.stdout:(46/147): libaio-devel-0.3.111-13.el9.x86_64.rp 246 kB/s | 10 kB 00:00 2023-11-24T06:24:31.326 INFO:teuthology.orchestra.run.smithi173.stdout:(47/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.2 MB/s | 159 kB 00:00 2023-11-24T06:24:31.329 INFO:teuthology.orchestra.run.smithi016.stdout:(20/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 2.1 MB/s | 162 kB 00:00 2023-11-24T06:24:31.380 INFO:teuthology.orchestra.run.smithi173.stdout:(48/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 840 kB/s | 44 kB 00:00 2023-11-24T06:24:31.438 INFO:teuthology.orchestra.run.smithi016.stdout:(21/147): python3-rados-18.0.0-7484.ga13ea5ff.e 2.9 MB/s | 319 kB 00:00 2023-11-24T06:24:31.438 INFO:teuthology.orchestra.run.smithi173.stdout:(49/147): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2023-11-24T06:24:31.497 INFO:teuthology.orchestra.run.smithi173.stdout:(50/147): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2023-11-24T06:24:31.538 INFO:teuthology.orchestra.run.smithi016.stdout:(22/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 2.9 MB/s | 298 kB 00:00 2023-11-24T06:24:31.539 INFO:teuthology.orchestra.run.smithi173.stdout:(51/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 601 kB/s | 24 kB 00:00 2023-11-24T06:24:31.581 INFO:teuthology.orchestra.run.smithi173.stdout:(52/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2023-11-24T06:24:31.597 INFO:teuthology.orchestra.run.smithi016.stdout:(23/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 1.7 MB/s | 99 kB 00:00 2023-11-24T06:24:31.631 INFO:teuthology.orchestra.run.smithi173.stdout:(53/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2023-11-24T06:24:31.655 INFO:teuthology.orchestra.run.smithi016.stdout:(24/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.5 MB/s | 85 kB 00:00 2023-11-24T06:24:31.707 INFO:teuthology.orchestra.run.smithi173.stdout:(54/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2023-11-24T06:24:31.749 INFO:teuthology.orchestra.run.smithi173.stdout:(55/147): openblas-0.3.21-2.el9.x86_64.rpm 839 kB/s | 34 kB 00:00 2023-11-24T06:24:31.899 INFO:teuthology.orchestra.run.smithi173.stdout:(56/147): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2023-11-24T06:24:32.041 INFO:teuthology.orchestra.run.smithi173.stdout:(57/147): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2023-11-24T06:24:32.108 INFO:teuthology.orchestra.run.smithi173.stdout:(58/147): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2023-11-24T06:24:32.232 INFO:teuthology.orchestra.run.smithi016.stdout:(25/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 5.2 MB/s | 3.0 MB 00:00 2023-11-24T06:24:32.289 INFO:teuthology.orchestra.run.smithi016.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 2.9 MB/s | 170 kB 00:00 2023-11-24T06:24:32.348 INFO:teuthology.orchestra.run.smithi016.stdout:(27/147): ceph-grafana-dashboards-18.0.0-7484.g 488 kB/s | 28 kB 00:00 2023-11-24T06:24:32.415 INFO:teuthology.orchestra.run.smithi016.stdout:(28/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 2.0 MB/s | 133 kB 00:00 2023-11-24T06:24:32.610 INFO:teuthology.orchestra.run.smithi173.stdout:(59/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 7.1 MB/s | 41 MB 00:05 2023-11-24T06:24:32.633 INFO:teuthology.orchestra.run.smithi173.stdout:(60/147): postgresql-private-libs-13.11-1.el9.x 255 kB/s | 134 kB 00:00 2023-11-24T06:24:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:(29/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 6.6 MB/s | 1.8 MB 00:00 2023-11-24T06:24:32.792 INFO:teuthology.orchestra.run.smithi173.stdout:(61/147): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2023-11-24T06:24:32.942 INFO:teuthology.orchestra.run.smithi173.stdout:(62/147): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2023-11-24T06:24:32.967 INFO:teuthology.orchestra.run.smithi173.stdout:(63/147): python3-jinja2-2.11.3-4.el9.noarch.rp 1.4 MB/s | 228 kB 00:00 2023-11-24T06:24:32.990 INFO:teuthology.orchestra.run.smithi016.stdout:(30/147): ceph-common-18.0.0-7484.ga13ea5ff.el9 2.3 MB/s | 19 MB 00:08 2023-11-24T06:24:33.101 INFO:teuthology.orchestra.run.smithi173.stdout:(64/147): ceph-mgr-diskprediction-local-18.0.0- 2.2 MB/s | 7.4 MB 00:03 2023-11-24T06:24:33.118 INFO:teuthology.orchestra.run.smithi173.stdout:(65/147): python3-jsonpointer-2.0-4.el9.noarch. 121 kB/s | 18 kB 00:00 2023-11-24T06:24:33.143 INFO:teuthology.orchestra.run.smithi173.stdout:(66/147): python3-jsonpatch-1.21-16.el9.noarch. 140 kB/s | 24 kB 00:00 2023-11-24T06:24:33.169 INFO:teuthology.orchestra.run.smithi173.stdout:(67/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2023-11-24T06:24:33.194 INFO:teuthology.orchestra.run.smithi173.stdout:(68/147): python3-markupsafe-1.1.1-12.el9.x86_6 423 kB/s | 32 kB 00:00 2023-11-24T06:24:33.328 INFO:teuthology.orchestra.run.smithi173.stdout:(69/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2023-11-24T06:24:33.352 INFO:teuthology.orchestra.run.smithi173.stdout:(70/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2023-11-24T06:24:33.377 INFO:teuthology.orchestra.run.smithi173.stdout:(71/147): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2023-11-24T06:24:33.428 INFO:teuthology.orchestra.run.smithi173.stdout:(72/147): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 41 kB 00:00 2023-11-24T06:24:33.453 INFO:teuthology.orchestra.run.smithi173.stdout:(73/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2023-11-24T06:24:33.487 INFO:teuthology.orchestra.run.smithi173.stdout:(74/147): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 211 kB 00:00 2023-11-24T06:24:33.512 INFO:teuthology.orchestra.run.smithi173.stdout:(75/147): python3-pytz-2021.1-5.el9.noarch.rpm 573 kB/s | 47 kB 00:00 2023-11-24T06:24:33.516 INFO:teuthology.orchestra.run.smithi016.stdout:(31/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 9.2 MB/s | 41 MB 00:04 2023-11-24T06:24:33.549 INFO:teuthology.orchestra.run.smithi016.stdout:(32/147): ceph-mgr-modules-core-18.0.0-7484.ga1 436 kB/s | 243 kB 00:00 2023-11-24T06:24:33.554 INFO:teuthology.orchestra.run.smithi173.stdout:(76/147): python3-requests-oauthlib-1.3.0-12.el 654 kB/s | 43 kB 00:00 2023-11-24T06:24:33.574 INFO:teuthology.orchestra.run.smithi016.stdout:(33/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 877 kB/s | 50 kB 00:00 2023-11-24T06:24:33.608 INFO:teuthology.orchestra.run.smithi016.stdout:(34/147): ceph-prometheus-alerts-18.0.0-7484.ga 262 kB/s | 15 kB 00:00 2023-11-24T06:24:33.668 INFO:teuthology.orchestra.run.smithi016.stdout:(35/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 2.8 MB/s | 262 kB 00:00 2023-11-24T06:24:33.720 INFO:teuthology.orchestra.run.smithi016.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 372 kB/s | 18 kB 00:00 2023-11-24T06:24:33.925 INFO:teuthology.orchestra.run.smithi016.stdout:(37/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 2.4 MB/s | 772 kB 00:00 2023-11-24T06:24:33.955 INFO:teuthology.orchestra.run.smithi173.stdout:(77/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2023-11-24T06:24:33.980 INFO:teuthology.orchestra.run.smithi173.stdout:(78/147): python3-toml-0.10.2-6.el9.noarch.rpm 86 kB/s | 39 kB 00:00 2023-11-24T06:24:34.013 INFO:teuthology.orchestra.run.smithi173.stdout:(79/147): socat-1.7.4.1-5.el9.x86_64.rpm 674 kB/s | 300 kB 00:00 2023-11-24T06:24:34.038 INFO:teuthology.orchestra.run.smithi173.stdout:(80/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.5 MB/s | 85 kB 00:00 2023-11-24T06:24:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:(38/147): ceph-mgr-diskprediction-local-18.0.0- 5.4 MB/s | 7.4 MB 00:01 2023-11-24T06:24:34.063 INFO:teuthology.orchestra.run.smithi173.stdout:(81/147): uuid-1.6.2-55.el9.x86_64.rpm 698 kB/s | 56 kB 00:00 2023-11-24T06:24:34.089 INFO:teuthology.orchestra.run.smithi173.stdout:(82/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 845 kB/s | 63 kB 00:00 2023-11-24T06:24:34.101 INFO:teuthology.orchestra.run.smithi016.stdout:(39/147): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 662 kB 00:00 2023-11-24T06:24:34.114 INFO:teuthology.orchestra.run.smithi173.stdout:(83/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 600 kB/s | 44 kB 00:00 2023-11-24T06:24:34.148 INFO:teuthology.orchestra.run.smithi173.stdout:(84/147): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2023-11-24T06:24:34.181 INFO:teuthology.orchestra.run.smithi173.stdout:(85/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.7 MB/s | 803 kB 00:00 2023-11-24T06:24:34.193 INFO:teuthology.orchestra.run.smithi016.stdout:(40/147): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 920 kB 00:00 2023-11-24T06:24:34.214 INFO:teuthology.orchestra.run.smithi173.stdout:(86/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2023-11-24T06:24:34.218 INFO:teuthology.orchestra.run.smithi016.stdout:(41/147): boost-program-options-1.75.0-8.el9.x8 687 kB/s | 106 kB 00:00 2023-11-24T06:24:34.243 INFO:teuthology.orchestra.run.smithi016.stdout:(42/147): elfutils-libelf-devel-0.190-1.el9.x86 155 kB/s | 21 kB 00:00 2023-11-24T06:24:34.257 INFO:teuthology.orchestra.run.smithi173.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2023-11-24T06:24:34.277 INFO:teuthology.orchestra.run.smithi016.stdout:(43/147): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 307 kB 00:00 2023-11-24T06:24:34.318 INFO:teuthology.orchestra.run.smithi173.stdout:(88/147): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.1 MB 00:00 2023-11-24T06:24:34.340 INFO:teuthology.orchestra.run.smithi173.stdout:(89/147): python3-ply-3.11-14.el9.noarch.rpm 860 kB/s | 103 kB 00:00 2023-11-24T06:24:34.344 INFO:teuthology.orchestra.run.smithi016.stdout:(44/147): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 32 kB 00:00 2023-11-24T06:24:34.358 INFO:teuthology.orchestra.run.smithi173.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 3.0 MB/s | 124 kB 00:00 2023-11-24T06:24:34.382 INFO:teuthology.orchestra.run.smithi173.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 1.7 MB/s | 113 kB 00:00 2023-11-24T06:24:34.416 INFO:teuthology.orchestra.run.smithi173.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 2.5 MB/s | 187 kB 00:00 2023-11-24T06:24:34.528 INFO:teuthology.orchestra.run.smithi016.stdout:(45/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2023-11-24T06:24:34.553 INFO:teuthology.orchestra.run.smithi016.stdout:(46/147): flexiblas-openblas-openmp-3.0.4-8.el9 63 kB/s | 17 kB 00:00 2023-11-24T06:24:34.587 INFO:teuthology.orchestra.run.smithi016.stdout:(47/147): jq-1.6-16.el9.x86_64.rpm 802 kB/s | 186 kB 00:00 2023-11-24T06:24:34.612 INFO:teuthology.orchestra.run.smithi016.stdout:(48/147): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 10 kB 00:00 2023-11-24T06:24:34.645 INFO:teuthology.orchestra.run.smithi016.stdout:(49/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2023-11-24T06:24:34.671 INFO:teuthology.orchestra.run.smithi016.stdout:(50/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 791 kB/s | 44 kB 00:00 2023-11-24T06:24:34.683 INFO:teuthology.orchestra.run.smithi173.stdout:(93/147): python3-devel-3.9.17-2.el9.x86_64.rpm 825 kB/s | 245 kB 00:00 2023-11-24T06:24:34.737 INFO:teuthology.orchestra.run.smithi016.stdout:(51/147): librdkafka-1.6.1-102.el9.x86_64.rpm 5.3 MB/s | 661 kB 00:00 2023-11-24T06:24:34.787 INFO:teuthology.orchestra.run.smithi016.stdout:(52/147): libtool-2.4.6-45.el9.x86_64.rpm 4.1 MB/s | 578 kB 00:00 2023-11-24T06:24:34.805 INFO:teuthology.orchestra.run.smithi016.stdout:(53/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 191 kB/s | 24 kB 00:00 2023-11-24T06:24:34.821 INFO:teuthology.orchestra.run.smithi016.stdout:(54/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 595 kB/s | 46 kB 00:00 2023-11-24T06:24:34.872 INFO:teuthology.orchestra.run.smithi016.stdout:(55/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2023-11-24T06:24:34.905 INFO:teuthology.orchestra.run.smithi016.stdout:(56/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.1 MB/s | 217 kB 00:00 2023-11-24T06:24:34.908 INFO:teuthology.orchestra.run.smithi173.stdout:(94/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 629 kB/s | 309 kB 00:00 2023-11-24T06:24:34.922 INFO:teuthology.orchestra.run.smithi016.stdout:(57/147): openblas-0.3.21-2.el9.x86_64.rpm 342 kB/s | 34 kB 00:00 2023-11-24T06:24:35.025 INFO:teuthology.orchestra.run.smithi173.stdout:(95/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 210 kB/s | 24 kB 00:00 2023-11-24T06:24:35.143 INFO:teuthology.orchestra.run.smithi173.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 419 kB/s | 49 kB 00:00 2023-11-24T06:24:35.164 INFO:teuthology.orchestra.run.smithi016.stdout:(58/147): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2023-11-24T06:24:35.231 INFO:teuthology.orchestra.run.smithi016.stdout:(59/147): postgresql-13.11-1.el9.x86_64.rpm 5.0 MB/s | 1.5 MB 00:00 2023-11-24T06:24:35.259 INFO:teuthology.orchestra.run.smithi173.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 581 kB/s | 67 kB 00:00 2023-11-24T06:24:35.265 INFO:teuthology.orchestra.run.smithi016.stdout:(60/147): postgresql-contrib-13.11-1.el9.x86_64 2.4 MB/s | 811 kB 00:00 2023-11-24T06:24:35.290 INFO:teuthology.orchestra.run.smithi016.stdout:(61/147): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 134 kB 00:00 2023-11-24T06:24:35.501 INFO:teuthology.orchestra.run.smithi173.stdout:(98/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.4 MB/s | 837 kB 00:00 2023-11-24T06:24:35.630 INFO:teuthology.orchestra.run.smithi016.stdout:(62/147): postgresql-server-13.11-1.el9.x86_64. 19 MB/s | 5.7 MB 00:00 2023-11-24T06:24:35.683 INFO:teuthology.orchestra.run.smithi016.stdout:(63/147): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2023-11-24T06:24:35.707 INFO:teuthology.orchestra.run.smithi016.stdout:(64/147): python3-jinja2-2.11.3-4.el9.noarch.rp 556 kB/s | 228 kB 00:00 2023-11-24T06:24:35.719 INFO:teuthology.orchestra.run.smithi173.stdout:(99/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 3.1 MB/s | 4.1 MB 00:01 2023-11-24T06:24:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:(65/147): python3-jsonpatch-1.21-16.el9.noarch. 152 kB/s | 24 kB 00:00 2023-11-24T06:24:35.758 INFO:teuthology.orchestra.run.smithi016.stdout:(66/147): python3-jsonpointer-2.0-4.el9.noarch. 371 kB/s | 18 kB 00:00 2023-11-24T06:24:35.777 INFO:teuthology.orchestra.run.smithi173.stdout:(100/147): python3-asyncssh-2.13.2-3.el9.noarch 2.0 MB/s | 548 kB 00:00 2023-11-24T06:24:35.783 INFO:teuthology.orchestra.run.smithi016.stdout:(67/147): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-11-24T06:24:35.808 INFO:teuthology.orchestra.run.smithi016.stdout:(68/147): python3-markupsafe-1.1.1-12.el9.x86_6 432 kB/s | 32 kB 00:00 2023-11-24T06:24:35.842 INFO:teuthology.orchestra.run.smithi016.stdout:(69/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 217 kB 00:00 2023-11-24T06:24:35.885 INFO:teuthology.orchestra.run.smithi173.stdout:(101/147): libarrow-9.0.0-9.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2023-11-24T06:24:35.910 INFO:teuthology.orchestra.run.smithi173.stdout:(102/147): python3-cachetools-4.2.4-1.el9.noarc 242 kB/s | 32 kB 00:00 2023-11-24T06:24:35.977 INFO:teuthology.orchestra.run.smithi173.stdout:(103/147): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2023-11-24T06:24:35.992 INFO:teuthology.orchestra.run.smithi016.stdout:(70/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2023-11-24T06:24:36.011 INFO:teuthology.orchestra.run.smithi173.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2023-11-24T06:24:36.017 INFO:teuthology.orchestra.run.smithi016.stdout:(71/147): python3-oauthlib-3.1.1-5.el9.noarch.r 873 kB/s | 174 kB 00:00 2023-11-24T06:24:36.042 INFO:teuthology.orchestra.run.smithi016.stdout:(72/147): python3-prettytable-0.7.2-27.el9.noar 222 kB/s | 41 kB 00:00 2023-11-24T06:24:36.093 INFO:teuthology.orchestra.run.smithi016.stdout:(73/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2023-11-24T06:24:36.103 INFO:teuthology.orchestra.run.smithi173.stdout:(105/147): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2023-11-24T06:24:36.118 INFO:teuthology.orchestra.run.smithi016.stdout:(74/147): python3-pytz-2021.1-5.el9.noarch.rpm 646 kB/s | 47 kB 00:00 2023-11-24T06:24:36.136 INFO:teuthology.orchestra.run.smithi173.stdout:(106/147): python3-google-auth-2.23.4-1.el9.noa 1.5 MB/s | 197 kB 00:00 2023-11-24T06:24:36.151 INFO:teuthology.orchestra.run.smithi016.stdout:(75/147): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 211 kB 00:00 2023-11-24T06:24:36.162 INFO:teuthology.orchestra.run.smithi173.stdout:(107/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 98 kB/s | 43 kB 00:00 2023-11-24T06:24:36.193 INFO:teuthology.orchestra.run.smithi016.stdout:(76/147): python3-requests-oauthlib-1.3.0-12.el 593 kB/s | 43 kB 00:00 2023-11-24T06:24:36.196 INFO:teuthology.orchestra.run.smithi173.stdout:(108/147): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2023-11-24T06:24:36.229 INFO:teuthology.orchestra.run.smithi173.stdout:(109/147): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2023-11-24T06:24:36.263 INFO:teuthology.orchestra.run.smithi173.stdout:(110/147): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2023-11-24T06:24:36.297 INFO:teuthology.orchestra.run.smithi173.stdout:(111/147): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2023-11-24T06:24:36.330 INFO:teuthology.orchestra.run.smithi173.stdout:(112/147): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2023-11-24T06:24:36.364 INFO:teuthology.orchestra.run.smithi173.stdout:(113/147): python3-jwt+crypto-2.4.0-1.el9.noarc 90 kB/s | 9.0 kB 00:00 2023-11-24T06:24:36.397 INFO:teuthology.orchestra.run.smithi173.stdout:(114/147): python3-jwt-2.4.0-1.el9.noarch.rpm 409 kB/s | 41 kB 00:00 2023-11-24T06:24:36.494 INFO:teuthology.orchestra.run.smithi016.stdout:(77/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2023-11-24T06:24:36.514 INFO:teuthology.orchestra.run.smithi173.stdout:(115/147): python3-logutils-0.3.5-21.el9.noarch 309 kB/s | 46 kB 00:00 2023-11-24T06:24:36.527 INFO:teuthology.orchestra.run.smithi016.stdout:(78/147): python3-toml-0.10.2-6.el9.noarch.rpm 108 kB/s | 39 kB 00:00 2023-11-24T06:24:36.540 INFO:teuthology.orchestra.run.smithi173.stdout:(116/147): python3-more-itertools-8.12.0-2.el9. 558 kB/s | 79 kB 00:00 2023-11-24T06:24:36.561 INFO:teuthology.orchestra.run.smithi016.stdout:(79/147): socat-1.7.4.1-5.el9.x86_64.rpm 850 kB/s | 300 kB 00:00 2023-11-24T06:24:36.581 INFO:teuthology.orchestra.run.smithi173.stdout:(117/147): python3-kubernetes-26.1.0-2.el9.noar 4.1 MB/s | 1.0 MB 00:00 2023-11-24T06:24:36.586 INFO:teuthology.orchestra.run.smithi016.stdout:(80/147): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2023-11-24T06:24:36.611 INFO:teuthology.orchestra.run.smithi016.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.0 MB/s | 85 kB 00:00 2023-11-24T06:24:36.637 INFO:teuthology.orchestra.run.smithi016.stdout:(82/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 856 kB/s | 63 kB 00:00 2023-11-24T06:24:36.640 INFO:teuthology.orchestra.run.smithi173.stdout:(118/147): python3-natsort-7.1.1-5.el9.noarch.r 463 kB/s | 58 kB 00:00 2023-11-24T06:24:36.662 INFO:teuthology.orchestra.run.smithi016.stdout:(83/147): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2023-11-24T06:24:36.673 INFO:teuthology.orchestra.run.smithi173.stdout:(119/147): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2023-11-24T06:24:36.695 INFO:teuthology.orchestra.run.smithi016.stdout:(84/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 536 kB/s | 44 kB 00:00 2023-11-24T06:24:36.698 INFO:teuthology.orchestra.run.smithi173.stdout:(120/147): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2023-11-24T06:24:36.729 INFO:teuthology.orchestra.run.smithi016.stdout:(85/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.7 MB/s | 803 kB 00:00 2023-11-24T06:24:36.788 INFO:teuthology.orchestra.run.smithi016.stdout:(86/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2023-11-24T06:24:36.803 INFO:teuthology.orchestra.run.smithi173.stdout:(121/147): python3-repoze-lru-0.7-16.el9.noarch 239 kB/s | 31 kB 00:00 2023-11-24T06:24:36.833 INFO:teuthology.orchestra.run.smithi173.stdout:(122/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 469 kB/s | 90 kB 00:00 2023-11-24T06:24:36.838 INFO:teuthology.orchestra.run.smithi016.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 241 kB 00:00 2023-11-24T06:24:36.867 INFO:teuthology.orchestra.run.smithi173.stdout:(123/147): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2023-11-24T06:24:36.879 INFO:teuthology.orchestra.run.smithi016.stdout:(88/147): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.1 MB 00:00 2023-11-24T06:24:36.905 INFO:teuthology.orchestra.run.smithi016.stdout:(89/147): python3-ply-3.11-14.el9.noarch.rpm 995 kB/s | 103 kB 00:00 2023-11-24T06:24:36.916 INFO:teuthology.orchestra.run.smithi173.stdout:(124/147): python3-rsa-4.9-2.el9.noarch.rpm 527 kB/s | 59 kB 00:00 2023-11-24T06:24:36.938 INFO:teuthology.orchestra.run.smithi016.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2023-11-24T06:24:36.950 INFO:teuthology.orchestra.run.smithi173.stdout:(125/147): python3-tempora-5.0.0-2.el9.noarch.r 307 kB/s | 36 kB 00:00 2023-11-24T06:24:36.964 INFO:teuthology.orchestra.run.smithi016.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 1.4 MB/s | 113 kB 00:00 2023-11-24T06:24:36.975 INFO:teuthology.orchestra.run.smithi173.stdout:(126/147): python3-typing-extensions-4.4.0-2.el 471 kB/s | 51 kB 00:00 2023-11-24T06:24:36.997 INFO:teuthology.orchestra.run.smithi016.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 2.1 MB/s | 187 kB 00:00 2023-11-24T06:24:37.026 INFO:teuthology.orchestra.run.smithi173.stdout:(127/147): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2023-11-24T06:24:37.093 INFO:teuthology.orchestra.run.smithi173.stdout:(128/147): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2023-11-24T06:24:37.126 INFO:teuthology.orchestra.run.smithi173.stdout:(129/147): python3-zc-lockfile-2.0-10.el9.noarc 201 kB/s | 20 kB 00:00 2023-11-24T06:24:37.176 INFO:teuthology.orchestra.run.smithi173.stdout:(130/147): python3-websocket-client-1.2.3-2.el9 398 kB/s | 90 kB 00:00 2023-11-24T06:24:37.243 INFO:teuthology.orchestra.run.smithi173.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2023-11-24T06:24:37.289 INFO:teuthology.orchestra.run.smithi016.stdout:(93/147): python3-devel-3.9.17-2.el9.x86_64.rpm 762 kB/s | 245 kB 00:00 2023-11-24T06:24:37.460 INFO:teuthology.orchestra.run.smithi173.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2023-11-24T06:24:37.535 INFO:teuthology.orchestra.run.smithi173.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 912 kB/s | 68 kB 00:00 2023-11-24T06:24:37.590 INFO:teuthology.orchestra.run.smithi016.stdout:(94/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 523 kB/s | 309 kB 00:00 2023-11-24T06:24:37.644 INFO:teuthology.orchestra.run.smithi173.stdout:(134/147): NetworkManager-1.45.6-1.el9.x86_64.r 21 MB/s | 2.2 MB 00:00 2023-11-24T06:24:37.706 INFO:teuthology.orchestra.run.smithi016.stdout:(95/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 210 kB/s | 24 kB 00:00 2023-11-24T06:24:37.736 INFO:teuthology.orchestra.run.smithi173.stdout:(135/147): NetworkManager-libnm-1.45.6-1.el9.x8 19 MB/s | 1.8 MB 00:00 2023-11-24T06:24:37.786 INFO:teuthology.orchestra.run.smithi173.stdout:(136/147): NetworkManager-team-1.45.6-1.el9.x86 856 kB/s | 42 kB 00:00 2023-11-24T06:24:37.815 INFO:teuthology.orchestra.run.smithi016.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 452 kB/s | 49 kB 00:00 2023-11-24T06:24:37.828 INFO:teuthology.orchestra.run.smithi173.stdout:(137/147): NetworkManager-tui-1.45.6-1.el9.x86_ 5.9 MB/s | 245 kB 00:00 2023-11-24T06:24:37.920 INFO:teuthology.orchestra.run.smithi173.stdout:(138/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 4.5 MB/s | 3.0 MB 00:00 2023-11-24T06:24:37.932 INFO:teuthology.orchestra.run.smithi016.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 581 kB/s | 67 kB 00:00 2023-11-24T06:24:37.946 INFO:teuthology.orchestra.run.smithi173.stdout:(139/147): acl-2.3.1-4.el9.x86_64.rpm 591 kB/s | 69 kB 00:00 2023-11-24T06:24:37.971 INFO:teuthology.orchestra.run.smithi173.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 748 kB/s | 36 kB 00:00 2023-11-24T06:24:38.005 INFO:teuthology.orchestra.run.smithi173.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2023-11-24T06:24:38.038 INFO:teuthology.orchestra.run.smithi173.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 3.9 MB/s | 255 kB 00:00 2023-11-24T06:24:38.064 INFO:teuthology.orchestra.run.smithi173.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 394 kB/s | 22 kB 00:00 2023-11-24T06:24:38.097 INFO:teuthology.orchestra.run.smithi173.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 321 kB 00:00 2023-11-24T06:24:38.166 INFO:teuthology.orchestra.run.smithi016.stdout:(98/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 3.4 MB/s | 4.1 MB 00:01 2023-11-24T06:24:38.291 INFO:teuthology.orchestra.run.smithi016.stdout:(99/147): libarrow-9.0.0-9.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2023-11-24T06:24:38.322 INFO:teuthology.orchestra.run.smithi173.stdout:(145/147): librados2-18.0.0-7484.ga13ea5ff.el9. 2.9 MB/s | 3.3 MB 00:01 2023-11-24T06:24:38.324 INFO:teuthology.orchestra.run.smithi016.stdout:(100/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.1 MB/s | 837 kB 00:00 2023-11-24T06:24:38.400 INFO:teuthology.orchestra.run.smithi016.stdout:(101/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 404 kB/s | 43 kB 00:00 2023-11-24T06:24:38.460 INFO:teuthology.orchestra.run.smithi016.stdout:(102/147): python3-cachetools-4.2.4-1.el9.noarc 239 kB/s | 32 kB 00:00 2023-11-24T06:24:38.509 INFO:teuthology.orchestra.run.smithi016.stdout:(103/147): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2023-11-24T06:24:38.567 INFO:teuthology.orchestra.run.smithi016.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2023-11-24T06:24:38.617 INFO:teuthology.orchestra.run.smithi016.stdout:(105/147): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2023-11-24T06:24:38.693 INFO:teuthology.orchestra.run.smithi016.stdout:(106/147): python3-google-auth-2.23.4-1.el9.noa 1.5 MB/s | 197 kB 00:00 2023-11-24T06:24:38.710 INFO:teuthology.orchestra.run.smithi016.stdout:(107/147): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2023-11-24T06:24:38.785 INFO:teuthology.orchestra.run.smithi016.stdout:(108/147): python3-jaraco-classes-3.2.1-5.el9.n 194 kB/s | 18 kB 00:00 2023-11-24T06:24:38.810 INFO:teuthology.orchestra.run.smithi016.stdout:(109/147): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2023-11-24T06:24:38.872 INFO:teuthology.orchestra.run.smithi016.stdout:(110/147): python3-asyncssh-2.13.2-3.el9.noarch 777 kB/s | 548 kB 00:00 2023-11-24T06:24:38.903 INFO:teuthology.orchestra.run.smithi016.stdout:(111/147): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2023-11-24T06:24:38.928 INFO:teuthology.orchestra.run.smithi016.stdout:(112/147): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2023-11-24T06:24:38.930 INFO:teuthology.orchestra.run.smithi173.stdout:(146/147): openssl-3.0.7-25.el9.x86_64.rpm 1.4 MB/s | 1.2 MB 00:00 2023-11-24T06:24:38.978 INFO:teuthology.orchestra.run.smithi016.stdout:(113/147): python3-jwt+crypto-2.4.0-1.el9.noarc 86 kB/s | 9.0 kB 00:00 2023-11-24T06:24:39.003 INFO:teuthology.orchestra.run.smithi016.stdout:(114/147): python3-jwt-2.4.0-1.el9.noarch.rpm 410 kB/s | 41 kB 00:00 2023-11-24T06:24:39.006 INFO:teuthology.orchestra.run.smithi173.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 2.4 MB/s | 2.2 MB 00:00 2023-11-24T06:24:39.015 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:24:39.015 INFO:teuthology.orchestra.run.smithi173.stdout:Total 13 MB/s | 204 MB 00:15 2023-11-24T06:24:39.070 INFO:teuthology.orchestra.run.smithi016.stdout:(115/147): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2023-11-24T06:24:39.096 INFO:teuthology.orchestra.run.smithi016.stdout:(116/147): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2023-11-24T06:24:39.121 INFO:teuthology.orchestra.run.smithi016.stdout:(117/147): python3-more-itertools-8.12.0-2.el9. 676 kB/s | 79 kB 00:00 2023-11-24T06:24:39.180 INFO:teuthology.orchestra.run.smithi016.stdout:(118/147): python3-natsort-7.1.1-5.el9.noarch.r 534 kB/s | 58 kB 00:00 2023-11-24T06:24:39.221 INFO:teuthology.orchestra.run.smithi016.stdout:(119/147): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2023-11-24T06:24:39.234 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2023-11-24T06:24:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:(120/147): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2023-11-24T06:24:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:(121/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 635 kB/s | 90 kB 00:00 2023-11-24T06:24:39.347 INFO:teuthology.orchestra.run.smithi016.stdout:(122/147): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2023-11-24T06:24:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:(123/147): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2023-11-24T06:24:39.439 INFO:teuthology.orchestra.run.smithi016.stdout:(124/147): python3-rsa-4.9-2.el9.noarch.rpm 508 kB/s | 59 kB 00:00 2023-11-24T06:24:39.465 INFO:teuthology.orchestra.run.smithi016.stdout:(125/147): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2023-11-24T06:24:39.512 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2023-11-24T06:24:39.512 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2023-11-24T06:24:39.515 INFO:teuthology.orchestra.run.smithi016.stdout:(126/147): python3-typing-extensions-4.4.0-2.el 511 kB/s | 51 kB 00:00 2023-11-24T06:24:39.549 INFO:teuthology.orchestra.run.smithi016.stdout:(127/147): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2023-11-24T06:24:39.565 INFO:teuthology.orchestra.run.smithi016.stdout:(128/147): python3-websocket-client-1.2.3-2.el9 898 kB/s | 90 kB 00:00 2023-11-24T06:24:39.649 INFO:teuthology.orchestra.run.smithi016.stdout:(129/147): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2023-11-24T06:24:39.708 INFO:teuthology.orchestra.run.smithi016.stdout:(130/147): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2023-11-24T06:24:39.733 INFO:teuthology.orchestra.run.smithi016.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2023-11-24T06:24:39.816 INFO:teuthology.orchestra.run.smithi016.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2023-11-24T06:24:39.858 INFO:teuthology.orchestra.run.smithi016.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-11-24T06:24:39.959 INFO:teuthology.orchestra.run.smithi016.stdout:(134/147): NetworkManager-1.45.6-1.el9.x86_64.r 23 MB/s | 2.2 MB 00:00 2023-11-24T06:24:40.051 INFO:teuthology.orchestra.run.smithi016.stdout:(135/147): NetworkManager-libnm-1.45.6-1.el9.x8 19 MB/s | 1.8 MB 00:00 2023-11-24T06:24:40.118 INFO:teuthology.orchestra.run.smithi016.stdout:(136/147): NetworkManager-team-1.45.6-1.el9.x86 639 kB/s | 42 kB 00:00 2023-11-24T06:24:40.168 INFO:teuthology.orchestra.run.smithi016.stdout:(137/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.9 MB/s | 245 kB 00:00 2023-11-24T06:24:40.251 INFO:teuthology.orchestra.run.smithi016.stdout:(138/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 5.8 MB/s | 3.0 MB 00:00 2023-11-24T06:24:40.279 INFO:teuthology.orchestra.run.smithi016.stdout:(139/147): acl-2.3.1-4.el9.x86_64.rpm 626 kB/s | 69 kB 00:00 2023-11-24T06:24:40.304 INFO:teuthology.orchestra.run.smithi016.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 716 kB/s | 36 kB 00:00 2023-11-24T06:24:40.338 INFO:teuthology.orchestra.run.smithi016.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2023-11-24T06:24:40.371 INFO:teuthology.orchestra.run.smithi016.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 3.9 MB/s | 255 kB 00:00 2023-11-24T06:24:40.397 INFO:teuthology.orchestra.run.smithi016.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 394 kB/s | 22 kB 00:00 2023-11-24T06:24:40.430 INFO:teuthology.orchestra.run.smithi016.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2023-11-24T06:24:40.547 INFO:teuthology.orchestra.run.smithi016.stdout:(145/147): librados2-18.0.0-7484.ga13ea5ff.el9. 4.0 MB/s | 3.3 MB 00:00 2023-11-24T06:24:40.689 INFO:teuthology.orchestra.run.smithi016.stdout:(146/147): openssl-3.0.7-25.el9.x86_64.rpm 4.2 MB/s | 1.2 MB 00:00 2023-11-24T06:24:40.773 INFO:teuthology.orchestra.run.smithi016.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 6.3 MB/s | 2.2 MB 00:00 2023-11-24T06:24:40.783 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:24:40.783 INFO:teuthology.orchestra.run.smithi016.stdout:Total 12 MB/s | 204 MB 00:16 2023-11-24T06:24:41.018 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-11-24T06:24:41.318 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-11-24T06:24:41.318 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-11-24T06:24:41.680 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2023-11-24T06:24:41.680 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2023-11-24T06:24:43.541 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-11-24T06:24:43.541 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-11-24T06:24:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2023-11-24T06:24:48.376 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T06:24:48.414 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T06:24:48.487 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T06:24:48.559 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T06:24:48.589 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T06:24:48.686 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T06:24:49.105 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T06:24:49.145 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:24:49.384 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:24:49.421 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:24:49.544 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:24:49.609 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:24:49.673 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T06:24:49.959 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T06:24:50.026 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T06:24:50.051 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:24:50.102 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-11-24T06:24:50.208 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:24:50.236 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:24:50.250 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T06:24:50.334 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T06:24:50.352 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:24:50.550 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T06:24:50.615 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T06:24:50.649 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T06:24:50.660 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T06:24:50.742 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T06:24:50.875 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:24:50.961 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:24:51.136 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:24:51.190 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T06:24:51.194 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:24:51.272 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:24:51.436 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:24:51.464 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:24:51.599 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:24:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:24:51.651 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:24:51.682 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T06:24:51.728 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T06:24:51.828 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T06:24:51.905 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T06:24:51.936 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T06:24:51.991 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T06:24:52.046 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T06:24:52.054 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T06:24:52.111 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T06:24:52.136 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:24:52.190 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T06:24:52.230 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T06:24:52.263 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T06:24:52.309 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:24:52.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:24:52.374 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T06:24:52.410 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T06:24:52.454 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:24:52.486 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T06:24:52.593 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T06:24:52.770 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T06:24:52.779 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T06:24:52.854 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T06:24:52.972 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:24:53.056 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:24:53.254 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:24:53.312 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:24:53.561 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T06:24:53.761 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T06:24:53.792 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:24:53.808 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T06:24:53.843 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T06:24:53.863 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T06:24:53.901 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T06:24:53.952 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T06:24:53.987 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T06:24:54.035 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T06:24:54.072 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T06:24:54.130 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T06:24:54.194 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T06:24:54.195 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T06:24:54.294 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T06:24:54.312 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T06:24:54.358 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T06:24:54.392 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T06:24:54.452 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T06:24:54.491 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T06:24:54.498 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T06:24:54.546 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T06:24:54.557 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T06:24:54.598 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T06:24:54.648 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T06:24:54.691 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T06:24:54.720 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T06:24:54.730 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T06:24:54.779 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T06:24:54.835 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T06:24:54.896 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T06:24:54.922 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T06:24:54.991 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T06:24:55.083 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T06:24:55.131 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T06:24:55.174 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T06:24:55.308 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T06:24:55.378 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T06:24:55.432 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T06:24:55.471 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T06:24:55.630 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T06:24:55.707 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T06:24:55.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T06:24:55.797 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T06:24:55.852 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T06:24:55.879 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T06:24:55.899 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T06:24:55.955 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T06:24:56.003 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T06:24:56.069 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T06:24:56.138 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T06:24:56.195 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T06:24:56.272 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T06:24:56.322 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T06:24:56.373 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T06:24:56.432 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T06:24:56.456 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T06:24:56.458 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T06:24:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T06:24:56.540 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T06:24:56.568 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T06:24:56.665 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T06:24:56.710 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T06:24:56.750 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T06:24:56.829 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T06:24:56.894 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T06:24:56.926 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T06:24:56.953 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T06:24:56.971 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T06:24:57.022 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T06:24:57.083 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T06:24:57.153 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T06:24:57.157 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T06:24:57.203 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T06:24:57.205 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T06:24:57.260 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T06:24:57.358 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T06:24:57.394 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T06:24:57.397 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T06:24:57.439 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T06:24:57.440 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T06:24:57.504 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T06:24:57.512 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T06:24:57.545 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T06:24:57.551 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T06:24:57.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T06:24:57.730 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T06:24:57.766 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T06:24:57.791 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T06:24:57.837 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T06:24:57.936 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T06:24:57.944 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T06:24:58.022 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T06:24:58.221 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T06:24:58.309 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T06:24:58.501 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T06:24:58.557 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T06:24:58.637 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T06:24:58.999 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T06:24:59.056 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T06:24:59.061 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T06:24:59.095 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T06:24:59.119 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T06:24:59.138 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T06:24:59.163 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T06:24:59.215 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T06:24:59.287 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T06:24:59.354 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T06:24:59.375 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T06:24:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T06:24:59.408 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T06:24:59.510 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T06:24:59.547 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T06:24:59.588 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T06:24:59.653 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T06:24:59.726 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T06:24:59.769 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T06:24:59.920 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T06:24:59.975 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T06:25:00.128 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T06:25:00.198 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T06:25:00.297 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T06:25:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T06:25:01.211 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T06:25:01.263 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T06:25:01.322 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T06:25:01.355 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T06:25:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T06:25:01.651 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T06:25:01.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T06:25:01.969 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T06:25:02.062 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T06:25:02.100 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T06:25:02.184 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T06:25:02.259 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T06:25:02.294 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T06:25:02.640 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T06:25:02.663 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T06:25:02.695 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:25:02.777 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T06:25:02.798 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:25:02.834 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T06:25:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T06:25:03.392 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T06:25:03.432 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:25:03.523 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:25:03.550 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T06:25:04.432 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T06:25:04.515 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T06:25:04.574 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T06:25:04.617 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T06:25:04.662 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T06:25:04.705 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T06:25:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T06:25:05.130 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:25:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:25:05.270 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T06:25:05.324 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T06:25:05.826 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T06:25:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:25:05.941 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:25:05.960 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T06:25:06.039 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:06.100 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:06.727 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:06.766 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:25:06.961 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:25:06.961 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T06:25:06.961 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T06:25:06.961 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:07.000 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:25:08.526 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:08.557 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:09.217 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:25:09.261 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:25:09.422 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:25:09.422 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T06:25:09.422 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T06:25:09.422 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:09.478 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:25:19.686 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:25:19.686 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2023-11-24T06:25:19.686 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2023-11-24T06:25:19.686 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2023-11-24T06:25:19.686 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2023-11-24T06:25:19.686 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2023-11-24T06:25:19.686 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /root 2023-11-24T06:25:19.687 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2023-11-24T06:25:19.687 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:19.895 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:25:19.952 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:25:19.952 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:19.952 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T06:25:19.952 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:21.643 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:25:21.745 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:25:21.745 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:21.745 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T06:25:21.745 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:25:21.745 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:25:21.745 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:21.973 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:25:22.035 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:25:22.035 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T06:25:22.036 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:25:22.037 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:22.524 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:25:22.607 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:25:22.607 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:22.608 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T06:25:22.608 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:25:22.608 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:25:22.608 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:22.734 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T06:25:22.972 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:25:22.973 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2023-11-24T06:25:22.973 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2023-11-24T06:25:22.973 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2023-11-24T06:25:22.973 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2023-11-24T06:25:22.973 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2023-11-24T06:25:22.973 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2023-11-24T06:25:22.973 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2023-11-24T06:25:22.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:23.209 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:25:23.265 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:25:23.265 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:23.265 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T06:25:23.265 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:23.543 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:25:23.585 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:25:23.630 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T06:25:23.727 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T06:25:23.756 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:25:24.075 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:25:24.107 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:25:25.054 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:25:25.146 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:25:25.146 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:25.146 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T06:25:25.146 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:25:25.147 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:25:25.147 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:25.234 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:25:25.265 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:25:25.388 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:25:25.406 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:25:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:25:25.463 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:25.463 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T06:25:25.463 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:25:25.463 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:25:25.463 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:25.573 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T06:25:25.609 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:25:25.687 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:25:25.688 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:25.688 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T06:25:25.688 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:25:25.688 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:25:25.688 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:25.752 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:25:25.791 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:25:25.887 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T06:25:25.972 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:25:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:25:26.047 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:26.047 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T06:25:26.047 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:25:26.047 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:25:26.047 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:26.174 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T06:25:26.848 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T06:25:26.872 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:25:26.880 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:25:26.900 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:25:26.952 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:25:26.952 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:26.952 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T06:25:26.952 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:25:26.953 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:25:26.953 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:26.973 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T06:25:27.023 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:25:27.072 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T06:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:27.094 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T06:25:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:27.104 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:25:27.156 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:25:27.232 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:25:27.232 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:27.232 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T06:25:27.233 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:25:27.233 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:25:27.233 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:27.433 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:25:27.472 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:25:27.565 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:25:27.635 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:25:27.636 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:27.636 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T06:25:27.636 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:25:27.636 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:25:27.636 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:28.646 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:25:28.680 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:25:28.830 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:25:29.013 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T06:25:29.049 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:25:29.127 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:25:29.128 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:29.128 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T06:25:29.128 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:25:29.128 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:25:29.128 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:29.183 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:25:29.225 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:25:29.305 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T06:25:30.214 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T06:25:30.252 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:25:30.348 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:25:30.348 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:30.348 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T06:25:30.348 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:25:30.348 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:25:30.348 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:30.394 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:25:30.492 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:25:30.492 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:30.492 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T06:25:30.492 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:30.561 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:25:30.665 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:25:30.665 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:30.665 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T06:25:30.665 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:25:30.666 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:25:30.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:31.037 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:25:31.124 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:25:31.124 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:25:31.124 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T06:25:31.124 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:25:31.124 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:25:31.124 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:31.780 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T06:25:31.833 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T06:25:31.889 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T06:25:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T06:25:32.246 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T06:25:32.380 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T06:25:32.423 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T06:25:32.457 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T06:25:32.497 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T06:25:32.536 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T06:25:32.637 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T06:25:32.734 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T06:25:32.775 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T06:25:33.580 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:25:33.623 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:25:33.680 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:25:35.045 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T06:25:35.200 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T06:25:35.261 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T06:25:35.339 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T06:25:35.524 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T06:25:35.526 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T06:25:35.574 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T06:25:35.582 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T06:25:35.608 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T06:25:35.638 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T06:25:35.642 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T06:25:35.676 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T06:25:35.684 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T06:25:35.684 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:25:35.807 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:25:35.882 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T06:25:35.946 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T06:25:35.979 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T06:25:35.979 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:25:35.986 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T06:25:36.114 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T06:25:36.115 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:25:36.146 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T06:25:36.146 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T06:25:36.180 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T06:25:36.186 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:25:36.186 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:25:36.236 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T06:25:36.282 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T06:25:36.430 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T06:25:36.496 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T06:25:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:25:36.512 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:25:36.532 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T06:25:36.662 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:25:36.698 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T06:25:36.732 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T06:25:36.767 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T06:25:36.893 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:25:37.168 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:25:37.168 INFO:teuthology.orchestra.run.smithi173.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T06:25:37.168 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:37.507 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:25:37.584 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:25:37.646 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:25:39.101 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T06:25:39.266 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T06:25:39.321 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T06:25:39.427 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T06:25:39.603 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T06:25:39.684 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T06:25:39.718 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T06:25:39.753 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T06:25:39.796 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T06:25:39.796 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:25:39.943 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:25:40.003 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T06:25:40.098 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T06:25:40.131 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T06:25:40.131 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:25:40.268 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:25:40.299 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T06:25:40.340 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:25:40.340 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:25:40.665 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:25:40.666 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:25:40.816 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:25:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T06:25:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T06:25:40.987 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T06:25:41.121 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:25:41.398 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:25:41.398 INFO:teuthology.orchestra.run.smithi016.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T06:25:41.398 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:42.074 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T06:25:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T06:25:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T06:25:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T06:25:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T06:25:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T06:25:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T06:25:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:25:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:25:42.075 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:25:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T06:25:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T06:25:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T06:25:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T06:25:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T06:25:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T06:25:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T06:25:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T06:25:42.076 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T06:25:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T06:25:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T06:25:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T06:25:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T06:25:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T06:25:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T06:25:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T06:25:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T06:25:42.077 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T06:25:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T06:25:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T06:25:42.078 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T06:25:42.089 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T06:25:42.089 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T06:25:42.089 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T06:25:42.090 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T06:25:42.091 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T06:25:42.092 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T06:25:42.093 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T06:25:42.094 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T06:25:42.094 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T06:25:42.094 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T06:25:42.094 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T06:25:42.094 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T06:25:42.094 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T06:25:42.094 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T06:25:42.094 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T06:25:42.094 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T06:25:42.095 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T06:25:42.096 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T06:25:42.097 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T06:25:42.098 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T06:25:42.099 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T06:25:42.100 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T06:25:42.101 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T06:25:42.101 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T06:25:42.101 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T06:25:42.101 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T06:25:42.101 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T06:25:44.383 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T06:25:44.383 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:44.383 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2023-11-24T06:25:44.383 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T06:25:44.383 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T06:25:44.383 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T06:25:44.384 INFO:teuthology.orchestra.run.smithi173.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.385 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.386 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T06:25:44.387 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T06:25:44.388 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T06:25:44.388 INFO:teuthology.orchestra.run.smithi173.stdout: jq-1.6-16.el9.x86_64 2023-11-24T06:25:44.388 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T06:25:44.388 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T06:25:44.388 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T06:25:44.388 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.388 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.388 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.388 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T06:25:44.389 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T06:25:44.390 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T06:25:44.391 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T06:25:44.392 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T06:25:44.393 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T06:25:44.394 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T06:25:44.395 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T06:25:44.395 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.395 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T06:25:44.395 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T06:25:44.395 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T06:25:44.395 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T06:25:44.395 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T06:25:44.397 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T06:25:44.397 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T06:25:44.397 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T06:25:44.397 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T06:25:44.397 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T06:25:44.398 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T06:25:44.398 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.398 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.398 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:44.398 INFO:teuthology.orchestra.run.smithi173.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T06:25:44.398 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T06:25:44.398 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T06:25:44.398 INFO:teuthology.orchestra.run.smithi173.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T06:25:44.398 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T06:25:44.399 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T06:25:44.399 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T06:25:44.399 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:44.399 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2023-11-24T06:25:45.079 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T06:25:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T06:25:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T06:25:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T06:25:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T06:25:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T06:25:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T06:25:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:25:46.301 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T06:25:46.302 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T06:25:46.303 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T06:25:46.303 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T06:25:46.303 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T06:25:46.303 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T06:25:46.303 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T06:25:46.303 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T06:25:46.303 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T06:25:46.303 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T06:25:46.303 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T06:25:46.315 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T06:25:46.315 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T06:25:46.315 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T06:25:46.315 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T06:25:46.315 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T06:25:46.316 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T06:25:46.317 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T06:25:46.318 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T06:25:46.319 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T06:25:46.320 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T06:25:46.321 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T06:25:46.322 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T06:25:46.323 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T06:25:46.324 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T06:25:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T06:25:46.326 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T06:25:46.327 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T06:25:46.327 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T06:25:46.327 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T06:25:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T06:25:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:48.719 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-11-24T06:25:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T06:25:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T06:25:48.719 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T06:25:48.720 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.721 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.722 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-16.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T06:25:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.724 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T06:25:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T06:25:48.726 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T06:25:48.727 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T06:25:48.728 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T06:25:48.729 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T06:25:48.730 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T06:25:48.731 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T06:25:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T06:25:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:48.732 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-11-24T06:25:49.369 DEBUG:teuthology.parallel:result is None 2023-11-24T06:25:49.369 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:25:49.586 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:25:49.665 INFO:teuthology.orchestra.run.smithi016.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:25:49.665 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:25:49.666 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:25:49.666 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:25:49.806 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:25:49.866 INFO:teuthology.orchestra.run.smithi173.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:25:49.867 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:25:49.867 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:25:49.867 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T06:25:49.868 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:49.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:25:49.961 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:25:49.962 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:25:50.044 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T06:25:50.045 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:50.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:25:50.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:25:50.241 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:25:50.241 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:25:50.321 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:25:50.444 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T06:25:50.444 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:50.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:25:50.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:25:50.639 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:25:50.639 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:25:50.715 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:25:50.830 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T06:25:50.830 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:50.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:25:50.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:25:51.010 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:25:51.010 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:25:51.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:25:51.212 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T06:25:51.295 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T06:25:51.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2023-11-24T06:25:51.305 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /var/log/ceph 2023-11-24T06:25:51.396 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T06:25:51.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:25:51.403 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:25:51.502 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T06:25:51.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:25:51.511 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:25:51.610 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T06:25:51.610 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T06:25:51.610 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T06:25:51.611 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:51.611 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:25:51.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:25:51.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:25:51.954 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:25:51.955 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:25:52.012 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:25:52.275 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:25:52.411 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T06:25:52.411 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T06:25:52.412 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}', 'email': None, 'first_in_suite': False, 'job_id': '7465555', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-mgr-vol-misc', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3305, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-mgr-vol-misc', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGD20YXGiNkytCHcjhlA6pBSfrL5G0UeUJxAQF/SQdBLaGQ6PMVANaB6OXial3GG4ffe7UR6oBNGhVUHZMKliLk=', 'smithi173.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIt7kG84kgBObmDZ7+jmRlniMGqurplR3EL62tMOa+cq2ZU5Agt28ZQNwGdsUXFv+G1bQUPSmbQKlSBOCkcQT6M='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots', 'tasks.cephfs.test_volumes.TestSubvolumeSnapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_05:41:33', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T06:25:52.412 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:25:52.417 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:25:52.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:25:52.502 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:25:52.603 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:52.603 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:25:52.648 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:25:52.649 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2023-11-24T06:25:52.732 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:25:52.732 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:25:52.732 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 634 Links: 1 2023-11-24T06:25:52.732 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:25:52.732 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:25:52.732 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-11-24 06:25:41.888424707 +0000 2023-11-24T06:25:52.732 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-11-24 06:23:01.188891162 +0000 2023-11-24T06:25:52.732 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-11-24 06:23:01.189891145 +0000 2023-11-24T06:25:52.732 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-11-24 06:23:01.188891162 +0000 2023-11-24T06:25:52.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:25:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-24T06:25:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-24T06:25:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000372265 s, 1.4 MB/s 2023-11-24T06:25:52.847 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:25:52.892 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2023-11-24T06:25:52.979 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:25:52.979 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:25:52.979 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 650 Links: 1 2023-11-24T06:25:52.979 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:25:52.979 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:25:52.979 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-11-24 06:25:41.889424691 +0000 2023-11-24T06:25:52.979 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-11-24 06:23:02.253872532 +0000 2023-11-24T06:25:52.979 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-11-24 06:23:02.253872532 +0000 2023-11-24T06:25:52.979 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-11-24 06:23:02.253872532 +0000 2023-11-24T06:25:52.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:25:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-24T06:25:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-24T06:25:53.101 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000187749 s, 2.7 MB/s 2023-11-24T06:25:53.105 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:25:53.152 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2023-11-24T06:25:53.237 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:25:53.237 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:25:53.237 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 670 Links: 1 2023-11-24T06:25:53.238 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:25:53.238 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:25:53.238 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-11-24 06:25:41.890424674 +0000 2023-11-24T06:25:53.238 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-11-24 06:23:03.336853586 +0000 2023-11-24T06:25:53.238 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-11-24 06:23:03.336853586 +0000 2023-11-24T06:25:53.239 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-11-24 06:23:03.336853586 +0000 2023-11-24T06:25:53.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:25:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-24T06:25:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-24T06:25:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000177951 s, 2.9 MB/s 2023-11-24T06:25:53.350 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:25:53.393 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2023-11-24T06:25:53.471 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:25:53.472 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:25:53.472 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-11-24T06:25:53.472 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:25:53.472 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:25:53.472 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-11-24 06:25:41.891424657 +0000 2023-11-24T06:25:53.472 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-11-24 06:23:04.398835008 +0000 2023-11-24T06:25:53.473 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-11-24 06:23:04.398835008 +0000 2023-11-24T06:25:53.473 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-11-24 06:23:04.398835008 +0000 2023-11-24T06:25:53.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:25:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-11-24T06:25:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-11-24T06:25:53.584 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.00018268 s, 2.8 MB/s 2023-11-24T06:25:53.587 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:25:53.629 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:25:53.629 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:25:53.629 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:25:53.688 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:25:53.688 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2023-11-24T06:25:53.739 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:25:53.739 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:25:53.739 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 634 Links: 1 2023-11-24T06:25:53.739 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:25:53.739 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:25:53.739 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2023-11-24 06:25:37.667752032 +0000 2023-11-24T06:25:53.740 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2023-11-24 06:22:38.730075881 +0000 2023-11-24T06:25:53.740 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2023-11-24 06:22:38.730075881 +0000 2023-11-24T06:25:53.740 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2023-11-24 06:22:38.730075881 +0000 2023-11-24T06:25:53.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:25:53.833 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2023-11-24T06:25:53.833 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2023-11-24T06:25:53.833 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000353189 s, 1.4 MB/s 2023-11-24T06:25:53.838 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:25:53.895 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2023-11-24T06:25:53.947 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:25:53.947 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:25:53.947 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-11-24T06:25:53.947 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:25:53.947 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:25:53.947 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2023-11-24 06:25:37.668752010 +0000 2023-11-24T06:25:53.947 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2023-11-24 06:22:39.757052938 +0000 2023-11-24T06:25:53.948 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2023-11-24 06:22:39.758052916 +0000 2023-11-24T06:25:53.948 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2023-11-24 06:22:39.757052938 +0000 2023-11-24T06:25:53.948 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:25:54.033 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2023-11-24T06:25:54.033 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2023-11-24T06:25:54.034 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000344244 s, 1.5 MB/s 2023-11-24T06:25:54.036 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:25:54.088 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2023-11-24T06:25:54.139 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:25:54.140 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:25:54.140 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-11-24T06:25:54.140 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:25:54.140 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:25:54.140 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2023-11-24 06:25:37.669751988 +0000 2023-11-24T06:25:54.140 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2023-11-24 06:22:40.852028476 +0000 2023-11-24T06:25:54.141 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2023-11-24 06:22:40.852028476 +0000 2023-11-24T06:25:54.141 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2023-11-24 06:22:40.852028476 +0000 2023-11-24T06:25:54.141 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:25:54.231 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2023-11-24T06:25:54.231 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2023-11-24T06:25:54.232 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000368724 s, 1.4 MB/s 2023-11-24T06:25:54.236 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:25:54.295 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2023-11-24T06:25:54.343 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:25:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:25:54.344 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 690 Links: 1 2023-11-24T06:25:54.344 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:25:54.344 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:25:54.344 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2023-11-24 06:25:37.670751966 +0000 2023-11-24T06:25:54.345 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2023-11-24 06:22:41.849006203 +0000 2023-11-24T06:25:54.345 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2023-11-24 06:22:41.849006203 +0000 2023-11-24T06:25:54.345 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2023-11-24 06:22:41.849006203 +0000 2023-11-24T06:25:54.345 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:25:54.428 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2023-11-24T06:25:54.429 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2023-11-24T06:25:54.429 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000335226 s, 1.5 MB/s 2023-11-24T06:25:54.432 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:25:54.492 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:25:54.492 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:25:54.492 INFO:tasks.ceph:Generating config... 2023-11-24T06:25:54.493 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T06:25:54.494 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T06:25:54.494 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T06:25:54.494 INFO:tasks.ceph:[client] pid file = 2023-11-24T06:25:54.494 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T06:25:54.494 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T06:25:54.494 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T06:25:54.494 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T06:25:54.494 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T06:25:54.494 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T06:25:54.494 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T06:25:54.495 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T06:25:54.496 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T06:25:54.497 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T06:25:54.497 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T06:25:54.497 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T06:25:54.497 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T06:25:54.497 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T06:25:54.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T06:25:54.653 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T06:25:54.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T06:25:54.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:25:54.904 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.173'), ('mon.c', '[v2:172.21.15.173:3301,v1:172.21.15.173:6790]')] 2023-11-24T06:25:54.904 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T06:25:54.905 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:54.905 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:25:54.963 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.173 --addv c '[v2:172.21.15.173:3301,v1:172.21.15.173:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:25:55.123 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid 6dc1668b-58b0-47dd-b8dc-4164e5a228ab 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = pacific 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 6dc1668b-58b0-47dd-b8dc-4164e5a228ab 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2023-11-24T06:25:55.122056+0000 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:created 2023-11-24T06:25:55.122056+0000 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 16 (pacific) 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2023-11-24T06:25:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2023-11-24T06:25:55.125 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] mon.c 2023-11-24T06:25:55.125 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T06:25:55.127 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:25:55.181 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6dc1668b-58b0-47dd-b8dc-4164e5a228ab... 2023-11-24T06:25:55.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:25:55.189 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:25:55.360 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2023-11-24T06:25:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2023-11-24T06:25:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:25:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2023-11-24T06:25:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2023-11-24T06:25:55.360 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.361 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:25:55.362 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.363 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.16,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 6dc1668b-58b0-47dd-b8dc-4164e5a228ab 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:25:55.364 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug trim objects = true 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2023-11-24T06:25:55.365 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2023-11-24T06:25:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:25:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2023-11-24T06:25:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2023-11-24T06:25:55.366 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 1/20 2023-11-24T06:25:55.367 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 1/20 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 20 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 4/10 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.368 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: debug finisher = 20 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2023-11-24T06:25:55.369 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2023-11-24T06:25:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2023-11-24T06:25:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2023-11-24T06:25:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2023-11-24T06:25:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2023-11-24T06:25:55.370 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:25:55.371 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: # 1m isn't quite enough 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2023-11-24T06:25:55.372 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 900 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 900 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2023-11-24T06:25:55.373 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds balancer = 20 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2023-11-24T06:25:55.374 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2023-11-24T06:25:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2023-11-24T06:25:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2023-11-24T06:25:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 900 2023-11-24T06:25:55.375 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 900 2023-11-24T06:25:55.375 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-11-24T06:25:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.376 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-11-24T06:25:55.377 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:25:55.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 6dc1668b-58b0-47dd-b8dc-4164e5a228ab 2023-11-24T06:25:55.379 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.380 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-11-24T06:25:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-11-24T06:25:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-11-24T06:25:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-11-24T06:25:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:25:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-11-24T06:25:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-11-24T06:25:55.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2023-11-24T06:25:55.383 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-11-24T06:25:55.384 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:25:55.385 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: debug finisher = 20 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-11-24T06:25:55.386 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:25:55.387 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-11-24T06:25:55.388 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-11-24T06:25:55.389 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2023-11-24T06:25:55.390 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-11-24T06:25:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-11-24T06:25:55.391 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-11-24T06:25:55.418 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T06:25:55.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T06:25:55.575 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T06:25:55.575 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:55.575 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T06:25:55.620 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:55.620 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T06:25:55.721 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:25:55.721 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:55.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:25:55.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:25:55.885 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:25:55.885 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:25:55.987 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:25:55.987 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:25:55.987 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:25:55.987 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:25:56.142 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:25:56.142 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:25:56.241 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T06:25:56.242 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T06:25:56.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:25:56.446 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:25:56.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:25:56.650 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:25:56.661 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T06:25:56.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:25:56.865 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:25:56.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T06:25:56.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:25:57.167 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:25:57.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T06:25:57.273 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:25:57.467 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:25:57.476 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T06:25:57.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:25:57.744 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:25:57.753 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T06:25:57.849 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T06:25:57.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T06:25:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T06:25:58.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T06:25:58.198 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T06:25:58.269 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T06:25:58.402 INFO:teuthology.orchestra.run.smithi173.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T06:25:58.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T06:25:58.581 INFO:teuthology.orchestra.run.smithi173.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T06:25:58.636 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T06:25:58.636 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:25:58.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T06:25:58.747 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:25:58.747 INFO:tasks.ceph:role: osd.0 2023-11-24T06:25:58.747 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:25:58.747 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:25:58.846 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:25:58.846 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:25:58.847 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:25:58.847 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:25:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:25:58.847 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:25:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:25:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:25:58.847 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:25:58.847 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:25:58.881 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-11-24T06:25:58.887 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-11-24T06:25:58.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T06:25:59.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T06:25:59.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T06:25:59.185 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:25:59.186 INFO:tasks.ceph:role: osd.1 2023-11-24T06:25:59.186 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:25:59.187 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:25:59.288 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:25:59.289 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:25:59.322 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-11-24T06:25:59.328 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-11-24T06:25:59.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T06:25:59.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T06:25:59.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T06:25:59.643 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:25:59.643 INFO:tasks.ceph:role: osd.2 2023-11-24T06:25:59.644 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:25:59.644 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:25:59.744 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:25:59.745 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:25:59.745 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:25:59.745 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:25:59.745 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:25:59.745 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:25:59.745 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:25:59.745 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:25:59.745 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:25:59.745 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:25:59.778 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-11-24T06:25:59.784 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-11-24T06:25:59.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T06:25:59.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T06:25:59.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T06:26:00.091 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:00.092 INFO:tasks.ceph:role: osd.3 2023-11-24T06:26:00.092 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:26:00.092 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:26:00.189 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:00.189 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:00.189 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:00.190 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:00.190 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:00.190 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:00.190 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:00.190 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:00.190 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:00.190 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:00.222 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-11-24T06:26:00.227 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-11-24T06:26:00.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T06:26:00.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T06:26:00.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:00.597+0000 7f623a21f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T06:26:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:00.597+0000 7f623a21f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T06:26:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:00.597+0000 7f623a21f500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:26:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:00.598+0000 7f623a21f500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:26:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:00.598+0000 7f623a21f500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:26:00.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:00.601+0000 7f623a21f500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T06:26:03.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T06:26:03.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:04.033+0000 7fabe451a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T06:26:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:04.034+0000 7fabe451a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T06:26:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:04.034+0000 7fabe451a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:26:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:04.034+0000 7fabe451a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:26:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:04.034+0000 7fabe451a500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:26:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:04.036+0000 7fabe451a500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T06:26:06.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T06:26:07.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:07.224+0000 7fdc7281a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T06:26:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:07.224+0000 7fdc7281a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T06:26:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:07.224+0000 7fdc7281a500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:26:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:07.224+0000 7fdc7281a500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:26:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:07.224+0000 7fdc7281a500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:26:07.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:07.226+0000 7fdc7281a500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T06:26:10.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T06:26:10.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:10.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:10.656+0000 7f5fc0425500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T06:26:10.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:10.657+0000 7f5fc0425500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T06:26:10.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:10.657+0000 7f5fc0425500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:26:10.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:10.657+0000 7f5fc0425500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:26:10.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:10.657+0000 7f5fc0425500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:26:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:10.659+0000 7f5fc0425500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T06:26:13.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T06:26:13.963 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T06:26:14.068 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:14.068 INFO:tasks.ceph:role: osd.4 2023-11-24T06:26:14.068 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:26:14.068 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:26:14.170 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:14.170 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:14.170 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:14.170 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:14.170 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:14.170 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:14.171 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:14.171 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:14.171 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:14.171 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:14.683 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2023-11-24T06:26:14.690 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2023-11-24T06:26:14.690 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T06:26:14.874 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T06:26:14.980 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T06:26:15.078 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:15.078 INFO:tasks.ceph:role: osd.5 2023-11-24T06:26:15.078 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:26:15.079 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:26:15.178 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:15.179 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:15.179 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:15.179 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:15.179 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:15.179 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:15.179 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:15.179 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:15.179 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:15.179 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:15.672 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2023-11-24T06:26:15.679 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2023-11-24T06:26:15.679 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T06:26:15.823 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T06:26:15.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T06:26:16.017 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:16.017 INFO:tasks.ceph:role: osd.6 2023-11-24T06:26:16.017 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:26:16.017 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:26:16.119 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:16.119 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:16.119 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:16.119 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:16.119 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:16.119 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:16.119 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:16.119 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:16.120 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:16.120 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:16.634 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2023-11-24T06:26:16.640 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2023-11-24T06:26:16.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T06:26:16.846 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T06:26:16.957 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T06:26:17.052 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:26:17.052 INFO:tasks.ceph:role: osd.7 2023-11-24T06:26:17.052 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:26:17.052 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:26:17.154 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:26:17.155 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:26:17.155 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:26:17.155 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:26:17.155 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:26:17.155 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:26:17.155 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:26:17.155 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:26:17.155 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:26:17.155 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:26:17.625 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2023-11-24T06:26:17.631 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2023-11-24T06:26:17.632 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T06:26:17.777 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T06:26:17.872 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:18.026 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:18.023+0000 7f5906e20500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T06:26:18.027 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:18.024+0000 7f5906e20500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T06:26:18.027 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:18.024+0000 7f5906e20500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:26:18.027 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:18.025+0000 7f5906e20500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:26:18.028 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:18.025+0000 7f5906e20500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:26:18.037 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:18.034+0000 7f5906e20500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T06:26:21.238 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T06:26:21.339 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:21.477 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:21.474+0000 7f3de12d8500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T06:26:21.478 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:21.475+0000 7f3de12d8500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T06:26:21.478 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:21.475+0000 7f3de12d8500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:26:21.478 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:21.475+0000 7f3de12d8500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:26:21.478 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:21.476+0000 7f3de12d8500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:26:21.480 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:21.477+0000 7f3de12d8500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T06:26:22.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:22.422 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:24.674 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T06:26:24.774 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:24.918 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:24.915+0000 7f14a3b17500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T06:26:24.918 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:24.916+0000 7f14a3b17500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T06:26:24.918 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:24.916+0000 7f14a3b17500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:26:24.919 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:24.916+0000 7f14a3b17500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:26:24.919 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:24.916+0000 7f14a3b17500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:26:24.920 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:24.918+0000 7f14a3b17500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T06:26:27.854 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T06:26:27.957 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:28.100 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:28.098+0000 7f49d9a0c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T06:26:28.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:28.099+0000 7f49d9a0c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T06:26:28.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:28.099+0000 7f49d9a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:26:28.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:28.099+0000 7f49d9a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:26:28.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:28.099+0000 7f49d9a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:26:28.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:28.101+0000 7f49d9a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T06:26:31.016 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T06:26:31.113 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T06:26:31.113 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:31.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T06:26:31.226 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:31.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T06:26:31.331 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:31.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T06:26:31.437 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:31.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T06:26:31.541 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:31.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T06:26:31.644 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:31.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T06:26:31.750 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:31.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T06:26:31.856 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:31.856 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T06:26:31.960 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:31.960 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T06:26:32.061 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:32.061 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T06:26:32.158 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:32.158 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T06:26:32.252 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:32.252 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T06:26:32.350 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:32.350 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T06:26:32.448 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:32.448 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T06:26:32.538 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:32.538 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T06:26:32.602 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:32.602 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T06:26:32.663 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:32.663 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T06:26:32.721 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:32.721 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T06:26:32.773 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T06:26:32.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:26:32.782 DEBUG:teuthology.orchestra.run.smithi173:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:26:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2023-11-24T06:26:32.877 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD0QWBlN4GDGhAAF9YIRqu011Xj6oWUv9LYlw== 2023-11-24T06:26:32.877 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-11-24T06:26:32.877 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD0QWBl2seAMxAAL8jbBlU0yStjuIivg2ehGQ== 2023-11-24T06:26:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-11-24T06:26:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD1QWBl79LhCRAALZnwoYyKju4aeWNS+jF31A== 2023-11-24T06:26:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2023-11-24T06:26:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD4QWBlarihIxAAczpFVQAODZogIKH0TTdMhw== 2023-11-24T06:26:32.878 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2023-11-24T06:26:32.878 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD8QWBlITYFAhAAMlJyCTojY7WZihUegTcBcQ== 2023-11-24T06:26:32.881 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2023-11-24T06:26:32.882 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD/QWBlA/N0DRAAThU0hfAL0+MggufKZC4K8g== 2023-11-24T06:26:32.882 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2023-11-24T06:26:32.882 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQACQmBlM7E3JxAAdmfhQHbjK/JGHgeD0urDZw== 2023-11-24T06:26:32.882 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2023-11-24T06:26:32.882 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD0QWBlz+KlJhAAXtiyVVA72YrbDKpNSBUnwA== 2023-11-24T06:26:32.882 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-11-24T06:26:32.882 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD1QWBlbyW+GxAAy15inYIiNwnGVHj7Ghs5hg== 2023-11-24T06:26:32.882 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-11-24T06:26:32.882 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD1QWBl/NBDLBAAR4qYgD5THE/yNeZT9XgK5g== 2023-11-24T06:26:32.883 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2023-11-24T06:26:32.883 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQAKQmBlrM97ARAApYaqyASXJqB5ITNLeBec/w== 2023-11-24T06:26:32.884 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2023-11-24T06:26:32.884 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQANQmBl3s5dHBAA3spWOZQiqazGS6isl8ft3Q== 2023-11-24T06:26:32.884 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2023-11-24T06:26:32.885 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQAQQmBlKVmeNhAAuV5j9qZ2NNvEePQx77EM6Q== 2023-11-24T06:26:32.885 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2023-11-24T06:26:32.886 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQAUQmBlwADrBRAAvS3M6+H3X+OQD1h6xwhnBQ== 2023-11-24T06:26:32.886 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-24T06:26:32.888 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA== 2023-11-24T06:26:32.888 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2023-11-24T06:26:32.888 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD2QWBliwu9CxAAPcBaB9CWK2QqRD1k2x+4ig== 2023-11-24T06:26:32.888 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2023-11-24T06:26:32.888 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD2QWBlKhfoFxAA8A+MObA/cjawPN7aclqS9Q== 2023-11-24T06:26:32.890 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2023-11-24T06:26:32.890 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD2QWBl8HWRIhAAqzUXni0Uqgbmi/imvnNIZg== 2023-11-24T06:26:32.892 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.y] 2023-11-24T06:26:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD0QWBlN4GDGhAAF9YIRqu011Xj6oWUv9LYlw== 2023-11-24T06:26:32.892 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2023-11-24T06:26:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD0QWBl2seAMxAAL8jbBlU0yStjuIivg2ehGQ== 2023-11-24T06:26:32.892 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2023-11-24T06:26:32.892 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD1QWBl79LhCRAALZnwoYyKju4aeWNS+jF31A== 2023-11-24T06:26:32.892 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.0] 2023-11-24T06:26:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD4QWBlarihIxAAczpFVQAODZogIKH0TTdMhw== 2023-11-24T06:26:32.893 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.1] 2023-11-24T06:26:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD8QWBlITYFAhAAMlJyCTojY7WZihUegTcBcQ== 2023-11-24T06:26:32.893 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.2] 2023-11-24T06:26:32.893 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD/QWBlA/N0DRAAThU0hfAL0+MggufKZC4K8g== 2023-11-24T06:26:32.894 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.3] 2023-11-24T06:26:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQACQmBlM7E3JxAAdmfhQHbjK/JGHgeD0urDZw== 2023-11-24T06:26:32.894 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.x] 2023-11-24T06:26:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD0QWBlz+KlJhAAXtiyVVA72YrbDKpNSBUnwA== 2023-11-24T06:26:32.894 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2023-11-24T06:26:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD1QWBlbyW+GxAAy15inYIiNwnGVHj7Ghs5hg== 2023-11-24T06:26:32.894 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2023-11-24T06:26:32.894 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD1QWBl/NBDLBAAR4qYgD5THE/yNeZT9XgK5g== 2023-11-24T06:26:32.894 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.4] 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQAKQmBlrM97ARAApYaqyASXJqB5ITNLeBec/w== 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.5] 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQANQmBl3s5dHBAA3spWOZQiqazGS6isl8ft3Q== 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.6] 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQAQQmBlKVmeNhAAuV5j9qZ2NNvEePQx77EM6Q== 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.7] 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQAUQmBlwADrBRAAvS3M6+H3X+OQD1h6xwhnBQ== 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout:[client.0] 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA== 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout:[client.2] 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD2QWBliwu9CxAAPcBaB9CWK2QqRD1k2x+4ig== 2023-11-24T06:26:32.895 INFO:teuthology.orchestra.run.smithi173.stdout:[client.1] 2023-11-24T06:26:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD2QWBlKhfoFxAA8A+MObA/cjawPN7aclqS9Q== 2023-11-24T06:26:32.896 INFO:teuthology.orchestra.run.smithi173.stdout:[client.3] 2023-11-24T06:26:32.896 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQD2QWBl8HWRIhAAqzUXni0Uqgbmi/imvnNIZg== 2023-11-24T06:26:32.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:26:32.905 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:26:33.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:26:33.062 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:26:33.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:26:33.213 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:26:33.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:33.360 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:33.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:33.513 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:33.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:33.663 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:33.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:33.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:33.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:26:33.972 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:26:34.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:26:34.125 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:26:34.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:26:34.266 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:26:34.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:34.414 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:34.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:34.567 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:34.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:34.718 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:34.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:34.875 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:26:35.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:26:35.030 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:26:35.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:26:35.183 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:26:35.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:26:35.336 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:26:35.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:26:35.487 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:26:35.630 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T06:26:35.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T06:26:35.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:26:35.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T06:26:36.008 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T06:26:36.111 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:26:36.270 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T06:26:36.348 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T06:26:36.437 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:26:36.594 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T06:26:36.680 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:36.688 DEBUG:teuthology.orchestra.run.smithi173:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:26:36.741 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T06:26:36.741 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T06:26:36.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T06:26:36.747 INFO:tasks.ceph.mon.a:Started 2023-11-24T06:26:36.747 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T06:26:36.747 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T06:26:36.752 INFO:tasks.ceph.mon.b:Started 2023-11-24T06:26:36.752 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T06:26:36.752 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T06:26:36.757 INFO:tasks.ceph.mon.c:Started 2023-11-24T06:26:36.757 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T06:26:36.757 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T06:26:36.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T06:26:36.763 INFO:tasks.ceph.mgr.y:Started 2023-11-24T06:26:36.763 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T06:26:36.763 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T06:26:36.767 INFO:tasks.ceph.mgr.x:Started 2023-11-24T06:26:36.768 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T06:26:36.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T06:26:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.954+0000 7f41c3ddc640 1 Processor -- start 2023-11-24T06:26:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.954+0000 7f41c3ddc640 1 -- start start 2023-11-24T06:26:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.955+0000 7f41c3ddc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41bc0ce8f0 0x7f41bc0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c3ddc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc0d0de0 con 0x7f41bc0b2a00 2023-11-24T06:26:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c3ddc640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc0d10c0 con 0x7f41bc160f80 2023-11-24T06:26:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c3ddc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc0d13a0 con 0x7f41bc0ce8f0 2023-11-24T06:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c1377640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f41bc0ce8f0 0x7f41bc0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c1b78640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f41bc160f80 0x7f41bc161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:60028/0 (socket says 172.21.15.16:60028) 2023-11-24T06:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c1b78640 1 -- 172.21.15.16:0/4138025703 learned_addr learned my addr 172.21.15.16:0/4138025703 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c1377640 1 -- 172.21.15.16:0/4138025703 >> v2:172.21.15.16:3300/0 conn(0x7f41bc0ce8f0 msgr2=0x7f41bc0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c1377640 1 -- 172.21.15.16:0/4138025703 >> v2:172.21.15.16:3300/0 conn(0x7f41bc0ce8f0 msgr2=0x7f41bc0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c1377640 1 --2- 172.21.15.16:0/4138025703 >> v2:172.21.15.16:3300/0 conn(0x7f41bc0ce8f0 0x7f41bc0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:26:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.956+0000 7f41c1377640 1 --2- 172.21.15.16:0/4138025703 >> v2:172.21.15.16:3300/0 conn(0x7f41bc0ce8f0 0x7f41bc0ced30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:26:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435319426 0 0) 0x7f41bc0d0de0 con 0x7f41bc0b2a00 2023-11-24T06:26:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a8003520 con 0x7f41bc0b2a00 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2036287865 0 0) 0x7f41a8003520 con 0x7f41bc0b2a00 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 >> v2:172.21.15.16:3300/0 conn(0x7f41bc0ce8f0 msgr2=0x7f41bc0ced30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 --2- 172.21.15.16:0/4138025703 >> v2:172.21.15.16:3300/0 conn(0x7f41bc0ce8f0 0x7f41bc0ced30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 >> v1:172.21.15.173:6789/0 conn(0x7f41bc160f80 legacy=0x7f41bc161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc0d1680 con 0x7f41bc0b2a00 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f41b40019f0 con 0x7f41bc0b2a00 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 >> v1:172.21.15.16:6789/0 conn(0x7f41bc0b2a00 legacy=0x7f41bc0b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 --2- 172.21.15.16:0/4138025703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a80039c0 0x7f41a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 --2- 172.21.15.16:0/4138025703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80047c0 0x7f41a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 --2- 172.21.15.16:0/4138025703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41a80051d0 0x7f41a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0d13a0 con 0x7f41a80039c0 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a800c120 con 0x7f41a80051d0 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.960+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a800c2e0 con 0x7f41a80047c0 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.961+0000 7f41c1b78640 1 --2- 172.21.15.16:0/4138025703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a80039c0 0x7f41a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.961+0000 7f41c1b78640 1 -- 172.21.15.16:0/4138025703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80047c0 msgr2=0x7f41a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.961+0000 7f41c1b78640 1 --2- 172.21.15.16:0/4138025703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80047c0 0x7f41a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.961+0000 7f41c1b78640 1 -- 172.21.15.16:0/4138025703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41a80051d0 msgr2=0x7f41a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.961+0000 7f41c1b78640 1 --2- 172.21.15.16:0/4138025703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41a80051d0 0x7f41a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.961+0000 7f41c1b78640 1 -- 172.21.15.16:0/4138025703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a8003520 con 0x7f41a80039c0 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.961+0000 7f41c2379640 1 -- 172.21.15.16:0/4138025703 reap_dead start 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.962+0000 7f41c1b78640 1 --2- 172.21.15.16:0/4138025703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a80039c0 0x7f41a8003de0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f41b8001810 tx=0x7f41a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7627418c3e0d177 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.962+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b802f070 con 0x7f41a80039c0 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.962+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b80048a0 con 0x7f41a80039c0 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.962+0000 7f41c0b76640 1 -- 172.21.15.16:0/4138025703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8004c10 con 0x7f41a80039c0 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.963+0000 7f41c3ddc640 1 -- 172.21.15.16:0/4138025703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a80039c0 msgr2=0x7f41a8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.963+0000 7f41c3ddc640 1 --2- 172.21.15.16:0/4138025703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a80039c0 0x7f41a8003de0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f41b8001810 tx=0x7f41a80057f0 comp rx=0 tx=0).stop 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.963+0000 7f41c3ddc640 1 -- 172.21.15.16:0/4138025703 shutdown_connections 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.963+0000 7f41c3ddc640 1 --2- 172.21.15.16:0/4138025703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41a80039c0 0x7f41a8003de0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.963+0000 7f41c3ddc640 1 -- 172.21.15.16:0/4138025703 >> 172.21.15.16:0/4138025703 conn(0x7f41bc09c550 msgr2=0x7f41bc0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.963+0000 7f41c3ddc640 1 -- 172.21.15.16:0/4138025703 shutdown_connections 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.963+0000 7f41c3ddc640 1 -- 172.21.15.16:0/4138025703 wait complete. 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.965+0000 7f41c3ddc640 1 Processor -- start 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.965+0000 7f41c3ddc640 1 -- start start 2023-11-24T06:26:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.966+0000 7f41c3ddc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80039c0 0x7f41bc1601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.966+0000 7f41c3ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41bc15d6c0 0x7f41bc0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.966+0000 7f41c3ddc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41bc0b15a0 0x7f41bc1f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.966+0000 7f41c3ddc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0b1a20 con 0x7f41bc15d6c0 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.966+0000 7f41c3ddc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc1f7540 con 0x7f41bc0b15a0 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.966+0000 7f41c3ddc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0cbf10 con 0x7f41a80039c0 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c1b78640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80039c0 0x7f41bc1601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c1b78640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80039c0 0x7f41bc1601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:49744/0 (socket says 172.21.15.16:49744) 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c1b78640 1 -- 172.21.15.16:0/2646911122 learned_addr learned my addr 172.21.15.16:0/2646911122 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c2379640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41bc0b15a0 0x7f41bc1f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c1377640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41bc15d6c0 0x7f41bc0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c2379640 1 -- 172.21.15.16:0/2646911122 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80039c0 msgr2=0x7f41bc1601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c2379640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80039c0 0x7f41bc1601f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c2379640 1 -- 172.21.15.16:0/2646911122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41bc15d6c0 msgr2=0x7f41bc0b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c2379640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41bc15d6c0 0x7f41bc0b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c2379640 1 -- 172.21.15.16:0/2646911122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc1f7260 con 0x7f41bc0b15a0 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c1b78640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80039c0 0x7f41bc1601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.967+0000 7f41c1377640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41bc15d6c0 0x7f41bc0b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.968+0000 7f41c2379640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41bc0b15a0 0x7f41bc1f6600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41bc08b520 tx=0x7f41b400ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.968+0000 7f41b27fc640 1 -- 172.21.15.16:0/2646911122 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b4007c10 con 0x7f41bc0b15a0 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.968+0000 7f41b27fc640 1 -- 172.21.15.16:0/2646911122 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b40070d0 con 0x7f41bc0b15a0 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.968+0000 7f41c3ddc640 1 -- 172.21.15.16:0/2646911122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc0cb8c0 con 0x7f41bc0b15a0 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.968+0000 7f41b27fc640 1 -- 172.21.15.16:0/2646911122 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b4007480 con 0x7f41bc0b15a0 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.968+0000 7f41c3ddc640 1 -- 172.21.15.16:0/2646911122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc160730 con 0x7f41bc0b15a0 2023-11-24T06:26:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.968+0000 7f41b27fc640 1 -- 172.21.15.16:0/2646911122 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f41b400a5b0 con 0x7f41bc0b15a0 2023-11-24T06:26:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.968+0000 7f41b27fc640 1 -- 172.21.15.16:0/2646911122 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f41b401c810 con 0x7f41bc0b15a0 2023-11-24T06:26:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.969+0000 7f41c3ddc640 1 -- 172.21.15.16:0/2646911122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc0675d0 con 0x7f41bc0b15a0 2023-11-24T06:26:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:36.973+0000 7f41b27fc640 1 -- 172.21.15.16:0/2646911122 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f41b4017070 con 0x7f41bc0b15a0 2023-11-24T06:26:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.046+0000 7f41c3ddc640 1 -- 172.21.15.16:0/2646911122 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f41bc0cc050 con 0x7f41bc0b15a0 2023-11-24T06:26:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.058+0000 7f41b27fc640 1 -- 172.21.15.16:0/2646911122 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f41b400aa40 con 0x7f41bc0b15a0 2023-11-24T06:26:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2023-11-24T06:26:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.059+0000 7f41c3ddc640 1 -- 172.21.15.16:0/2646911122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41bc0b15a0 msgr2=0x7f41bc1f6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.059+0000 7f41c3ddc640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41bc0b15a0 0x7f41bc1f6600 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41bc08b520 tx=0x7f41b400ccd0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.059+0000 7f41c3ddc640 1 -- 172.21.15.16:0/2646911122 shutdown_connections 2023-11-24T06:26:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.059+0000 7f41c3ddc640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41bc0b15a0 0x7f41bc1f6600 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.059+0000 7f41c3ddc640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41bc15d6c0 0x7f41bc0b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.059+0000 7f41c3ddc640 1 --2- 172.21.15.16:0/2646911122 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f41a80039c0 0x7f41bc1601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.059+0000 7f41c3ddc640 1 -- 172.21.15.16:0/2646911122 >> 172.21.15.16:0/2646911122 conn(0x7f41bc09c550 msgr2=0x7f41bc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.059+0000 7f41c3ddc640 1 -- 172.21.15.16:0/2646911122 shutdown_connections 2023-11-24T06:26:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:37.060+0000 7f41c3ddc640 1 -- 172.21.15.16:0/2646911122 wait complete. 2023-11-24T06:26:37.082 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T06:26:37.083 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T06:26:37.084 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:37.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T06:26:37.174 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:37.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T06:26:37.254 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:37.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T06:26:37.397 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:26:37.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T06:26:37.494 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:37.495 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T06:26:37.584 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:37.584 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T06:26:37.675 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:37.676 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T06:26:37.725 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:37.721+0000 7fd33683a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:26:37.727 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:37.724+0000 7f25e8a13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:26:37.790 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:37.787+0000 7fd33683a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:26:37.793 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:37.790+0000 7f25e8a13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:26:37.806 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:26:37.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T06:26:37.891 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 147be2d1-6231-4031-8f98-d0b50bcd068f 0 2023-11-24T06:26:38.126 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.123+0000 7f91f62d0640 1 Processor -- start 2023-11-24T06:26:38.126 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.123+0000 7f91f62d0640 1 -- start start 2023-11-24T06:26:38.128 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.125+0000 7f91f62d0640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f91f0159040 0x7f91f00cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.128 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.125+0000 7f91f62d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91f00b34e0 0x7f91f00d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.128 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.125+0000 7f91f62d0640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f00c6060 con 0x7f91f015c300 2023-11-24T06:26:38.128 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.125+0000 7f91f62d0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00c62d0 con 0x7f91f00b34e0 2023-11-24T06:26:38.128 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.125+0000 7f91f62d0640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f00c65b0 con 0x7f91f0159040 2023-11-24T06:26:38.128 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.125+0000 7f91effff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f91f0159040 0x7f91f00cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.129 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.125+0000 7f91effff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f91f0159040 0x7f91f00cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:47986/0 (socket says 172.21.15.173:47986) 2023-11-24T06:26:38.129 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.125+0000 7f91effff640 1 -- 172.21.15.173:0/3837592448 learned_addr learned my addr 172.21.15.173:0/3837592448 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:38.130 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.126+0000 7f91effff640 1 -- 172.21.15.173:0/3837592448 >> v2:172.21.15.16:3300/0 conn(0x7f91f00b34e0 msgr2=0x7f91f00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:38.130 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.126+0000 7f91f486d640 1 --2- 172.21.15.173:0/3837592448 >> v2:172.21.15.16:3300/0 conn(0x7f91f00b34e0 0x7f91f00d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.130 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.126+0000 7f91effff640 1 --2- 172.21.15.173:0/3837592448 >> v2:172.21.15.16:3300/0 conn(0x7f91f00b34e0 0x7f91f00d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.130 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.126+0000 7f91effff640 1 -- 172.21.15.173:0/3837592448 >> v1:172.21.15.173:6789/0 conn(0x7f91f015c300 legacy=0x7f91f015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.130 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.126+0000 7f91effff640 1 -- 172.21.15.173:0/3837592448 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f015eec0 con 0x7f91f0159040 2023-11-24T06:26:38.130 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.126+0000 7f91eeffd640 1 -- 172.21.15.173:0/3837592448 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879526265 0 0) 0x7f91f00c6060 con 0x7f91f015c300 2023-11-24T06:26:38.130 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.126+0000 7f91f486d640 1 --2- 172.21.15.173:0/3837592448 >> v2:172.21.15.16:3300/0 conn(0x7f91f00b34e0 0x7f91f00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:38.130 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.127+0000 7f91effff640 1 --2- 172.21.15.173:0/3837592448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f0159040 0x7f91f00cf000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91f00d1e90 tx=0x7f91e002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b37086397338fe5f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:38.130 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.127+0000 7f91eeffd640 1 -- 172.21.15.173:0/3837592448 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0030860 con 0x7f91f0159040 2023-11-24T06:26:38.131 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.127+0000 7f91eeffd640 1 -- 172.21.15.173:0/3837592448 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e00309c0 con 0x7f91f0159040 2023-11-24T06:26:38.131 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.127+0000 7f91eeffd640 1 -- 172.21.15.173:0/3837592448 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0030d50 con 0x7f91f0159040 2023-11-24T06:26:38.131 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.128+0000 7f91f62d0640 1 -- 172.21.15.173:0/3837592448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f0159040 msgr2=0x7f91f00cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.131 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.128+0000 7f91f62d0640 1 --2- 172.21.15.173:0/3837592448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f0159040 0x7f91f00cf000 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91f00d1e90 tx=0x7f91e002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.132 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.128+0000 7f91f62d0640 1 -- 172.21.15.173:0/3837592448 shutdown_connections 2023-11-24T06:26:38.132 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.128+0000 7f91f62d0640 1 --2- 172.21.15.173:0/3837592448 >> v2:172.21.15.16:3300/0 conn(0x7f91f00b34e0 0x7f91f00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.132 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.128+0000 7f91f62d0640 1 --2- 172.21.15.173:0/3837592448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f0159040 0x7f91f00cf000 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.132 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.128+0000 7f91f62d0640 1 -- 172.21.15.173:0/3837592448 >> 172.21.15.173:0/3837592448 conn(0x7f91f0066690 msgr2=0x7f91f006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:38.132 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.128+0000 7f91f62d0640 1 -- 172.21.15.173:0/3837592448 shutdown_connections 2023-11-24T06:26:38.132 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.128+0000 7f91f62d0640 1 -- 172.21.15.173:0/3837592448 wait complete. 2023-11-24T06:26:38.132 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.130+0000 7f91f62d0640 1 Processor -- start 2023-11-24T06:26:38.133 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.130+0000 7f91f62d0640 1 -- start start 2023-11-24T06:26:38.136 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f62d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f00b34e0 0x7f91f00b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.137 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f62d0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91f0159040 0x7f91f00b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.137 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f62d0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f015c300 0x7f91f014f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.137 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f62d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00b2020 con 0x7f91f00b34e0 2023-11-24T06:26:38.137 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f62d0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00b2300 con 0x7f91f015c300 2023-11-24T06:26:38.138 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f62d0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f00b25e0 con 0x7f91f0159040 2023-11-24T06:26:38.138 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f486d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f015c300 0x7f91f014f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.138 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91f0159040 0x7f91f00b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.138 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f486d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f015c300 0x7f91f014f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:47994/0 (socket says 172.21.15.173:47994) 2023-11-24T06:26:38.138 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f486d640 1 -- 172.21.15.173:0/2269895446 learned_addr learned my addr 172.21.15.173:0/2269895446 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:38.138 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91effff640 1 --2- 172.21.15.173:0/2269895446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f00b34e0 0x7f91f00b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.139 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f486d640 1 -- 172.21.15.173:0/2269895446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91f0159040 msgr2=0x7f91f00b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.139 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f486d640 1 --2- 172.21.15.173:0/2269895446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91f0159040 0x7f91f00b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.139 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f486d640 1 -- 172.21.15.173:0/2269895446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f00b34e0 msgr2=0x7f91f00b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.139 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f486d640 1 --2- 172.21.15.173:0/2269895446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f00b34e0 0x7f91f00b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.139 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.131+0000 7f91f486d640 1 -- 172.21.15.173:0/2269895446 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00094d0 con 0x7f91f015c300 2023-11-24T06:26:38.140 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.132+0000 7f91ef7fe640 1 --2- 172.21.15.173:0/2269895446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91f0159040 0x7f91f00b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:26:38.140 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.132+0000 7f91f486d640 1 --2- 172.21.15.173:0/2269895446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f015c300 0x7f91f014f870 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f91e800cda0 tx=0x7f91e800a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:38.140 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.132+0000 7f91ecff9640 1 -- 172.21.15.173:0/2269895446 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e800d840 con 0x7f91f015c300 2023-11-24T06:26:38.140 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.132+0000 7f91ecff9640 1 -- 172.21.15.173:0/2269895446 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e800d9a0 con 0x7f91f015c300 2023-11-24T06:26:38.140 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.132+0000 7f91ecff9640 1 -- 172.21.15.173:0/2269895446 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e800dd90 con 0x7f91f015c300 2023-11-24T06:26:38.140 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.132+0000 7f91f62d0640 1 -- 172.21.15.173:0/2269895446 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f00b28c0 con 0x7f91f015c300 2023-11-24T06:26:38.141 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.132+0000 7f91f62d0640 1 -- 172.21.15.173:0/2269895446 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f00b4fe0 con 0x7f91f015c300 2023-11-24T06:26:38.141 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.132+0000 7f91ecff9640 1 -- 172.21.15.173:0/2269895446 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f91e801f740 con 0x7f91f015c300 2023-11-24T06:26:38.141 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.133+0000 7f91ecff9640 1 -- 172.21.15.173:0/2269895446 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f91e8015990 con 0x7f91f015c300 2023-11-24T06:26:38.141 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.133+0000 7f91f62d0640 1 -- 172.21.15.173:0/2269895446 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f0074a50 con 0x7f91f015c300 2023-11-24T06:26:38.142 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.136+0000 7f91ecff9640 1 -- 172.21.15.173:0/2269895446 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f91e801a070 con 0x7f91f015c300 2023-11-24T06:26:38.217 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.214+0000 7f91f62d0640 1 -- 172.21.15.173:0/2269895446 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "147be2d1-6231-4031-8f98-d0b50bcd068f", "id": 0} v 0) v1 -- 0x7f91f00675d0 con 0x7f91f015c300 2023-11-24T06:26:38.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.221+0000 7f91ecff9640 1 -- 172.21.15.173:0/2269895446 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "147be2d1-6231-4031-8f98-d0b50bcd068f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f91e8015b50 con 0x7f91f015c300 2023-11-24T06:26:38.225 INFO:teuthology.orchestra.run.smithi173.stdout:0 2023-11-24T06:26:38.226 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.223+0000 7f91f62d0640 1 -- 172.21.15.173:0/2269895446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f015c300 msgr2=0x7f91f014f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.226 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.223+0000 7f91f62d0640 1 --2- 172.21.15.173:0/2269895446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f015c300 0x7f91f014f870 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f91e800cda0 tx=0x7f91e800a710 comp rx=0 tx=0).stop 2023-11-24T06:26:38.226 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.223+0000 7f91f62d0640 1 -- 172.21.15.173:0/2269895446 shutdown_connections 2023-11-24T06:26:38.226 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.223+0000 7f91f62d0640 1 --2- 172.21.15.173:0/2269895446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91f015c300 0x7f91f014f870 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.226 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.223+0000 7f91f62d0640 1 --2- 172.21.15.173:0/2269895446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91f0159040 0x7f91f00b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.226 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.223+0000 7f91f62d0640 1 --2- 172.21.15.173:0/2269895446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91f00b34e0 0x7f91f00b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.227 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.223+0000 7f91f62d0640 1 -- 172.21.15.173:0/2269895446 >> 172.21.15.173:0/2269895446 conn(0x7f91f0066690 msgr2=0x7f91f00ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:38.227 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.223+0000 7f91f62d0640 1 -- 172.21.15.173:0/2269895446 shutdown_connections 2023-11-24T06:26:38.227 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.223+0000 7f91f62d0640 1 -- 172.21.15.173:0/2269895446 wait complete. 2023-11-24T06:26:38.243 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 55e17526-cd19-4a3d-a36b-c75b35e02906 1 2023-11-24T06:26:38.414 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:38.411+0000 7f25e8a13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:26:38.425 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:38.422+0000 7fd33683a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:26:38.431 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.428+0000 7fb08e516640 1 Processor -- start 2023-11-24T06:26:38.431 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.429+0000 7fb08e516640 1 -- start start 2023-11-24T06:26:38.433 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.430+0000 7fb08e516640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb088160a20 0x7fb0880cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.433 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.430+0000 7fb08e516640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb0880b3b20 0x7fb0880cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.433 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.430+0000 7fb08e516640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0880d1e00 con 0x7fb088160e00 2023-11-24T06:26:38.433 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.430+0000 7fb08e516640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0880d3c40 con 0x7fb088160a20 2023-11-24T06:26:38.433 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.430+0000 7fb08e516640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0880d3e50 con 0x7fb0880b3b20 2023-11-24T06:26:38.433 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.430+0000 7fb08dd15640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb0880b3b20 0x7fb0880cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.433 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.430+0000 7fb08dd15640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb0880b3b20 0x7fb0880cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48002/0 (socket says 172.21.15.173:48002) 2023-11-24T06:26:38.434 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.430+0000 7fb08dd15640 1 -- 172.21.15.173:0/2957601599 learned_addr learned my addr 172.21.15.173:0/2957601599 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:38.434 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.431+0000 7fb08d514640 1 --2- 172.21.15.173:0/2957601599 >> v2:172.21.15.16:3300/0 conn(0x7fb088160a20 0x7fb0880cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.434 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.431+0000 7fb08dd15640 1 -- 172.21.15.173:0/2957601599 >> v2:172.21.15.16:3300/0 conn(0x7fb088160a20 msgr2=0x7fb0880cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:38.434 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.431+0000 7fb08dd15640 1 --2- 172.21.15.173:0/2957601599 >> v2:172.21.15.16:3300/0 conn(0x7fb088160a20 0x7fb0880cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.434 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.431+0000 7fb08dd15640 1 -- 172.21.15.173:0/2957601599 >> v1:172.21.15.16:6789/0 conn(0x7fb088160e00 legacy=0x7fb0880b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:38.434 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.431+0000 7fb08dd15640 1 -- 172.21.15.173:0/2957601599 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0880d1a90 con 0x7fb0880b3b20 2023-11-24T06:26:38.435 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.432+0000 7fb08dd15640 1 --2- 172.21.15.173:0/2957601599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0880b3b20 0x7fb0880cfd90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb08808d550 tx=0x7fb08002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7716cb1f19171 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:38.435 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.432+0000 7fb07ffff640 1 -- 172.21.15.173:0/2957601599 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08002ca80 con 0x7fb0880b3b20 2023-11-24T06:26:38.435 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.432+0000 7fb07ffff640 1 -- 172.21.15.173:0/2957601599 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08002cbe0 con 0x7fb0880b3b20 2023-11-24T06:26:38.435 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.432+0000 7fb07ffff640 1 -- 172.21.15.173:0/2957601599 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800343f0 con 0x7fb0880b3b20 2023-11-24T06:26:38.436 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.433+0000 7fb08e516640 1 -- 172.21.15.173:0/2957601599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0880b3b20 msgr2=0x7fb0880cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.436 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.433+0000 7fb08e516640 1 --2- 172.21.15.173:0/2957601599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0880b3b20 0x7fb0880cfd90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb08808d550 tx=0x7fb08002bee0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.436 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.433+0000 7fb08e516640 1 -- 172.21.15.173:0/2957601599 shutdown_connections 2023-11-24T06:26:38.436 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.433+0000 7fb08e516640 1 --2- 172.21.15.173:0/2957601599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0880b3b20 0x7fb0880cfd90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.436 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.433+0000 7fb08e516640 1 --2- 172.21.15.173:0/2957601599 >> v2:172.21.15.16:3300/0 conn(0x7fb088160a20 0x7fb0880cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.436 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.433+0000 7fb08e516640 1 -- 172.21.15.173:0/2957601599 >> 172.21.15.173:0/2957601599 conn(0x7fb0880d3120 msgr2=0x7fb0880d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:38.437 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.433+0000 7fb08e516640 1 -- 172.21.15.173:0/2957601599 shutdown_connections 2023-11-24T06:26:38.437 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.433+0000 7fb08e516640 1 -- 172.21.15.173:0/2957601599 wait complete. 2023-11-24T06:26:38.437 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.435+0000 7fb08e516640 1 Processor -- start 2023-11-24T06:26:38.437 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.435+0000 7fb08e516640 1 -- start start 2023-11-24T06:26:38.440 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08e516640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0880b3b20 0x7fb0880b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.440 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08e516640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb088160a20 0x7fb0880b8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.440 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08e516640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb088160e00 0x7fb08815c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.440 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08e516640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880b8ba0 con 0x7fb088160e00 2023-11-24T06:26:38.440 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08e516640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880cb2a0 con 0x7fb088160a20 2023-11-24T06:26:38.440 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08e516640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0880b7940 con 0x7fb0880b3b20 2023-11-24T06:26:38.440 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08cd13640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb088160a20 0x7fb0880b8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08d514640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0880b3b20 0x7fb0880b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08cd13640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb088160a20 0x7fb0880b8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48016/0 (socket says 172.21.15.173:48016) 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.436+0000 7fb08cd13640 1 -- 172.21.15.173:0/1247445852 learned_addr learned my addr 172.21.15.173:0/1247445852 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08dd15640 1 --2- 172.21.15.173:0/1247445852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb088160e00 0x7fb08815c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08cd13640 1 -- 172.21.15.173:0/1247445852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0880b3b20 msgr2=0x7fb0880b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08cd13640 1 --2- 172.21.15.173:0/1247445852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0880b3b20 0x7fb0880b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08cd13640 1 -- 172.21.15.173:0/1247445852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb088160e00 msgr2=0x7fb08815c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08cd13640 1 --2- 172.21.15.173:0/1247445852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb088160e00 0x7fb08815c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08cd13640 1 -- 172.21.15.173:0/1247445852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb080009b30 con 0x7fb088160a20 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08dd15640 1 --2- 172.21.15.173:0/1247445852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb088160e00 0x7fb08815c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:38.441 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08cd13640 1 --2- 172.21.15.173:0/1247445852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb088160a20 0x7fb0880b8660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb07800b3f0 tx=0x7fb07800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:38.442 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb07dffb640 1 -- 172.21.15.173:0/1247445852 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb078011830 con 0x7fb088160a20 2023-11-24T06:26:38.442 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb07dffb640 1 -- 172.21.15.173:0/1247445852 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb078011990 con 0x7fb088160a20 2023-11-24T06:26:38.442 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08e516640 1 -- 172.21.15.173:0/1247445852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0880b6dc0 con 0x7fb088160a20 2023-11-24T06:26:38.442 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb07dffb640 1 -- 172.21.15.173:0/1247445852 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb078011d40 con 0x7fb088160a20 2023-11-24T06:26:38.442 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.437+0000 7fb08e516640 1 -- 172.21.15.173:0/1247445852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0880b7380 con 0x7fb088160a20 2023-11-24T06:26:38.442 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.438+0000 7fb07dffb640 1 -- 172.21.15.173:0/1247445852 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb078004320 con 0x7fb088160a20 2023-11-24T06:26:38.442 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.438+0000 7fb07dffb640 1 -- 172.21.15.173:0/1247445852 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fb07801a400 con 0x7fb088160a20 2023-11-24T06:26:38.442 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.438+0000 7fb08e516640 1 -- 172.21.15.173:0/1247445852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb088067b60 con 0x7fb088160a20 2023-11-24T06:26:38.443 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.441+0000 7fb07dffb640 1 -- 172.21.15.173:0/1247445852 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb078038ce0 con 0x7fb088160a20 2023-11-24T06:26:38.479 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:38.475+0000 7f25e8a13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:26:38.490 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:38.487+0000 7fd33683a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:26:38.522 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.519+0000 7fb08e516640 1 -- 172.21.15.173:0/1247445852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "55e17526-cd19-4a3d-a36b-c75b35e02906", "id": 1} v 0) v1 -- 0x7fb0880d2820 con 0x7fb088160a20 2023-11-24T06:26:38.530 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.528+0000 7fb07dffb640 1 -- 172.21.15.173:0/1247445852 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "55e17526-cd19-4a3d-a36b-c75b35e02906", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb078027070 con 0x7fb088160a20 2023-11-24T06:26:38.530 INFO:teuthology.orchestra.run.smithi173.stdout:1 2023-11-24T06:26:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.529+0000 7fb08e516640 1 -- 172.21.15.173:0/1247445852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb088160a20 msgr2=0x7fb0880b8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.529+0000 7fb08e516640 1 --2- 172.21.15.173:0/1247445852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb088160a20 0x7fb0880b8660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb07800b3f0 tx=0x7fb07800b8b0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.529+0000 7fb08e516640 1 -- 172.21.15.173:0/1247445852 shutdown_connections 2023-11-24T06:26:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.529+0000 7fb08e516640 1 --2- 172.21.15.173:0/1247445852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb088160e00 0x7fb08815c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.529+0000 7fb08e516640 1 --2- 172.21.15.173:0/1247445852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb088160a20 0x7fb0880b8660 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.529+0000 7fb08e516640 1 --2- 172.21.15.173:0/1247445852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0880b3b20 0x7fb0880b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.529+0000 7fb08e516640 1 -- 172.21.15.173:0/1247445852 >> 172.21.15.173:0/1247445852 conn(0x7fb0880d3120 msgr2=0x7fb08806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.529+0000 7fb08e516640 1 -- 172.21.15.173:0/1247445852 shutdown_connections 2023-11-24T06:26:38.532 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.529+0000 7fb08e516640 1 -- 172.21.15.173:0/1247445852 wait complete. 2023-11-24T06:26:38.533 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:38.530+0000 7f25e8a13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:26:38.544 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:38.541+0000 7fd33683a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:26:38.548 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new b4036fde-c18b-4aab-8fbb-5e735b8e5371 2 2023-11-24T06:26:38.598 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:38.595+0000 7f25e8a13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:26:38.610 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:38.607+0000 7fd33683a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:26:38.650 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:38.647+0000 7f25e8a13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:26:38.662 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:38.660+0000 7fd33683a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:26:38.737 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.734+0000 7fb606d53640 1 Processor -- start 2023-11-24T06:26:38.737 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.734+0000 7fb606d53640 1 -- start start 2023-11-24T06:26:38.739 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb606d53640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb6000ce8f0 0x7fb6000ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.739 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb606d53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6000b2a00 0x7fb6000b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.739 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb606d53640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000d0de0 con 0x7fb600160f80 2023-11-24T06:26:38.739 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb606d53640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6000d10c0 con 0x7fb6000b2a00 2023-11-24T06:26:38.739 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb606d53640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6000d13a0 con 0x7fb6000ce8f0 2023-11-24T06:26:38.739 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb5fffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb6000ce8f0 0x7fb6000ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.739 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb5fffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb6000ce8f0 0x7fb6000ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48022/0 (socket says 172.21.15.173:48022) 2023-11-24T06:26:38.739 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb5fffff640 1 -- 172.21.15.173:0/480082769 learned_addr learned my addr 172.21.15.173:0/480082769 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:38.740 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb5fffff640 1 -- 172.21.15.173:0/480082769 >> v2:172.21.15.16:3300/0 conn(0x7fb6000b2a00 msgr2=0x7fb6000b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:38.740 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.736+0000 7fb5fffff640 1 --2- 172.21.15.173:0/480082769 >> v2:172.21.15.16:3300/0 conn(0x7fb6000b2a00 0x7fb6000b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.740 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.737+0000 7fb5fffff640 1 -- 172.21.15.173:0/480082769 >> v1:172.21.15.173:6789/0 conn(0x7fb600160f80 legacy=0x7fb600161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.740 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.737+0000 7fb5fffff640 1 -- 172.21.15.173:0/480082769 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000d1680 con 0x7fb6000ce8f0 2023-11-24T06:26:38.740 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.738+0000 7fb5fffff640 1 --2- 172.21.15.173:0/480082769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000ce8f0 0x7fb6000ced30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb6000b5d10 tx=0x7fb5ec02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9f7143890f14a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:38.740 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.738+0000 7fb5ff7fe640 1 -- 172.21.15.173:0/480082769 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec030860 con 0x7fb6000ce8f0 2023-11-24T06:26:38.741 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.738+0000 7fb5ff7fe640 1 -- 172.21.15.173:0/480082769 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec0309c0 con 0x7fb6000ce8f0 2023-11-24T06:26:38.741 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.738+0000 7fb5ff7fe640 1 -- 172.21.15.173:0/480082769 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec030d50 con 0x7fb6000ce8f0 2023-11-24T06:26:38.741 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.739+0000 7fb606d53640 1 -- 172.21.15.173:0/480082769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000ce8f0 msgr2=0x7fb6000ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.742 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.739+0000 7fb606d53640 1 --2- 172.21.15.173:0/480082769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000ce8f0 0x7fb6000ced30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb6000b5d10 tx=0x7fb5ec02b9a0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.742 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.739+0000 7fb606d53640 1 -- 172.21.15.173:0/480082769 shutdown_connections 2023-11-24T06:26:38.742 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.739+0000 7fb606d53640 1 --2- 172.21.15.173:0/480082769 >> v2:172.21.15.16:3300/0 conn(0x7fb6000b2a00 0x7fb6000b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.742 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.739+0000 7fb606d53640 1 --2- 172.21.15.173:0/480082769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000ce8f0 0x7fb6000ced30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.742 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.739+0000 7fb606d53640 1 -- 172.21.15.173:0/480082769 >> 172.21.15.173:0/480082769 conn(0x7fb60009c550 msgr2=0x7fb6000d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:38.742 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.739+0000 7fb606d53640 1 -- 172.21.15.173:0/480082769 shutdown_connections 2023-11-24T06:26:38.742 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.739+0000 7fb606d53640 1 -- 172.21.15.173:0/480082769 wait complete. 2023-11-24T06:26:38.743 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.740+0000 7fb606d53640 1 Processor -- start 2023-11-24T06:26:38.743 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.741+0000 7fb606d53640 1 -- start start 2023-11-24T06:26:38.745 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.741+0000 7fb606d53640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000b2a00 0x7fb6001582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.745 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.741+0000 7fb606d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6000ce8f0 0x7fb6001600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.745 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb606d53640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb600160f80 0x7fb60015f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:38.745 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb606d53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000c7810 con 0x7fb6000ce8f0 2023-11-24T06:26:38.745 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb606d53640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000c90b0 con 0x7fb6000b2a00 2023-11-24T06:26:38.745 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb606d53640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60015f9b0 con 0x7fb600160f80 2023-11-24T06:26:38.745 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb604aef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000b2a00 0x7fb6001582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.745 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb6052f0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb600160f80 0x7fb60015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.745 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb604aef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000b2a00 0x7fb6001582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48036/0 (socket says 172.21.15.173:48036) 2023-11-24T06:26:38.746 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb604aef640 1 -- 172.21.15.173:0/4091983725 learned_addr learned my addr 172.21.15.173:0/4091983725 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:38.746 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb5fffff640 1 --2- 172.21.15.173:0/4091983725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6000ce8f0 0x7fb6001600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:38.746 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb604aef640 1 -- 172.21.15.173:0/4091983725 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb600160f80 msgr2=0x7fb60015f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.746 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb604aef640 1 --2- 172.21.15.173:0/4091983725 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb600160f80 0x7fb60015f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.746 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb604aef640 1 -- 172.21.15.173:0/4091983725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6000ce8f0 msgr2=0x7fb6001600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.746 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb604aef640 1 --2- 172.21.15.173:0/4091983725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6000ce8f0 0x7fb6001600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.746 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.742+0000 7fb604aef640 1 -- 172.21.15.173:0/4091983725 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0094d0 con 0x7fb6000b2a00 2023-11-24T06:26:38.746 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.743+0000 7fb604aef640 1 --2- 172.21.15.173:0/4091983725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000b2a00 0x7fb6001582b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c430 tx=0x7fb5f400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:38.747 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.743+0000 7fb5fd7fa640 1 -- 172.21.15.173:0/4091983725 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40052d0 con 0x7fb6000b2a00 2023-11-24T06:26:38.747 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.743+0000 7fb5fd7fa640 1 -- 172.21.15.173:0/4091983725 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4005430 con 0x7fb6000b2a00 2023-11-24T06:26:38.747 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.743+0000 7fb5fd7fa640 1 -- 172.21.15.173:0/4091983725 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4003830 con 0x7fb6000b2a00 2023-11-24T06:26:38.747 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.743+0000 7fb606d53640 1 -- 172.21.15.173:0/4091983725 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60015fc30 con 0x7fb6000b2a00 2023-11-24T06:26:38.747 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.743+0000 7fb606d53640 1 -- 172.21.15.173:0/4091983725 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6000c7150 con 0x7fb6000b2a00 2023-11-24T06:26:38.747 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.743+0000 7fb5fd7fa640 1 -- 172.21.15.173:0/4091983725 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb5f4003c90 con 0x7fb6000b2a00 2023-11-24T06:26:38.747 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.743+0000 7fb5fd7fa640 1 -- 172.21.15.173:0/4091983725 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fb5f40119b0 con 0x7fb6000b2a00 2023-11-24T06:26:38.747 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.744+0000 7fb606d53640 1 -- 172.21.15.173:0/4091983725 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb600161cd0 con 0x7fb6000b2a00 2023-11-24T06:26:38.750 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.748+0000 7fb5fd7fa640 1 -- 172.21.15.173:0/4091983725 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb5f4017070 con 0x7fb6000b2a00 2023-11-24T06:26:38.768 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:38.765+0000 7f25e8a13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:26:38.795 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:38.792+0000 7fd33683a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:26:38.827 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.825+0000 7fb606d53640 1 -- 172.21.15.173:0/4091983725 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b4036fde-c18b-4aab-8fbb-5e735b8e5371", "id": 2} v 0) v1 -- 0x7fb6000675d0 con 0x7fb6000b2a00 2023-11-24T06:26:38.834 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.831+0000 7fb5fd7fa640 1 -- 172.21.15.173:0/4091983725 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4036fde-c18b-4aab-8fbb-5e735b8e5371", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb5f4011e50 con 0x7fb6000b2a00 2023-11-24T06:26:38.834 INFO:teuthology.orchestra.run.smithi173.stdout:2 2023-11-24T06:26:38.835 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.832+0000 7fb606d53640 1 -- 172.21.15.173:0/4091983725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000b2a00 msgr2=0x7fb6001582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:38.835 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.832+0000 7fb606d53640 1 --2- 172.21.15.173:0/4091983725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000b2a00 0x7fb6001582b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c430 tx=0x7fb5f400c8f0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.836 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.833+0000 7fb606d53640 1 -- 172.21.15.173:0/4091983725 shutdown_connections 2023-11-24T06:26:38.836 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.833+0000 7fb606d53640 1 --2- 172.21.15.173:0/4091983725 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb600160f80 0x7fb60015f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.836 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.833+0000 7fb606d53640 1 --2- 172.21.15.173:0/4091983725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6000ce8f0 0x7fb6001600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.836 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.833+0000 7fb606d53640 1 --2- 172.21.15.173:0/4091983725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6000b2a00 0x7fb6001582b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:38.836 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.833+0000 7fb606d53640 1 -- 172.21.15.173:0/4091983725 >> 172.21.15.173:0/4091983725 conn(0x7fb60009c550 msgr2=0x7fb6000d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:38.836 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.833+0000 7fb606d53640 1 -- 172.21.15.173:0/4091983725 shutdown_connections 2023-11-24T06:26:38.836 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:38.833+0000 7fb606d53640 1 -- 172.21.15.173:0/4091983725 wait complete. 2023-11-24T06:26:38.851 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new e4695946-b031-4353-b5b2-de60bbf183e4 3 2023-11-24T06:26:38.853 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:38.850+0000 7f25e8a13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:26:38.881 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:38.878+0000 7fd33683a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:26:38.909 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:38.906+0000 7f25e8a13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:26:38.937 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:38.934+0000 7fd33683a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:26:38.973 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:38.970+0000 7f25e8a13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:26:39.002 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:38.999+0000 7fd33683a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:26:39.031 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:39.027+0000 7f25e8a13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:26:39.043 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.041+0000 7f49dffff640 1 Processor -- start 2023-11-24T06:26:39.043 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.041+0000 7f49dffff640 1 -- start start 2023-11-24T06:26:39.045 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.042+0000 7f49dffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49e00cf9a0 0x7f49e00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.045 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.042+0000 7f49dffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00d0320 con 0x7f49e00b31d0 2023-11-24T06:26:39.045 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.042+0000 7f49dffff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00d1c60 con 0x7f49e015d4b0 2023-11-24T06:26:39.045 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.042+0000 7f49dffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e00b2730 con 0x7f49e00cf9a0 2023-11-24T06:26:39.045 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49deffd640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f49e015d4b0 0x7f49e00d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:45902/0 (socket says 172.21.15.173:45902) 2023-11-24T06:26:39.045 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49deffd640 1 -- 172.21.15.173:0/327728792 learned_addr learned my addr 172.21.15.173:0/327728792 (peer_addr_for_me v1:172.21.15.173:0/0) 2023-11-24T06:26:39.047 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501361882 0 0) 0x7f49e00d1c60 con 0x7f49e015d4b0 2023-11-24T06:26:39.047 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49de7fc640 1 --2- 172.21.15.173:0/327728792 >> v2:172.21.15.16:3300/0 conn(0x7f49e00cf9a0 0x7f49e00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.047 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49c8003520 con 0x7f49e015d4b0 2023-11-24T06:26:39.047 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 <== mon.1 v1:172.21.15.173:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1168457903 0 0) 0x7f49c8003520 con 0x7f49e015d4b0 2023-11-24T06:26:39.047 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 >> v2:172.21.15.16:3300/0 conn(0x7f49e00cf9a0 msgr2=0x7f49e00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 --2- 172.21.15.173:0/327728792 >> v2:172.21.15.16:3300/0 conn(0x7f49e00cf9a0 0x7f49e00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 >> v1:172.21.15.16:6789/0 conn(0x7f49e00b31d0 legacy=0x7f49e00b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 --> v1:172.21.15.173:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e00d1ec0 con 0x7f49e015d4b0 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 <== mon.1 v1:172.21.15.173:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f49d0001670 con 0x7f49e015d4b0 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 >> v1:172.21.15.173:6789/0 conn(0x7f49e015d4b0 legacy=0x7f49e00d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8003970 0x7f49c8003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49c80047c0 0x7f49c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49c8005180 0x7f49c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00b2730 con 0x7f49c8003970 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c800c0b0 con 0x7f49c8005180 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.043+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c800c270 con 0x7f49c80047c0 2023-11-24T06:26:39.048 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.044+0000 7f49df7fe640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49c8005180 0x7f49c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.044+0000 7f49deffd640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8003970 0x7f49c8003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.044+0000 7f49de7fc640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49c80047c0 0x7f49c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.044+0000 7f49df7fe640 1 -- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49c80047c0 msgr2=0x7f49c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:39.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.044+0000 7f49df7fe640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49c80047c0 0x7f49c8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.044+0000 7f49df7fe640 1 -- 172.21.15.173:0/327728792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8003970 msgr2=0x7f49c8003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:39.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.044+0000 7f49df7fe640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8003970 0x7f49c8003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.044+0000 7f49df7fe640 1 -- 172.21.15.173:0/327728792 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c8003520 con 0x7f49c8005180 2023-11-24T06:26:39.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.044+0000 7f49deffd640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8003970 0x7f49c8003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:39.050 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.046+0000 7f49df7fe640 1 -- 172.21.15.173:0/327728792 reap_dead start 2023-11-24T06:26:39.050 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.046+0000 7f49df7fe640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49c8005180 0x7f49c800b980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49d8004480 tx=0x7f49c80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceb074ea8ad92fc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:39.050 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.046+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8006b50 con 0x7f49c8005180 2023-11-24T06:26:39.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.046+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d8006cd0 con 0x7f49c8005180 2023-11-24T06:26:39.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.046+0000 7f49ddffb640 1 -- 172.21.15.173:0/327728792 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8007260 con 0x7f49c8005180 2023-11-24T06:26:39.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.047+0000 7f49dffff640 1 -- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49c8005180 msgr2=0x7f49c800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.047+0000 7f49dffff640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49c8005180 0x7f49c800b980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49d8004480 tx=0x7f49c80057f0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.047+0000 7f49dffff640 1 -- 172.21.15.173:0/327728792 shutdown_connections 2023-11-24T06:26:39.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.047+0000 7f49dffff640 1 --2- 172.21.15.173:0/327728792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49c8005180 0x7f49c800b980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.047+0000 7f49dffff640 1 -- 172.21.15.173:0/327728792 >> 172.21.15.173:0/327728792 conn(0x7f49e00d3000 msgr2=0x7f49e00d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:39.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.047+0000 7f49dffff640 1 -- 172.21.15.173:0/327728792 shutdown_connections 2023-11-24T06:26:39.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.047+0000 7f49dffff640 1 -- 172.21.15.173:0/327728792 wait complete. 2023-11-24T06:26:39.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.049+0000 7f49dffff640 1 Processor -- start 2023-11-24T06:26:39.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.049+0000 7f49dffff640 1 -- start start 2023-11-24T06:26:39.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8005180 0x7f49e0155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49dffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e00ca0d0 0x7f49e01f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49dffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e00cb650 0x7f49e00a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49dffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0155a70 con 0x7f49c8005180 2023-11-24T06:26:39.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49dffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e01f3940 con 0x7f49e00ca0d0 2023-11-24T06:26:39.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49dffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00cc5e0 con 0x7f49e00cb650 2023-11-24T06:26:39.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49de7fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e00ca0d0 0x7f49e01f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49de7fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e00ca0d0 0x7f49e01f3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48056/0 (socket says 172.21.15.173:48056) 2023-11-24T06:26:39.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49de7fc640 1 -- 172.21.15.173:0/3685390560 learned_addr learned my addr 172.21.15.173:0/3685390560 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49df7fe640 1 --2- 172.21.15.173:0/3685390560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e00cb650 0x7f49e00a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49deffd640 1 --2- 172.21.15.173:0/3685390560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8005180 0x7f49e0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49de7fc640 1 -- 172.21.15.173:0/3685390560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e00cb650 msgr2=0x7f49e00a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49de7fc640 1 --2- 172.21.15.173:0/3685390560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e00cb650 0x7f49e00a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49de7fc640 1 -- 172.21.15.173:0/3685390560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8005180 msgr2=0x7f49e0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49de7fc640 1 --2- 172.21.15.173:0/3685390560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8005180 0x7f49e0155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.050+0000 7f49de7fc640 1 -- 172.21.15.173:0/3685390560 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e00ca5b0 con 0x7f49e00ca0d0 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.051+0000 7f49de7fc640 1 --2- 172.21.15.173:0/3685390560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e00ca0d0 0x7f49e01f3400 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49d400b7c0 tx=0x7f49d400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.051+0000 7f49c7fff640 1 -- 172.21.15.173:0/3685390560 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4010830 con 0x7f49e00ca0d0 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.051+0000 7f49c7fff640 1 -- 172.21.15.173:0/3685390560 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4010990 con 0x7f49e00ca0d0 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.051+0000 7f49dffff640 1 -- 172.21.15.173:0/3685390560 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e00ca890 con 0x7f49e00ca0d0 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.051+0000 7f49c7fff640 1 -- 172.21.15.173:0/3685390560 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4010d20 con 0x7f49e00ca0d0 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.051+0000 7f49dffff640 1 -- 172.21.15.173:0/3685390560 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e00cae50 con 0x7f49e00ca0d0 2023-11-24T06:26:39.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.051+0000 7f49c7fff640 1 -- 172.21.15.173:0/3685390560 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f49d401d750 con 0x7f49e00ca0d0 2023-11-24T06:26:39.056 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.051+0000 7f49c7fff640 1 -- 172.21.15.173:0/3685390560 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f49d40145c0 con 0x7f49e00ca0d0 2023-11-24T06:26:39.056 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.052+0000 7f49dffff640 1 -- 172.21.15.173:0/3685390560 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e0067450 con 0x7f49e00ca0d0 2023-11-24T06:26:39.057 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.055+0000 7f49c7fff640 1 -- 172.21.15.173:0/3685390560 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f49d4022060 con 0x7f49e00ca0d0 2023-11-24T06:26:39.061 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:39.059+0000 7fd33683a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:26:39.135 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.133+0000 7f49dffff640 1 -- 172.21.15.173:0/3685390560 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e4695946-b031-4353-b5b2-de60bbf183e4", "id": 3} v 0) v1 -- 0x7f49e00cff90 con 0x7f49e00ca0d0 2023-11-24T06:26:39.143 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.141+0000 7f49c7fff640 1 -- 172.21.15.173:0/3685390560 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e4695946-b031-4353-b5b2-de60bbf183e4", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f49d4018070 con 0x7f49e00ca0d0 2023-11-24T06:26:39.144 INFO:teuthology.orchestra.run.smithi173.stdout:3 2023-11-24T06:26:39.145 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.142+0000 7f49dffff640 1 -- 172.21.15.173:0/3685390560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e00ca0d0 msgr2=0x7f49e01f3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.145 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.142+0000 7f49dffff640 1 --2- 172.21.15.173:0/3685390560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e00ca0d0 0x7f49e01f3400 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49d400b7c0 tx=0x7f49d400bc80 comp rx=0 tx=0).stop 2023-11-24T06:26:39.146 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.142+0000 7f49dffff640 1 -- 172.21.15.173:0/3685390560 shutdown_connections 2023-11-24T06:26:39.146 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.142+0000 7f49dffff640 1 --2- 172.21.15.173:0/3685390560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e00cb650 0x7f49e00a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.146 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.142+0000 7f49dffff640 1 --2- 172.21.15.173:0/3685390560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e00ca0d0 0x7f49e01f3400 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.146 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.142+0000 7f49dffff640 1 --2- 172.21.15.173:0/3685390560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c8005180 0x7f49e0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.146 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.142+0000 7f49dffff640 1 -- 172.21.15.173:0/3685390560 >> 172.21.15.173:0/3685390560 conn(0x7f49e00d3000 msgr2=0x7f49e006b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:39.146 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.143+0000 7f49dffff640 1 -- 172.21.15.173:0/3685390560 shutdown_connections 2023-11-24T06:26:39.146 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.143+0000 7f49dffff640 1 -- 172.21.15.173:0/3685390560 wait complete. 2023-11-24T06:26:39.160 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 0132f098-065d-4b24-9340-025b067ab6e4 4 2023-11-24T06:26:39.398 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.395+0000 7feaebe94640 1 Processor -- start 2023-11-24T06:26:39.398 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.396+0000 7feaebe94640 1 -- start start 2023-11-24T06:26:39.400 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feaebe94640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7feae40ce8f0 0x7feae40ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.400 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feaebe94640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae40b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.400 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feaebe94640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae40d0de0 con 0x7feae4160f80 2023-11-24T06:26:39.400 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feaebe94640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae40d10c0 con 0x7feae40ce8f0 2023-11-24T06:26:39.400 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feaebe94640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae40d13a0 con 0x7feae40b2a00 2023-11-24T06:26:39.400 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feae942f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7feae40ce8f0 0x7feae40ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.401 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feaea431640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae40b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.401 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feae942f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7feae40ce8f0 0x7feae40ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48062/0 (socket says 172.21.15.173:48062) 2023-11-24T06:26:39.401 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feaea431640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae40b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:52518/0 (socket says 172.21.15.173:52518) 2023-11-24T06:26:39.401 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.397+0000 7feae942f640 1 -- 172.21.15.173:0/1599037275 learned_addr learned my addr 172.21.15.173:0/1599037275 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:39.402 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.398+0000 7feaea431640 1 -- 172.21.15.173:0/1599037275 >> v2:172.21.15.173:3300/0 conn(0x7feae40ce8f0 msgr2=0x7feae40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:39.402 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.398+0000 7feaea431640 1 --2- 172.21.15.173:0/1599037275 >> v2:172.21.15.173:3300/0 conn(0x7feae40ce8f0 0x7feae40ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.402 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.398+0000 7feaea431640 1 -- 172.21.15.173:0/1599037275 >> v1:172.21.15.16:6789/0 conn(0x7feae4160f80 legacy=0x7feae4161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.402 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.398+0000 7feaea431640 1 -- 172.21.15.173:0/1599037275 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae40d1680 con 0x7feae40b2a00 2023-11-24T06:26:39.402 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.399+0000 7feaea431640 1 --2- 172.21.15.173:0/1599037275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae40b2e90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feadc009d90 tx=0x7feadc02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f245c4e83bf1af5b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:39.402 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.399+0000 7feae8c2e640 1 -- 172.21.15.173:0/1599037275 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc02cc50 con 0x7feae40b2a00 2023-11-24T06:26:39.403 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.399+0000 7feae8c2e640 1 -- 172.21.15.173:0/1599037275 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc02cdb0 con 0x7feae40b2a00 2023-11-24T06:26:39.403 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.399+0000 7feae8c2e640 1 -- 172.21.15.173:0/1599037275 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc034660 con 0x7feae40b2a00 2023-11-24T06:26:39.403 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.400+0000 7feaebe94640 1 -- 172.21.15.173:0/1599037275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 msgr2=0x7feae40b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.403 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.400+0000 7feaebe94640 1 --2- 172.21.15.173:0/1599037275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae40b2e90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feadc009d90 tx=0x7feadc02bea0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.404 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.400+0000 7feaebe94640 1 -- 172.21.15.173:0/1599037275 shutdown_connections 2023-11-24T06:26:39.404 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.400+0000 7feaebe94640 1 --2- 172.21.15.173:0/1599037275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae40b2e90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.404 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.400+0000 7feaebe94640 1 --2- 172.21.15.173:0/1599037275 >> v2:172.21.15.173:3300/0 conn(0x7feae40ce8f0 0x7feae40ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.404 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.400+0000 7feaebe94640 1 -- 172.21.15.173:0/1599037275 >> 172.21.15.173:0/1599037275 conn(0x7feae409c550 msgr2=0x7feae40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:39.404 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.400+0000 7feaebe94640 1 -- 172.21.15.173:0/1599037275 shutdown_connections 2023-11-24T06:26:39.404 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.400+0000 7feaebe94640 1 -- 172.21.15.173:0/1599037275 wait complete. 2023-11-24T06:26:39.404 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.402+0000 7feaebe94640 1 Processor -- start 2023-11-24T06:26:39.405 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.402+0000 7feaebe94640 1 -- start start 2023-11-24T06:26:39.407 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feaebe94640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae4158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.407 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feaebe94640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae40ce8f0 0x7feae40b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.407 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feaebe94640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4160f80 0x7feae406be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.407 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feaebe94640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41605c0 con 0x7feae4160f80 2023-11-24T06:26:39.407 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feaebe94640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4158cd0 con 0x7feae40ce8f0 2023-11-24T06:26:39.407 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feaebe94640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae415f6d0 con 0x7feae40b2a00 2023-11-24T06:26:39.407 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae9c30640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae4158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.407 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae942f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae40ce8f0 0x7feae40b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.407 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae942f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae40ce8f0 0x7feae40b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48068/0 (socket says 172.21.15.173:48068) 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae942f640 1 -- 172.21.15.173:0/414809550 learned_addr learned my addr 172.21.15.173:0/414809550 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae9c30640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae4158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:52520/0 (socket says 172.21.15.173:52520) 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feaea431640 1 --2- 172.21.15.173:0/414809550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4160f80 0x7feae406be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae9c30640 1 -- 172.21.15.173:0/414809550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae40ce8f0 msgr2=0x7feae40b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae9c30640 1 --2- 172.21.15.173:0/414809550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae40ce8f0 0x7feae40b7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae9c30640 1 -- 172.21.15.173:0/414809550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4160f80 msgr2=0x7feae406be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae9c30640 1 --2- 172.21.15.173:0/414809550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4160f80 0x7feae406be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.403+0000 7feae9c30640 1 -- 172.21.15.173:0/414809550 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc009a70 con 0x7feae40b2a00 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.404+0000 7feae9c30640 1 --2- 172.21.15.173:0/414809550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae4158790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feae000c390 tx=0x7feae000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.404+0000 7fead67fc640 1 -- 172.21.15.173:0/414809550 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae00052d0 con 0x7feae40b2a00 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.404+0000 7fead67fc640 1 -- 172.21.15.173:0/414809550 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae0005430 con 0x7feae40b2a00 2023-11-24T06:26:39.408 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.404+0000 7feaebe94640 1 -- 172.21.15.173:0/414809550 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae415f9b0 con 0x7feae40b2a00 2023-11-24T06:26:39.409 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.404+0000 7fead67fc640 1 -- 172.21.15.173:0/414809550 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0003830 con 0x7feae40b2a00 2023-11-24T06:26:39.409 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.404+0000 7feaebe94640 1 -- 172.21.15.173:0/414809550 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae415ff70 con 0x7feae40b2a00 2023-11-24T06:26:39.409 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.404+0000 7fead67fc640 1 -- 172.21.15.173:0/414809550 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7feae0003c90 con 0x7feae40b2a00 2023-11-24T06:26:39.409 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.404+0000 7fead67fc640 1 -- 172.21.15.173:0/414809550 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7feae0011e00 con 0x7feae40b2a00 2023-11-24T06:26:39.409 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.405+0000 7feaebe94640 1 -- 172.21.15.173:0/414809550 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae40675d0 con 0x7feae40b2a00 2023-11-24T06:26:39.410 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.408+0000 7fead67fc640 1 -- 172.21.15.173:0/414809550 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7feae0017070 con 0x7feae40b2a00 2023-11-24T06:26:39.489 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.486+0000 7feaebe94640 1 -- 172.21.15.173:0/414809550 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0132f098-065d-4b24-9340-025b067ab6e4", "id": 4} v 0) v1 -- 0x7feae40d3920 con 0x7feae40b2a00 2023-11-24T06:26:39.497 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.495+0000 7fead67fc640 1 -- 172.21.15.173:0/414809550 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0132f098-065d-4b24-9340-025b067ab6e4", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7feae0011590 con 0x7feae40b2a00 2023-11-24T06:26:39.497 INFO:teuthology.orchestra.run.smithi173.stdout:4 2023-11-24T06:26:39.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.496+0000 7feaebe94640 1 -- 172.21.15.173:0/414809550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 msgr2=0x7feae4158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.496+0000 7feaebe94640 1 --2- 172.21.15.173:0/414809550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae4158790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feae000c390 tx=0x7feae000c850 comp rx=0 tx=0).stop 2023-11-24T06:26:39.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.496+0000 7feaebe94640 1 -- 172.21.15.173:0/414809550 shutdown_connections 2023-11-24T06:26:39.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.496+0000 7feaebe94640 1 --2- 172.21.15.173:0/414809550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae4160f80 0x7feae406be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.496+0000 7feaebe94640 1 --2- 172.21.15.173:0/414809550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae40ce8f0 0x7feae40b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.496+0000 7feaebe94640 1 --2- 172.21.15.173:0/414809550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae40b2a00 0x7feae4158790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.496+0000 7feaebe94640 1 -- 172.21.15.173:0/414809550 >> 172.21.15.173:0/414809550 conn(0x7feae409c550 msgr2=0x7feae4161360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:39.500 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.496+0000 7feaebe94640 1 -- 172.21.15.173:0/414809550 shutdown_connections 2023-11-24T06:26:39.500 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.496+0000 7feaebe94640 1 -- 172.21.15.173:0/414809550 wait complete. 2023-11-24T06:26:39.506 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:39.503+0000 7f25e8a13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:26:39.514 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 36be1f62-f8bb-4c5d-a94c-1b59a67cbead 5 2023-11-24T06:26:39.538 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:39.535+0000 7fd33683a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:26:39.674 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:39.671+0000 7f25e8a13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:26:39.696 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.694+0000 7ff5cf568640 1 Processor -- start 2023-11-24T06:26:39.697 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.694+0000 7ff5cf568640 1 -- start start 2023-11-24T06:26:39.698 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.695+0000 7ff5cf568640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c8160f80 0x7ff5c8161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.698 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.695+0000 7ff5cf568640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff5c80ced50 0x7ff5c80d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.698 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.695+0000 7ff5cf568640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c80cf1b0 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.699 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.695+0000 7ff5cf568640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c81618a0 con 0x7ff5c80ced50 2023-11-24T06:26:39.699 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.695+0000 7ff5cf568640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80d0de0 con 0x7ff5c8160f80 2023-11-24T06:26:39.699 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.696+0000 7ff5cd304640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c8160f80 0x7ff5c8161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.699 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.696+0000 7ff5cdb05640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff5c80ced50 0x7ff5c80d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.699 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.696+0000 7ff5cdb05640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff5c80ced50 0x7ff5c80d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48080/0 (socket says 172.21.15.173:48080) 2023-11-24T06:26:39.699 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.696+0000 7ff5cdb05640 1 -- 172.21.15.173:0/74655651 learned_addr learned my addr 172.21.15.173:0/74655651 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:39.699 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.696+0000 7ff5cd304640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c8160f80 0x7ff5c8161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:52528/0 (socket says 172.21.15.173:52528) 2023-11-24T06:26:39.700 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.696+0000 7ff5cd304640 1 -- 172.21.15.173:0/74655651 >> v2:172.21.15.173:3300/0 conn(0x7ff5c80ced50 msgr2=0x7ff5c80d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:39.700 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.696+0000 7ff5cd304640 1 --2- 172.21.15.173:0/74655651 >> v2:172.21.15.173:3300/0 conn(0x7ff5c80ced50 0x7ff5c80d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.700 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.696+0000 7ff5cd304640 1 -- 172.21.15.173:0/74655651 >> v1:172.21.15.173:6789/0 conn(0x7ff5c80ce8f0 legacy=0x7ff5c80cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.700 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.696+0000 7ff5cd304640 1 -- 172.21.15.173:0/74655651 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80d10c0 con 0x7ff5c8160f80 2023-11-24T06:26:39.700 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.697+0000 7ff5cd304640 1 --2- 172.21.15.173:0/74655651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c8160f80 0x7ff5c8161360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097a0 tx=0x7ff5c402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a2254365f41c0e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:39.701 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5bffff640 1 -- 172.21.15.173:0/74655651 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c402d040 con 0x7ff5c8160f80 2023-11-24T06:26:39.701 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5bffff640 1 -- 172.21.15.173:0/74655651 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c4031810 con 0x7ff5c8160f80 2023-11-24T06:26:39.701 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5bffff640 1 -- 172.21.15.173:0/74655651 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4031ba0 con 0x7ff5c8160f80 2023-11-24T06:26:39.701 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5cf568640 1 -- 172.21.15.173:0/74655651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c8160f80 msgr2=0x7ff5c8161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.701 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5cf568640 1 --2- 172.21.15.173:0/74655651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c8160f80 0x7ff5c8161360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5c40097a0 tx=0x7ff5c402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.701 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5cf568640 1 -- 172.21.15.173:0/74655651 shutdown_connections 2023-11-24T06:26:39.702 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5cf568640 1 --2- 172.21.15.173:0/74655651 >> v2:172.21.15.173:3300/0 conn(0x7ff5c80ced50 0x7ff5c80d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.702 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5cf568640 1 --2- 172.21.15.173:0/74655651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c8160f80 0x7ff5c8161360 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.702 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5cf568640 1 -- 172.21.15.173:0/74655651 >> 172.21.15.173:0/74655651 conn(0x7ff5c809c550 msgr2=0x7ff5c80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:39.702 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.698+0000 7ff5cf568640 1 -- 172.21.15.173:0/74655651 shutdown_connections 2023-11-24T06:26:39.702 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.699+0000 7ff5cf568640 1 -- 172.21.15.173:0/74655651 wait complete. 2023-11-24T06:26:39.702 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.700+0000 7ff5cf568640 1 Processor -- start 2023-11-24T06:26:39.703 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.700+0000 7ff5cf568640 1 -- start start 2023-11-24T06:26:39.705 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cf568640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c80ce8f0 0x7ff5c81f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.705 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cf568640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5c80ced50 0x7ff5c81f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.705 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cf568640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5c8160f80 0x7ff5c815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cf568640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c815d840 con 0x7ff5c80ced50 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cf568640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80b88a0 con 0x7ff5c8160f80 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cf568640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80b8b80 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cd304640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c80ce8f0 0x7ff5c81f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cd304640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c80ce8f0 0x7ff5c81f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:52544/0 (socket says 172.21.15.173:52544) 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cd304640 1 -- 172.21.15.173:0/917162472 learned_addr learned my addr 172.21.15.173:0/917162472 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.701+0000 7ff5cdb05640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5c8160f80 0x7ff5c815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.702+0000 7ff5ccb03640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5c80ced50 0x7ff5c81f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.702+0000 7ff5cd304640 1 -- 172.21.15.173:0/917162472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5c8160f80 msgr2=0x7ff5c815fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.702+0000 7ff5cd304640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5c8160f80 0x7ff5c815fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.702+0000 7ff5cd304640 1 -- 172.21.15.173:0/917162472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5c80ced50 msgr2=0x7ff5c81f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.706 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.702+0000 7ff5cd304640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5c80ced50 0x7ff5c81f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.702+0000 7ff5cd304640 1 -- 172.21.15.173:0/917162472 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c4009400 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.702+0000 7ff5ccb03640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5c80ced50 0x7ff5c81f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.702+0000 7ff5cd304640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c80ce8f0 0x7ff5c81f6a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4000c00 tx=0x7ff5c40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.702+0000 7ff5bdffb640 1 -- 172.21.15.173:0/917162472 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4006e10 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.703+0000 7ff5bdffb640 1 -- 172.21.15.173:0/917162472 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c403ad60 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.703+0000 7ff5cf568640 1 -- 172.21.15.173:0/917162472 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c815db00 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.703+0000 7ff5bdffb640 1 -- 172.21.15.173:0/917162472 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4044630 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.703+0000 7ff5cf568640 1 -- 172.21.15.173:0/917162472 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c815e110 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.703+0000 7ff5bdffb640 1 -- 172.21.15.173:0/917162472 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff5c4044a90 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.707 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.703+0000 7ff5bdffb640 1 -- 172.21.15.173:0/917162472 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7ff5c4044d40 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.708 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.704+0000 7ff5cf568640 1 -- 172.21.15.173:0/917162472 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c815dd10 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.708 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:39.704+0000 7fd33683a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:26:39.709 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.706+0000 7ff5bdffb640 1 -- 172.21.15.173:0/917162472 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff5c4035020 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.787 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.784+0000 7ff5cf568640 1 -- 172.21.15.173:0/917162472 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "36be1f62-f8bb-4c5d-a94c-1b59a67cbead", "id": 5} v 0) v1 -- 0x7ff5c80675d0 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.796 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.794+0000 7ff5bdffb640 1 -- 172.21.15.173:0/917162472 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "36be1f62-f8bb-4c5d-a94c-1b59a67cbead", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff5c404d450 con 0x7ff5c80ce8f0 2023-11-24T06:26:39.797 INFO:teuthology.orchestra.run.smithi173.stdout:5 2023-11-24T06:26:39.798 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.795+0000 7ff5cf568640 1 -- 172.21.15.173:0/917162472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c80ce8f0 msgr2=0x7ff5c81f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:39.798 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.795+0000 7ff5cf568640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c80ce8f0 0x7ff5c81f6a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4000c00 tx=0x7ff5c40046c0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.798 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.795+0000 7ff5cf568640 1 -- 172.21.15.173:0/917162472 shutdown_connections 2023-11-24T06:26:39.798 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.795+0000 7ff5cf568640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5c8160f80 0x7ff5c815fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.798 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.795+0000 7ff5cf568640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5c80ced50 0x7ff5c81f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.798 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.795+0000 7ff5cf568640 1 --2- 172.21.15.173:0/917162472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5c80ce8f0 0x7ff5c81f6a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:39.798 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.795+0000 7ff5cf568640 1 -- 172.21.15.173:0/917162472 >> 172.21.15.173:0/917162472 conn(0x7ff5c809c550 msgr2=0x7ff5c806c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:39.798 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.795+0000 7ff5cf568640 1 -- 172.21.15.173:0/917162472 shutdown_connections 2023-11-24T06:26:39.799 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:39.795+0000 7ff5cf568640 1 -- 172.21.15.173:0/917162472 wait complete. 2023-11-24T06:26:39.814 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 0fb31be6-68df-42ac-9002-e21325dd058e 6 2023-11-24T06:26:39.825 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:39.822+0000 7f25e8a13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:26:39.863 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:39.854+0000 7fd33683a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:26:39.881 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:39.878+0000 7f25e8a13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:26:39.912 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:39.910+0000 7fd33683a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:26:39.996 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:39.993+0000 7f25e8a13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:26:40.030 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:40.027+0000 7fd33683a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:26:40.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.046+0000 7f5a9b4bb640 1 Processor -- start 2023-11-24T06:26:40.049 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.047+0000 7f5a9b4bb640 1 -- start start 2023-11-24T06:26:40.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9b4bb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5a9415d4b0 0x7f5a940d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9b4bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a940b41e0 0x7f5a940cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9b4bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a940d0320 con 0x7f5a940cf9a0 2023-11-24T06:26:40.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9b4bb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a940b2730 con 0x7f5a9415d4b0 2023-11-24T06:26:40.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9b4bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a940d1ec0 con 0x7f5a940b41e0 2023-11-24T06:26:40.051 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9a4b9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5a9415d4b0 0x7f5a940d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9acba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a940b41e0 0x7f5a940cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9acba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a940b41e0 0x7f5a940cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:52558/0 (socket says 172.21.15.173:52558) 2023-11-24T06:26:40.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9acba640 1 -- 172.21.15.173:0/3753264338 learned_addr learned my addr 172.21.15.173:0/3753264338 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:40.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9a4b9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5a9415d4b0 0x7f5a940d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:48094/0 (socket says 172.21.15.173:48094) 2023-11-24T06:26:40.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9acba640 1 -- 172.21.15.173:0/3753264338 >> v2:172.21.15.173:3300/0 conn(0x7f5a9415d4b0 msgr2=0x7f5a940d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:40.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9acba640 1 --2- 172.21.15.173:0/3753264338 >> v2:172.21.15.173:3300/0 conn(0x7f5a9415d4b0 0x7f5a940d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9acba640 1 -- 172.21.15.173:0/3753264338 >> v1:172.21.15.16:6789/0 conn(0x7f5a940cf9a0 legacy=0x7f5a940cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:40.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9acba640 1 -- 172.21.15.173:0/3753264338 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940b29d0 con 0x7f5a940b41e0 2023-11-24T06:26:40.052 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.048+0000 7f5a9a4b9640 1 --2- 172.21.15.173:0/3753264338 >> v2:172.21.15.173:3300/0 conn(0x7f5a9415d4b0 0x7f5a940d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:26:40.053 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.050+0000 7f5a9acba640 1 --2- 172.21.15.173:0/3753264338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a940b41e0 0x7f5a940cfde0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009e50 tx=0x7f5a8c02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42c173ff4a484fc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.050+0000 7f5a994b7640 1 -- 172.21.15.173:0/3753264338 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c02ca80 con 0x7f5a940b41e0 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.050+0000 7f5a994b7640 1 -- 172.21.15.173:0/3753264338 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c02cbe0 con 0x7f5a940b41e0 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.050+0000 7f5a994b7640 1 -- 172.21.15.173:0/3753264338 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0343f0 con 0x7f5a940b41e0 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.051+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3753264338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a940b41e0 msgr2=0x7f5a940cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.051+0000 7f5a9b4bb640 1 --2- 172.21.15.173:0/3753264338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a940b41e0 0x7f5a940cfde0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009e50 tx=0x7f5a8c02bee0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.051+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3753264338 shutdown_connections 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.051+0000 7f5a9b4bb640 1 --2- 172.21.15.173:0/3753264338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a940b41e0 0x7f5a940cfde0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.051+0000 7f5a9b4bb640 1 --2- 172.21.15.173:0/3753264338 >> v2:172.21.15.173:3300/0 conn(0x7f5a9415d4b0 0x7f5a940d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.051+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3753264338 >> 172.21.15.173:0/3753264338 conn(0x7f5a940d3000 msgr2=0x7f5a940d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.051+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3753264338 shutdown_connections 2023-11-24T06:26:40.054 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.051+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3753264338 wait complete. 2023-11-24T06:26:40.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.052+0000 7f5a9b4bb640 1 Processor -- start 2023-11-24T06:26:40.055 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.053+0000 7f5a9b4bb640 1 -- start start 2023-11-24T06:26:40.058 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.053+0000 7f5a9b4bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940b41e0 0x7f5a9409cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.058 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.053+0000 7f5a9b4bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a940cf9a0 0x7f5a940ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.058 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9b4bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a9415d4b0 0x7f5a94151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.058 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9b4bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a940a0d50 con 0x7f5a940b41e0 2023-11-24T06:26:40.058 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9b4bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a940a0a90 con 0x7f5a940cf9a0 2023-11-24T06:26:40.058 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9b4bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94151dd0 con 0x7f5a9415d4b0 2023-11-24T06:26:40.058 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9acba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a9415d4b0 0x7f5a94151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.058 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a99cb8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a940cf9a0 0x7f5a940ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.058 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9acba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a9415d4b0 0x7f5a94151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:52566/0 (socket says 172.21.15.173:52566) 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9acba640 1 -- 172.21.15.173:0/3526953233 learned_addr learned my addr 172.21.15.173:0/3526953233 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9a4b9640 1 --2- 172.21.15.173:0/3526953233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940b41e0 0x7f5a9409cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9acba640 1 -- 172.21.15.173:0/3526953233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a940cf9a0 msgr2=0x7f5a940ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9acba640 1 --2- 172.21.15.173:0/3526953233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a940cf9a0 0x7f5a940ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9acba640 1 -- 172.21.15.173:0/3526953233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940b41e0 msgr2=0x7f5a9409cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9acba640 1 --2- 172.21.15.173:0/3526953233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940b41e0 0x7f5a9409cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a9acba640 1 -- 172.21.15.173:0/3526953233 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a90009400 con 0x7f5a9415d4b0 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.054+0000 7f5a99cb8640 1 --2- 172.21.15.173:0/3526953233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a940cf9a0 0x7f5a940ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.055+0000 7f5a9acba640 1 --2- 172.21.15.173:0/3526953233 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a9415d4b0 0x7f5a94151070 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c036040 tx=0x7f5a8c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.055+0000 7f5a8affd640 1 -- 172.21.15.173:0/3526953233 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c009510 con 0x7f5a9415d4b0 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.055+0000 7f5a8affd640 1 -- 172.21.15.173:0/3526953233 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c03e450 con 0x7f5a9415d4b0 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.055+0000 7f5a8affd640 1 -- 172.21.15.173:0/3526953233 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c03e800 con 0x7f5a9415d4b0 2023-11-24T06:26:40.059 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.055+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3526953233 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c009b30 con 0x7f5a9415d4b0 2023-11-24T06:26:40.060 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.055+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3526953233 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9415da10 con 0x7f5a9415d4b0 2023-11-24T06:26:40.060 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.055+0000 7f5a8affd640 1 -- 172.21.15.173:0/3526953233 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5a8c02cd50 con 0x7f5a9415d4b0 2023-11-24T06:26:40.060 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.055+0000 7f5a8affd640 1 -- 172.21.15.173:0/3526953233 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f5a8c003c90 con 0x7f5a9415d4b0 2023-11-24T06:26:40.060 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.056+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3526953233 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a940b3fd0 con 0x7f5a9415d4b0 2023-11-24T06:26:40.061 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.058+0000 7f5a8affd640 1 -- 172.21.15.173:0/3526953233 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5a8c043020 con 0x7f5a9415d4b0 2023-11-24T06:26:40.112 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:40.109+0000 7f25e8a13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:26:40.139 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.136+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3526953233 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0fb31be6-68df-42ac-9002-e21325dd058e", "id": 6} v 0) v1 -- 0x7f5a940cd3c0 con 0x7f5a9415d4b0 2023-11-24T06:26:40.145 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.143+0000 7f5a8affd640 1 -- 172.21.15.173:0/3526953233 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0fb31be6-68df-42ac-9002-e21325dd058e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5a8c03ec80 con 0x7f5a9415d4b0 2023-11-24T06:26:40.145 INFO:teuthology.orchestra.run.smithi173.stdout:6 2023-11-24T06:26:40.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.144+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3526953233 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a9415d4b0 msgr2=0x7f5a94151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.144+0000 7f5a9b4bb640 1 --2- 172.21.15.173:0/3526953233 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a9415d4b0 0x7f5a94151070 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c036040 tx=0x7f5a8c004a40 comp rx=0 tx=0).stop 2023-11-24T06:26:40.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.144+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3526953233 shutdown_connections 2023-11-24T06:26:40.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.144+0000 7f5a9b4bb640 1 --2- 172.21.15.173:0/3526953233 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a9415d4b0 0x7f5a94151070 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.144+0000 7f5a9b4bb640 1 --2- 172.21.15.173:0/3526953233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a940cf9a0 0x7f5a940ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.147 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.144+0000 7f5a9b4bb640 1 --2- 172.21.15.173:0/3526953233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a940b41e0 0x7f5a9409cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.148 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.144+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3526953233 >> 172.21.15.173:0/3526953233 conn(0x7f5a940d3000 msgr2=0x7f5a9406ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:40.148 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.145+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3526953233 shutdown_connections 2023-11-24T06:26:40.148 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.145+0000 7f5a9b4bb640 1 -- 172.21.15.173:0/3526953233 wait complete. 2023-11-24T06:26:40.148 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:40.145+0000 7fd33683a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:26:40.164 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new b8fc590d-7dcb-4691-9d07-7ec1b870d761 7 2023-11-24T06:26:40.280 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:40.276+0000 7f25e8a13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:26:40.315 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:40.312+0000 7fd33683a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:26:40.331 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:40.328+0000 7f25e8a13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:26:40.357 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.354+0000 7f6857631640 1 Processor -- start 2023-11-24T06:26:40.357 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.355+0000 7f6857631640 1 -- start start 2023-11-24T06:26:40.359 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.356+0000 7f6857631640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6850160e10 0x7f68500d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.360 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.356+0000 7f6857631640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500d4160 con 0x7f68500cfcf0 2023-11-24T06:26:40.360 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.356+0000 7f6857631640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500b2420 con 0x7f68500d3910 2023-11-24T06:26:40.360 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.356+0000 7f6857631640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500ceff0 con 0x7f6850160e10 2023-11-24T06:26:40.360 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.356+0000 7f6855bce640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6850160e10 0x7f68500d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.360 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.356+0000 7f6854bcc640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f68500d3910 0x7f68500d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:45916/0 (socket says 172.21.15.173:45916) 2023-11-24T06:26:40.360 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.356+0000 7f6854bcc640 1 -- 172.21.15.173:0/1433196451 learned_addr learned my addr 172.21.15.173:0/1433196451 (peer_addr_for_me v1:172.21.15.173:0/0) 2023-11-24T06:26:40.361 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.357+0000 7f6855bce640 1 -- 172.21.15.173:0/1433196451 >> v1:172.21.15.173:6789/0 conn(0x7f68500d3910 legacy=0x7f68500d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.361 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.357+0000 7f6855bce640 1 -- 172.21.15.173:0/1433196451 >> v1:172.21.15.16:6789/0 conn(0x7f68500cfcf0 legacy=0x7f6850160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:40.361 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.357+0000 7f6847fff640 1 -- 172.21.15.173:0/1433196451 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863794781 0 0) 0x7f68500b2420 con 0x7f68500d3910 2023-11-24T06:26:40.361 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.357+0000 7f6855bce640 1 -- 172.21.15.173:0/1433196451 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500b2140 con 0x7f6850160e10 2023-11-24T06:26:40.363 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.358+0000 7f6855bce640 1 --2- 172.21.15.173:0/1433196451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6850160e10 0x7f68500d1810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6848009dc0 tx=0x7f684802be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b068c5b69e60be server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:40.363 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.358+0000 7f6847fff640 1 -- 172.21.15.173:0/1433196451 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684802cbb0 con 0x7f6850160e10 2023-11-24T06:26:40.363 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.358+0000 7f6847fff640 1 -- 172.21.15.173:0/1433196451 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684802cd10 con 0x7f6850160e10 2023-11-24T06:26:40.363 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.358+0000 7f6847fff640 1 -- 172.21.15.173:0/1433196451 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848034660 con 0x7f6850160e10 2023-11-24T06:26:40.363 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.359+0000 7f6857631640 1 -- 172.21.15.173:0/1433196451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6850160e10 msgr2=0x7f68500d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.363 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.359+0000 7f6857631640 1 --2- 172.21.15.173:0/1433196451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6850160e10 0x7f68500d1810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6848009dc0 tx=0x7f684802be00 comp rx=0 tx=0).stop 2023-11-24T06:26:40.364 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.359+0000 7f6857631640 1 -- 172.21.15.173:0/1433196451 shutdown_connections 2023-11-24T06:26:40.364 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.359+0000 7f6857631640 1 --2- 172.21.15.173:0/1433196451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6850160e10 0x7f68500d1810 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.364 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.359+0000 7f6857631640 1 -- 172.21.15.173:0/1433196451 >> 172.21.15.173:0/1433196451 conn(0x7f685009c530 msgr2=0x7f68500d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:40.364 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.359+0000 7f6857631640 1 -- 172.21.15.173:0/1433196451 shutdown_connections 2023-11-24T06:26:40.364 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.359+0000 7f6857631640 1 -- 172.21.15.173:0/1433196451 wait complete. 2023-11-24T06:26:40.364 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.361+0000 7f6857631640 1 Processor -- start 2023-11-24T06:26:40.364 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.361+0000 7f6857631640 1 -- start start 2023-11-24T06:26:40.366 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6857631640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68500cfcf0 0x7f685014a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.366 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6857631640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68500d3910 0x7f685014a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.366 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6857631640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6850160e10 0x7f685015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.366 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6857631640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500b61e0 con 0x7f68500cfcf0 2023-11-24T06:26:40.366 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6857631640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500c7140 con 0x7f6850160e10 2023-11-24T06:26:40.367 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6857631640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500b6430 con 0x7f68500d3910 2023-11-24T06:26:40.367 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6854bcc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68500d3910 0x7f685014a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.367 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6855bce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6850160e10 0x7f685015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.367 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6854bcc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68500d3910 0x7f685014a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:52576/0 (socket says 172.21.15.173:52576) 2023-11-24T06:26:40.367 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6854bcc640 1 -- 172.21.15.173:0/1701995297 learned_addr learned my addr 172.21.15.173:0/1701995297 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-11-24T06:26:40.367 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f68553cd640 1 --2- 172.21.15.173:0/1701995297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68500cfcf0 0x7f685014a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.367 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6854bcc640 1 -- 172.21.15.173:0/1701995297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6850160e10 msgr2=0x7f685015cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.368 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6854bcc640 1 --2- 172.21.15.173:0/1701995297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6850160e10 0x7f685015cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.368 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6854bcc640 1 -- 172.21.15.173:0/1701995297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68500cfcf0 msgr2=0x7f685014a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.368 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6854bcc640 1 --2- 172.21.15.173:0/1701995297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68500cfcf0 0x7f685014a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.368 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6854bcc640 1 -- 172.21.15.173:0/1701995297 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009a70 con 0x7f68500d3910 2023-11-24T06:26:40.368 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.362+0000 7f6855bce640 1 --2- 172.21.15.173:0/1701995297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6850160e10 0x7f685015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:26:40.368 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.363+0000 7f6854bcc640 1 --2- 172.21.15.173:0/1701995297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68500d3910 0x7f685014a8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f684000c4d0 tx=0x7f684000c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:40.368 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.363+0000 7f6845ffb640 1 -- 172.21.15.173:0/1701995297 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400074e0 con 0x7f68500d3910 2023-11-24T06:26:40.368 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.363+0000 7f6845ffb640 1 -- 172.21.15.173:0/1701995297 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840007660 con 0x7f68500d3910 2023-11-24T06:26:40.368 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.363+0000 7f6845ffb640 1 -- 172.21.15.173:0/1701995297 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840004a20 con 0x7f68500d3910 2023-11-24T06:26:40.369 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.363+0000 7f6857631640 1 -- 172.21.15.173:0/1701995297 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68500b6710 con 0x7f68500d3910 2023-11-24T06:26:40.369 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.363+0000 7f6857631640 1 -- 172.21.15.173:0/1701995297 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6850159be0 con 0x7f68500d3910 2023-11-24T06:26:40.369 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.363+0000 7f6845ffb640 1 -- 172.21.15.173:0/1701995297 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6840012400 con 0x7f68500d3910 2023-11-24T06:26:40.369 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.363+0000 7f6845ffb640 1 -- 172.21.15.173:0/1701995297 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f684001b9c0 con 0x7f68500d3910 2023-11-24T06:26:40.370 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.364+0000 7f6857631640 1 -- 172.21.15.173:0/1701995297 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68500675d0 con 0x7f68500d3910 2023-11-24T06:26:40.370 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:40.366+0000 7fd33683a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:26:40.370 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.366+0000 7f6845ffb640 1 -- 172.21.15.173:0/1701995297 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6840018070 con 0x7f68500d3910 2023-11-24T06:26:40.447 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.444+0000 7f6857631640 1 -- 172.21.15.173:0/1701995297 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b8fc590d-7dcb-4691-9d07-7ec1b870d761", "id": 7} v 0) v1 -- 0x7f68500b2ab0 con 0x7f68500d3910 2023-11-24T06:26:40.456 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.453+0000 7f6845ffb640 1 -- 172.21.15.173:0/1701995297 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b8fc590d-7dcb-4691-9d07-7ec1b870d761", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f684001bcf0 con 0x7f68500d3910 2023-11-24T06:26:40.456 INFO:teuthology.orchestra.run.smithi173.stdout:7 2023-11-24T06:26:40.457 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.454+0000 7f6857631640 1 -- 172.21.15.173:0/1701995297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68500d3910 msgr2=0x7f685014a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.458 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.454+0000 7f6857631640 1 --2- 172.21.15.173:0/1701995297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68500d3910 0x7f685014a8e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f684000c4d0 tx=0x7f684000c990 comp rx=0 tx=0).stop 2023-11-24T06:26:40.458 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.455+0000 7f6857631640 1 -- 172.21.15.173:0/1701995297 shutdown_connections 2023-11-24T06:26:40.458 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.455+0000 7f6857631640 1 --2- 172.21.15.173:0/1701995297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6850160e10 0x7f685015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.458 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.455+0000 7f6857631640 1 --2- 172.21.15.173:0/1701995297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68500d3910 0x7f685014a8e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.458 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.455+0000 7f6857631640 1 --2- 172.21.15.173:0/1701995297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68500cfcf0 0x7f685014a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.458 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.455+0000 7f6857631640 1 -- 172.21.15.173:0/1701995297 >> 172.21.15.173:0/1701995297 conn(0x7f685009c530 msgr2=0x7f68500b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:40.458 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.455+0000 7f6857631640 1 -- 172.21.15.173:0/1701995297 shutdown_connections 2023-11-24T06:26:40.458 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:26:40.455+0000 7f6857631640 1 -- 172.21.15.173:0/1701995297 wait complete. 2023-11-24T06:26:40.473 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T06:26:40.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T06:26:40.485 INFO:tasks.ceph.osd.0:Started 2023-11-24T06:26:40.485 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T06:26:40.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T06:26:40.494 INFO:tasks.ceph.osd.1:Started 2023-11-24T06:26:40.495 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T06:26:40.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T06:26:40.502 INFO:tasks.ceph.osd.2:Started 2023-11-24T06:26:40.502 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T06:26:40.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T06:26:40.503 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:40.495+0000 7f25e8a13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:26:40.508 INFO:tasks.ceph.osd.3:Started 2023-11-24T06:26:40.509 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T06:26:40.509 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T06:26:40.516 INFO:tasks.ceph.osd.4:Started 2023-11-24T06:26:40.516 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T06:26:40.516 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T06:26:40.522 INFO:tasks.ceph.osd.5:Started 2023-11-24T06:26:40.523 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T06:26:40.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T06:26:40.529 INFO:tasks.ceph.osd.6:Started 2023-11-24T06:26:40.529 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T06:26:40.530 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T06:26:40.534 INFO:tasks.ceph.osd.7:Started 2023-11-24T06:26:40.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:26:40.541 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:40.538+0000 7fd33683a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:26:40.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.766+0000 7fee4f67c640 1 Processor -- start 2023-11-24T06:26:40.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.766+0000 7fee4f67c640 1 -- start start 2023-11-24T06:26:40.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4f67c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee480cf9a0 0x7fee4815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4f67c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee4815e1e0 0x7fee480702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4f67c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee480b1ee0 con 0x7fee48143ae0 2023-11-24T06:26:40.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4f67c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480b2780 con 0x7fee4815e1e0 2023-11-24T06:26:40.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4f67c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4815e990 con 0x7fee480cf9a0 2023-11-24T06:26:40.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4dc19640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee4815e1e0 0x7fee480702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4cc17640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee480cf9a0 0x7fee4815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4dc19640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee4815e1e0 0x7fee480702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59522/0 (socket says 172.21.15.16:59522) 2023-11-24T06:26:40.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4dc19640 1 -- 172.21.15.16:0/3473653075 learned_addr learned my addr 172.21.15.16:0/3473653075 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:40.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.767+0000 7fee4d418640 1 --1- 172.21.15.16:0/3473653075 >> v1:172.21.15.173:6789/0 conn(0x7fee48143ae0 0x7fee480cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:60068/0 (socket says 172.21.15.16:60068) 2023-11-24T06:26:40.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.768+0000 7fee4cc17640 1 -- 172.21.15.16:0/3473653075 >> v2:172.21.15.16:3300/0 conn(0x7fee4815e1e0 msgr2=0x7fee480702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:40.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.768+0000 7fee4cc17640 1 --2- 172.21.15.16:0/3473653075 >> v2:172.21.15.16:3300/0 conn(0x7fee4815e1e0 0x7fee480702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.768+0000 7fee4cc17640 1 -- 172.21.15.16:0/3473653075 >> v1:172.21.15.173:6789/0 conn(0x7fee48143ae0 legacy=0x7fee480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.768+0000 7fee4cc17640 1 -- 172.21.15.16:0/3473653075 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee480b2130 con 0x7fee480cf9a0 2023-11-24T06:26:40.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.768+0000 7fee4dc19640 1 --2- 172.21.15.16:0/3473653075 >> v2:172.21.15.16:3300/0 conn(0x7fee4815e1e0 0x7fee480702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:26:40.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.768+0000 7fee3ffff640 1 -- 172.21.15.16:0/3473653075 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787160426 0 0) 0x7fee480b1ee0 con 0x7fee48143ae0 2023-11-24T06:26:40.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.769+0000 7fee4cc17640 1 --2- 172.21.15.16:0/3473653075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee480cf9a0 0x7fee4815dca0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee380097a0 tx=0x7fee3802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80eb56573137c9ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:40.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.769+0000 7fee3ffff640 1 -- 172.21.15.16:0/3473653075 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3802d040 con 0x7fee480cf9a0 2023-11-24T06:26:40.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.769+0000 7fee3ffff640 1 -- 172.21.15.16:0/3473653075 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee38031810 con 0x7fee480cf9a0 2023-11-24T06:26:40.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.769+0000 7fee3ffff640 1 -- 172.21.15.16:0/3473653075 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38031ba0 con 0x7fee480cf9a0 2023-11-24T06:26:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.770+0000 7fee4f67c640 1 -- 172.21.15.16:0/3473653075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee480cf9a0 msgr2=0x7fee4815dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.770+0000 7fee4f67c640 1 --2- 172.21.15.16:0/3473653075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee480cf9a0 0x7fee4815dca0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee380097a0 tx=0x7fee3802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.770+0000 7fee4f67c640 1 -- 172.21.15.16:0/3473653075 shutdown_connections 2023-11-24T06:26:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.770+0000 7fee4f67c640 1 --2- 172.21.15.16:0/3473653075 >> v2:172.21.15.16:3300/0 conn(0x7fee4815e1e0 0x7fee480702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.770+0000 7fee4f67c640 1 --2- 172.21.15.16:0/3473653075 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee480cf9a0 0x7fee4815dca0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.770+0000 7fee4f67c640 1 -- 172.21.15.16:0/3473653075 >> 172.21.15.16:0/3473653075 conn(0x7fee4806a800 msgr2=0x7fee4806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:40.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.770+0000 7fee4f67c640 1 -- 172.21.15.16:0/3473653075 shutdown_connections 2023-11-24T06:26:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.770+0000 7fee4f67c640 1 -- 172.21.15.16:0/3473653075 wait complete. 2023-11-24T06:26:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.772+0000 7fee4f67c640 1 Processor -- start 2023-11-24T06:26:40.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.772+0000 7fee4f67c640 1 -- start start 2023-11-24T06:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4f67c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee480cf9a0 0x7fee48158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4f67c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee48143ae0 0x7fee481373d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4f67c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee4815e1e0 0x7fee48137910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:40.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4f67c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480af420 con 0x7fee480cf9a0 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4f67c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48159380 con 0x7fee4815e1e0 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4f67c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee480cd520 con 0x7fee48143ae0 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4d418640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee480cf9a0 0x7fee48158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4d418640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee480cf9a0 0x7fee48158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59536/0 (socket says 172.21.15.16:59536) 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4d418640 1 -- 172.21.15.16:0/1899020525 learned_addr learned my addr 172.21.15.16:0/1899020525 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4cc17640 1 --2- 172.21.15.16:0/1899020525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee48143ae0 0x7fee481373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4dc19640 1 --2- 172.21.15.16:0/1899020525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee4815e1e0 0x7fee48137910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4d418640 1 -- 172.21.15.16:0/1899020525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee48143ae0 msgr2=0x7fee481373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4d418640 1 --2- 172.21.15.16:0/1899020525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee48143ae0 0x7fee481373d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4d418640 1 -- 172.21.15.16:0/1899020525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee4815e1e0 msgr2=0x7fee48137910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4d418640 1 --2- 172.21.15.16:0/1899020525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee4815e1e0 0x7fee48137910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.773+0000 7fee4d418640 1 -- 172.21.15.16:0/1899020525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee40009b30 con 0x7fee480cf9a0 2023-11-24T06:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.774+0000 7fee4d418640 1 --2- 172.21.15.16:0/1899020525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee480cf9a0 0x7fee48158e40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee4400c4d0 tx=0x7fee4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.774+0000 7fee3dffb640 1 -- 172.21.15.16:0/1899020525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee440074e0 con 0x7fee480cf9a0 2023-11-24T06:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.774+0000 7fee3dffb640 1 -- 172.21.15.16:0/1899020525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee44007660 con 0x7fee480cf9a0 2023-11-24T06:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.774+0000 7fee4f67c640 1 -- 172.21.15.16:0/1899020525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee38009400 con 0x7fee480cf9a0 2023-11-24T06:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.774+0000 7fee3dffb640 1 -- 172.21.15.16:0/1899020525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee44004a20 con 0x7fee480cf9a0 2023-11-24T06:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.774+0000 7fee4f67c640 1 -- 172.21.15.16:0/1899020525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee480cdae0 con 0x7fee480cf9a0 2023-11-24T06:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.774+0000 7fee3dffb640 1 -- 172.21.15.16:0/1899020525 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fee44012400 con 0x7fee480cf9a0 2023-11-24T06:26:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.774+0000 7fee3dffb640 1 -- 172.21.15.16:0/1899020525 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fee4401b360 con 0x7fee480cf9a0 2023-11-24T06:26:40.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.775+0000 7fee4f67c640 1 -- 172.21.15.16:0/1899020525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee480af590 con 0x7fee480cf9a0 2023-11-24T06:26:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.778+0000 7fee3dffb640 1 -- 172.21.15.16:0/1899020525 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fee480af590 con 0x7fee480cf9a0 2023-11-24T06:26:40.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.844+0000 7fee4f67c640 1 -- 172.21.15.16:0/1899020525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee480675d0 con 0x7fee480cf9a0 2023-11-24T06:26:40.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.845+0000 7fee3dffb640 1 -- 172.21.15.16:0/1899020525 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fee44018070 con 0x7fee480cf9a0 2023-11-24T06:26:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:26:40.848 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:40.447025+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:26:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.846+0000 7fee4f67c640 1 -- 172.21.15.16:0/1899020525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee480cf9a0 msgr2=0x7fee48158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.846+0000 7fee4f67c640 1 --2- 172.21.15.16:0/1899020525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee480cf9a0 0x7fee48158e40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee4400c4d0 tx=0x7fee4400c990 comp rx=0 tx=0).stop 2023-11-24T06:26:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.846+0000 7fee4f67c640 1 -- 172.21.15.16:0/1899020525 shutdown_connections 2023-11-24T06:26:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.846+0000 7fee4f67c640 1 --2- 172.21.15.16:0/1899020525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee4815e1e0 0x7fee48137910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.846+0000 7fee4f67c640 1 --2- 172.21.15.16:0/1899020525 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee48143ae0 0x7fee481373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.846+0000 7fee4f67c640 1 --2- 172.21.15.16:0/1899020525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee480cf9a0 0x7fee48158e40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:40.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.846+0000 7fee4f67c640 1 -- 172.21.15.16:0/1899020525 >> 172.21.15.16:0/1899020525 conn(0x7fee4806a800 msgr2=0x7fee48140320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:40.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.846+0000 7fee4f67c640 1 -- 172.21.15.16:0/1899020525 shutdown_connections 2023-11-24T06:26:40.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:40.846+0000 7fee4f67c640 1 -- 172.21.15.16:0/1899020525 wait complete. 2023-11-24T06:26:40.866 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T06:26:40.866 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T06:26:41.202 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-11-24T06:26:41.199+0000 7f582572b500 -1 Falling back to public interface 2023-11-24T06:26:41.217 INFO:tasks.ceph.osd.4.smithi173.stderr:2023-11-24T06:26:41.215+0000 7f08316fa500 -1 Falling back to public interface 2023-11-24T06:26:41.219 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-11-24T06:26:41.216+0000 7f7ba7018500 -1 Falling back to public interface 2023-11-24T06:26:41.219 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-11-24T06:26:41.217+0000 7f4d4421a500 -1 Falling back to public interface 2023-11-24T06:26:41.228 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-11-24T06:26:41.225+0000 7f79e011c500 -1 Falling back to public interface 2023-11-24T06:26:41.229 INFO:tasks.ceph.osd.6.smithi173.stderr:2023-11-24T06:26:41.226+0000 7f5a12a20500 -1 Falling back to public interface 2023-11-24T06:26:41.244 INFO:tasks.ceph.osd.5.smithi173.stderr:2023-11-24T06:26:41.241+0000 7efcd560d500 -1 Falling back to public interface 2023-11-24T06:26:41.251 INFO:tasks.ceph.osd.7.smithi173.stderr:2023-11-24T06:26:41.248+0000 7fa53e823500 -1 Falling back to public interface 2023-11-24T06:26:41.593 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:41.590+0000 7f25e8a13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:26:41.604 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:41.601+0000 7fd33683a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:26:41.672 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:26:41.851 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.848+0000 7f7d134cf640 1 Processor -- start 2023-11-24T06:26:41.851 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.848+0000 7f7d134cf640 1 -- start start 2023-11-24T06:26:41.853 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.849+0000 7f7d134cf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d0c0d2030 0x7f7d0c0d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:41.853 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.849+0000 7f7d134cf640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7d0c0d3720 0x7f7d0c0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:41.853 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.849+0000 7f7d134cf640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c0d3b50 con 0x7f7d0c160c80 2023-11-24T06:26:41.853 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.849+0000 7f7d134cf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d0c0b3230 con 0x7f7d0c0d2030 2023-11-24T06:26:41.853 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.849+0000 7f7d134cf640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d0c0ce860 con 0x7f7d0c0d3720 2023-11-24T06:26:41.853 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.849+0000 7f7d10a6a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d0c0d2030 0x7f7d0c0d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:41.853 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.849+0000 7f7d10a6a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d0c0d2030 0x7f7d0c0d31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59546/0 (socket says 172.21.15.16:59546) 2023-11-24T06:26:41.853 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.849+0000 7f7d10a6a640 1 -- 172.21.15.16:0/2020034224 learned_addr learned my addr 172.21.15.16:0/2020034224 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:41.853 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.849+0000 7f7d11a6c640 1 --2- 172.21.15.16:0/2020034224 >> v2:172.21.15.173:3300/0 conn(0x7f7d0c0d3720 0x7f7d0c0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:41.854 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.850+0000 7f7d10a6a640 1 -- 172.21.15.16:0/2020034224 >> v2:172.21.15.173:3300/0 conn(0x7f7d0c0d3720 msgr2=0x7f7d0c0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:41.854 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.850+0000 7f7d10a6a640 1 --2- 172.21.15.16:0/2020034224 >> v2:172.21.15.173:3300/0 conn(0x7f7d0c0d3720 0x7f7d0c0b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:41.854 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.850+0000 7f7d10a6a640 1 -- 172.21.15.16:0/2020034224 >> v1:172.21.15.173:6789/0 conn(0x7f7d0c160c80 legacy=0x7f7d0c0cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:41.854 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.850+0000 7f7d10a6a640 1 -- 172.21.15.16:0/2020034224 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c0ceb40 con 0x7f7d0c0d2030 2023-11-24T06:26:41.855 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.851+0000 7f7d10a6a640 1 --2- 172.21.15.16:0/2020034224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d2030 0x7f7d0c0d31e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0b3090 tx=0x7f7cfc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e37af5a9ad65d15 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:41.855 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.851+0000 7f7d03fff640 1 -- 172.21.15.16:0/2020034224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc030860 con 0x7f7d0c0d2030 2023-11-24T06:26:41.855 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.851+0000 7f7d03fff640 1 -- 172.21.15.16:0/2020034224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0309c0 con 0x7f7d0c0d2030 2023-11-24T06:26:41.855 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.851+0000 7f7d03fff640 1 -- 172.21.15.16:0/2020034224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc030d50 con 0x7f7d0c0d2030 2023-11-24T06:26:41.855 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.851+0000 7f7d134cf640 1 -- 172.21.15.16:0/2020034224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d2030 msgr2=0x7f7d0c0d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:41.855 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.852+0000 7f7d134cf640 1 --2- 172.21.15.16:0/2020034224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d2030 0x7f7d0c0d31e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0b3090 tx=0x7f7cfc02b9a0 comp rx=0 tx=0).stop 2023-11-24T06:26:41.855 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.852+0000 7f7d134cf640 1 -- 172.21.15.16:0/2020034224 shutdown_connections 2023-11-24T06:26:41.855 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.852+0000 7f7d134cf640 1 --2- 172.21.15.16:0/2020034224 >> v2:172.21.15.173:3300/0 conn(0x7f7d0c0d3720 0x7f7d0c0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:41.855 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.852+0000 7f7d134cf640 1 --2- 172.21.15.16:0/2020034224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d2030 0x7f7d0c0d31e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:41.856 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.852+0000 7f7d134cf640 1 -- 172.21.15.16:0/2020034224 >> 172.21.15.16:0/2020034224 conn(0x7f7d0c08aff0 msgr2=0x7f7d0c09c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:41.856 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.852+0000 7f7d134cf640 1 -- 172.21.15.16:0/2020034224 shutdown_connections 2023-11-24T06:26:41.856 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.852+0000 7f7d134cf640 1 -- 172.21.15.16:0/2020034224 wait complete. 2023-11-24T06:26:41.856 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.853+0000 7f7d134cf640 1 Processor -- start 2023-11-24T06:26:41.856 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.853+0000 7f7d134cf640 1 -- start start 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.854+0000 7f7d134cf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d0c0d2030 0x7f7d0c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.854+0000 7f7d134cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d3720 0x7f7d0c0c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.854+0000 7f7d134cf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d0c160c80 0x7f7d0c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.854+0000 7f7d134cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0cbdd0 con 0x7f7d0c0d3720 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.854+0000 7f7d134cf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0cb150 con 0x7f7d0c0d2030 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.854+0000 7f7d134cf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0cb430 con 0x7f7d0c160c80 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d10a6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d3720 0x7f7d0c0c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d10a6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d3720 0x7f7d0c0c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59562/0 (socket says 172.21.15.16:59562) 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d10a6a640 1 -- 172.21.15.16:0/15046264 learned_addr learned my addr 172.21.15.16:0/15046264 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:41.860 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d1126b640 1 --2- 172.21.15.16:0/15046264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d0c0d2030 0x7f7d0c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:41.861 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d11a6c640 1 --2- 172.21.15.16:0/15046264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d0c160c80 0x7f7d0c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:41.861 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d10a6a640 1 -- 172.21.15.16:0/15046264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d0c160c80 msgr2=0x7f7d0c1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:41.861 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d10a6a640 1 --2- 172.21.15.16:0/15046264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d0c160c80 0x7f7d0c1555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:41.861 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d10a6a640 1 -- 172.21.15.16:0/15046264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d0c0d2030 msgr2=0x7f7d0c0cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:41.861 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d10a6a640 1 --2- 172.21.15.16:0/15046264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d0c0d2030 0x7f7d0c0cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:41.861 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.855+0000 7f7d10a6a640 1 -- 172.21.15.16:0/15046264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0094d0 con 0x7f7d0c0d3720 2023-11-24T06:26:41.861 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.856+0000 7f7d10a6a640 1 --2- 172.21.15.16:0/15046264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d3720 0x7f7d0c0c9750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc02b9e0 tx=0x7f7cfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:41.861 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.856+0000 7f7d01ffb640 1 -- 172.21.15.16:0/15046264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc039c90 con 0x7f7d0c0d3720 2023-11-24T06:26:41.861 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.856+0000 7f7d01ffb640 1 -- 172.21.15.16:0/15046264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc039e10 con 0x7f7d0c0d3720 2023-11-24T06:26:41.862 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.856+0000 7f7d134cf640 1 -- 172.21.15.16:0/15046264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c0cb710 con 0x7f7d0c0d3720 2023-11-24T06:26:41.862 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.856+0000 7f7d01ffb640 1 -- 172.21.15.16:0/15046264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc043640 con 0x7f7d0c0d3720 2023-11-24T06:26:41.862 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.856+0000 7f7d134cf640 1 -- 172.21.15.16:0/15046264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c0c9c90 con 0x7f7d0c0d3720 2023-11-24T06:26:41.862 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.856+0000 7f7d01ffb640 1 -- 172.21.15.16:0/15046264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7cfc043aa0 con 0x7f7d0c0d3720 2023-11-24T06:26:41.862 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.856+0000 7f7d01ffb640 1 -- 172.21.15.16:0/15046264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f7cfc03e070 con 0x7f7d0c0d3720 2023-11-24T06:26:41.862 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.857+0000 7f7d134cf640 1 -- 172.21.15.16:0/15046264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c0cbf10 con 0x7f7d0c0d3720 2023-11-24T06:26:41.862 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.860+0000 7f7d01ffb640 1 -- 172.21.15.16:0/15046264 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f7d0c0cbf10 con 0x7f7d0c0d3720 2023-11-24T06:26:41.909 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:41.906+0000 7fd33683a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:26:41.917 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:41.914+0000 7f25e8a13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:26:41.929 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.926+0000 7f7d134cf640 1 -- 172.21.15.16:0/15046264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d0c0675d0 con 0x7f7d0c0d3720 2023-11-24T06:26:41.930 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.927+0000 7f7d01ffb640 1 -- 172.21.15.16:0/15046264 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f7cfc034070 con 0x7f7d0c0d3720 2023-11-24T06:26:41.931 INFO:teuthology.misc.health.smithi016.stdout: 2023-11-24T06:26:41.931 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:40.447025+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:26:41.932 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.929+0000 7f7d134cf640 1 -- 172.21.15.16:0/15046264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d3720 msgr2=0x7f7d0c0c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:41.932 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.929+0000 7f7d134cf640 1 --2- 172.21.15.16:0/15046264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d3720 0x7f7d0c0c9750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc02b9e0 tx=0x7f7cfc004730 comp rx=0 tx=0).stop 2023-11-24T06:26:41.933 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.929+0000 7f7d134cf640 1 -- 172.21.15.16:0/15046264 shutdown_connections 2023-11-24T06:26:41.933 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.929+0000 7f7d134cf640 1 --2- 172.21.15.16:0/15046264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d0c160c80 0x7f7d0c1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:41.933 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.929+0000 7f7d134cf640 1 --2- 172.21.15.16:0/15046264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0c0d3720 0x7f7d0c0c9750 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:41.933 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.929+0000 7f7d134cf640 1 --2- 172.21.15.16:0/15046264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d0c0d2030 0x7f7d0c0cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:41.933 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.929+0000 7f7d134cf640 1 -- 172.21.15.16:0/15046264 >> 172.21.15.16:0/15046264 conn(0x7f7d0c08aff0 msgr2=0x7f7d0c0b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:41.933 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.929+0000 7f7d134cf640 1 -- 172.21.15.16:0/15046264 shutdown_connections 2023-11-24T06:26:41.933 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:41.929+0000 7f7d134cf640 1 -- 172.21.15.16:0/15046264 wait complete. 2023-11-24T06:26:41.945 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T06:26:41.969 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:41.967+0000 7fd33683a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:26:41.982 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:41.979+0000 7f25e8a13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:26:42.084 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:42.081+0000 7fd33683a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:26:42.103 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-11-24T06:26:42.100+0000 7f25e8a13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:26:43.125 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:43.122+0000 7fd2d7bc2640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T06:26:43.125 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:26:43.123+0000 7fd2d7bc2640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T06:26:44.571 INFO:tasks.ceph.osd.7.smithi173.stderr:2023-11-24T06:26:44.569+0000 7fa53e823500 -1 osd.7 0 log_to_monitors true 2023-11-24T06:26:44.764 INFO:tasks.ceph.osd.4.smithi173.stderr:2023-11-24T06:26:44.762+0000 7f08316fa500 -1 osd.4 0 log_to_monitors true 2023-11-24T06:26:44.769 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-11-24T06:26:44.766+0000 7f582572b500 -1 osd.0 0 log_to_monitors true 2023-11-24T06:26:44.807 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-11-24T06:26:44.804+0000 7f4d4421a500 -1 osd.1 0 log_to_monitors true 2023-11-24T06:26:44.821 INFO:tasks.ceph.osd.6.smithi173.stderr:2023-11-24T06:26:44.819+0000 7f5a12a20500 -1 osd.6 0 log_to_monitors true 2023-11-24T06:26:44.824 INFO:tasks.ceph.osd.5.smithi173.stderr:2023-11-24T06:26:44.821+0000 7efcd560d500 -1 osd.5 0 log_to_monitors true 2023-11-24T06:26:44.832 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-11-24T06:26:44.829+0000 7f7ba7018500 -1 osd.2 0 log_to_monitors true 2023-11-24T06:26:44.836 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-11-24T06:26:44.833+0000 7f79e011c500 -1 osd.3 0 log_to_monitors true 2023-11-24T06:26:47.923 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-11-24T06:26:47.921+0000 7f7ba21e2640 -1 osd.2 0 waiting for initial osdmap 2023-11-24T06:26:47.931 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-11-24T06:26:47.929+0000 7f7b9a81b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:26:47.940 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-11-24T06:26:47.938+0000 7f58226ce640 -1 osd.0 0 waiting for initial osdmap 2023-11-24T06:26:47.946 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-11-24T06:26:47.944+0000 7f58194cf640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:26:47.947 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-11-24T06:26:47.945+0000 7f79db887640 -1 osd.3 0 waiting for initial osdmap 2023-11-24T06:26:47.966 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-11-24T06:26:47.964+0000 7f79d3ec0640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:26:47.977 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-11-24T06:26:47.975+0000 7f4d40c92640 -1 osd.1 0 waiting for initial osdmap 2023-11-24T06:26:47.993 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-11-24T06:26:47.991+0000 7f4d37a93640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:26:48.504 INFO:tasks.ceph.osd.6.smithi173.stderr:2023-11-24T06:26:48.502+0000 7f5a0dbcd640 -1 osd.6 0 waiting for initial osdmap 2023-11-24T06:26:48.512 INFO:tasks.ceph.osd.6.smithi173.stderr:2023-11-24T06:26:48.509+0000 7f5a06206640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:26:48.531 INFO:tasks.ceph.osd.5.smithi173.stderr:2023-11-24T06:26:48.529+0000 7efcd0677640 -1 osd.5 0 waiting for initial osdmap 2023-11-24T06:26:48.539 INFO:tasks.ceph.osd.5.smithi173.stderr:2023-11-24T06:26:48.537+0000 7efcc8cb0640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:26:48.543 INFO:tasks.ceph.osd.4.smithi173.stderr:2023-11-24T06:26:48.541+0000 7f082ce67640 -1 osd.4 0 waiting for initial osdmap 2023-11-24T06:26:48.551 INFO:tasks.ceph.osd.4.smithi173.stderr:2023-11-24T06:26:48.549+0000 7f08254a0640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:26:48.554 INFO:tasks.ceph.osd.7.smithi173.stderr:2023-11-24T06:26:48.552+0000 7fa5398d1640 -1 osd.7 0 waiting for initial osdmap 2023-11-24T06:26:48.566 INFO:tasks.ceph.osd.7.smithi173.stderr:2023-11-24T06:26:48.563+0000 7fa531f0a640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:26:48.751 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:26:48.948 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.946+0000 7f2038ada640 1 Processor -- start 2023-11-24T06:26:48.949 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.946+0000 7f2038ada640 1 -- start start 2023-11-24T06:26:48.950 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.948+0000 7f2038ada640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f203415d5d0 0x7f20340d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.950 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.948+0000 7f2038ada640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340d1c60 0x7f20340b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.951 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.948+0000 7f2038ada640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20340afd50 con 0x7f20340b21a0 2023-11-24T06:26:48.951 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.948+0000 7f2038ada640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20340d11e0 con 0x7f203415d5d0 2023-11-24T06:26:48.951 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.948+0000 7f2038ada640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20340cef90 con 0x7f20340d1c60 2023-11-24T06:26:48.952 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.950+0000 7f20337fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f203415d5d0 0x7f20340d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.952 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.950+0000 7f20337fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f203415d5d0 0x7f20340d3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:57070/0 (socket says 172.21.15.16:57070) 2023-11-24T06:26:48.952 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.950+0000 7f20337fe640 1 -- 172.21.15.16:0/1247694917 learned_addr learned my addr 172.21.15.16:0/1247694917 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:48.953 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.951+0000 7f20337fe640 1 -- 172.21.15.16:0/1247694917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340d1c60 msgr2=0x7f20340b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:48.953 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.951+0000 7f20337fe640 1 --2- 172.21.15.16:0/1247694917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340d1c60 0x7f20340b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.953 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.951+0000 7f20337fe640 1 -- 172.21.15.16:0/1247694917 >> v1:172.21.15.173:6789/0 conn(0x7f20340b21a0 legacy=0x7f20340d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:48.953 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.951+0000 7f20337fe640 1 -- 172.21.15.16:0/1247694917 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340cf270 con 0x7f203415d5d0 2023-11-24T06:26:48.954 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.953+0000 7f20337fe640 1 --2- 172.21.15.16:0/1247694917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203415d5d0 0x7f20340d3b80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2028009870 tx=0x7f202802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb83384439d29f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:48.957 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.955+0000 7f20327fc640 1 -- 172.21.15.16:0/1247694917 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028031640 con 0x7f203415d5d0 2023-11-24T06:26:48.957 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.955+0000 7f20327fc640 1 -- 172.21.15.16:0/1247694917 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20280317a0 con 0x7f203415d5d0 2023-11-24T06:26:48.957 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.955+0000 7f20327fc640 1 -- 172.21.15.16:0/1247694917 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028031b30 con 0x7f203415d5d0 2023-11-24T06:26:48.958 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.956+0000 7f2038ada640 1 -- 172.21.15.16:0/1247694917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203415d5d0 msgr2=0x7f20340d3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:48.958 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.956+0000 7f2038ada640 1 --2- 172.21.15.16:0/1247694917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203415d5d0 0x7f20340d3b80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2028009870 tx=0x7f202802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.959 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.956+0000 7f2038ada640 1 -- 172.21.15.16:0/1247694917 shutdown_connections 2023-11-24T06:26:48.959 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.956+0000 7f2038ada640 1 --2- 172.21.15.16:0/1247694917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340d1c60 0x7f20340b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.959 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.956+0000 7f2038ada640 1 --2- 172.21.15.16:0/1247694917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f203415d5d0 0x7f20340d3b80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.959 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.956+0000 7f2038ada640 1 -- 172.21.15.16:0/1247694917 >> 172.21.15.16:0/1247694917 conn(0x7f20340d3090 msgr2=0x7f20340d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:48.959 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.956+0000 7f2038ada640 1 -- 172.21.15.16:0/1247694917 shutdown_connections 2023-11-24T06:26:48.959 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.956+0000 7f2038ada640 1 -- 172.21.15.16:0/1247694917 wait complete. 2023-11-24T06:26:48.961 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.959+0000 7f2038ada640 1 Processor -- start 2023-11-24T06:26:48.961 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.959+0000 7f2038ada640 1 -- start start 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.960+0000 7f2038ada640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340b21a0 0x7f20341f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.960+0000 7f2038ada640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340d1c60 0x7f203415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.960+0000 7f2038ada640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203415d5d0 0x7f20341513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.960+0000 7f2038ada640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20341f3cc0 con 0x7f20340b21a0 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.960+0000 7f2038ada640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2034160410 con 0x7f20340d1c60 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.960+0000 7f2038ada640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203415ee30 con 0x7f203415d5d0 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.960+0000 7f20337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340b21a0 0x7f20341f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.960+0000 7f20337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340b21a0 0x7f20341f3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39798/0 (socket says 172.21.15.16:39798) 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.960+0000 7f20337fe640 1 -- 172.21.15.16:0/2587367800 learned_addr learned my addr 172.21.15.16:0/2587367800 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:48.963 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.961+0000 7f2032ffd640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340d1c60 0x7f203415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.964 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.961+0000 7f2033fff640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203415d5d0 0x7f20341513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.964 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.961+0000 7f20337fe640 1 -- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203415d5d0 msgr2=0x7f20341513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.961+0000 7f20337fe640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203415d5d0 0x7f20341513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.961+0000 7f20337fe640 1 -- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340d1c60 msgr2=0x7f203415fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.961+0000 7f20337fe640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340d1c60 0x7f203415fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.961+0000 7f20337fe640 1 -- 172.21.15.16:0/2587367800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20280094d0 con 0x7f20340b21a0 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.961+0000 7f20337fe640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340b21a0 0x7f20341f3780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f20280090b0 tx=0x7f20280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.962+0000 7f201bfff640 1 -- 172.21.15.16:0/2587367800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028043d20 con 0x7f20340b21a0 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.962+0000 7f201bfff640 1 -- 172.21.15.16:0/2587367800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2028047070 con 0x7f20340b21a0 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.962+0000 7f201bfff640 1 -- 172.21.15.16:0/2587367800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202803a910 con 0x7f20340b21a0 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.962+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203415f110 con 0x7f20340b21a0 2023-11-24T06:26:48.965 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.962+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203415f6d0 con 0x7f20340b21a0 2023-11-24T06:26:48.966 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.963+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20340b1fc0 con 0x7f20340b21a0 2023-11-24T06:26:48.966 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.964+0000 7f201bfff640 1 -- 172.21.15.16:0/2587367800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f202802f050 con 0x7f20340b21a0 2023-11-24T06:26:48.968 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.965+0000 7f201bfff640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f202006adb0 0x7f202006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:48.968 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.965+0000 7f201bfff640 1 -- 172.21.15.16:0/2587367800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f20280b3350 con 0x7f20340b21a0 2023-11-24T06:26:48.968 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.966+0000 7f2032ffd640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f202006adb0 0x7f202006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:48.968 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.966+0000 7f2032ffd640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f202006adb0 0x7f202006d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2024007fd0 tx=0x7f2024009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:48.972 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:48.971+0000 7f201bfff640 1 -- 172.21.15.16:0/2587367800 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f202807e9a0 con 0x7f20340b21a0 2023-11-24T06:26:49.161 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.159+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2034067450 con 0x7f20340b21a0 2023-11-24T06:26:49.163 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.161+0000 7f201bfff640 1 -- 172.21.15.16:0/2587367800 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10908 (secure 0 0 0) 0x7f2028081fc0 con 0x7f20340b21a0 2023-11-24T06:26:49.164 INFO:teuthology.misc.health.smithi016.stdout: 2023-11-24T06:26:49.164 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":13,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:48.927403+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:26:49.167 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.164+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f202006adb0 msgr2=0x7f202006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.168 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.164+0000 7f2038ada640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f202006adb0 0x7f202006d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2024007fd0 tx=0x7f2024009040 comp rx=0 tx=0).stop 2023-11-24T06:26:49.168 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340b21a0 msgr2=0x7f20341f3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.168 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340b21a0 0x7f20341f3780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f20280090b0 tx=0x7f20280046c0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.168 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 shutdown_connections 2023-11-24T06:26:49.168 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f202006adb0 0x7f202006d270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.168 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f203415d5d0 0x7f20341513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.168 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340d1c60 0x7f203415fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.168 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 --2- 172.21.15.16:0/2587367800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340b21a0 0x7f20341f3780 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.169 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 >> 172.21.15.16:0/2587367800 conn(0x7f20340d3090 msgr2=0x7f20340d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:49.169 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 shutdown_connections 2023-11-24T06:26:49.169 INFO:teuthology.misc.health.smithi016.stderr:2023-11-24T06:26:49.165+0000 7f2038ada640 1 -- 172.21.15.16:0/2587367800 wait complete. 2023-11-24T06:26:49.182 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T06:26:49.182 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T06:26:49.182 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:26:49.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:26:49.190 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:26:49.191 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:26:49.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:26:49.199 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:26:49.200 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:26:49.200 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:26:49.208 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:26:49.209 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:26:49.209 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:26:49.217 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:26:49.217 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T06:26:49.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:26:49.342 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-24T06:26:49.343 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:26:49.363 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2023-11-24T06:26:49.372 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2023-11-24T06:26:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.395+0000 7fd3520c9640 1 Processor -- start 2023-11-24T06:26:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.395+0000 7fd3520c9640 1 -- start start 2023-11-24T06:26:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd3520c9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c160f80 0x7fd34c161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd3520c9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd34c0ce8f0 0x7fd34c0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd3520c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd34c0cfbe0 0x7fd34c0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd3520c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34c1618a0 con 0x7fd34c0cfbe0 2023-11-24T06:26:49.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd3520c9640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34c0d0de0 con 0x7fd34c0ce8f0 2023-11-24T06:26:49.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd3520c9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c0d10c0 con 0x7fd34c160f80 2023-11-24T06:26:49.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd34bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd34c0cfbe0 0x7fd34c0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd34bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd34c0cfbe0 0x7fd34c0d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39822/0 (socket says 172.21.15.16:39822) 2023-11-24T06:26:49.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd343fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd34c0ce8f0 0x7fd34c0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd34bfff640 1 -- 172.21.15.16:0/3671034123 learned_addr learned my addr 172.21.15.16:0/3671034123 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:49.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.396+0000 7fd34b7fe640 1 --2- 172.21.15.16:0/3671034123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c160f80 0x7fd34c161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.397+0000 7fd343fff640 1 -- 172.21.15.16:0/3671034123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c160f80 msgr2=0x7fd34c161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.397+0000 7fd343fff640 1 --2- 172.21.15.16:0/3671034123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c160f80 0x7fd34c161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.397+0000 7fd343fff640 1 -- 172.21.15.16:0/3671034123 >> v2:172.21.15.16:3300/0 conn(0x7fd34c0cfbe0 msgr2=0x7fd34c0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.397+0000 7fd343fff640 1 --2- 172.21.15.16:0/3671034123 >> v2:172.21.15.16:3300/0 conn(0x7fd34c0cfbe0 0x7fd34c0d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.397+0000 7fd343fff640 1 -- 172.21.15.16:0/3671034123 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c0d13a0 con 0x7fd34c0ce8f0 2023-11-24T06:26:49.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.398+0000 7fd343fff640 1 --2- 172.21.15.16:0/3671034123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c0ce8f0 0x7fd34c0ced50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd34c08b520 tx=0x7fd33402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=799006c7d71ee4f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:49.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.398+0000 7fd34affd640 1 -- 172.21.15.16:0/3671034123 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33402e070 con 0x7fd34c0ce8f0 2023-11-24T06:26:49.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.398+0000 7fd34affd640 1 -- 172.21.15.16:0/3671034123 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3340315a0 con 0x7fd34c0ce8f0 2023-11-24T06:26:49.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.398+0000 7fd34affd640 1 -- 172.21.15.16:0/3671034123 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334031930 con 0x7fd34c0ce8f0 2023-11-24T06:26:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.398+0000 7fd3520c9640 1 -- 172.21.15.16:0/3671034123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c0ce8f0 msgr2=0x7fd34c0ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.398+0000 7fd3520c9640 1 --2- 172.21.15.16:0/3671034123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c0ce8f0 0x7fd34c0ced50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd34c08b520 tx=0x7fd33402b840 comp rx=0 tx=0).stop 2023-11-24T06:26:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.399+0000 7fd3520c9640 1 -- 172.21.15.16:0/3671034123 shutdown_connections 2023-11-24T06:26:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.399+0000 7fd3520c9640 1 --2- 172.21.15.16:0/3671034123 >> v2:172.21.15.16:3300/0 conn(0x7fd34c0cfbe0 0x7fd34c0d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.399+0000 7fd3520c9640 1 --2- 172.21.15.16:0/3671034123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c0ce8f0 0x7fd34c0ced50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.399+0000 7fd3520c9640 1 --2- 172.21.15.16:0/3671034123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c160f80 0x7fd34c161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.399+0000 7fd3520c9640 1 -- 172.21.15.16:0/3671034123 >> 172.21.15.16:0/3671034123 conn(0x7fd34c09c550 msgr2=0x7fd34c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.399+0000 7fd3520c9640 1 -- 172.21.15.16:0/3671034123 shutdown_connections 2023-11-24T06:26:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.399+0000 7fd3520c9640 1 -- 172.21.15.16:0/3671034123 wait complete. 2023-11-24T06:26:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.400+0000 7fd3520c9640 1 Processor -- start 2023-11-24T06:26:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.400+0000 7fd3520c9640 1 -- start start 2023-11-24T06:26:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.401+0000 7fd3520c9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c0ce8f0 0x7fd34c160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.401+0000 7fd3520c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34c0cfbe0 0x7fd34c152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.401+0000 7fd3520c9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c160f80 0x7fd34c1527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.401+0000 7fd3520c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c152cf0 con 0x7fd34c0cfbe0 2023-11-24T06:26:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.401+0000 7fd3520c9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c0c9020 con 0x7fd34c160f80 2023-11-24T06:26:49.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.401+0000 7fd3520c9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c0c9300 con 0x7fd34c0ce8f0 2023-11-24T06:26:49.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd343fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34c0cfbe0 0x7fd34c152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd343fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34c0cfbe0 0x7fd34c152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39828/0 (socket says 172.21.15.16:39828) 2023-11-24T06:26:49.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd343fff640 1 -- 172.21.15.16:0/2118960119 learned_addr learned my addr 172.21.15.16:0/2118960119 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:49.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd34bfff640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c160f80 0x7fd34c1527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd34b7fe640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c0ce8f0 0x7fd34c160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd34bfff640 1 -- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c0ce8f0 msgr2=0x7fd34c160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd34bfff640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c0ce8f0 0x7fd34c160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd34bfff640 1 -- 172.21.15.16:0/2118960119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34c0cfbe0 msgr2=0x7fd34c152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd34bfff640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34c0cfbe0 0x7fd34c152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd34bfff640 1 -- 172.21.15.16:0/2118960119 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd334009400 con 0x7fd34c160f80 2023-11-24T06:26:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.402+0000 7fd34bfff640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c160f80 0x7fd34c1527b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd34400eed0 tx=0x7fd34400b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.403+0000 7fd348ff9640 1 -- 172.21.15.16:0/2118960119 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344010040 con 0x7fd34c160f80 2023-11-24T06:26:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.403+0000 7fd348ff9640 1 -- 172.21.15.16:0/2118960119 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34400bd10 con 0x7fd34c160f80 2023-11-24T06:26:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.403+0000 7fd348ff9640 1 -- 172.21.15.16:0/2118960119 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344003bd0 con 0x7fd34c160f80 2023-11-24T06:26:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.403+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34c0c95e0 con 0x7fd34c160f80 2023-11-24T06:26:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.403+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c0c98c0 con 0x7fd34c160f80 2023-11-24T06:26:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.404+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34c15e660 con 0x7fd34c160f80 2023-11-24T06:26:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.405+0000 7fd348ff9640 1 -- 172.21.15.16:0/2118960119 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fd344003d30 con 0x7fd34c160f80 2023-11-24T06:26:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.406+0000 7fd348ff9640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd32c06adb0 0x7fd32c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.406+0000 7fd348ff9640 1 -- 172.21.15.16:0/2118960119 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fd3440892c0 con 0x7fd34c160f80 2023-11-24T06:26:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.406+0000 7fd34b7fe640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd32c06adb0 0x7fd32c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.407+0000 7fd34b7fe640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd32c06adb0 0x7fd32c06d270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009aa0 tx=0x7fd33c009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.413+0000 7fd348ff9640 1 -- 172.21.15.16:0/2118960119 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3440549c0 con 0x7fd34c160f80 2023-11-24T06:26:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.599+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd34c15ea40 con 0x7fd34c160f80 2023-11-24T06:26:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.600+0000 7fd348ff9640 1 -- 172.21.15.16:0/2118960119 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10908 (secure 0 0 0) 0x7fd34c15ea40 con 0x7fd34c160f80 2023-11-24T06:26:49.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:26:49.603 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:48.927403+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:26:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd32c06adb0 msgr2=0x7fd32c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd32c06adb0 0x7fd32c06d270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009aa0 tx=0x7fd33c009190 comp rx=0 tx=0).stop 2023-11-24T06:26:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c160f80 msgr2=0x7fd34c1527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c160f80 0x7fd34c1527b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd34400eed0 tx=0x7fd34400b480 comp rx=0 tx=0).stop 2023-11-24T06:26:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 shutdown_connections 2023-11-24T06:26:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd32c06adb0 0x7fd32c06d270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd34c160f80 0x7fd34c1527b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34c0cfbe0 0x7fd34c152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 --2- 172.21.15.16:0/2118960119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd34c0ce8f0 0x7fd34c160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 >> 172.21.15.16:0/2118960119 conn(0x7fd34c09c550 msgr2=0x7fd34c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 shutdown_connections 2023-11-24T06:26:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.603+0000 7fd3520c9640 1 -- 172.21.15.16:0/2118960119 wait complete. 2023-11-24T06:26:49.622 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-11-24T06:26:49.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:49.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.835+0000 7f2ee1659640 1 Processor -- start 2023-11-24T06:26:49.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.835+0000 7f2ee1659640 1 -- start start 2023-11-24T06:26:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.836+0000 7f2ee1659640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2edc160920 0x7f2edc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2ee1659640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc0d0a10 con 0x7f2edc0cf9f0 2023-11-24T06:26:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2ee1659640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc161e30 con 0x7f2edc0cfea0 2023-11-24T06:26:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2ee1659640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2edc0d1f50 con 0x7f2edc160920 2023-11-24T06:26:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2edbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2edc160920 0x7f2edc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2edb7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2edc0cf9f0 0x7f2edc161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44418/0 (socket says 172.21.15.16:44418) 2023-11-24T06:26:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2edb7fe640 1 -- 172.21.15.16:0/1192521837 learned_addr learned my addr 172.21.15.16:0/1192521837 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:26:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2edbfff640 1 -- 172.21.15.16:0/1192521837 >> v1:172.21.15.173:6789/0 conn(0x7f2edc0cfea0 legacy=0x7f2edc161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2edbfff640 1 -- 172.21.15.16:0/1192521837 >> v1:172.21.15.16:6789/0 conn(0x7f2edc0cf9f0 legacy=0x7f2edc161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2edbfff640 1 -- 172.21.15.16:0/1192521837 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc0cf080 con 0x7f2edc160920 2023-11-24T06:26:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.837+0000 7f2edaffd640 1 -- 172.21.15.16:0/1192521837 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205329198 0 0) 0x7f2edc0d0a10 con 0x7f2edc0cf9f0 2023-11-24T06:26:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.839+0000 7f2edbfff640 1 --2- 172.21.15.16:0/1192521837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc160920 0x7f2edc0b3f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0097a0 tx=0x7f2ecc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbaee9e3624d6e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.839+0000 7f2edaffd640 1 -- 172.21.15.16:0/1192521837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc031830 con 0x7f2edc160920 2023-11-24T06:26:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.839+0000 7f2edaffd640 1 -- 172.21.15.16:0/1192521837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc031990 con 0x7f2edc160920 2023-11-24T06:26:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.839+0000 7f2edaffd640 1 -- 172.21.15.16:0/1192521837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc031d20 con 0x7f2edc160920 2023-11-24T06:26:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.840+0000 7f2ee1659640 1 -- 172.21.15.16:0/1192521837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc160920 msgr2=0x7f2edc0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.840+0000 7f2ee1659640 1 --2- 172.21.15.16:0/1192521837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc160920 0x7f2edc0b3f30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0097a0 tx=0x7f2ecc02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.840+0000 7f2ee1659640 1 -- 172.21.15.16:0/1192521837 shutdown_connections 2023-11-24T06:26:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.840+0000 7f2ee1659640 1 --2- 172.21.15.16:0/1192521837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc160920 0x7f2edc0b3f30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.840+0000 7f2ee1659640 1 -- 172.21.15.16:0/1192521837 >> 172.21.15.16:0/1192521837 conn(0x7f2edc0d3090 msgr2=0x7f2edc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.840+0000 7f2ee1659640 1 -- 172.21.15.16:0/1192521837 shutdown_connections 2023-11-24T06:26:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.840+0000 7f2ee1659640 1 -- 172.21.15.16:0/1192521837 wait complete. 2023-11-24T06:26:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.842+0000 7f2ee1659640 1 Processor -- start 2023-11-24T06:26:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.842+0000 7f2ee1659640 1 -- start start 2023-11-24T06:26:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.843+0000 7f2ee1659640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2edc0cf9f0 0x7f2edc0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.843+0000 7f2ee1659640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc0cfea0 0x7f2edc0cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.843+0000 7f2ee1659640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2edc160920 0x7f2edc15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.843+0000 7f2ee1659640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0ca160 con 0x7f2edc0cfea0 2023-11-24T06:26:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.843+0000 7f2ee1659640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc0caf80 con 0x7f2edc160920 2023-11-24T06:26:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.843+0000 7f2ee1659640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc15f510 con 0x7f2edc0cf9f0 2023-11-24T06:26:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.843+0000 7f2edb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc0cfea0 0x7f2edc0cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.843+0000 7f2edb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc0cfea0 0x7f2edc0cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39840/0 (socket says 172.21.15.16:39840) 2023-11-24T06:26:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.843+0000 7f2edb7fe640 1 -- 172.21.15.16:0/2250839271 learned_addr learned my addr 172.21.15.16:0/2250839271 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2ee0e58640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2edc160920 0x7f2edc15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2edbfff640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2edc0cf9f0 0x7f2edc0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2edb7fe640 1 -- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2edc0cf9f0 msgr2=0x7f2edc0b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2edb7fe640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2edc0cf9f0 0x7f2edc0b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2edb7fe640 1 -- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2edc160920 msgr2=0x7f2edc15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2edb7fe640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2edc160920 0x7f2edc15cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2edb7fe640 1 -- 172.21.15.16:0/2250839271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc009400 con 0x7f2edc0cfea0 2023-11-24T06:26:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2ee0e58640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2edc160920 0x7f2edc15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2edb7fe640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc0cfea0 0x7f2edc0cb410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c430 tx=0x7f2ed000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2ed8ff9640 1 -- 172.21.15.16:0/2250839271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00074e0 con 0x7f2edc0cfea0 2023-11-24T06:26:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2ed8ff9640 1 -- 172.21.15.16:0/2250839271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed0007660 con 0x7f2edc0cfea0 2023-11-24T06:26:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.844+0000 7f2ed8ff9640 1 -- 172.21.15.16:0/2250839271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0004a20 con 0x7f2edc0cfea0 2023-11-24T06:26:49.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.845+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc0ca370 con 0x7f2edc0cfea0 2023-11-24T06:26:49.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.845+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc0ca930 con 0x7f2edc0cfea0 2023-11-24T06:26:49.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.846+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc067f30 con 0x7f2edc0cfea0 2023-11-24T06:26:49.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.847+0000 7f2ed8ff9640 1 -- 172.21.15.16:0/2250839271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f2ed0004b80 con 0x7f2edc0cfea0 2023-11-24T06:26:49.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.848+0000 7f2ed8ff9640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2ec806adb0 0x7f2ec806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:49.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.848+0000 7f2ed8ff9640 1 -- 172.21.15.16:0/2250839271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f2ed0088e70 con 0x7f2edc0cfea0 2023-11-24T06:26:49.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.848+0000 7f2edbfff640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2ec806adb0 0x7f2ec806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:49.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.850+0000 7f2edbfff640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2ec806adb0 0x7f2ec806d270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc036000 tx=0x7f2ecc03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:49.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:49.853+0000 7f2ed8ff9640 1 -- 172.21.15.16:0/2250839271 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ed00544c0 con 0x7f2edc0cfea0 2023-11-24T06:26:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.085+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2edc0cf5c0 con 0x7f2edc0cfea0 2023-11-24T06:26:50.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.085+0000 7f2ed8ff9640 1 -- 172.21.15.16:0/2250839271 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f2ed0057ae0 con 0x7f2edc0cfea0 2023-11-24T06:26:50.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:26:50.088 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-24T06:26:50.088 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2023-11-24T06:26:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.087+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2ec806adb0 msgr2=0x7f2ec806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.087+0000 7f2ee1659640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2ec806adb0 0x7f2ec806d270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc036000 tx=0x7f2ecc03c040 comp rx=0 tx=0).stop 2023-11-24T06:26:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.087+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc0cfea0 msgr2=0x7f2edc0cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.087+0000 7f2ee1659640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc0cfea0 0x7f2edc0cb410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000c430 tx=0x7f2ed000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:26:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.088+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 shutdown_connections 2023-11-24T06:26:50.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.088+0000 7f2ee1659640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2ec806adb0 0x7f2ec806d270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:50.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.088+0000 7f2ee1659640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2edc160920 0x7f2edc15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:50.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.088+0000 7f2ee1659640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2edc0cfea0 0x7f2edc0cb410 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:50.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.088+0000 7f2ee1659640 1 --2- 172.21.15.16:0/2250839271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2edc0cf9f0 0x7f2edc0b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:50.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.088+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 >> 172.21.15.16:0/2250839271 conn(0x7f2edc0d3090 msgr2=0x7f2edc06b300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:50.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.088+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 shutdown_connections 2023-11-24T06:26:50.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:50.088+0000 7f2ee1659640 1 -- 172.21.15.16:0/2250839271 wait complete. 2023-11-24T06:26:52.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:26:52.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.316+0000 7f4bbd0ff640 1 Processor -- start 2023-11-24T06:26:52.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.316+0000 7f4bbd0ff640 1 -- start start 2023-11-24T06:26:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.317+0000 7f4bbd0ff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4bb815d3c0 0x7f4bb80d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.317+0000 7f4bbd0ff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 0x7f4bb80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.317+0000 7f4bbd0ff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb80d12f0 con 0x7f4bb80d0e90 2023-11-24T06:26:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.317+0000 7f4bbd0ff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb80ce5d0 con 0x7f4bb815d3c0 2023-11-24T06:26:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.317+0000 7f4bbd0ff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80b23c0 con 0x7f4bb80ce9f0 2023-11-24T06:26:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.317+0000 7f4bb77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4bb80d0e90 0x7f4bb80afc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44428/0 (socket says 172.21.15.16:44428) 2023-11-24T06:26:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.318+0000 7f4bb77fe640 1 -- 172.21.15.16:0/3404022878 learned_addr learned my addr 172.21.15.16:0/3404022878 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:26:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.318+0000 7f4bb7fff640 1 --2- 172.21.15.16:0/3404022878 >> v2:172.21.15.16:3300/0 conn(0x7f4bb815d3c0 0x7f4bb80d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.318+0000 7f4bbc8fe640 1 --2- 172.21.15.16:0/3404022878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 0x7f4bb80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.318+0000 7f4bb7fff640 1 -- 172.21.15.16:0/3404022878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 msgr2=0x7f4bb80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.318+0000 7f4bb7fff640 1 --2- 172.21.15.16:0/3404022878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 0x7f4bb80cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.318+0000 7f4bb7fff640 1 -- 172.21.15.16:0/3404022878 >> v1:172.21.15.16:6789/0 conn(0x7f4bb80d0e90 legacy=0x7f4bb80afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:52.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.318+0000 7f4bb7fff640 1 -- 172.21.15.16:0/3404022878 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80d0800 con 0x7f4bb815d3c0 2023-11-24T06:26:52.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.318+0000 7f4bb6ffd640 1 -- 172.21.15.16:0/3404022878 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591030787 0 0) 0x7f4ba0001110 con 0x7f4bb80d0e90 2023-11-24T06:26:52.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.319+0000 7f4bb7fff640 1 --2- 172.21.15.16:0/3404022878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 0x7f4bb80d0220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009d90 tx=0x7f4ba802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b69adc6807a333e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bb6ffd640 1 -- 172.21.15.16:0/3404022878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba802cbc0 con 0x7f4bb815d3c0 2023-11-24T06:26:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bb6ffd640 1 -- 172.21.15.16:0/3404022878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba802cd20 con 0x7f4bb815d3c0 2023-11-24T06:26:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bb6ffd640 1 -- 172.21.15.16:0/3404022878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8034660 con 0x7f4bb815d3c0 2023-11-24T06:26:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/3404022878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 msgr2=0x7f4bb80d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bbd0ff640 1 --2- 172.21.15.16:0/3404022878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 0x7f4bb80d0220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009d90 tx=0x7f4ba802be10 comp rx=0 tx=0).stop 2023-11-24T06:26:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/3404022878 shutdown_connections 2023-11-24T06:26:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bbd0ff640 1 --2- 172.21.15.16:0/3404022878 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 0x7f4bb80cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bbd0ff640 1 --2- 172.21.15.16:0/3404022878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 0x7f4bb80d0220 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/3404022878 >> 172.21.15.16:0/3404022878 conn(0x7f4bb80d30b0 msgr2=0x7f4bb80d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/3404022878 shutdown_connections 2023-11-24T06:26:52.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.320+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/3404022878 wait complete. 2023-11-24T06:26:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.322+0000 7f4bbd0ff640 1 Processor -- start 2023-11-24T06:26:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.322+0000 7f4bbd0ff640 1 -- start start 2023-11-24T06:26:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.323+0000 7f4bbd0ff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 0x7f4bb8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.323+0000 7f4bbd0ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb80d0e90 0x7f4bb8156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.323+0000 7f4bbd0ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 0x7f4bb80c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.323+0000 7f4bbd0ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb8157470 con 0x7f4bb815d3c0 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.323+0000 7f4bbd0ff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80b6fe0 con 0x7f4bb80d0e90 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.323+0000 7f4bbd0ff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80b72c0 con 0x7f4bb80ce9f0 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bbc8fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 0x7f4bb80c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bbc8fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 0x7f4bb80c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39844/0 (socket says 172.21.15.16:39844) 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bbc8fe640 1 -- 172.21.15.16:0/860645151 learned_addr learned my addr 172.21.15.16:0/860645151 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bb77fe640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb80d0e90 0x7f4bb8156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bb7fff640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 0x7f4bb8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bbc8fe640 1 -- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 msgr2=0x7f4bb8156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bbc8fe640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 0x7f4bb8156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bbc8fe640 1 -- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb80d0e90 msgr2=0x7f4bb8156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bbc8fe640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb80d0e90 0x7f4bb8156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.324+0000 7f4bbc8fe640 1 -- 172.21.15.16:0/860645151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba8009a70 con 0x7f4bb815d3c0 2023-11-24T06:26:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.325+0000 7f4bbc8fe640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 0x7f4bb80c9060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000d5d0 tx=0x7f4bb000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.325+0000 7f4bb4ff9640 1 -- 172.21.15.16:0/860645151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00125f0 con 0x7f4bb815d3c0 2023-11-24T06:26:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.325+0000 7f4bb4ff9640 1 -- 172.21.15.16:0/860645151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb0012770 con 0x7f4bb815d3c0 2023-11-24T06:26:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.325+0000 7f4bb4ff9640 1 -- 172.21.15.16:0/860645151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0012b20 con 0x7f4bb815d3c0 2023-11-24T06:26:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.325+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb80b75a0 con 0x7f4bb815d3c0 2023-11-24T06:26:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.325+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb80b7b60 con 0x7f4bb815d3c0 2023-11-24T06:26:52.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.326+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb8067b60 con 0x7f4bb815d3c0 2023-11-24T06:26:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.327+0000 7f4bb4ff9640 1 -- 172.21.15.16:0/860645151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f4bb0012c80 con 0x7f4bb815d3c0 2023-11-24T06:26:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.329+0000 7f4bb4ff9640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4b9806adb0 0x7f4b9806d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.329+0000 7f4bb4ff9640 1 -- 172.21.15.16:0/860645151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4bb0089720 con 0x7f4bb815d3c0 2023-11-24T06:26:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.329+0000 7f4bb7fff640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4b9806adb0 0x7f4b9806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.330+0000 7f4bb7fff640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4b9806adb0 0x7f4b9806d270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8036000 tx=0x7f4ba8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:52.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.334+0000 7f4bb4ff9640 1 -- 172.21.15.16:0/860645151 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4bb0054b70 con 0x7f4bb815d3c0 2023-11-24T06:26:52.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.575+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bb808c800 con 0x7f4bb815d3c0 2023-11-24T06:26:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.576+0000 7f4bb4ff9640 1 -- 172.21.15.16:0/860645151 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f4bb0058190 con 0x7f4bb815d3c0 2023-11-24T06:26:52.579 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:26:52.579 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3019663874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3019663874},{"type":"v1","addr":"172.21.15.173:6837","nonce":3019663874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2239024365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2239024365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2239024365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4239,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/915142029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":915142029},{"type":"v1","addr":"172.21.15.173:6835","nonce":915142029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T06:26:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2023-11-24T06:26:52.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4b9806adb0 msgr2=0x7f4b9806d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4b9806adb0 0x7f4b9806d270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8036000 tx=0x7f4ba8030000 comp rx=0 tx=0).stop 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 msgr2=0x7f4bb80c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 0x7f4bb80c9060 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000d5d0 tx=0x7f4bb000da90 comp rx=0 tx=0).stop 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 shutdown_connections 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4b9806adb0 0x7f4b9806d270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb815d3c0 0x7f4bb80c9060 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bb80d0e90 0x7f4bb8156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 --2- 172.21.15.16:0/860645151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bb80ce9f0 0x7f4bb8156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 >> 172.21.15.16:0/860645151 conn(0x7f4bb80d30b0 msgr2=0x7f4bb806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 shutdown_connections 2023-11-24T06:26:52.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.579+0000 7f4bbd0ff640 1 -- 172.21.15.16:0/860645151 wait complete. 2023-11-24T06:26:52.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:52.593 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:26:52.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:26:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.840+0000 7f530f651640 1 Processor -- start 2023-11-24T06:26:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.841+0000 7f530f651640 1 -- start start 2023-11-24T06:26:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.842+0000 7f530f651640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5308160f80 0x7f5308161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.842+0000 7f530f651640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f53080ce8f0 0x7f53080ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.842+0000 7f530f651640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 0x7f53080d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.842+0000 7f530f651640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53081618a0 con 0x7f5308160f80 2023-11-24T06:26:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.842+0000 7f530f651640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53080d0de0 con 0x7f53080ce8f0 2023-11-24T06:26:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.842+0000 7f530f651640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53080d10c0 con 0x7f53080cfbe0 2023-11-24T06:26:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530d3ed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5308160f80 0x7f5308161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530d3ed640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5308160f80 0x7f5308161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40428/0 (socket says 172.21.15.16:40428) 2023-11-24T06:26:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530d3ed640 1 -- 172.21.15.16:0/3375824705 learned_addr learned my addr 172.21.15.16:0/3375824705 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530dbee640 1 --2- 172.21.15.16:0/3375824705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 0x7f53080d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530d3ed640 1 -- 172.21.15.16:0/3375824705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 msgr2=0x7f53080d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530cbec640 1 --2- 172.21.15.16:0/3375824705 >> v2:172.21.15.173:3300/0 conn(0x7f53080ce8f0 0x7f53080ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530d3ed640 1 --2- 172.21.15.16:0/3375824705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 0x7f53080d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530d3ed640 1 -- 172.21.15.16:0/3375824705 >> v2:172.21.15.173:3300/0 conn(0x7f53080ce8f0 msgr2=0x7f53080ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530d3ed640 1 --2- 172.21.15.16:0/3375824705 >> v2:172.21.15.173:3300/0 conn(0x7f53080ce8f0 0x7f53080ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530d3ed640 1 -- 172.21.15.16:0/3375824705 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53080d13a0 con 0x7f5308160f80 2023-11-24T06:26:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.843+0000 7f530cbec640 1 --2- 172.21.15.16:0/3375824705 >> v2:172.21.15.173:3300/0 conn(0x7f53080ce8f0 0x7f53080ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.844+0000 7f530d3ed640 1 --2- 172.21.15.16:0/3375824705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 0x7f5308161360 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53080b5d10 tx=0x7f530402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a442e69db76fb6b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f52fffff640 1 -- 172.21.15.16:0/3375824705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304031860 con 0x7f5308160f80 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f52fffff640 1 -- 172.21.15.16:0/3375824705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53040319c0 con 0x7f5308160f80 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f52fffff640 1 -- 172.21.15.16:0/3375824705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304031d50 con 0x7f5308160f80 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f530f651640 1 -- 172.21.15.16:0/3375824705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 msgr2=0x7f5308161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f530f651640 1 --2- 172.21.15.16:0/3375824705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 0x7f5308161360 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53080b5d10 tx=0x7f530402b900 comp rx=0 tx=0).stop 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f530f651640 1 -- 172.21.15.16:0/3375824705 shutdown_connections 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f530f651640 1 --2- 172.21.15.16:0/3375824705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 0x7f53080d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f530f651640 1 --2- 172.21.15.16:0/3375824705 >> v2:172.21.15.173:3300/0 conn(0x7f53080ce8f0 0x7f53080ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f530f651640 1 --2- 172.21.15.16:0/3375824705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 0x7f5308161360 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.845+0000 7f530f651640 1 -- 172.21.15.16:0/3375824705 >> 172.21.15.16:0/3375824705 conn(0x7f530809c550 msgr2=0x7f53080d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.846+0000 7f530f651640 1 -- 172.21.15.16:0/3375824705 shutdown_connections 2023-11-24T06:26:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.846+0000 7f530f651640 1 -- 172.21.15.16:0/3375824705 wait complete. 2023-11-24T06:26:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.847+0000 7f530f651640 1 Processor -- start 2023-11-24T06:26:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.847+0000 7f530f651640 1 -- start start 2023-11-24T06:26:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.848+0000 7f530f651640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53080ce8f0 0x7f5308151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.848+0000 7f530f651640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 0x7f53080cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.848+0000 7f530f651640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 0x7f53080ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.848+0000 7f530f651640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080b1dd0 con 0x7f5308160f80 2023-11-24T06:26:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.848+0000 7f530f651640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080cd170 con 0x7f53080ce8f0 2023-11-24T06:26:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.848+0000 7f530f651640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53080cce90 con 0x7f53080cfbe0 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530dbee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 0x7f53080ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530dbee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 0x7f53080ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40442/0 (socket says 172.21.15.16:40442) 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530dbee640 1 -- 172.21.15.16:0/4235265287 learned_addr learned my addr 172.21.15.16:0/4235265287 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530d3ed640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53080ce8f0 0x7f5308151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530cbec640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 0x7f53080cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530dbee640 1 -- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 msgr2=0x7f53080cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530dbee640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 0x7f53080cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530dbee640 1 -- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53080ce8f0 msgr2=0x7f5308151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530dbee640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53080ce8f0 0x7f5308151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.849+0000 7f530dbee640 1 -- 172.21.15.16:0/4235265287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53040094d0 con 0x7f5308160f80 2023-11-24T06:26:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.850+0000 7f530dbee640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 0x7f53080ca4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f530000cd70 tx=0x7f530000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.850+0000 7f52fdffb640 1 -- 172.21.15.16:0/4235265287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300003b90 con 0x7f5308160f80 2023-11-24T06:26:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.850+0000 7f52fdffb640 1 -- 172.21.15.16:0/4235265287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5300003cf0 con 0x7f5308160f80 2023-11-24T06:26:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.850+0000 7f52fdffb640 1 -- 172.21.15.16:0/4235265287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300004240 con 0x7f5308160f80 2023-11-24T06:26:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.850+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53080c77b0 con 0x7f5308160f80 2023-11-24T06:26:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.850+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53080c7d70 con 0x7f5308160f80 2023-11-24T06:26:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.851+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53080c7f90 con 0x7f5308160f80 2023-11-24T06:26:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.852+0000 7f52fdffb640 1 -- 172.21.15.16:0/4235265287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f53000043a0 con 0x7f5308160f80 2023-11-24T06:26:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.853+0000 7f52fdffb640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f52ec06adb0 0x7f52ec06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.853+0000 7f52fdffb640 1 -- 172.21.15.16:0/4235265287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5300012070 con 0x7f5308160f80 2023-11-24T06:26:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.854+0000 7f530d3ed640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f52ec06adb0 0x7f52ec06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.855+0000 7f530d3ed640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f52ec06adb0 0x7f52ec06d270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5304036000 tx=0x7f530403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.858+0000 7f52fdffb640 1 -- 172.21.15.16:0/4235265287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53000550c0 con 0x7f5308160f80 2023-11-24T06:26:52.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:52.955+0000 7f52fdffb640 1 -- 172.21.15.16:0/4235265287 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5300055490 con 0x7f5308160f80 2023-11-24T06:26:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.062+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53080675d0 con 0x7f5308160f80 2023-11-24T06:26:53.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.063+0000 7f52fdffb640 1 -- 172.21.15.16:0/4235265287 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f5300076f80 con 0x7f5308160f80 2023-11-24T06:26:53.065 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:26:53.066 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:52.365717+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:26:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.066+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f52ec06adb0 msgr2=0x7f52ec06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.066+0000 7f530f651640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f52ec06adb0 0x7f52ec06d270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5304036000 tx=0x7f530403c040 comp rx=0 tx=0).stop 2023-11-24T06:26:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.066+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 msgr2=0x7f53080ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.066+0000 7f530f651640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 0x7f53080ca4a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f530000cd70 tx=0x7f530000a660 comp rx=0 tx=0).stop 2023-11-24T06:26:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.066+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 shutdown_connections 2023-11-24T06:26:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.066+0000 7f530f651640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f52ec06adb0 0x7f52ec06d270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.067+0000 7f530f651640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5308160f80 0x7f53080ca4a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.067+0000 7f530f651640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53080cfbe0 0x7f53080cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.067+0000 7f530f651640 1 --2- 172.21.15.16:0/4235265287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53080ce8f0 0x7f5308151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.067+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 >> 172.21.15.16:0/4235265287 conn(0x7f530809c550 msgr2=0x7f53080b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.067+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 shutdown_connections 2023-11-24T06:26:53.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.067+0000 7f530f651640 1 -- 172.21.15.16:0/4235265287 wait complete. 2023-11-24T06:26:53.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:26:53.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:26:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.285+0000 7f3e37fff640 1 Processor -- start 2023-11-24T06:26:53.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.286+0000 7f3e37fff640 1 -- start start 2023-11-24T06:26:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.288+0000 7f3e37fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e380715d0 0x7f3e381642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.288+0000 7f3e37fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e38159a70 con 0x7f3e3806d110 2023-11-24T06:26:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.288+0000 7f3e37fff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e380cc2e0 con 0x7f3e38159690 2023-11-24T06:26:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.288+0000 7f3e37fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3806d640 con 0x7f3e380715d0 2023-11-24T06:26:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.288+0000 7f3e367fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3e3806d110 0x7f3e38070ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34948/0 (socket says 172.21.15.16:34948) 2023-11-24T06:26:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.288+0000 7f3e367fc640 1 -- 172.21.15.16:0/2171343275 learned_addr learned my addr 172.21.15.16:0/2171343275 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:26:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.288+0000 7f3e377fe640 1 --2- 172.21.15.16:0/2171343275 >> v2:172.21.15.16:3300/0 conn(0x7f3e380715d0 0x7f3e381642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.288+0000 7f3e35ffb640 1 -- 172.21.15.16:0/2171343275 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944706075 0 0) 0x7f3e38159a70 con 0x7f3e3806d110 2023-11-24T06:26:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.288+0000 7f3e35ffb640 1 -- 172.21.15.16:0/2171343275 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e24003520 con 0x7f3e3806d110 2023-11-24T06:26:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.289+0000 7f3e377fe640 1 -- 172.21.15.16:0/2171343275 >> v1:172.21.15.173:6789/0 conn(0x7f3e38159690 legacy=0x7f3e3806ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.289+0000 7f3e377fe640 1 -- 172.21.15.16:0/2171343275 >> v1:172.21.15.16:6789/0 conn(0x7f3e3806d110 legacy=0x7f3e38070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.289+0000 7f3e377fe640 1 -- 172.21.15.16:0/2171343275 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3806d920 con 0x7f3e380715d0 2023-11-24T06:26:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.290+0000 7f3e377fe640 1 --2- 172.21.15.16:0/2171343275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e380715d0 0x7f3e381642e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000a8c0 tx=0x7f3e3002cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65416b0cbeb20738 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.290+0000 7f3e35ffb640 1 -- 172.21.15.16:0/2171343275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3002d940 con 0x7f3e380715d0 2023-11-24T06:26:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.290+0000 7f3e35ffb640 1 -- 172.21.15.16:0/2171343275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3002daa0 con 0x7f3e380715d0 2023-11-24T06:26:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.290+0000 7f3e35ffb640 1 -- 172.21.15.16:0/2171343275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3002de30 con 0x7f3e380715d0 2023-11-24T06:26:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.291+0000 7f3e37fff640 1 -- 172.21.15.16:0/2171343275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e380715d0 msgr2=0x7f3e381642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.291+0000 7f3e37fff640 1 --2- 172.21.15.16:0/2171343275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e380715d0 0x7f3e381642e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000a8c0 tx=0x7f3e3002cb90 comp rx=0 tx=0).stop 2023-11-24T06:26:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.291+0000 7f3e37fff640 1 -- 172.21.15.16:0/2171343275 shutdown_connections 2023-11-24T06:26:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.291+0000 7f3e37fff640 1 --2- 172.21.15.16:0/2171343275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e380715d0 0x7f3e381642e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.291+0000 7f3e37fff640 1 -- 172.21.15.16:0/2171343275 >> 172.21.15.16:0/2171343275 conn(0x7f3e380b15b0 msgr2=0x7f3e380b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.291+0000 7f3e37fff640 1 -- 172.21.15.16:0/2171343275 shutdown_connections 2023-11-24T06:26:53.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.291+0000 7f3e37fff640 1 -- 172.21.15.16:0/2171343275 wait complete. 2023-11-24T06:26:53.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.293+0000 7f3e37fff640 1 Processor -- start 2023-11-24T06:26:53.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.293+0000 7f3e37fff640 1 -- start start 2023-11-24T06:26:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e37fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e3806d110 0x7f3e3815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e37fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e380715d0 0x7f3e381e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e38159690 0x7f3e381f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e37fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381e6000 con 0x7f3e38159690 2023-11-24T06:26:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e37fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381e5760 con 0x7f3e3806d110 2023-11-24T06:26:53.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e37fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381e5a40 con 0x7f3e380715d0 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e38159690 0x7f3e381f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e38159690 0x7f3e381f27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40454/0 (socket says 172.21.15.16:40454) 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e377fe640 1 -- 172.21.15.16:0/1220444609 learned_addr learned my addr 172.21.15.16:0/1220444609 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e36ffd640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e3806d110 0x7f3e3815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.294+0000 7f3e367fc640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e380715d0 0x7f3e381e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e377fe640 1 -- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e380715d0 msgr2=0x7f3e381e4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e377fe640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e380715d0 0x7f3e381e4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e377fe640 1 -- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e3806d110 msgr2=0x7f3e3815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e377fe640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e3806d110 0x7f3e3815a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e377fe640 1 -- 172.21.15.16:0/1220444609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3000a5a0 con 0x7f3e38159690 2023-11-24T06:26:53.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e377fe640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e38159690 0x7f3e381f27b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e30038040 tx=0x7f3e300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e2f7fe640 1 -- 172.21.15.16:0/1220444609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3003e630 con 0x7f3e38159690 2023-11-24T06:26:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e2f7fe640 1 -- 172.21.15.16:0/1220444609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3003e7b0 con 0x7f3e38159690 2023-11-24T06:26:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e2f7fe640 1 -- 172.21.15.16:0/1220444609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3003eb60 con 0x7f3e38159690 2023-11-24T06:26:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e381e5d20 con 0x7f3e38159690 2023-11-24T06:26:53.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.295+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e381e5450 con 0x7f3e38159690 2023-11-24T06:26:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.297+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3814e8d0 con 0x7f3e38159690 2023-11-24T06:26:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.298+0000 7f3e2f7fe640 1 -- 172.21.15.16:0/1220444609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e3003c040 con 0x7f3e38159690 2023-11-24T06:26:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.299+0000 7f3e2f7fe640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e1806ae40 0x7f3e1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.299+0000 7f3e2f7fe640 1 -- 172.21.15.16:0/1220444609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3e300ab3c0 con 0x7f3e38159690 2023-11-24T06:26:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.300+0000 7f3e36ffd640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e1806ae40 0x7f3e1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.301+0000 7f3e36ffd640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e1806ae40 0x7f3e1806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e28007fd0 tx=0x7f3e28009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:53.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.304+0000 7f3e2f7fe640 1 -- 172.21.15.16:0/1220444609 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e3814e8d0 con 0x7f3e38159690 2023-11-24T06:26:53.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.498+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e3814e8d0 con 0x7f3e38159690 2023-11-24T06:26:53.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.498+0000 7f3e2f7fe640 1 -- 172.21.15.16:0/1220444609 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e3814e8d0 con 0x7f3e38159690 2023-11-24T06:26:53.501 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:26:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e1806ae40 msgr2=0x7f3e1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e1806ae40 0x7f3e1806d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3e28007fd0 tx=0x7f3e28009040 comp rx=0 tx=0).stop 2023-11-24T06:26:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e38159690 msgr2=0x7f3e381f27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e38159690 0x7f3e381f27b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e30038040 tx=0x7f3e300046c0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 shutdown_connections 2023-11-24T06:26:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e1806ae40 0x7f3e1806d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e38159690 0x7f3e381f27b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e380715d0 0x7f3e381e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 --2- 172.21.15.16:0/1220444609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e3806d110 0x7f3e3815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 >> 172.21.15.16:0/1220444609 conn(0x7f3e380b15b0 msgr2=0x7f3e380af100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.501+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 shutdown_connections 2023-11-24T06:26:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.502+0000 7f3e37fff640 1 -- 172.21.15.16:0/1220444609 wait complete. 2023-11-24T06:26:53.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:26:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.739+0000 7f5546ce7640 1 Processor -- start 2023-11-24T06:26:53.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.740+0000 7f5546ce7640 1 -- start start 2023-11-24T06:26:53.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.741+0000 7f5546ce7640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f55400cf9a0 0x7f554015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.741+0000 7f5546ce7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f554015e1e0 0x7f55400702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.741+0000 7f5546ce7640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55400b1ee0 con 0x7f5540143ae0 2023-11-24T06:26:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.741+0000 7f5546ce7640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55400b2780 con 0x7f55400cf9a0 2023-11-24T06:26:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.741+0000 7f5546ce7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554015e990 con 0x7f554015e1e0 2023-11-24T06:26:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.742+0000 7f5545284640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f554015e1e0 0x7f55400702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.742+0000 7f5537fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f55400cf9a0 0x7f554015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.742+0000 7f5545284640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f554015e1e0 0x7f55400702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:35348/0 (socket says 172.21.15.16:35348) 2023-11-24T06:26:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.742+0000 7f5545284640 1 -- 172.21.15.16:0/2818228423 learned_addr learned my addr 172.21.15.16:0/2818228423 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.742+0000 7f5537fff640 1 -- 172.21.15.16:0/2818228423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f554015e1e0 msgr2=0x7f55400702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.742+0000 7f5537fff640 1 --2- 172.21.15.16:0/2818228423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f554015e1e0 0x7f55400702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.742+0000 7f5537fff640 1 -- 172.21.15.16:0/2818228423 >> v1:172.21.15.173:6789/0 conn(0x7f5540143ae0 legacy=0x7f55400cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.742+0000 7f5537fff640 1 -- 172.21.15.16:0/2818228423 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400b2130 con 0x7f55400cf9a0 2023-11-24T06:26:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.743+0000 7f5537fff640 1 --2- 172.21.15.16:0/2818228423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55400cf9a0 0x7f554015dca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55300097a0 tx=0x7f553002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89cd7dd4fcccb777 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.744+0000 7f55377fe640 1 -- 172.21.15.16:0/2818228423 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553002e070 con 0x7f55400cf9a0 2023-11-24T06:26:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.744+0000 7f55377fe640 1 -- 172.21.15.16:0/2818228423 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55300315a0 con 0x7f55400cf9a0 2023-11-24T06:26:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.744+0000 7f55377fe640 1 -- 172.21.15.16:0/2818228423 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530031930 con 0x7f55400cf9a0 2023-11-24T06:26:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.745+0000 7f5546ce7640 1 -- 172.21.15.16:0/2818228423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55400cf9a0 msgr2=0x7f554015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.745+0000 7f5546ce7640 1 --2- 172.21.15.16:0/2818228423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55400cf9a0 0x7f554015dca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55300097a0 tx=0x7f553002b840 comp rx=0 tx=0).stop 2023-11-24T06:26:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.745+0000 7f5546ce7640 1 -- 172.21.15.16:0/2818228423 shutdown_connections 2023-11-24T06:26:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.745+0000 7f5546ce7640 1 --2- 172.21.15.16:0/2818228423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f554015e1e0 0x7f55400702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.745+0000 7f5546ce7640 1 --2- 172.21.15.16:0/2818228423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55400cf9a0 0x7f554015dca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.745+0000 7f5546ce7640 1 -- 172.21.15.16:0/2818228423 >> 172.21.15.16:0/2818228423 conn(0x7f554006a800 msgr2=0x7f554006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.745+0000 7f5546ce7640 1 -- 172.21.15.16:0/2818228423 shutdown_connections 2023-11-24T06:26:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.745+0000 7f5546ce7640 1 -- 172.21.15.16:0/2818228423 wait complete. 2023-11-24T06:26:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.747+0000 7f5546ce7640 1 Processor -- start 2023-11-24T06:26:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.747+0000 7f5546ce7640 1 -- start start 2023-11-24T06:26:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5546ce7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55400cf9a0 0x7f55400adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5546ce7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5540143ae0 0x7f5540131330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5546ce7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554015e1e0 0x7f5540141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5546ce7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5540132310 con 0x7f554015e1e0 2023-11-24T06:26:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5546ce7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55400ae3e0 con 0x7f55400cf9a0 2023-11-24T06:26:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5546ce7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5540131870 con 0x7f5540143ae0 2023-11-24T06:26:53.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5545284640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554015e1e0 0x7f5540141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5545284640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554015e1e0 0x7f5540141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40456/0 (socket says 172.21.15.16:40456) 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5545284640 1 -- 172.21.15.16:0/2199156770 learned_addr learned my addr 172.21.15.16:0/2199156770 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5544a83640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55400cf9a0 0x7f55400adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5537fff640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5540143ae0 0x7f5540131330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5545284640 1 -- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5540143ae0 msgr2=0x7f5540131330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5545284640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5540143ae0 0x7f5540131330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5545284640 1 -- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55400cf9a0 msgr2=0x7f55400adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5545284640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55400cf9a0 0x7f55400adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.748+0000 7f5545284640 1 -- 172.21.15.16:0/2199156770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530009400 con 0x7f554015e1e0 2023-11-24T06:26:53.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.749+0000 7f5537fff640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5540143ae0 0x7f5540131330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.749+0000 7f5545284640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554015e1e0 0x7f5540141430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f553800efc0 tx=0x7f553800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.749+0000 7f55357fa640 1 -- 172.21.15.16:0/2199156770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538009b60 con 0x7f554015e1e0 2023-11-24T06:26:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.749+0000 7f55357fa640 1 -- 172.21.15.16:0/2199156770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553800f040 con 0x7f554015e1e0 2023-11-24T06:26:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.749+0000 7f55357fa640 1 -- 172.21.15.16:0/2199156770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538003990 con 0x7f554015e1e0 2023-11-24T06:26:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.749+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5540131a70 con 0x7f554015e1e0 2023-11-24T06:26:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.749+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5540132030 con 0x7f554015e1e0 2023-11-24T06:26:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.750+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55400a84c0 con 0x7f554015e1e0 2023-11-24T06:26:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.751+0000 7f55357fa640 1 -- 172.21.15.16:0/2199156770 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5538003af0 con 0x7f554015e1e0 2023-11-24T06:26:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.752+0000 7f55357fa640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f552c06ae40 0x7f552c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.752+0000 7f55357fa640 1 -- 172.21.15.16:0/2199156770 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5538089840 con 0x7f554015e1e0 2023-11-24T06:26:53.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.753+0000 7f5544a83640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f552c06ae40 0x7f552c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:53.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.754+0000 7f5544a83640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f552c06ae40 0x7f552c06d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55400cec50 tx=0x7f553c007e30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:53.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.757+0000 7f55357fa640 1 -- 172.21.15.16:0/2199156770 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55400a84c0 con 0x7f554015e1e0 2023-11-24T06:26:53.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.952+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f55400a88a0 con 0x7f554015e1e0 2023-11-24T06:26:53.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.954+0000 7f55357fa640 1 -- 172.21.15.16:0/2199156770 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f55400a88a0 con 0x7f554015e1e0 2023-11-24T06:26:53.956 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:26:53.956 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:26:53.956 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:26:53.956 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-24T06:26:53.956 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-24T06:26:53.956 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:26:53.956 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:26:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f552c06ae40 msgr2=0x7f552c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f552c06ae40 0x7f552c06d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55400cec50 tx=0x7f553c007e30 comp rx=0 tx=0).stop 2023-11-24T06:26:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554015e1e0 msgr2=0x7f5540141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:53.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554015e1e0 0x7f5540141430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f553800efc0 tx=0x7f553800c490 comp rx=0 tx=0).stop 2023-11-24T06:26:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 shutdown_connections 2023-11-24T06:26:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f552c06ae40 0x7f552c06d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554015e1e0 0x7f5540141430 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5540143ae0 0x7f5540131330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 --2- 172.21.15.16:0/2199156770 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55400cf9a0 0x7f55400adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.956+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 >> 172.21.15.16:0/2199156770 conn(0x7f554006a800 msgr2=0x7f554015a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.957+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 shutdown_connections 2023-11-24T06:26:53.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:53.957+0000 7f5546ce7640 1 -- 172.21.15.16:0/2199156770 wait complete. 2023-11-24T06:26:53.979 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:26:53.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:26:54.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.210+0000 7fab3ffff640 1 Processor -- start 2023-11-24T06:26:54.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.211+0000 7fab3ffff640 1 -- start start 2023-11-24T06:26:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.213+0000 7fab3ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab40143c40 0x7fab40071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.213+0000 7fab3ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fab400717d0 0x7fab400cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.213+0000 7fab3ffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab40070e50 con 0x7fab400cca40 2023-11-24T06:26:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.213+0000 7fab3ffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab400723c0 con 0x7fab40143c40 2023-11-24T06:26:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.213+0000 7fab3ffff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab400cce40 con 0x7fab400717d0 2023-11-24T06:26:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.213+0000 7fab3f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fab400cca40 0x7fab4015eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34964/0 (socket says 172.21.15.16:34964) 2023-11-24T06:26:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.213+0000 7fab3f7fe640 1 -- 172.21.15.16:0/1346489179 learned_addr learned my addr 172.21.15.16:0/1346489179 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:26:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.213+0000 7fab3effd640 1 --2- 172.21.15.16:0/1346489179 >> v2:172.21.15.16:3300/0 conn(0x7fab40143c40 0x7fab40071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.213+0000 7fab37fff640 1 --2- 172.21.15.16:0/1346489179 >> v2:172.21.15.173:3300/0 conn(0x7fab400717d0 0x7fab400cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.214+0000 7fab3e7fc640 1 -- 172.21.15.16:0/1346489179 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729994036 0 0) 0x7fab40070e50 con 0x7fab400cca40 2023-11-24T06:26:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.214+0000 7fab3effd640 1 -- 172.21.15.16:0/1346489179 >> v2:172.21.15.173:3300/0 conn(0x7fab400717d0 msgr2=0x7fab400cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.214+0000 7fab3effd640 1 --2- 172.21.15.16:0/1346489179 >> v2:172.21.15.173:3300/0 conn(0x7fab400717d0 0x7fab400cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.214+0000 7fab3effd640 1 -- 172.21.15.16:0/1346489179 >> v1:172.21.15.16:6789/0 conn(0x7fab400cca40 legacy=0x7fab4015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.214+0000 7fab3effd640 1 -- 172.21.15.16:0/1346489179 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4006e170 con 0x7fab40143c40 2023-11-24T06:26:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.215+0000 7fab3effd640 1 --2- 172.21.15.16:0/1346489179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab40143c40 0x7fab40071290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab400b44a0 tx=0x7fab3002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af4276abac1906b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.216+0000 7fab3e7fc640 1 -- 172.21.15.16:0/1346489179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3002e070 con 0x7fab40143c40 2023-11-24T06:26:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.216+0000 7fab3e7fc640 1 -- 172.21.15.16:0/1346489179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab300315a0 con 0x7fab40143c40 2023-11-24T06:26:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.216+0000 7fab3e7fc640 1 -- 172.21.15.16:0/1346489179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30031930 con 0x7fab40143c40 2023-11-24T06:26:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.216+0000 7fab3ffff640 1 -- 172.21.15.16:0/1346489179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab40143c40 msgr2=0x7fab40071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.216+0000 7fab3ffff640 1 --2- 172.21.15.16:0/1346489179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab40143c40 0x7fab40071290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fab400b44a0 tx=0x7fab3002b840 comp rx=0 tx=0).stop 2023-11-24T06:26:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.216+0000 7fab3ffff640 1 -- 172.21.15.16:0/1346489179 shutdown_connections 2023-11-24T06:26:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.216+0000 7fab3ffff640 1 --2- 172.21.15.16:0/1346489179 >> v2:172.21.15.173:3300/0 conn(0x7fab400717d0 0x7fab400cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.216+0000 7fab3ffff640 1 --2- 172.21.15.16:0/1346489179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab40143c40 0x7fab40071290 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.216+0000 7fab3ffff640 1 -- 172.21.15.16:0/1346489179 >> 172.21.15.16:0/1346489179 conn(0x7fab4006a800 msgr2=0x7fab4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.217+0000 7fab3ffff640 1 -- 172.21.15.16:0/1346489179 shutdown_connections 2023-11-24T06:26:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.217+0000 7fab3ffff640 1 -- 172.21.15.16:0/1346489179 wait complete. 2023-11-24T06:26:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.218+0000 7fab3ffff640 1 Processor -- start 2023-11-24T06:26:54.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.218+0000 7fab3ffff640 1 -- start start 2023-11-24T06:26:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.219+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab400717d0 0x7fab40139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.219+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab400cca40 0x7fab400ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.219+0000 7fab3ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40143c40 0x7fab400abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.219+0000 7fab3ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab400ac330 con 0x7fab400cca40 2023-11-24T06:26:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.219+0000 7fab3ffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4013a200 con 0x7fab40143c40 2023-11-24T06:26:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.219+0000 7fab3ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab400a8fa0 con 0x7fab400717d0 2023-11-24T06:26:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab400cca40 0x7fab400ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab400cca40 0x7fab400ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40472/0 (socket says 172.21.15.16:40472) 2023-11-24T06:26:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab37fff640 1 -- 172.21.15.16:0/755798845 learned_addr learned my addr 172.21.15.16:0/755798845 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab3effd640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab400717d0 0x7fab40139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab3f7fe640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40143c40 0x7fab400abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab37fff640 1 -- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab400717d0 msgr2=0x7fab40139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab37fff640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab400717d0 0x7fab40139cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab37fff640 1 -- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40143c40 msgr2=0x7fab400abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab37fff640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40143c40 0x7fab400abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab37fff640 1 -- 172.21.15.16:0/755798845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30009400 con 0x7fab400cca40 2023-11-24T06:26:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab3f7fe640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40143c40 0x7fab400abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:26:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.220+0000 7fab37fff640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab400cca40 0x7fab400ab8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9b0 tx=0x7fab2800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.221+0000 7fab377fe640 1 -- 172.21.15.16:0/755798845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28009670 con 0x7fab400cca40 2023-11-24T06:26:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.221+0000 7fab377fe640 1 -- 172.21.15.16:0/755798845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2800c9a0 con 0x7fab400cca40 2023-11-24T06:26:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.221+0000 7fab377fe640 1 -- 172.21.15.16:0/755798845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2800cd50 con 0x7fab400cca40 2023-11-24T06:26:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.221+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab400a9280 con 0x7fab400cca40 2023-11-24T06:26:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.221+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab400a9840 con 0x7fab400cca40 2023-11-24T06:26:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.222+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab40067450 con 0x7fab400cca40 2023-11-24T06:26:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.223+0000 7fab377fe640 1 -- 172.21.15.16:0/755798845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab28013470 con 0x7fab400cca40 2023-11-24T06:26:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.224+0000 7fab377fe640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab2006ae70 0x7fab2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.224+0000 7fab377fe640 1 -- 172.21.15.16:0/755798845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fab28089460 con 0x7fab400cca40 2023-11-24T06:26:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.224+0000 7fab3effd640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab2006ae70 0x7fab2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.226+0000 7fab3effd640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab2006ae70 0x7fab2006d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab30036000 tx=0x7fab3003e040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.228+0000 7fab377fe640 1 -- 172.21.15.16:0/755798845 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab28054870 con 0x7fab400cca40 2023-11-24T06:26:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.423+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fab400ceb30 con 0x7fab400cca40 2023-11-24T06:26:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.963+0000 7fab377fe640 1 -- 172.21.15.16:0/755798845 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fab28057e90 con 0x7fab400cca40 2023-11-24T06:26:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-24T06:26:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab2006ae70 msgr2=0x7fab2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab2006ae70 0x7fab2006d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab30036000 tx=0x7fab3003e040 comp rx=0 tx=0).stop 2023-11-24T06:26:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab400cca40 msgr2=0x7fab400ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab400cca40 0x7fab400ab8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fab2800e9b0 tx=0x7fab2800ee70 comp rx=0 tx=0).stop 2023-11-24T06:26:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 shutdown_connections 2023-11-24T06:26:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab2006ae70 0x7fab2006d330 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab30036000 tx=0x7fab3003e040 comp rx=0 tx=0).stop 2023-11-24T06:26:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40143c40 0x7fab400abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab400cca40 0x7fab400ab8b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 --2- 172.21.15.16:0/755798845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab400717d0 0x7fab40139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.968+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 >> 172.21.15.16:0/755798845 conn(0x7fab4006a800 msgr2=0x7fab4006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.970+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 shutdown_connections 2023-11-24T06:26:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:54.970+0000 7fab3ffff640 1 -- 172.21.15.16:0/755798845 wait complete. 2023-11-24T06:26:54.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:26:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.209+0000 7f3daf0ca640 1 Processor -- start 2023-11-24T06:26:55.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.209+0000 7f3daf0ca640 1 -- start start 2023-11-24T06:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.211+0000 7f3daf0ca640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da8160920 0x7f3da80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.211+0000 7f3daf0ca640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3da80cf9f0 0x7f3da80cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.211+0000 7f3daf0ca640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da80d0a10 con 0x7f3da8161010 2023-11-24T06:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.211+0000 7f3daf0ca640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da80d03b0 con 0x7f3da80cf9f0 2023-11-24T06:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.211+0000 7f3daf0ca640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da80d1f50 con 0x7f3da8160920 2023-11-24T06:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.212+0000 7f3dae0c8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da8160920 0x7f3da80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.212+0000 7f3dae0c8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da8160920 0x7f3da80b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:35372/0 (socket says 172.21.15.16:35372) 2023-11-24T06:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.212+0000 7f3dae0c8640 1 -- 172.21.15.16:0/2636257248 learned_addr learned my addr 172.21.15.16:0/2636257248 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.212+0000 7f3dad8c7640 1 --2- 172.21.15.16:0/2636257248 >> v2:172.21.15.173:3300/0 conn(0x7f3da80cf9f0 0x7f3da80cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.212+0000 7f3dae0c8640 1 -- 172.21.15.16:0/2636257248 >> v2:172.21.15.173:3300/0 conn(0x7f3da80cf9f0 msgr2=0x7f3da80cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.212+0000 7f3dae0c8640 1 --2- 172.21.15.16:0/2636257248 >> v2:172.21.15.173:3300/0 conn(0x7f3da80cf9f0 0x7f3da80cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.212+0000 7f3dae0c8640 1 -- 172.21.15.16:0/2636257248 >> v1:172.21.15.173:6789/0 conn(0x7f3da8161010 legacy=0x7f3da81614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.212+0000 7f3dae0c8640 1 -- 172.21.15.16:0/2636257248 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da80cf080 con 0x7f3da8160920 2023-11-24T06:26:55.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.213+0000 7f3dae0c8640 1 --2- 172.21.15.16:0/2636257248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da8160920 0x7f3da80b3f30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009e50 tx=0x7f3da402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1a4c3b829ca7e98 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.213+0000 7f3dad0c6640 1 -- 172.21.15.16:0/2636257248 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da402cb00 con 0x7f3da8160920 2023-11-24T06:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.214+0000 7f3dad0c6640 1 -- 172.21.15.16:0/2636257248 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da402cc60 con 0x7f3da8160920 2023-11-24T06:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.214+0000 7f3dad0c6640 1 -- 172.21.15.16:0/2636257248 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da40343f0 con 0x7f3da8160920 2023-11-24T06:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.214+0000 7f3daf0ca640 1 -- 172.21.15.16:0/2636257248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da8160920 msgr2=0x7f3da80b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.214+0000 7f3daf0ca640 1 --2- 172.21.15.16:0/2636257248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da8160920 0x7f3da80b3f30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009e50 tx=0x7f3da402bf60 comp rx=0 tx=0).stop 2023-11-24T06:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.214+0000 7f3daf0ca640 1 -- 172.21.15.16:0/2636257248 shutdown_connections 2023-11-24T06:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.214+0000 7f3daf0ca640 1 --2- 172.21.15.16:0/2636257248 >> v2:172.21.15.173:3300/0 conn(0x7f3da80cf9f0 0x7f3da80cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.214+0000 7f3daf0ca640 1 --2- 172.21.15.16:0/2636257248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da8160920 0x7f3da80b3f30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.214+0000 7f3daf0ca640 1 -- 172.21.15.16:0/2636257248 >> 172.21.15.16:0/2636257248 conn(0x7f3da80d3090 msgr2=0x7f3da80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.215+0000 7f3daf0ca640 1 -- 172.21.15.16:0/2636257248 shutdown_connections 2023-11-24T06:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.215+0000 7f3daf0ca640 1 -- 172.21.15.16:0/2636257248 wait complete. 2023-11-24T06:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.218+0000 7f3daf0ca640 1 Processor -- start 2023-11-24T06:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.218+0000 7f3daf0ca640 1 -- start start 2023-11-24T06:26:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.219+0000 7f3daf0ca640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da80cf9f0 0x7f3da8154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.219+0000 7f3daf0ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da8160920 0x7f3da8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.220+0000 7f3daf0ca640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3da8161010 0x7f3da8159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.220+0000 7f3daf0ca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da81576d0 con 0x7f3da8160920 2023-11-24T06:26:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.220+0000 7f3daf0ca640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8154b80 con 0x7f3da8161010 2023-11-24T06:26:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.220+0000 7f3daf0ca640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8156630 con 0x7f3da80cf9f0 2023-11-24T06:26:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.220+0000 7f3dad8c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da8160920 0x7f3da8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.220+0000 7f3dae0c8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da80cf9f0 0x7f3da8154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.220+0000 7f3dad8c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da8160920 0x7f3da8155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40478/0 (socket says 172.21.15.16:40478) 2023-11-24T06:26:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.220+0000 7f3dad8c7640 1 -- 172.21.15.16:0/1568521377 learned_addr learned my addr 172.21.15.16:0/1568521377 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.220+0000 7f3dae8c9640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3da8161010 0x7f3da8159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3dad8c7640 1 -- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da80cf9f0 msgr2=0x7f3da8154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3dad8c7640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da80cf9f0 0x7f3da8154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3dad8c7640 1 -- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3da8161010 msgr2=0x7f3da8159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3dad8c7640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3da8161010 0x7f3da8159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3dad8c7640 1 -- 172.21.15.16:0/1568521377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da4009b30 con 0x7f3da8160920 2023-11-24T06:26:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3dae0c8640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da80cf9f0 0x7f3da8154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:26:55.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3dae8c9640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3da8161010 0x7f3da8159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:26:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3dad8c7640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da8160920 0x7f3da8155d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800d800 tx=0x7f3d9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3d9effd640 1 -- 172.21.15.16:0/1568521377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800b6c0 con 0x7f3da8160920 2023-11-24T06:26:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3d9effd640 1 -- 172.21.15.16:0/1568521377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9800b820 con 0x7f3da8160920 2023-11-24T06:26:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3d9effd640 1 -- 172.21.15.16:0/1568521377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800bbb0 con 0x7f3da8160920 2023-11-24T06:26:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.221+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da8156910 con 0x7f3da8160920 2023-11-24T06:26:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.222+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da8156ed0 con 0x7f3da8160920 2023-11-24T06:26:55.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.223+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da8160180 con 0x7f3da8160920 2023-11-24T06:26:55.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.224+0000 7f3d9effd640 1 -- 172.21.15.16:0/1568521377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d9800bd10 con 0x7f3da8160920 2023-11-24T06:26:55.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.225+0000 7f3d9effd640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d8c06ae90 0x7f3d8c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:55.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.225+0000 7f3d9effd640 1 -- 172.21.15.16:0/1568521377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f3d98089800 con 0x7f3da8160920 2023-11-24T06:26:55.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.226+0000 7f3dae0c8640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d8c06ae90 0x7f3d8c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:55.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.226+0000 7f3dae0c8640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d8c06ae90 0x7f3d8c06d350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3da4035000 tx=0x7f3da4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:55.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.231+0000 7f3d9effd640 1 -- 172.21.15.16:0/1568521377 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d98054bb0 con 0x7f3da8160920 2023-11-24T06:26:55.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.430+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3da8067450 con 0x7f3da8160920 2023-11-24T06:26:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.969+0000 7f3d9effd640 1 -- 172.21.15.16:0/1568521377 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3d980581d0 con 0x7f3da8160920 2023-11-24T06:26:55.973 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-24T06:26:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d8c06ae90 msgr2=0x7f3d8c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d8c06ae90 0x7f3d8c06d350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3da4035000 tx=0x7f3da4030000 comp rx=0 tx=0).stop 2023-11-24T06:26:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da8160920 msgr2=0x7f3da8155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:55.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da8160920 0x7f3da8155d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800d800 tx=0x7f3d9800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 shutdown_connections 2023-11-24T06:26:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d8c06ae90 0x7f3d8c06d350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3da8161010 0x7f3da8159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da8160920 0x7f3da8155d90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 --2- 172.21.15.16:0/1568521377 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3da80cf9f0 0x7f3da8154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:55.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.974+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 >> 172.21.15.16:0/1568521377 conn(0x7f3da80d3090 msgr2=0x7f3da80d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.981+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 shutdown_connections 2023-11-24T06:26:55.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:55.981+0000 7f3daf0ca640 1 -- 172.21.15.16:0/1568521377 wait complete. 2023-11-24T06:26:56.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:26:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.206+0000 7f83b4940640 1 Processor -- start 2023-11-24T06:26:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.207+0000 7f83b4940640 1 -- start start 2023-11-24T06:26:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.208+0000 7f83b4940640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.208+0000 7f83b4940640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f83b00cf9a0 0x7f83b00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:56.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.208+0000 7f83b4940640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b00d0320 con 0x7f83b00cddf0 2023-11-24T06:26:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.208+0000 7f83b4940640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b00b2730 con 0x7f83b00cf9a0 2023-11-24T06:26:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.208+0000 7f83b4940640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00d1ec0 con 0x7f83b015d4b0 2023-11-24T06:26:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.209+0000 7f83aed9d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.209+0000 7f83aed9d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:35398/0 (socket says 172.21.15.16:35398) 2023-11-24T06:26:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.209+0000 7f83aed9d640 1 -- 172.21.15.16:0/1120120784 learned_addr learned my addr 172.21.15.16:0/1120120784 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.209+0000 7f83ae59c640 1 --2- 172.21.15.16:0/1120120784 >> v2:172.21.15.173:3300/0 conn(0x7f83b00cf9a0 0x7f83b00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.209+0000 7f83aed9d640 1 -- 172.21.15.16:0/1120120784 >> v2:172.21.15.173:3300/0 conn(0x7f83b00cf9a0 msgr2=0x7f83b00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.209+0000 7f83aed9d640 1 --2- 172.21.15.16:0/1120120784 >> v2:172.21.15.173:3300/0 conn(0x7f83b00cf9a0 0x7f83b00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.209+0000 7f83aed9d640 1 -- 172.21.15.16:0/1120120784 >> v1:172.21.15.173:6789/0 conn(0x7f83b00cddf0 legacy=0x7f83b00ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.209+0000 7f83aed9d640 1 -- 172.21.15.16:0/1120120784 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b00b29d0 con 0x7f83b015d4b0 2023-11-24T06:26:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.211+0000 7f83aed9d640 1 --2- 172.21.15.16:0/1120120784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00d1550 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009e80 tx=0x7f83a002bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16645edbd5c1605 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.211+0000 7f83add9b640 1 -- 172.21.15.16:0/1120120784 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a002cae0 con 0x7f83b015d4b0 2023-11-24T06:26:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.211+0000 7f83add9b640 1 -- 172.21.15.16:0/1120120784 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a002cc40 con 0x7f83b015d4b0 2023-11-24T06:26:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.211+0000 7f83add9b640 1 -- 172.21.15.16:0/1120120784 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a00343f0 con 0x7f83b015d4b0 2023-11-24T06:26:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.212+0000 7f83b4940640 1 -- 172.21.15.16:0/1120120784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 msgr2=0x7f83b00d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.212+0000 7f83b4940640 1 --2- 172.21.15.16:0/1120120784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00d1550 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009e80 tx=0x7f83a002bf40 comp rx=0 tx=0).stop 2023-11-24T06:26:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.212+0000 7f83b4940640 1 -- 172.21.15.16:0/1120120784 shutdown_connections 2023-11-24T06:26:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.212+0000 7f83b4940640 1 --2- 172.21.15.16:0/1120120784 >> v2:172.21.15.173:3300/0 conn(0x7f83b00cf9a0 0x7f83b00cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.212+0000 7f83b4940640 1 --2- 172.21.15.16:0/1120120784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00d1550 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:56.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.212+0000 7f83b4940640 1 -- 172.21.15.16:0/1120120784 >> 172.21.15.16:0/1120120784 conn(0x7f83b00d3000 msgr2=0x7f83b00d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:56.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.212+0000 7f83b4940640 1 -- 172.21.15.16:0/1120120784 shutdown_connections 2023-11-24T06:26:56.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.212+0000 7f83b4940640 1 -- 172.21.15.16:0/1120120784 wait complete. 2023-11-24T06:26:56.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.214+0000 7f83b4940640 1 Processor -- start 2023-11-24T06:26:56.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.214+0000 7f83b4940640 1 -- start start 2023-11-24T06:26:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.215+0000 7f83b4940640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b00cddf0 0x7f83b00caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.215+0000 7f83b4940640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83b00cf9a0 0x7f83b01f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:56.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.215+0000 7f83b4940640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.215+0000 7f83b4940640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01f2c30 con 0x7f83b00cddf0 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.215+0000 7f83b4940640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00cd140 con 0x7f83b00cf9a0 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.215+0000 7f83b4940640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00cb170 con 0x7f83b015d4b0 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.215+0000 7f83aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b00cddf0 0x7f83b00caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b00cddf0 0x7f83b00caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40492/0 (socket says 172.21.15.16:40492) 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83aed9d640 1 -- 172.21.15.16:0/3393055689 learned_addr learned my addr 172.21.15.16:0/3393055689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83ae59c640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83b00cf9a0 0x7f83b01f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83af59e640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83aed9d640 1 -- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 msgr2=0x7f83b00ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83aed9d640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83aed9d640 1 -- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83b00cf9a0 msgr2=0x7f83b01f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83aed9d640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83b00cf9a0 0x7f83b01f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83aed9d640 1 -- 172.21.15.16:0/3393055689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009b30 con 0x7f83b00cddf0 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.216+0000 7f83aed9d640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b00cddf0 0x7f83b00caae0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83a0000f80 tx=0x7f83a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.217+0000 7f83a77fe640 1 -- 172.21.15.16:0/3393055689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0040670 con 0x7f83b00cddf0 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.217+0000 7f83a77fe640 1 -- 172.21.15.16:0/3393055689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a00407d0 con 0x7f83b00cddf0 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.217+0000 7f83a77fe640 1 -- 172.21.15.16:0/3393055689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0040b80 con 0x7f83b00cddf0 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.217+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b015da10 con 0x7f83b00cddf0 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.217+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b00cb730 con 0x7f83b00cddf0 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.218+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b0067450 con 0x7f83b00cddf0 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.219+0000 7f83a77fe640 1 -- 172.21.15.16:0/3393055689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f83a0040ce0 con 0x7f83b00cddf0 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.220+0000 7f83a77fe640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f839006ae40 0x7f839006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:56.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.220+0000 7f83a77fe640 1 -- 172.21.15.16:0/3393055689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f83a00ac580 con 0x7f83b00cddf0 2023-11-24T06:26:56.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.221+0000 7f83ae59c640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f839006ae40 0x7f839006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:56.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.222+0000 7f83ae59c640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f839006ae40 0x7f839006d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f83b015c590 tx=0x7f8398008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:56.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.225+0000 7f83a77fe640 1 -- 172.21.15.16:0/3393055689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83a00777c0 con 0x7f83b00cddf0 2023-11-24T06:26:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.459+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f83b00c9b60 con 0x7f83b00cddf0 2023-11-24T06:26:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.978+0000 7f83a77fe640 1 -- 172.21.15.16:0/3393055689 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f83a0009510 con 0x7f83b00cddf0 2023-11-24T06:26:56.982 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T06:26:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.982+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f839006ae40 msgr2=0x7f839006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.982+0000 7f83b4940640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f839006ae40 0x7f839006d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f83b015c590 tx=0x7f8398008040 comp rx=0 tx=0).stop 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.982+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b00cddf0 msgr2=0x7f83b00caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.982+0000 7f83b4940640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b00cddf0 0x7f83b00caae0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83a0000f80 tx=0x7f83a0004a40 comp rx=0 tx=0).stop 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.983+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 shutdown_connections 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.983+0000 7f83b4940640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f839006ae40 0x7f839006d300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.983+0000 7f83b4940640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83b015d4b0 0x7f83b00ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.983+0000 7f83b4940640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83b00cf9a0 0x7f83b01f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.983+0000 7f83b4940640 1 --2- 172.21.15.16:0/3393055689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b00cddf0 0x7f83b00caae0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.983+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 >> 172.21.15.16:0/3393055689 conn(0x7f83b00d3000 msgr2=0x7f83b006c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.983+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 shutdown_connections 2023-11-24T06:26:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:56.983+0000 7f83b4940640 1 -- 172.21.15.16:0/3393055689 wait complete. 2023-11-24T06:26:57.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:26:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.195+0000 7ffbdeeda640 1 Processor -- start 2023-11-24T06:26:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.195+0000 7ffbdeeda640 1 -- start start 2023-11-24T06:26:57.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.197+0000 7ffbdeeda640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffbd80cc500 0x7ffbd80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.197+0000 7ffbdeeda640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd8070e50 con 0x7ffbd806d160 2023-11-24T06:26:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.197+0000 7ffbdeeda640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd8144020 con 0x7ffbd8143c40 2023-11-24T06:26:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.197+0000 7ffbdeeda640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd80723c0 con 0x7ffbd80cc500 2023-11-24T06:26:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.197+0000 7ffbdd6d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffbd80cc500 0x7ffbd80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.197+0000 7ffbde6d9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffbd806d160 0x7ffbd806d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34978/0 (socket says 172.21.15.16:34978) 2023-11-24T06:26:57.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.197+0000 7ffbde6d9640 1 -- 172.21.15.16:0/426418948 learned_addr learned my addr 172.21.15.16:0/426418948 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:26:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.198+0000 7ffbdd6d7640 1 -- 172.21.15.16:0/426418948 >> v1:172.21.15.173:6789/0 conn(0x7ffbd8143c40 legacy=0x7ffbd8071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.198+0000 7ffbdd6d7640 1 -- 172.21.15.16:0/426418948 >> v1:172.21.15.16:6789/0 conn(0x7ffbd806d160 legacy=0x7ffbd806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.198+0000 7ffbdd6d7640 1 -- 172.21.15.16:0/426418948 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8071e00 con 0x7ffbd80cc500 2023-11-24T06:26:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.199+0000 7ffbdd6d7640 1 --2- 172.21.15.16:0/426418948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd80cc500 0x7ffbd80cc990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80097a0 tx=0x7ffbc802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770cc3c071ddd981 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.199+0000 7ffbdced6640 1 -- 172.21.15.16:0/426418948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc802e070 con 0x7ffbd80cc500 2023-11-24T06:26:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.199+0000 7ffbdced6640 1 -- 172.21.15.16:0/426418948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc80315a0 con 0x7ffbd80cc500 2023-11-24T06:26:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.199+0000 7ffbdced6640 1 -- 172.21.15.16:0/426418948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8031930 con 0x7ffbd80cc500 2023-11-24T06:26:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.200+0000 7ffbdeeda640 1 -- 172.21.15.16:0/426418948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd80cc500 msgr2=0x7ffbd80cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.200+0000 7ffbdeeda640 1 --2- 172.21.15.16:0/426418948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd80cc500 0x7ffbd80cc990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80097a0 tx=0x7ffbc802b840 comp rx=0 tx=0).stop 2023-11-24T06:26:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.200+0000 7ffbdeeda640 1 -- 172.21.15.16:0/426418948 shutdown_connections 2023-11-24T06:26:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.200+0000 7ffbdeeda640 1 --2- 172.21.15.16:0/426418948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd80cc500 0x7ffbd80cc990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.200+0000 7ffbdeeda640 1 -- 172.21.15.16:0/426418948 >> 172.21.15.16:0/426418948 conn(0x7ffbd806a800 msgr2=0x7ffbd806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.200+0000 7ffbdeeda640 1 -- 172.21.15.16:0/426418948 shutdown_connections 2023-11-24T06:26:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.200+0000 7ffbdeeda640 1 -- 172.21.15.16:0/426418948 wait complete. 2023-11-24T06:26:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.202+0000 7ffbdeeda640 1 Processor -- start 2023-11-24T06:26:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.202+0000 7ffbdeeda640 1 -- start start 2023-11-24T06:26:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.203+0000 7ffbdeeda640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbd806d160 0x7ffbd813c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.203+0000 7ffbdeeda640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd80cc500 0x7ffbd813ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.203+0000 7ffbdeeda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd8143c40 0x7ffbd80aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.203+0000 7ffbdeeda640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd813e3d0 con 0x7ffbd8143c40 2023-11-24T06:26:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.203+0000 7ffbdeeda640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd813d7c0 con 0x7ffbd80cc500 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.203+0000 7ffbdeeda640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd813daa0 con 0x7ffbd806d160 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.203+0000 7ffbde6d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd8143c40 0x7ffbd80aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.203+0000 7ffbde6d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd8143c40 0x7ffbd80aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40506/0 (socket says 172.21.15.16:40506) 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.203+0000 7ffbde6d9640 1 -- 172.21.15.16:0/128795734 learned_addr learned my addr 172.21.15.16:0/128795734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbdd6d7640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd80cc500 0x7ffbd813ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbdded8640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbd806d160 0x7ffbd813c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbde6d9640 1 -- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbd806d160 msgr2=0x7ffbd813c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbde6d9640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbd806d160 0x7ffbd813c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbde6d9640 1 -- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd80cc500 msgr2=0x7ffbd813ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbde6d9640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd80cc500 0x7ffbd813ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbde6d9640 1 -- 172.21.15.16:0/128795734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8009400 con 0x7ffbd8143c40 2023-11-24T06:26:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbdd6d7640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd80cc500 0x7ffbd813ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbdded8640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbd806d160 0x7ffbd813c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbde6d9640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd8143c40 0x7ffbd80aed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000ca80 tx=0x7ffbd000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbce7fc640 1 -- 172.21.15.16:0/128795734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd000ac30 con 0x7ffbd8143c40 2023-11-24T06:26:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbce7fc640 1 -- 172.21.15.16:0/128795734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd000ad90 con 0x7ffbd8143c40 2023-11-24T06:26:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbce7fc640 1 -- 172.21.15.16:0/128795734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0012600 con 0x7ffbd8143c40 2023-11-24T06:26:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.204+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd813dd80 con 0x7ffbd8143c40 2023-11-24T06:26:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.205+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd813cbc0 con 0x7ffbd8143c40 2023-11-24T06:26:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.206+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd8070710 con 0x7ffbd8143c40 2023-11-24T06:26:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.207+0000 7ffbce7fc640 1 -- 172.21.15.16:0/128795734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffbd0007070 con 0x7ffbd8143c40 2023-11-24T06:26:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.208+0000 7ffbce7fc640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffbbc06ae40 0x7ffbbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.208+0000 7ffbce7fc640 1 -- 172.21.15.16:0/128795734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbd00894c0 con 0x7ffbd8143c40 2023-11-24T06:26:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.208+0000 7ffbdded8640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffbbc06ae40 0x7ffbbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.209+0000 7ffbdded8640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffbbc06ae40 0x7ffbbc06d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4003eb0 tx=0x7ffbd4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:57.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.212+0000 7ffbce7fc640 1 -- 172.21.15.16:0/128795734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffbd0054580 con 0x7ffbd8143c40 2023-11-24T06:26:57.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.404+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbd8067450 con 0x7ffbd8143c40 2023-11-24T06:26:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.406+0000 7ffbce7fc640 1 -- 172.21.15.16:0/128795734 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16518 (secure 0 0 0) 0x7ffbd0057ba0 con 0x7ffbd8143c40 2023-11-24T06:26:57.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:26:57.409 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:56.460138+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:26:57.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffbbc06ae40 msgr2=0x7ffbbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffbbc06ae40 0x7ffbbc06d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4003eb0 tx=0x7ffbd4009040 comp rx=0 tx=0).stop 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd8143c40 msgr2=0x7ffbd80aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd8143c40 0x7ffbd80aed50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000ca80 tx=0x7ffbd000cf40 comp rx=0 tx=0).stop 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 shutdown_connections 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffbbc06ae40 0x7ffbbc06d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbd8143c40 0x7ffbd80aed50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbd80cc500 0x7ffbd813ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 --2- 172.21.15.16:0/128795734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbd806d160 0x7ffbd813c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 >> 172.21.15.16:0/128795734 conn(0x7ffbd806a800 msgr2=0x7ffbd80b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 shutdown_connections 2023-11-24T06:26:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.409+0000 7ffbdeeda640 1 -- 172.21.15.16:0/128795734 wait complete. 2023-11-24T06:26:57.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:26:57.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.645+0000 7fb736cd1640 1 Processor -- start 2023-11-24T06:26:57.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.646+0000 7fb736cd1640 1 -- start start 2023-11-24T06:26:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.647+0000 7fb736cd1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb73015d3c0 0x7fb7300d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.647+0000 7fb736cd1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7300cfa40 0x7fb7300cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.647+0000 7fb736cd1640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7300b2c70 con 0x7fb7300cde90 2023-11-24T06:26:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.647+0000 7fb736cd1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300d03c0 con 0x7fb7300cfa40 2023-11-24T06:26:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.647+0000 7fb736cd1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300cf6d0 con 0x7fb73015d3c0 2023-11-24T06:26:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.648+0000 7fb7354ce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7300cfa40 0x7fb7300cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.648+0000 7fb7354ce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7300cfa40 0x7fb7300cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40518/0 (socket says 172.21.15.16:40518) 2023-11-24T06:26:57.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.648+0000 7fb7354ce640 1 -- 172.21.15.16:0/3666726292 learned_addr learned my addr 172.21.15.16:0/3666726292 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.648+0000 7fb735ccf640 1 --2- 172.21.15.16:0/3666726292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb73015d3c0 0x7fb7300d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.648+0000 7fb7354ce640 1 -- 172.21.15.16:0/3666726292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb73015d3c0 msgr2=0x7fb7300d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.648+0000 7fb7354ce640 1 --2- 172.21.15.16:0/3666726292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb73015d3c0 0x7fb7300d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.648+0000 7fb7354ce640 1 -- 172.21.15.16:0/3666726292 >> v1:172.21.15.173:6789/0 conn(0x7fb7300cde90 legacy=0x7fb7300ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.648+0000 7fb7354ce640 1 -- 172.21.15.16:0/3666726292 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300cf3f0 con 0x7fb7300cfa40 2023-11-24T06:26:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.649+0000 7fb7354ce640 1 --2- 172.21.15.16:0/3666726292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cfa40 0x7fb7300cfe80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb720009870 tx=0x7fb72002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a875e4570613aa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:57.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.650+0000 7fb734ccd640 1 -- 172.21.15.16:0/3666726292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720031640 con 0x7fb7300cfa40 2023-11-24T06:26:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.650+0000 7fb734ccd640 1 -- 172.21.15.16:0/3666726292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7200317a0 con 0x7fb7300cfa40 2023-11-24T06:26:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.650+0000 7fb734ccd640 1 -- 172.21.15.16:0/3666726292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720031b30 con 0x7fb7300cfa40 2023-11-24T06:26:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.650+0000 7fb736cd1640 1 -- 172.21.15.16:0/3666726292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cfa40 msgr2=0x7fb7300cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.650+0000 7fb736cd1640 1 --2- 172.21.15.16:0/3666726292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cfa40 0x7fb7300cfe80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb720009870 tx=0x7fb72002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.650+0000 7fb736cd1640 1 -- 172.21.15.16:0/3666726292 shutdown_connections 2023-11-24T06:26:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.650+0000 7fb736cd1640 1 --2- 172.21.15.16:0/3666726292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cfa40 0x7fb7300cfe80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.650+0000 7fb736cd1640 1 --2- 172.21.15.16:0/3666726292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb73015d3c0 0x7fb7300d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.650+0000 7fb736cd1640 1 -- 172.21.15.16:0/3666726292 >> 172.21.15.16:0/3666726292 conn(0x7fb7300d3050 msgr2=0x7fb7300d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.651+0000 7fb736cd1640 1 -- 172.21.15.16:0/3666726292 shutdown_connections 2023-11-24T06:26:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.651+0000 7fb736cd1640 1 -- 172.21.15.16:0/3666726292 wait complete. 2023-11-24T06:26:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.652+0000 7fb736cd1640 1 Processor -- start 2023-11-24T06:26:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.652+0000 7fb736cd1640 1 -- start start 2023-11-24T06:26:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.653+0000 7fb736cd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cde90 0x7fb73014a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.653+0000 7fb736cd1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7300cfa40 0x7fb7300b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.653+0000 7fb736cd1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb73015d3c0 0x7fb7300cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.653+0000 7fb736cd1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300ca4e0 con 0x7fb7300cde90 2023-11-24T06:26:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.653+0000 7fb736cd1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300ca7c0 con 0x7fb73015d3c0 2023-11-24T06:26:57.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.653+0000 7fb736cd1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300caaa0 con 0x7fb7300cfa40 2023-11-24T06:26:57.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb735ccf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cde90 0x7fb73014a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb735ccf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cde90 0x7fb73014a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40526/0 (socket says 172.21.15.16:40526) 2023-11-24T06:26:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb735ccf640 1 -- 172.21.15.16:0/2101145742 learned_addr learned my addr 172.21.15.16:0/2101145742 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb735ccf640 1 -- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7300cfa40 msgr2=0x7fb7300b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb7354ce640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7300cfa40 0x7fb7300b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb735ccf640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7300cfa40 0x7fb7300b8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb735ccf640 1 -- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb73015d3c0 msgr2=0x7fb7300cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb7364d0640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb73015d3c0 0x7fb7300cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb735ccf640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb73015d3c0 0x7fb7300cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.654+0000 7fb735ccf640 1 -- 172.21.15.16:0/2101145742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200094d0 con 0x7fb7300cde90 2023-11-24T06:26:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.655+0000 7fb735ccf640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cde90 0x7fb73014a680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00d5f0 tx=0x7fb72c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.655+0000 7fb7267fc640 1 -- 172.21.15.16:0/2101145742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0125f0 con 0x7fb7300cde90 2023-11-24T06:26:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.655+0000 7fb7267fc640 1 -- 172.21.15.16:0/2101145742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c012750 con 0x7fb7300cde90 2023-11-24T06:26:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.655+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7300cad80 con 0x7fb7300cde90 2023-11-24T06:26:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.655+0000 7fb7267fc640 1 -- 172.21.15.16:0/2101145742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c012b00 con 0x7fb7300cde90 2023-11-24T06:26:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.655+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7300ca120 con 0x7fb7300cde90 2023-11-24T06:26:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.656+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73008ba80 con 0x7fb7300cde90 2023-11-24T06:26:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.657+0000 7fb7267fc640 1 -- 172.21.15.16:0/2101145742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb72c012c60 con 0x7fb7300cde90 2023-11-24T06:26:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.658+0000 7fb7267fc640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb71406ae40 0x7fb71406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.658+0000 7fb7267fc640 1 -- 172.21.15.16:0/2101145742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb72c089ba0 con 0x7fb7300cde90 2023-11-24T06:26:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.659+0000 7fb7354ce640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb71406ae40 0x7fb71406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.660+0000 7fb7354ce640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb71406ae40 0x7fb71406d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb720036000 tx=0x7fb72003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.663+0000 7fb7267fc640 1 -- 172.21.15.16:0/2101145742 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb72c054bb0 con 0x7fb7300cde90 2023-11-24T06:26:57.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.856+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb730067450 con 0x7fb7300cde90 2023-11-24T06:26:57.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.858+0000 7fb7267fc640 1 -- 172.21.15.16:0/2101145742 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16518 (secure 0 0 0) 0x7fb72c0581d0 con 0x7fb7300cde90 2023-11-24T06:26:57.861 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:26:57.861 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:56.460138+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:26:57.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.861+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb71406ae40 msgr2=0x7fb71406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.861+0000 7fb736cd1640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb71406ae40 0x7fb71406d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb720036000 tx=0x7fb72003c040 comp rx=0 tx=0).stop 2023-11-24T06:26:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.861+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cde90 msgr2=0x7fb73014a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.861+0000 7fb736cd1640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cde90 0x7fb73014a680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00d5f0 tx=0x7fb72c00dab0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.861+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 shutdown_connections 2023-11-24T06:26:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.861+0000 7fb736cd1640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb71406ae40 0x7fb71406d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.861+0000 7fb736cd1640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb73015d3c0 0x7fb7300cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.861+0000 7fb736cd1640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7300cfa40 0x7fb7300b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.861+0000 7fb736cd1640 1 --2- 172.21.15.16:0/2101145742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7300cde90 0x7fb73014a680 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:57.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.862+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 >> 172.21.15.16:0/2101145742 conn(0x7fb7300d3050 msgr2=0x7fb73006b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.862+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 shutdown_connections 2023-11-24T06:26:57.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:57.862+0000 7fb736cd1640 1 -- 172.21.15.16:0/2101145742 wait complete. 2023-11-24T06:26:57.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:26:58.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.110+0000 7f7bc906e640 1 Processor -- start 2023-11-24T06:26:58.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.110+0000 7f7bc906e640 1 -- start start 2023-11-24T06:26:58.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.111+0000 7f7bc906e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7bc415d4b0 0x7f7bc40d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:58.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.111+0000 7f7bc906e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7bc40cf9a0 0x7f7bc40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.111+0000 7f7bc906e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc40d0320 con 0x7f7bc40cddf0 2023-11-24T06:26:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.111+0000 7f7bc906e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc40b2730 con 0x7f7bc415d4b0 2023-11-24T06:26:58.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.111+0000 7f7bc906e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc40d1ec0 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.112+0000 7f7bc886d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7bc40cddf0 0x7f7bc40ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34994/0 (socket says 172.21.15.16:34994) 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.112+0000 7f7bc886d640 1 -- 172.21.15.16:0/28813129 learned_addr learned my addr 172.21.15.16:0/28813129 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.112+0000 7f7bc3fff640 1 --2- 172.21.15.16:0/28813129 >> v2:172.21.15.16:3300/0 conn(0x7f7bc415d4b0 0x7f7bc40d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.112+0000 7f7bc37fe640 1 --2- 172.21.15.16:0/28813129 >> v2:172.21.15.173:3300/0 conn(0x7f7bc40cf9a0 0x7f7bc40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.112+0000 7f7bc2ffd640 1 -- 172.21.15.16:0/28813129 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280040658 0 0) 0x7f7bc40d0320 con 0x7f7bc40cddf0 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.112+0000 7f7bc2ffd640 1 -- 172.21.15.16:0/28813129 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb0003520 con 0x7f7bc40cddf0 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.113+0000 7f7bc3fff640 1 -- 172.21.15.16:0/28813129 >> v2:172.21.15.173:3300/0 conn(0x7f7bc40cf9a0 msgr2=0x7f7bc40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.113+0000 7f7bc3fff640 1 --2- 172.21.15.16:0/28813129 >> v2:172.21.15.173:3300/0 conn(0x7f7bc40cf9a0 0x7f7bc40cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.113+0000 7f7bc3fff640 1 -- 172.21.15.16:0/28813129 >> v1:172.21.15.16:6789/0 conn(0x7f7bc40cddf0 legacy=0x7f7bc40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:58.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.113+0000 7f7bc3fff640 1 -- 172.21.15.16:0/28813129 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc40b29d0 con 0x7f7bc415d4b0 2023-11-24T06:26:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.114+0000 7f7bc3fff640 1 --2- 172.21.15.16:0/28813129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc415d4b0 0x7f7bc40d1550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7bc408b580 tx=0x7f7bb402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b69ca8e4f3c38ef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.114+0000 7f7bc2ffd640 1 -- 172.21.15.16:0/28813129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb402cc80 con 0x7f7bc415d4b0 2023-11-24T06:26:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.114+0000 7f7bc2ffd640 1 -- 172.21.15.16:0/28813129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb402cde0 con 0x7f7bc415d4b0 2023-11-24T06:26:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.114+0000 7f7bc2ffd640 1 -- 172.21.15.16:0/28813129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4034660 con 0x7f7bc415d4b0 2023-11-24T06:26:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.115+0000 7f7bc906e640 1 -- 172.21.15.16:0/28813129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc415d4b0 msgr2=0x7f7bc40d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.115+0000 7f7bc906e640 1 --2- 172.21.15.16:0/28813129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc415d4b0 0x7f7bc40d1550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7bc408b580 tx=0x7f7bb402bed0 comp rx=0 tx=0).stop 2023-11-24T06:26:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.115+0000 7f7bc906e640 1 -- 172.21.15.16:0/28813129 shutdown_connections 2023-11-24T06:26:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.115+0000 7f7bc906e640 1 --2- 172.21.15.16:0/28813129 >> v2:172.21.15.173:3300/0 conn(0x7f7bc40cf9a0 0x7f7bc40cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.115+0000 7f7bc906e640 1 --2- 172.21.15.16:0/28813129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc415d4b0 0x7f7bc40d1550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:58.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.115+0000 7f7bc906e640 1 -- 172.21.15.16:0/28813129 >> 172.21.15.16:0/28813129 conn(0x7f7bc40d3000 msgr2=0x7f7bc40d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:58.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.115+0000 7f7bc906e640 1 -- 172.21.15.16:0/28813129 shutdown_connections 2023-11-24T06:26:58.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.115+0000 7f7bc906e640 1 -- 172.21.15.16:0/28813129 wait complete. 2023-11-24T06:26:58.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.117+0000 7f7bc906e640 1 Processor -- start 2023-11-24T06:26:58.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.117+0000 7f7bc906e640 1 -- start start 2023-11-24T06:26:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc906e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7bc40cddf0 0x7f7bc4155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc906e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc40cf9a0 0x7f7bc40cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc906e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc415d4b0 0x7f7bc409de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:58.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc906e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc409db40 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc906e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc409cce0 con 0x7f7bc415d4b0 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc906e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc409cf30 con 0x7f7bc40cddf0 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc40cf9a0 0x7f7bc40cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc40cf9a0 0x7f7bc40cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40550/0 (socket says 172.21.15.16:40550) 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc37fe640 1 -- 172.21.15.16:0/1811328260 learned_addr learned my addr 172.21.15.16:0/1811328260 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc3fff640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7bc40cddf0 0x7f7bc4155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.118+0000 7f7bc37fe640 1 -- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7bc40cddf0 msgr2=0x7f7bc4155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc37fe640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7bc40cddf0 0x7f7bc4155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc37fe640 1 -- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc415d4b0 msgr2=0x7f7bc409de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc37fe640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc415d4b0 0x7f7bc409de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc37fe640 1 -- 172.21.15.16:0/1811328260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4009a70 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc37fe640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc40cf9a0 0x7f7bc40cbe40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00e9b0 tx=0x7f7bac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc0ff9640 1 -- 172.21.15.16:0/1811328260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac009670 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc0ff9640 1 -- 172.21.15.16:0/1811328260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00c9a0 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc0ff9640 1 -- 172.21.15.16:0/1811328260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00cd50 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc415da10 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.119+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc409d4f0 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.121+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc4067450 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.121+0000 7f7bc0ff9640 1 -- 172.21.15.16:0/1811328260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bac013470 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.122+0000 7f7bc0ff9640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ba406ae40 0x7f7ba406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:58.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.122+0000 7f7bc0ff9640 1 -- 172.21.15.16:0/1811328260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bac0898c0 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.123+0000 7f7bc3fff640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ba406ae40 0x7f7ba406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:58.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.124+0000 7f7bc3fff640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ba406ae40 0x7f7ba406d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4036000 tx=0x7f7bb4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:58.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.126+0000 7f7bc0ff9640 1 -- 172.21.15.16:0/1811328260 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bac054960 con 0x7f7bc40cf9a0 2023-11-24T06:26:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:58.363+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7bc415cba0 con 0x7f7bc40cf9a0 2023-11-24T06:26:59.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.007+0000 7f7bc0ff9640 1 -- 172.21.15.16:0/1811328260 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7bac0890b0 con 0x7f7bc40cf9a0 2023-11-24T06:26:59.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ba406ae40 msgr2=0x7f7ba406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ba406ae40 0x7f7ba406d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4036000 tx=0x7f7bb4030000 comp rx=0 tx=0).stop 2023-11-24T06:26:59.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc40cf9a0 msgr2=0x7f7bc40cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc40cf9a0 0x7f7bc40cbe40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00e9b0 tx=0x7f7bac00ee70 comp rx=0 tx=0).stop 2023-11-24T06:26:59.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 shutdown_connections 2023-11-24T06:26:59.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ba406ae40 0x7f7ba406d300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc415d4b0 0x7f7bc409de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bc40cf9a0 0x7f7bc40cbe40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 --2- 172.21.15.16:0/1811328260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7bc40cddf0 0x7f7bc4155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.010+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 >> 172.21.15.16:0/1811328260 conn(0x7f7bc40d3000 msgr2=0x7f7bc40cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:59.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.011+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 shutdown_connections 2023-11-24T06:26:59.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.011+0000 7f7bc906e640 1 -- 172.21.15.16:0/1811328260 wait complete. 2023-11-24T06:26:59.033 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:26:59.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:26:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.260+0000 7fc363163640 1 Processor -- start 2023-11-24T06:26:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.260+0000 7fc363163640 1 -- start start 2023-11-24T06:26:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.262+0000 7fc363163640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc35c0cf9a0 0x7fc35c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.262+0000 7fc363163640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b31d0 0x7fc35c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.262+0000 7fc363163640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35c0d0320 con 0x7fc35c15d4b0 2023-11-24T06:26:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.262+0000 7fc363163640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc35c0d1c60 con 0x7fc35c0cf9a0 2023-11-24T06:26:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.262+0000 7fc363163640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0b2730 con 0x7fc35c0b31d0 2023-11-24T06:26:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.262+0000 7fc362161640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc35c15d4b0 0x7fc35c0d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34998/0 (socket says 172.21.15.16:34998) 2023-11-24T06:26:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.262+0000 7fc362161640 1 -- 172.21.15.16:0/2884534853 learned_addr learned my addr 172.21.15.16:0/2884534853 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:26:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc362962640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b31d0 0x7fc35c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc361960640 1 --2- 172.21.15.16:0/2884534853 >> v2:172.21.15.173:3300/0 conn(0x7fc35c0cf9a0 0x7fc35c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698942010 0 0) 0x7fc35c0d0320 con 0x7fc35c15d4b0 2023-11-24T06:26:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc348003520 con 0x7fc35c15d4b0 2023-11-24T06:26:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 582851487 0 0) 0x7fc348003520 con 0x7fc35c15d4b0 2023-11-24T06:26:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b31d0 msgr2=0x7fc35c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b31d0 0x7fc35c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 >> v2:172.21.15.173:3300/0 conn(0x7fc35c0cf9a0 msgr2=0x7fc35c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 --2- 172.21.15.16:0/2884534853 >> v2:172.21.15.173:3300/0 conn(0x7fc35c0cf9a0 0x7fc35c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0d1ec0 con 0x7fc35c15d4b0 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fc358001670 con 0x7fc35c15d4b0 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 >> v1:172.21.15.16:6789/0 conn(0x7fc35c15d4b0 legacy=0x7fc35c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.263+0000 7fc36115f640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc348003970 0x7fc348003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc36115f640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc348004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc36115f640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b31d0 0x7fc348005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0d1c60 con 0x7fc3480047c0 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c0b2730 con 0x7fc348003970 2023-11-24T06:26:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34800b980 con 0x7fc35c0b31d0 2023-11-24T06:26:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc361960640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc348004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc362161640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc348003970 0x7fc348003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc362962640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b31d0 0x7fc348005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc361960640 1 -- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b31d0 msgr2=0x7fc348005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc361960640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b31d0 0x7fc348005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc361960640 1 -- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc348003970 msgr2=0x7fc348003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:26:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc361960640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc348003970 0x7fc348003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc361960640 1 -- 172.21.15.16:0/2884534853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc348003520 con 0x7fc3480047c0 2023-11-24T06:26:59.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.264+0000 7fc362962640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b31d0 0x7fc348005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:26:59.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.265+0000 7fc361960640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc348004c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009ac0 tx=0x7fc3480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c823a030154fed2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c02c710 con 0x7fc3480047c0 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c02c870 con 0x7fc3480047c0 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc36115f640 1 -- 172.21.15.16:0/2884534853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c02cc00 con 0x7fc3480047c0 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc363163640 1 -- 172.21.15.16:0/2884534853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 msgr2=0x7fc348004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc363163640 1 --2- 172.21.15.16:0/2884534853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc348004c40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009ac0 tx=0x7fc3480057f0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc362962640 1 -- 172.21.15.16:0/2884534853 reap_dead start 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc363163640 1 -- 172.21.15.16:0/2884534853 shutdown_connections 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc363163640 1 -- 172.21.15.16:0/2884534853 >> 172.21.15.16:0/2884534853 conn(0x7fc35c0d3000 msgr2=0x7fc35c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc363163640 1 -- 172.21.15.16:0/2884534853 shutdown_connections 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.266+0000 7fc363163640 1 -- 172.21.15.16:0/2884534853 wait complete. 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.268+0000 7fc363163640 1 Processor -- start 2023-11-24T06:26:59.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.268+0000 7fc363163640 1 -- start start 2023-11-24T06:26:59.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.269+0000 7fc363163640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc35c151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.269+0000 7fc363163640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc35c0cc810 0x7fc35c1f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.269+0000 7fc363163640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b49d0 0x7fc35c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.269+0000 7fc363163640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c152230 con 0x7fc3480047c0 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.269+0000 7fc363163640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c1f6f90 con 0x7fc35c0cc810 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.269+0000 7fc363163640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c1f7270 con 0x7fc35c0b49d0 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362161640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc35c151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362161640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc35c151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40566/0 (socket says 172.21.15.16:40566) 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362161640 1 -- 172.21.15.16:0/1920242637 learned_addr learned my addr 172.21.15.16:0/1920242637 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc361960640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc35c0cc810 0x7fc35c1f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362962640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b49d0 0x7fc35c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362161640 1 -- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b49d0 msgr2=0x7fc35c0b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362161640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b49d0 0x7fc35c0b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362161640 1 -- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc35c0cc810 msgr2=0x7fc35c1f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:26:59.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362161640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc35c0cc810 0x7fc35c1f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:26:59.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362161640 1 -- 172.21.15.16:0/1920242637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c1f7550 con 0x7fc3480047c0 2023-11-24T06:26:59.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.270+0000 7fc362161640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc35c151410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc35800e700 tx=0x7fc35800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:59.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.271+0000 7fc352ffd640 1 -- 172.21.15.16:0/1920242637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35800ee10 con 0x7fc3480047c0 2023-11-24T06:26:59.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.271+0000 7fc352ffd640 1 -- 172.21.15.16:0/1920242637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35800c850 con 0x7fc3480047c0 2023-11-24T06:26:59.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.271+0000 7fc352ffd640 1 -- 172.21.15.16:0/1920242637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35800cc00 con 0x7fc3480047c0 2023-11-24T06:26:59.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.271+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c151e50 con 0x7fc3480047c0 2023-11-24T06:26:59.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.271+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c1f6bd0 con 0x7fc3480047c0 2023-11-24T06:26:59.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.272+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35c067450 con 0x7fc3480047c0 2023-11-24T06:26:59.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.273+0000 7fc352ffd640 1 -- 172.21.15.16:0/1920242637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc35801d460 con 0x7fc3480047c0 2023-11-24T06:26:59.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.274+0000 7fc352ffd640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc34006ae40 0x7fc34006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:26:59.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.274+0000 7fc352ffd640 1 -- 172.21.15.16:0/1920242637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3580895a0 con 0x7fc3480047c0 2023-11-24T06:26:59.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.275+0000 7fc361960640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc34006ae40 0x7fc34006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:26:59.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.276+0000 7fc361960640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc34006ae40 0x7fc34006d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc34c036000 tx=0x7fc34c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:26:59.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.278+0000 7fc352ffd640 1 -- 172.21.15.16:0/1920242637 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc358058040 con 0x7fc3480047c0 2023-11-24T06:26:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:26:59.515+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc35c0cdde0 con 0x7fc3480047c0 2023-11-24T06:27:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.010+0000 7fc352ffd640 1 -- 172.21.15.16:0/1920242637 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc358057bd0 con 0x7fc3480047c0 2023-11-24T06:27:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc34006ae40 msgr2=0x7fc34006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc34006ae40 0x7fc34006d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc34c036000 tx=0x7fc34c030000 comp rx=0 tx=0).stop 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 msgr2=0x7fc35c151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc35c151410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc35800e700 tx=0x7fc35800ebc0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 shutdown_connections 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc34006ae40 0x7fc34006d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc35c0b49d0 0x7fc35c0b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc35c0cc810 0x7fc35c1f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 --2- 172.21.15.16:0/1920242637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3480047c0 0x7fc35c151410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 >> 172.21.15.16:0/1920242637 conn(0x7fc35c0d3000 msgr2=0x7fc35c15fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 shutdown_connections 2023-11-24T06:27:00.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.014+0000 7fc363163640 1 -- 172.21.15.16:0/1920242637 wait complete. 2023-11-24T06:27:00.037 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:27:00.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:00.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.270+0000 7fd93c1c9640 1 Processor -- start 2023-11-24T06:27:00.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.270+0000 7fd93c1c9640 1 -- start start 2023-11-24T06:27:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.272+0000 7fd93c1c9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd9340cf9a0 0x7fd934070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.272+0000 7fd93c1c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd934070ea0 0x7fd9340cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.272+0000 7fd93c1c9640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd934071690 con 0x7fd93414b2c0 2023-11-24T06:27:00.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.272+0000 7fd93c1c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93414b6a0 con 0x7fd934070ea0 2023-11-24T06:27:00.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.272+0000 7fd93c1c9640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd934071970 con 0x7fd9340cf9a0 2023-11-24T06:27:00.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.272+0000 7fd93a766640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd934070ea0 0x7fd9340cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.272+0000 7fd93a766640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd934070ea0 0x7fd9340cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40570/0 (socket says 172.21.15.16:40570) 2023-11-24T06:27:00.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.272+0000 7fd93a766640 1 -- 172.21.15.16:0/4167166072 learned_addr learned my addr 172.21.15.16:0/4167166072 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:00.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.273+0000 7fd939764640 1 --2- 172.21.15.16:0/4167166072 >> v2:172.21.15.173:3300/0 conn(0x7fd9340cf9a0 0x7fd934070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.273+0000 7fd93a766640 1 -- 172.21.15.16:0/4167166072 >> v2:172.21.15.173:3300/0 conn(0x7fd9340cf9a0 msgr2=0x7fd934070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.273+0000 7fd93a766640 1 --2- 172.21.15.16:0/4167166072 >> v2:172.21.15.173:3300/0 conn(0x7fd9340cf9a0 0x7fd934070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.273+0000 7fd93a766640 1 -- 172.21.15.16:0/4167166072 >> v1:172.21.15.173:6789/0 conn(0x7fd93414b2c0 legacy=0x7fd9340cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.273+0000 7fd93a766640 1 -- 172.21.15.16:0/4167166072 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd934071c50 con 0x7fd934070ea0 2023-11-24T06:27:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.274+0000 7fd93a766640 1 --2- 172.21.15.16:0/4167166072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 0x7fd9340cc160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9340cfea0 tx=0x7fd92c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff02a98a938afae5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.274+0000 7fd938f63640 1 -- 172.21.15.16:0/4167166072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c02ca80 con 0x7fd934070ea0 2023-11-24T06:27:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.274+0000 7fd938f63640 1 -- 172.21.15.16:0/4167166072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c02cbe0 con 0x7fd934070ea0 2023-11-24T06:27:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.274+0000 7fd938f63640 1 -- 172.21.15.16:0/4167166072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c0343f0 con 0x7fd934070ea0 2023-11-24T06:27:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.275+0000 7fd93c1c9640 1 -- 172.21.15.16:0/4167166072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 msgr2=0x7fd9340cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.275+0000 7fd93c1c9640 1 --2- 172.21.15.16:0/4167166072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 0x7fd9340cc160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9340cfea0 tx=0x7fd92c02bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.275+0000 7fd93c1c9640 1 -- 172.21.15.16:0/4167166072 shutdown_connections 2023-11-24T06:27:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.275+0000 7fd93c1c9640 1 --2- 172.21.15.16:0/4167166072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 0x7fd9340cc160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.275+0000 7fd93c1c9640 1 --2- 172.21.15.16:0/4167166072 >> v2:172.21.15.173:3300/0 conn(0x7fd9340cf9a0 0x7fd934070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.275+0000 7fd93c1c9640 1 -- 172.21.15.16:0/4167166072 >> 172.21.15.16:0/4167166072 conn(0x7fd93406a800 msgr2=0x7fd93406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.275+0000 7fd93c1c9640 1 -- 172.21.15.16:0/4167166072 shutdown_connections 2023-11-24T06:27:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.275+0000 7fd93c1c9640 1 -- 172.21.15.16:0/4167166072 wait complete. 2023-11-24T06:27:00.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.277+0000 7fd93c1c9640 1 Processor -- start 2023-11-24T06:27:00.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.277+0000 7fd93c1c9640 1 -- start start 2023-11-24T06:27:00.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.278+0000 7fd93c1c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 0x7fd9341645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.278+0000 7fd93c1c9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9340cf9a0 0x7fd934161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.278+0000 7fd93c1c9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd93414b2c0 0x7fd934160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.278+0000 7fd93c1c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341627e0 con 0x7fd934070ea0 2023-11-24T06:27:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.278+0000 7fd93c1c9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd934162ac0 con 0x7fd9340cf9a0 2023-11-24T06:27:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.278+0000 7fd93c1c9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd934162da0 con 0x7fd93414b2c0 2023-11-24T06:27:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939f65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 0x7fd9341645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939f65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 0x7fd9341645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40586/0 (socket says 172.21.15.16:40586) 2023-11-24T06:27:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939f65640 1 -- 172.21.15.16:0/2543766928 learned_addr learned my addr 172.21.15.16:0/2543766928 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939f65640 1 -- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd93414b2c0 msgr2=0x7fd934160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939f65640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd93414b2c0 0x7fd934160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939764640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9340cf9a0 0x7fd934161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939f65640 1 -- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9340cf9a0 msgr2=0x7fd934161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939f65640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9340cf9a0 0x7fd934161e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939f65640 1 -- 172.21.15.16:0/2543766928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c009b30 con 0x7fd934070ea0 2023-11-24T06:27:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd939f65640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 0x7fd9341645d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd93000b3f0 tx=0x7fd93000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.279+0000 7fd92a7fc640 1 -- 172.21.15.16:0/2543766928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930011830 con 0x7fd934070ea0 2023-11-24T06:27:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.280+0000 7fd92a7fc640 1 -- 172.21.15.16:0/2543766928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930011990 con 0x7fd934070ea0 2023-11-24T06:27:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.280+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd934163080 con 0x7fd934070ea0 2023-11-24T06:27:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.280+0000 7fd92a7fc640 1 -- 172.21.15.16:0/2543766928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930011d40 con 0x7fd934070ea0 2023-11-24T06:27:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.280+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd934164b10 con 0x7fd934070ea0 2023-11-24T06:27:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.281+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9340675d0 con 0x7fd934070ea0 2023-11-24T06:27:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.282+0000 7fd92a7fc640 1 -- 172.21.15.16:0/2543766928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd93001a400 con 0x7fd934070ea0 2023-11-24T06:27:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.283+0000 7fd92a7fc640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd91806ae40 0x7fd91806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.283+0000 7fd92a7fc640 1 -- 172.21.15.16:0/2543766928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9300894c0 con 0x7fd934070ea0 2023-11-24T06:27:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.284+0000 7fd939764640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd91806ae40 0x7fd91806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.285+0000 7fd939764640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd91806ae40 0x7fd91806d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd9341462f0 tx=0x7fd924008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:00.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.288+0000 7fd92a7fc640 1 -- 172.21.15.16:0/2543766928 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd930054570 con 0x7fd934070ea0 2023-11-24T06:27:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.525+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9340b2a60 con 0x7fd934070ea0 2023-11-24T06:27:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.527+0000 7fd92a7fc640 1 -- 172.21.15.16:0/2543766928 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fd930057b90 con 0x7fd934070ea0 2023-11-24T06:27:00.529 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:00.529 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3019663874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3019663874},{"type":"v1","addr":"172.21.15.173:6837","nonce":3019663874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2239024365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2239024365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2239024365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:26:56.460479+0000","modified":"2023-11-24T06:26:59.516602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/915142029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":915142029},{"type":"v1","addr":"172.21.15.173:6835","nonce":915142029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:27:00.529 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-11-24T06:27:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd91806ae40 msgr2=0x7fd91806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd91806ae40 0x7fd91806d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd9341462f0 tx=0x7fd924008040 comp rx=0 tx=0).stop 2023-11-24T06:27:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 msgr2=0x7fd9341645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 0x7fd9341645d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd93000b3f0 tx=0x7fd93000b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 shutdown_connections 2023-11-24T06:27:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd91806ae40 0x7fd91806d300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd93414b2c0 0x7fd934160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9340cf9a0 0x7fd934161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 --2- 172.21.15.16:0/2543766928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd934070ea0 0x7fd9341645d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 >> 172.21.15.16:0/2543766928 conn(0x7fd93406a800 msgr2=0x7fd9340ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 shutdown_connections 2023-11-24T06:27:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.530+0000 7fd93c1c9640 1 -- 172.21.15.16:0/2543766928 wait complete. 2023-11-24T06:27:00.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.772+0000 7f1066167640 1 Processor -- start 2023-11-24T06:27:00.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.772+0000 7f1066167640 1 -- start start 2023-11-24T06:27:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.774+0000 7f1066167640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106015d3c0 0x7f10600d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.774+0000 7f1066167640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f10600ce9f0 0x7f10600cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.774+0000 7f1066167640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10600d12f0 con 0x7f10600d0e90 2023-11-24T06:27:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.774+0000 7f1066167640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10600ce5d0 con 0x7f10600ce9f0 2023-11-24T06:27:00.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.774+0000 7f1066167640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10600cf390 con 0x7f106015d3c0 2023-11-24T06:27:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.774+0000 7f1065966640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f10600ce9f0 0x7f10600cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.774+0000 7f1065165640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106015d3c0 0x7f10600d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.774+0000 7f1065966640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f10600ce9f0 0x7f10600cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:52824/0 (socket says 172.21.15.16:52824) 2023-11-24T06:27:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.774+0000 7f1065966640 1 -- 172.21.15.16:0/3541711543 learned_addr learned my addr 172.21.15.16:0/3541711543 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.775+0000 7f1065966640 1 -- 172.21.15.16:0/3541711543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106015d3c0 msgr2=0x7f10600d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:00.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.775+0000 7f1065966640 1 --2- 172.21.15.16:0/3541711543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106015d3c0 0x7f10600d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.775+0000 7f1065966640 1 -- 172.21.15.16:0/3541711543 >> v1:172.21.15.173:6789/0 conn(0x7f10600d0e90 legacy=0x7f10600afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.775+0000 7f1065966640 1 -- 172.21.15.16:0/3541711543 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600b23c0 con 0x7f10600ce9f0 2023-11-24T06:27:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.777+0000 7f1065966640 1 --2- 172.21.15.16:0/3541711543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 0x7f10600cee50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1058009e50 tx=0x7f105802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6d0897eeea77a9b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.777+0000 7f1057fff640 1 -- 172.21.15.16:0/3541711543 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105802ca70 con 0x7f10600ce9f0 2023-11-24T06:27:00.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.777+0000 7f1057fff640 1 -- 172.21.15.16:0/3541711543 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105802cbd0 con 0x7f10600ce9f0 2023-11-24T06:27:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.777+0000 7f1057fff640 1 -- 172.21.15.16:0/3541711543 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10580343f0 con 0x7f10600ce9f0 2023-11-24T06:27:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.778+0000 7f1066167640 1 -- 172.21.15.16:0/3541711543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 msgr2=0x7f10600cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.778+0000 7f1066167640 1 --2- 172.21.15.16:0/3541711543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 0x7f10600cee50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1058009e50 tx=0x7f105802bed0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.778+0000 7f1066167640 1 -- 172.21.15.16:0/3541711543 shutdown_connections 2023-11-24T06:27:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.778+0000 7f1066167640 1 --2- 172.21.15.16:0/3541711543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 0x7f10600cee50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.778+0000 7f1066167640 1 --2- 172.21.15.16:0/3541711543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106015d3c0 0x7f10600d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.778+0000 7f1066167640 1 -- 172.21.15.16:0/3541711543 >> 172.21.15.16:0/3541711543 conn(0x7f10600d30b0 msgr2=0x7f10600d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.778+0000 7f1066167640 1 -- 172.21.15.16:0/3541711543 shutdown_connections 2023-11-24T06:27:00.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.778+0000 7f1066167640 1 -- 172.21.15.16:0/3541711543 wait complete. 2023-11-24T06:27:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.780+0000 7f1066167640 1 Processor -- start 2023-11-24T06:27:00.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.780+0000 7f1066167640 1 -- start start 2023-11-24T06:27:00.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.781+0000 7f1066167640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 0x7f10600b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.781+0000 7f1066167640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10600d0e90 0x7f10601f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.781+0000 7f1066167640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f106015d3c0 0x7f10601f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.781+0000 7f1066167640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10600b4d30 con 0x7f106015d3c0 2023-11-24T06:27:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.781+0000 7f1066167640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10600ccbd0 con 0x7f10600ce9f0 2023-11-24T06:27:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.781+0000 7f1066167640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10600b2f90 con 0x7f10600d0e90 2023-11-24T06:27:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.781+0000 7f1065966640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f106015d3c0 0x7f10601f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.781+0000 7f1065966640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f106015d3c0 0x7f10601f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40594/0 (socket says 172.21.15.16:40594) 2023-11-24T06:27:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.781+0000 7f1065966640 1 -- 172.21.15.16:0/2703090792 learned_addr learned my addr 172.21.15.16:0/2703090792 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1065165640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 0x7f10600b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1064964640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10600d0e90 0x7f10601f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1065966640 1 -- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10600d0e90 msgr2=0x7f10601f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1065966640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10600d0e90 0x7f10601f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1065966640 1 -- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 msgr2=0x7f10600b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1065966640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 0x7f10600b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1065966640 1 -- 172.21.15.16:0/2703090792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058009b30 con 0x7f106015d3c0 2023-11-24T06:27:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1065165640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 0x7f10600b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1065966640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f106015d3c0 0x7f10601f6890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1058036040 tx=0x7f1058004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.782+0000 7f1055ffb640 1 -- 172.21.15.16:0/2703090792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105803e450 con 0x7f106015d3c0 2023-11-24T06:27:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.783+0000 7f1055ffb640 1 -- 172.21.15.16:0/2703090792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105803e5b0 con 0x7f106015d3c0 2023-11-24T06:27:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.783+0000 7f1055ffb640 1 -- 172.21.15.16:0/2703090792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105803e960 con 0x7f106015d3c0 2023-11-24T06:27:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.783+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10600b3830 con 0x7f106015d3c0 2023-11-24T06:27:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.783+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10600b31e0 con 0x7f106015d3c0 2023-11-24T06:27:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.784+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10600b39c0 con 0x7f106015d3c0 2023-11-24T06:27:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.785+0000 7f1055ffb640 1 -- 172.21.15.16:0/2703090792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f105803eac0 con 0x7f106015d3c0 2023-11-24T06:27:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.786+0000 7f1055ffb640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f104406ae90 0x7f104406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.786+0000 7f1055ffb640 1 -- 172.21.15.16:0/2703090792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10580ac570 con 0x7f106015d3c0 2023-11-24T06:27:00.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.786+0000 7f1065165640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f104406ae90 0x7f104406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.787+0000 7f1065165640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f104406ae90 0x7f104406d350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f105c009b90 tx=0x7f105c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.790+0000 7f1055ffb640 1 -- 172.21.15.16:0/2703090792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1058077690 con 0x7f106015d3c0 2023-11-24T06:27:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.985+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1060067b60 con 0x7f106015d3c0 2023-11-24T06:27:00.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.988+0000 7f1055ffb640 1 -- 172.21.15.16:0/2703090792 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f1058033e80 con 0x7f106015d3c0 2023-11-24T06:27:00.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:00.991 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f104406ae90 msgr2=0x7f104406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f104406ae90 0x7f104406d350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f105c009b90 tx=0x7f105c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f106015d3c0 msgr2=0x7f10601f6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f106015d3c0 0x7f10601f6890 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1058036040 tx=0x7f1058004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 shutdown_connections 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f104406ae90 0x7f104406d350 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f106015d3c0 0x7f10601f6890 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10600d0e90 0x7f10601f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 --2- 172.21.15.16:0/2703090792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10600ce9f0 0x7f10600b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 >> 172.21.15.16:0/2703090792 conn(0x7f10600d30b0 msgr2=0x7f106006bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 shutdown_connections 2023-11-24T06:27:00.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:00.991+0000 7f1066167640 1 -- 172.21.15.16:0/2703090792 wait complete. 2023-11-24T06:27:01.013 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:27:01.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:01.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.220+0000 7f5d55e1e640 1 Processor -- start 2023-11-24T06:27:01.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.220+0000 7f5d55e1e640 1 -- start start 2023-11-24T06:27:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.222+0000 7f5d55e1e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5d5013df80 0x7f5d5015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.222+0000 7f5d55e1e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d5013e360 0x7f5d500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.222+0000 7f5d55e1e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5006c470 con 0x7f5d5015f9c0 2023-11-24T06:27:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.222+0000 7f5d55e1e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5006c750 con 0x7f5d5013e360 2023-11-24T06:27:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.222+0000 7f5d55e1e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5006ca30 con 0x7f5d5013df80 2023-11-24T06:27:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.223+0000 7f5d47fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d5013e360 0x7f5d500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.223+0000 7f5d47fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d5013e360 0x7f5d500cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40610/0 (socket says 172.21.15.16:40610) 2023-11-24T06:27:01.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.223+0000 7f5d47fff640 1 -- 172.21.15.16:0/1976693207 learned_addr learned my addr 172.21.15.16:0/1976693207 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:01.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.223+0000 7f5d4f7fe640 1 --2- 172.21.15.16:0/1976693207 >> v2:172.21.15.173:3300/0 conn(0x7f5d5013df80 0x7f5d5015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.223+0000 7f5d47fff640 1 -- 172.21.15.16:0/1976693207 >> v2:172.21.15.173:3300/0 conn(0x7f5d5013df80 msgr2=0x7f5d5015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:01.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.223+0000 7f5d47fff640 1 --2- 172.21.15.16:0/1976693207 >> v2:172.21.15.173:3300/0 conn(0x7f5d5013df80 0x7f5d5015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.223+0000 7f5d47fff640 1 -- 172.21.15.16:0/1976693207 >> v1:172.21.15.173:6789/0 conn(0x7f5d5015f9c0 legacy=0x7f5d501408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:01.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.223+0000 7f5d47fff640 1 -- 172.21.15.16:0/1976693207 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5006cd10 con 0x7f5d5013e360 2023-11-24T06:27:01.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.223+0000 7f5d4f7fe640 1 --2- 172.21.15.16:0/1976693207 >> v2:172.21.15.173:3300/0 conn(0x7f5d5013df80 0x7f5d5015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:01.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.224+0000 7f5d47fff640 1 --2- 172.21.15.16:0/1976693207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013e360 0x7f5d500cf290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d50072770 tx=0x7f5d3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77de4aa9e6785657 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:01.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.224+0000 7f5d4effd640 1 -- 172.21.15.16:0/1976693207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38031870 con 0x7f5d5013e360 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.225+0000 7f5d4effd640 1 -- 172.21.15.16:0/1976693207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d380319d0 con 0x7f5d5013e360 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.225+0000 7f5d4effd640 1 -- 172.21.15.16:0/1976693207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38031d60 con 0x7f5d5013e360 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.225+0000 7f5d55e1e640 1 -- 172.21.15.16:0/1976693207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013e360 msgr2=0x7f5d500cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.225+0000 7f5d55e1e640 1 --2- 172.21.15.16:0/1976693207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013e360 0x7f5d500cf290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d50072770 tx=0x7f5d3802b910 comp rx=0 tx=0).stop 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.225+0000 7f5d55e1e640 1 -- 172.21.15.16:0/1976693207 shutdown_connections 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.225+0000 7f5d55e1e640 1 --2- 172.21.15.16:0/1976693207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013e360 0x7f5d500cf290 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.225+0000 7f5d55e1e640 1 --2- 172.21.15.16:0/1976693207 >> v2:172.21.15.173:3300/0 conn(0x7f5d5013df80 0x7f5d5015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.225+0000 7f5d55e1e640 1 -- 172.21.15.16:0/1976693207 >> 172.21.15.16:0/1976693207 conn(0x7f5d5006a800 msgr2=0x7f5d5006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.225+0000 7f5d55e1e640 1 -- 172.21.15.16:0/1976693207 shutdown_connections 2023-11-24T06:27:01.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.226+0000 7f5d55e1e640 1 -- 172.21.15.16:0/1976693207 wait complete. 2023-11-24T06:27:01.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.227+0000 7f5d55e1e640 1 Processor -- start 2023-11-24T06:27:01.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.227+0000 7f5d55e1e640 1 -- start start 2023-11-24T06:27:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.228+0000 7f5d55e1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013df80 0x7f5d5013d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.228+0000 7f5d55e1e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d5013e360 0x7f5d5014fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d55e1e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d5015f9c0 0x7f5d5015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d55e1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50153aa0 con 0x7f5d5013df80 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d55e1e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5013d950 con 0x7f5d5015f9c0 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d55e1e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50152ce0 con 0x7f5d5013e360 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d4f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013df80 0x7f5d5013d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d4f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013df80 0x7f5d5013d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40620/0 (socket says 172.21.15.16:40620) 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d4f7fe640 1 -- 172.21.15.16:0/572801020 learned_addr learned my addr 172.21.15.16:0/572801020 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d4f7fe640 1 -- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d5013e360 msgr2=0x7f5d5014fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d4f7fe640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d5013e360 0x7f5d5014fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d4f7fe640 1 -- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d5015f9c0 msgr2=0x7f5d5015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d4f7fe640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d5015f9c0 0x7f5d5015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.229+0000 7f5d4f7fe640 1 -- 172.21.15.16:0/572801020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d380094d0 con 0x7f5d5013df80 2023-11-24T06:27:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.230+0000 7f5d4f7fe640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013df80 0x7f5d5013d410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c7f0 tx=0x7f5d4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.230+0000 7f5d4cff9640 1 -- 172.21.15.16:0/572801020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d400077c0 con 0x7f5d5013df80 2023-11-24T06:27:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.230+0000 7f5d4cff9640 1 -- 172.21.15.16:0/572801020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40007920 con 0x7f5d5013df80 2023-11-24T06:27:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.230+0000 7f5d4cff9640 1 -- 172.21.15.16:0/572801020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40007c90 con 0x7f5d5013df80 2023-11-24T06:27:01.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.230+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d50152fc0 con 0x7f5d5013df80 2023-11-24T06:27:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.230+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d50153580 con 0x7f5d5013df80 2023-11-24T06:27:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.231+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d500b03a0 con 0x7f5d5013df80 2023-11-24T06:27:01.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.232+0000 7f5d4cff9640 1 -- 172.21.15.16:0/572801020 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d400113f0 con 0x7f5d5013df80 2023-11-24T06:27:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.234+0000 7f5d4cff9640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3006ae40 0x7f5d3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.234+0000 7f5d4cff9640 1 -- 172.21.15.16:0/572801020 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d400899f0 con 0x7f5d5013df80 2023-11-24T06:27:01.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.234+0000 7f5d47fff640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3006ae40 0x7f5d3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.235+0000 7f5d47fff640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3006ae40 0x7f5d3006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d38036000 tx=0x7f5d38030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:01.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.238+0000 7f5d4cff9640 1 -- 172.21.15.16:0/572801020 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d40054970 con 0x7f5d5013df80 2023-11-24T06:27:01.374 INFO:tasks.ceph.mds.b.smithi173.stderr:2023-11-24T06:27:01.370+0000 7f0fc80d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:27:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.434+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d500ce850 con 0x7f5d3006ae40 2023-11-24T06:27:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.459+0000 7f5d4cff9640 1 -- 172.21.15.16:0/572801020 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264501 (secure 0 0 0) 0x7f5d38031c40 con 0x7f5d3006ae40 2023-11-24T06:27:01.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:27:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3006ae40 msgr2=0x7f5d3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3006ae40 0x7f5d3006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d38036000 tx=0x7f5d38030000 comp rx=0 tx=0).stop 2023-11-24T06:27:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013df80 msgr2=0x7f5d5013d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013df80 0x7f5d5013d410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000c7f0 tx=0x7f5d4000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 shutdown_connections 2023-11-24T06:27:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3006ae40 0x7f5d3006d300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d5015f9c0 0x7f5d5015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d5013e360 0x7f5d5014fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 --2- 172.21.15.16:0/572801020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5013df80 0x7f5d5013d410 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 >> 172.21.15.16:0/572801020 conn(0x7f5d5006a800 msgr2=0x7f5d500af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 shutdown_connections 2023-11-24T06:27:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.465+0000 7f5d55e1e640 1 -- 172.21.15.16:0/572801020 wait complete. 2023-11-24T06:27:01.490 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-11-24T06:27:00.101266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235876,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738844,"statfs":{"total":773094113280,"available":772852576256,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.133694"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948224+0000","last_change":"2023-11-24T06:26:56.969865+0000","last_active":"2023-11-24T06:26:57.948224+0000","last_peered":"2023-11-24T06:26:57.948224+0000","last_clean":"2023-11-24T06:26:57.948224+0000","last_became_active":"2023-11-24T06:26:56.969570+0000","last_became_peered":"2023-11-24T06:26:56.969570+0000","last_unstale":"2023-11-24T06:26:57.948224+0000","last_undegraded":"2023-11-24T06:26:57.948224+0000","last_fullsized":"2023-11-24T06:26:57.948224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:56.199949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944608+0000","last_change":"2023-11-24T06:26:56.979241+0000","last_active":"2023-11-24T06:26:57.944608+0000","last_peered":"2023-11-24T06:26:57.944608+0000","last_clean":"2023-11-24T06:26:57.944608+0000","last_became_active":"2023-11-24T06:26:56.978864+0000","last_became_peered":"2023-11-24T06:26:56.978864+0000","last_unstale":"2023-11-24T06:26:57.944608+0000","last_undegraded":"2023-11-24T06:26:57.944608+0000","last_fullsized":"2023-11-24T06:26:57.944608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218032+0000","last_change":"2023-11-24T06:26:56.976436+0000","last_active":"2023-11-24T06:26:58.218032+0000","last_peered":"2023-11-24T06:26:58.218032+0000","last_clean":"2023-11-24T06:26:58.218032+0000","last_became_active":"2023-11-24T06:26:56.976114+0000","last_became_peered":"2023-11-24T06:26:56.976114+0000","last_unstale":"2023-11-24T06:26:58.218032+0000","last_undegraded":"2023-11-24T06:26:58.218032+0000","last_fullsized":"2023-11-24T06:26:58.218032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222085+0000","last_change":"2023-11-24T06:26:56.992564+0000","last_active":"2023-11-24T06:26:58.222085+0000","last_peered":"2023-11-24T06:26:58.222085+0000","last_clean":"2023-11-24T06:26:58.222085+0000","last_became_active":"2023-11-24T06:26:56.992045+0000","last_became_peered":"2023-11-24T06:26:56.992045+0000","last_unstale":"2023-11-24T06:26:58.222085+0000","last_undegraded":"2023-11-24T06:26:58.222085+0000","last_fullsized":"2023-11-24T06:26:58.222085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947424+0000","last_change":"2023-11-24T06:26:56.964941+0000","last_active":"2023-11-24T06:26:57.947424+0000","last_peered":"2023-11-24T06:26:57.947424+0000","last_clean":"2023-11-24T06:26:57.947424+0000","last_became_active":"2023-11-24T06:26:56.964244+0000","last_became_peered":"2023-11-24T06:26:56.964244+0000","last_unstale":"2023-11-24T06:26:57.947424+0000","last_undegraded":"2023-11-24T06:26:57.947424+0000","last_fullsized":"2023-11-24T06:26:57.947424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219175+0000","last_change":"2023-11-24T06:26:56.993038+0000","last_active":"2023-11-24T06:26:58.219175+0000","last_peered":"2023-11-24T06:26:58.219175+0000","last_clean":"2023-11-24T06:26:58.219175+0000","last_became_active":"2023-11-24T06:26:56.986387+0000","last_became_peered":"2023-11-24T06:26:56.986387+0000","last_unstale":"2023-11-24T06:26:58.219175+0000","last_undegraded":"2023-11-24T06:26:58.219175+0000","last_fullsized":"2023-11-24T06:26:58.219175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946794+0000","last_change":"2023-11-24T06:26:56.977591+0000","last_active":"2023-11-24T06:26:57.946794+0000","last_peered":"2023-11-24T06:26:57.946794+0000","last_clean":"2023-11-24T06:26:57.946794+0000","last_became_active":"2023-11-24T06:26:56.976851+0000","last_became_peered":"2023-11-24T06:26:56.976851+0000","last_unstale":"2023-11-24T06:26:57.946794+0000","last_undegraded":"2023-11-24T06:26:57.946794+0000","last_fullsized":"2023-11-24T06:26:57.946794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944925+0000","last_change":"2023-11-24T06:26:56.991219+0000","last_active":"2023-11-24T06:26:57.944925+0000","last_peered":"2023-11-24T06:26:57.944925+0000","last_clean":"2023-11-24T06:26:57.944925+0000","last_became_active":"2023-11-24T06:26:56.990836+0000","last_became_peered":"2023-11-24T06:26:56.990836+0000","last_unstale":"2023-11-24T06:26:57.944925+0000","last_undegraded":"2023-11-24T06:26:57.944925+0000","last_fullsized":"2023-11-24T06:26:57.944925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948378+0000","last_change":"2023-11-24T06:26:57.001238+0000","last_active":"2023-11-24T06:26:57.948378+0000","last_peered":"2023-11-24T06:26:57.948378+0000","last_clean":"2023-11-24T06:26:57.948378+0000","last_became_active":"2023-11-24T06:26:57.000919+0000","last_became_peered":"2023-11-24T06:26:57.000919+0000","last_unstale":"2023-11-24T06:26:57.948378+0000","last_undegraded":"2023-11-24T06:26:57.948378+0000","last_fullsized":"2023-11-24T06:26:57.948378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:38.348575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219671+0000","last_change":"2023-11-24T06:26:56.998442+0000","last_active":"2023-11-24T06:26:58.219671+0000","last_peered":"2023-11-24T06:26:58.219671+0000","last_clean":"2023-11-24T06:26:58.219671+0000","last_became_active":"2023-11-24T06:26:56.998097+0000","last_became_peered":"2023-11-24T06:26:56.998097+0000","last_unstale":"2023-11-24T06:26:58.219671+0000","last_undegraded":"2023-11-24T06:26:58.219671+0000","last_fullsized":"2023-11-24T06:26:58.219671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218262+0000","last_change":"2023-11-24T06:26:57.000349+0000","last_active":"2023-11-24T06:26:58.218262+0000","last_peered":"2023-11-24T06:26:58.218262+0000","last_clean":"2023-11-24T06:26:58.218262+0000","last_became_active":"2023-11-24T06:26:57.000122+0000","last_became_peered":"2023-11-24T06:26:57.000122+0000","last_unstale":"2023-11-24T06:26:58.218262+0000","last_undegraded":"2023-11-24T06:26:58.218262+0000","last_fullsized":"2023-11-24T06:26:58.218262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221702+0000","last_change":"2023-11-24T06:26:56.969362+0000","last_active":"2023-11-24T06:26:58.221702+0000","last_peered":"2023-11-24T06:26:58.221702+0000","last_clean":"2023-11-24T06:26:58.221702+0000","last_became_active":"2023-11-24T06:26:56.969057+0000","last_became_peered":"2023-11-24T06:26:56.969057+0000","last_unstale":"2023-11-24T06:26:58.221702+0000","last_undegraded":"2023-11-24T06:26:58.221702+0000","last_fullsized":"2023-11-24T06:26:58.221702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:04.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945262+0000","last_change":"2023-11-24T06:26:56.970657+0000","last_active":"2023-11-24T06:26:57.945262+0000","last_peered":"2023-11-24T06:26:57.945262+0000","last_clean":"2023-11-24T06:26:57.945262+0000","last_became_active":"2023-11-24T06:26:56.970348+0000","last_became_peered":"2023-11-24T06:26:56.970348+0000","last_unstale":"2023-11-24T06:26:57.945262+0000","last_undegraded":"2023-11-24T06:26:57.945262+0000","last_fullsized":"2023-11-24T06:26:57.945262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217421+0000","last_change":"2023-11-24T06:26:56.972241+0000","last_active":"2023-11-24T06:26:58.217421+0000","last_peered":"2023-11-24T06:26:58.217421+0000","last_clean":"2023-11-24T06:26:58.217421+0000","last_became_active":"2023-11-24T06:26:56.971982+0000","last_became_peered":"2023-11-24T06:26:56.971982+0000","last_unstale":"2023-11-24T06:26:58.217421+0000","last_undegraded":"2023-11-24T06:26:58.217421+0000","last_fullsized":"2023-11-24T06:26:58.217421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:19.450311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944041+0000","last_change":"2023-11-24T06:26:56.978171+0000","last_active":"2023-11-24T06:26:57.944041+0000","last_peered":"2023-11-24T06:26:57.944041+0000","last_clean":"2023-11-24T06:26:57.944041+0000","last_became_active":"2023-11-24T06:26:56.977851+0000","last_became_peered":"2023-11-24T06:26:56.977851+0000","last_unstale":"2023-11-24T06:26:57.944041+0000","last_undegraded":"2023-11-24T06:26:57.944041+0000","last_fullsized":"2023-11-24T06:26:57.944041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218965+0000","last_change":"2023-11-24T06:26:56.991619+0000","last_active":"2023-11-24T06:26:58.218965+0000","last_peered":"2023-11-24T06:26:58.218965+0000","last_clean":"2023-11-24T06:26:58.218965+0000","last_became_active":"2023-11-24T06:26:56.986865+0000","last_became_peered":"2023-11-24T06:26:56.986865+0000","last_unstale":"2023-11-24T06:26:58.218965+0000","last_undegraded":"2023-11-24T06:26:58.218965+0000","last_fullsized":"2023-11-24T06:26:58.218965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.223090+0000","last_change":"2023-11-24T06:26:56.997282+0000","last_active":"2023-11-24T06:26:58.223090+0000","last_peered":"2023-11-24T06:26:58.223090+0000","last_clean":"2023-11-24T06:26:58.223090+0000","last_became_active":"2023-11-24T06:26:56.995631+0000","last_became_peered":"2023-11-24T06:26:56.995631+0000","last_unstale":"2023-11-24T06:26:58.223090+0000","last_undegraded":"2023-11-24T06:26:58.223090+0000","last_fullsized":"2023-11-24T06:26:58.223090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:38.348575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217200+0000","last_change":"2023-11-24T06:26:56.969721+0000","last_active":"2023-11-24T06:26:58.217200+0000","last_peered":"2023-11-24T06:26:58.217200+0000","last_clean":"2023-11-24T06:26:58.217200+0000","last_became_active":"2023-11-24T06:26:56.969475+0000","last_became_peered":"2023-11-24T06:26:56.969475+0000","last_unstale":"2023-11-24T06:26:58.217200+0000","last_undegraded":"2023-11-24T06:26:58.217200+0000","last_fullsized":"2023-11-24T06:26:58.217200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:04.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947802+0000","last_change":"2023-11-24T06:26:56.964126+0000","last_active":"2023-11-24T06:26:57.947802+0000","last_peered":"2023-11-24T06:26:57.947802+0000","last_clean":"2023-11-24T06:26:57.947802+0000","last_became_active":"2023-11-24T06:26:56.963622+0000","last_became_peered":"2023-11-24T06:26:56.963622+0000","last_unstale":"2023-11-24T06:26:57.947802+0000","last_undegraded":"2023-11-24T06:26:57.947802+0000","last_fullsized":"2023-11-24T06:26:57.947802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:17.662630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218464+0000","last_change":"2023-11-24T06:26:56.975222+0000","last_active":"2023-11-24T06:26:58.218464+0000","last_peered":"2023-11-24T06:26:58.218464+0000","last_clean":"2023-11-24T06:26:58.218464+0000","last_became_active":"2023-11-24T06:26:56.974069+0000","last_became_peered":"2023-11-24T06:26:56.974069+0000","last_unstale":"2023-11-24T06:26:58.218464+0000","last_undegraded":"2023-11-24T06:26:58.218464+0000","last_fullsized":"2023-11-24T06:26:58.218464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218120+0000","last_change":"2023-11-24T06:26:56.975253+0000","last_active":"2023-11-24T06:26:58.218120+0000","last_peered":"2023-11-24T06:26:58.218120+0000","last_clean":"2023-11-24T06:26:58.218120+0000","last_became_active":"2023-11-24T06:26:56.974888+0000","last_became_peered":"2023-11-24T06:26:56.974888+0000","last_unstale":"2023-11-24T06:26:58.218120+0000","last_undegraded":"2023-11-24T06:26:58.218120+0000","last_fullsized":"2023-11-24T06:26:58.218120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221143+0000","last_change":"2023-11-24T06:26:56.985344+0000","last_active":"2023-11-24T06:26:58.221143+0000","last_peered":"2023-11-24T06:26:58.221143+0000","last_clean":"2023-11-24T06:26:58.221143+0000","last_became_active":"2023-11-24T06:26:56.984528+0000","last_became_peered":"2023-11-24T06:26:56.984528+0000","last_unstale":"2023-11-24T06:26:58.221143+0000","last_undegraded":"2023-11-24T06:26:58.221143+0000","last_fullsized":"2023-11-24T06:26:58.221143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944974+0000","last_change":"2023-11-24T06:26:56.979949+0000","last_active":"2023-11-24T06:26:57.944974+0000","last_peered":"2023-11-24T06:26:57.944974+0000","last_clean":"2023-11-24T06:26:57.944974+0000","last_became_active":"2023-11-24T06:26:56.979635+0000","last_became_peered":"2023-11-24T06:26:56.979635+0000","last_unstale":"2023-11-24T06:26:57.944974+0000","last_undegraded":"2023-11-24T06:26:57.944974+0000","last_fullsized":"2023-11-24T06:26:57.944974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219328+0000","last_change":"2023-11-24T06:26:56.968934+0000","last_active":"2023-11-24T06:26:58.219328+0000","last_peered":"2023-11-24T06:26:58.219328+0000","last_clean":"2023-11-24T06:26:58.219328+0000","last_became_active":"2023-11-24T06:26:56.967198+0000","last_became_peered":"2023-11-24T06:26:56.967198+0000","last_unstale":"2023-11-24T06:26:58.219328+0000","last_undegraded":"2023-11-24T06:26:58.219328+0000","last_fullsized":"2023-11-24T06:26:58.219328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945713+0000","last_change":"2023-11-24T06:26:56.974003+0000","last_active":"2023-11-24T06:26:57.945713+0000","last_peered":"2023-11-24T06:26:57.945713+0000","last_clean":"2023-11-24T06:26:57.945713+0000","last_became_active":"2023-11-24T06:26:56.973631+0000","last_became_peered":"2023-11-24T06:26:56.973631+0000","last_unstale":"2023-11-24T06:26:57.945713+0000","last_undegraded":"2023-11-24T06:26:57.945713+0000","last_fullsized":"2023-11-24T06:26:57.945713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219990+0000","last_change":"2023-11-24T06:26:56.972613+0000","last_active":"2023-11-24T06:26:58.219990+0000","last_peered":"2023-11-24T06:26:58.219990+0000","last_clean":"2023-11-24T06:26:58.219990+0000","last_became_active":"2023-11-24T06:26:56.972280+0000","last_became_peered":"2023-11-24T06:26:56.972280+0000","last_unstale":"2023-11-24T06:26:58.219990+0000","last_undegraded":"2023-11-24T06:26:58.219990+0000","last_fullsized":"2023-11-24T06:26:58.219990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944265+0000","last_change":"2023-11-24T06:26:56.966680+0000","last_active":"2023-11-24T06:26:57.944265+0000","last_peered":"2023-11-24T06:26:57.944265+0000","last_clean":"2023-11-24T06:26:57.944265+0000","last_became_active":"2023-11-24T06:26:56.966315+0000","last_became_peered":"2023-11-24T06:26:56.966315+0000","last_unstale":"2023-11-24T06:26:57.944265+0000","last_undegraded":"2023-11-24T06:26:57.944265+0000","last_fullsized":"2023-11-24T06:26:57.944265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945586+0000","last_change":"2023-11-24T06:26:56.974920+0000","last_active":"2023-11-24T06:26:57.945586+0000","last_peered":"2023-11-24T06:26:57.945586+0000","last_clean":"2023-11-24T06:26:57.945586+0000","last_became_active":"2023-11-24T06:26:56.974594+0000","last_became_peered":"2023-11-24T06:26:56.974594+0000","last_unstale":"2023-11-24T06:26:57.945586+0000","last_undegraded":"2023-11-24T06:26:57.945586+0000","last_fullsized":"2023-11-24T06:26:57.945586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217994+0000","last_change":"2023-11-24T06:26:56.976017+0000","last_active":"2023-11-24T06:26:58.217994+0000","last_peered":"2023-11-24T06:26:58.217994+0000","last_clean":"2023-11-24T06:26:58.217994+0000","last_became_active":"2023-11-24T06:26:56.975709+0000","last_became_peered":"2023-11-24T06:26:56.975709+0000","last_unstale":"2023-11-24T06:26:58.217994+0000","last_undegraded":"2023-11-24T06:26:58.217994+0000","last_fullsized":"2023-11-24T06:26:58.217994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221250+0000","last_change":"2023-11-24T06:26:56.980269+0000","last_active":"2023-11-24T06:26:58.221250+0000","last_peered":"2023-11-24T06:26:58.221250+0000","last_clean":"2023-11-24T06:26:58.221250+0000","last_became_active":"2023-11-24T06:26:56.979929+0000","last_became_peered":"2023-11-24T06:26:56.979929+0000","last_unstale":"2023-11-24T06:26:58.221250+0000","last_undegraded":"2023-11-24T06:26:58.221250+0000","last_fullsized":"2023-11-24T06:26:58.221250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216766+0000","last_change":"2023-11-24T06:26:56.986717+0000","last_active":"2023-11-24T06:26:58.216766+0000","last_peered":"2023-11-24T06:26:58.216766+0000","last_clean":"2023-11-24T06:26:58.216766+0000","last_became_active":"2023-11-24T06:26:56.986398+0000","last_became_peered":"2023-11-24T06:26:56.986398+0000","last_unstale":"2023-11-24T06:26:58.216766+0000","last_undegraded":"2023-11-24T06:26:58.216766+0000","last_fullsized":"2023-11-24T06:26:58.216766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:47.926009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217958+0000","last_change":"2023-11-24T06:26:56.978275+0000","last_active":"2023-11-24T06:26:58.217958+0000","last_peered":"2023-11-24T06:26:58.217958+0000","last_clean":"2023-11-24T06:26:58.217958+0000","last_became_active":"2023-11-24T06:26:56.977924+0000","last_became_peered":"2023-11-24T06:26:56.977924+0000","last_unstale":"2023-11-24T06:26:58.217958+0000","last_undegraded":"2023-11-24T06:26:58.217958+0000","last_fullsized":"2023-11-24T06:26:58.217958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:58.210301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220024+0000","last_change":"2023-11-24T06:26:56.970266+0000","last_active":"2023-11-24T06:26:58.220024+0000","last_peered":"2023-11-24T06:26:58.220024+0000","last_clean":"2023-11-24T06:26:58.220024+0000","last_became_active":"2023-11-24T06:26:56.969973+0000","last_became_peered":"2023-11-24T06:26:56.969973+0000","last_unstale":"2023-11-24T06:26:58.220024+0000","last_undegraded":"2023-11-24T06:26:58.220024+0000","last_fullsized":"2023-11-24T06:26:58.220024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:28.630873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945477+0000","last_change":"2023-11-24T06:26:56.008228+0000","last_active":"2023-11-24T06:26:57.945477+0000","last_peered":"2023-11-24T06:26:57.945477+0000","last_clean":"2023-11-24T06:26:57.945477+0000","last_became_active":"2023-11-24T06:26:56.007916+0000","last_became_peered":"2023-11-24T06:26:56.007916+0000","last_unstale":"2023-11-24T06:26:57.945477+0000","last_undegraded":"2023-11-24T06:26:57.945477+0000","last_fullsized":"2023-11-24T06:26:57.945477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218141+0000","last_change":"2023-11-24T06:26:56.982794+0000","last_active":"2023-11-24T06:26:58.218141+0000","last_peered":"2023-11-24T06:26:58.218141+0000","last_clean":"2023-11-24T06:26:58.218141+0000","last_became_active":"2023-11-24T06:26:56.982455+0000","last_became_peered":"2023-11-24T06:26:56.982455+0000","last_unstale":"2023-11-24T06:26:58.218141+0000","last_undegraded":"2023-11-24T06:26:58.218141+0000","last_fullsized":"2023-11-24T06:26:58.218141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218348+0000","last_change":"2023-11-24T06:26:56.010407+0000","last_active":"2023-11-24T06:26:58.218348+0000","last_peered":"2023-11-24T06:26:58.218348+0000","last_clean":"2023-11-24T06:26:58.218348+0000","last_became_active":"2023-11-24T06:26:56.009548+0000","last_became_peered":"2023-11-24T06:26:56.009548+0000","last_unstale":"2023-11-24T06:26:58.218348+0000","last_undegraded":"2023-11-24T06:26:58.218348+0000","last_fullsized":"2023-11-24T06:26:58.218348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:58.487660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945323+0000","last_change":"2023-11-24T06:26:56.005804+0000","last_active":"2023-11-24T06:26:57.945323+0000","last_peered":"2023-11-24T06:26:57.945323+0000","last_clean":"2023-11-24T06:26:57.945323+0000","last_became_active":"2023-11-24T06:26:56.005473+0000","last_became_peered":"2023-11-24T06:26:56.005473+0000","last_unstale":"2023-11-24T06:26:57.945323+0000","last_undegraded":"2023-11-24T06:26:57.945323+0000","last_fullsized":"2023-11-24T06:26:57.945323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:18.225163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944935+0000","last_change":"2023-11-24T06:26:56.977459+0000","last_active":"2023-11-24T06:26:57.944935+0000","last_peered":"2023-11-24T06:26:57.944935+0000","last_clean":"2023-11-24T06:26:57.944935+0000","last_became_active":"2023-11-24T06:26:56.977160+0000","last_became_peered":"2023-11-24T06:26:56.977160+0000","last_unstale":"2023-11-24T06:26:57.944935+0000","last_undegraded":"2023-11-24T06:26:57.944935+0000","last_fullsized":"2023-11-24T06:26:57.944935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219546+0000","last_change":"2023-11-24T06:26:56.010202+0000","last_active":"2023-11-24T06:26:58.219546+0000","last_peered":"2023-11-24T06:26:58.219546+0000","last_clean":"2023-11-24T06:26:58.219546+0000","last_became_active":"2023-11-24T06:26:56.009924+0000","last_became_peered":"2023-11-24T06:26:56.009924+0000","last_unstale":"2023-11-24T06:26:58.219546+0000","last_undegraded":"2023-11-24T06:26:58.219546+0000","last_fullsized":"2023-11-24T06:26:58.219546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:17.355207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944060+0000","last_change":"2023-11-24T06:26:56.962258+0000","last_active":"2023-11-24T06:26:57.944060+0000","last_peered":"2023-11-24T06:26:57.944060+0000","last_clean":"2023-11-24T06:26:57.944060+0000","last_became_active":"2023-11-24T06:26:56.961665+0000","last_became_peered":"2023-11-24T06:26:56.961665+0000","last_unstale":"2023-11-24T06:26:57.944060+0000","last_undegraded":"2023-11-24T06:26:57.944060+0000","last_fullsized":"2023-11-24T06:26:57.944060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221161+0000","last_change":"2023-11-24T06:26:56.017642+0000","last_active":"2023-11-24T06:26:58.221161+0000","last_peered":"2023-11-24T06:26:58.221161+0000","last_clean":"2023-11-24T06:26:58.221161+0000","last_became_active":"2023-11-24T06:26:56.017386+0000","last_became_peered":"2023-11-24T06:26:56.017386+0000","last_unstale":"2023-11-24T06:26:58.221161+0000","last_undegraded":"2023-11-24T06:26:58.221161+0000","last_fullsized":"2023-11-24T06:26:58.221161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:18.225163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222938+0000","last_change":"2023-11-24T06:26:56.985441+0000","last_active":"2023-11-24T06:26:58.222938+0000","last_peered":"2023-11-24T06:26:58.222938+0000","last_clean":"2023-11-24T06:26:58.222938+0000","last_became_active":"2023-11-24T06:26:56.984430+0000","last_became_peered":"2023-11-24T06:26:56.984430+0000","last_unstale":"2023-11-24T06:26:58.222938+0000","last_undegraded":"2023-11-24T06:26:58.222938+0000","last_fullsized":"2023-11-24T06:26:58.222938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:56.199949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222514+0000","last_change":"2023-11-24T06:26:56.017588+0000","last_active":"2023-11-24T06:26:58.222514+0000","last_peered":"2023-11-24T06:26:58.222514+0000","last_clean":"2023-11-24T06:26:58.222514+0000","last_became_active":"2023-11-24T06:26:56.017128+0000","last_became_peered":"2023-11-24T06:26:56.017128+0000","last_unstale":"2023-11-24T06:26:58.222514+0000","last_undegraded":"2023-11-24T06:26:58.222514+0000","last_fullsized":"2023-11-24T06:26:58.222514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218742+0000","last_change":"2023-11-24T06:26:57.002819+0000","last_active":"2023-11-24T06:26:58.218742+0000","last_peered":"2023-11-24T06:26:58.218742+0000","last_clean":"2023-11-24T06:26:58.218742+0000","last_became_active":"2023-11-24T06:26:57.002515+0000","last_became_peered":"2023-11-24T06:26:57.002515+0000","last_unstale":"2023-11-24T06:26:58.218742+0000","last_undegraded":"2023-11-24T06:26:58.218742+0000","last_fullsized":"2023-11-24T06:26:58.218742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944342+0000","last_change":"2023-11-24T06:26:56.014991+0000","last_active":"2023-11-24T06:26:57.944342+0000","last_peered":"2023-11-24T06:26:57.944342+0000","last_clean":"2023-11-24T06:26:57.944342+0000","last_became_active":"2023-11-24T06:26:56.014706+0000","last_became_peered":"2023-11-24T06:26:56.014706+0000","last_unstale":"2023-11-24T06:26:57.944342+0000","last_undegraded":"2023-11-24T06:26:57.944342+0000","last_fullsized":"2023-11-24T06:26:57.944342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:56.985153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221316+0000","last_change":"2023-11-24T06:26:56.970097+0000","last_active":"2023-11-24T06:26:58.221316+0000","last_peered":"2023-11-24T06:26:58.221316+0000","last_clean":"2023-11-24T06:26:58.221316+0000","last_became_active":"2023-11-24T06:26:56.969183+0000","last_became_peered":"2023-11-24T06:26:56.969183+0000","last_unstale":"2023-11-24T06:26:58.221316+0000","last_undegraded":"2023-11-24T06:26:58.221316+0000","last_fullsized":"2023-11-24T06:26:58.221316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:47.926009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221734+0000","last_change":"2023-11-24T06:26:56.002890+0000","last_active":"2023-11-24T06:26:58.221734+0000","last_peered":"2023-11-24T06:26:58.221734+0000","last_clean":"2023-11-24T06:26:58.221734+0000","last_became_active":"2023-11-24T06:26:56.002603+0000","last_became_peered":"2023-11-24T06:26:56.002603+0000","last_unstale":"2023-11-24T06:26:58.221734+0000","last_undegraded":"2023-11-24T06:26:58.221734+0000","last_fullsized":"2023-11-24T06:26:58.221734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947457+0000","last_change":"2023-11-24T06:26:56.961409+0000","last_active":"2023-11-24T06:26:57.947457+0000","last_peered":"2023-11-24T06:26:57.947457+0000","last_clean":"2023-11-24T06:26:57.947457+0000","last_became_active":"2023-11-24T06:26:56.961107+0000","last_became_peered":"2023-11-24T06:26:56.961107+0000","last_unstale":"2023-11-24T06:26:57.947457+0000","last_undegraded":"2023-11-24T06:26:57.947457+0000","last_fullsized":"2023-11-24T06:26:57.947457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218244+0000","last_change":"2023-11-24T06:26:56.006966+0000","last_active":"2023-11-24T06:26:58.218244+0000","last_peered":"2023-11-24T06:26:58.218244+0000","last_clean":"2023-11-24T06:26:58.218244+0000","last_became_active":"2023-11-24T06:26:56.006596+0000","last_became_peered":"2023-11-24T06:26:56.006596+0000","last_unstale":"2023-11-24T06:26:58.218244+0000","last_undegraded":"2023-11-24T06:26:58.218244+0000","last_fullsized":"2023-11-24T06:26:58.218244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217572+0000","last_change":"2023-11-24T06:26:56.974192+0000","last_active":"2023-11-24T06:26:58.217572+0000","last_peered":"2023-11-24T06:26:58.217572+0000","last_clean":"2023-11-24T06:26:58.217572+0000","last_became_active":"2023-11-24T06:26:56.973850+0000","last_became_peered":"2023-11-24T06:26:56.973850+0000","last_unstale":"2023-11-24T06:26:58.217572+0000","last_undegraded":"2023-11-24T06:26:58.217572+0000","last_fullsized":"2023-11-24T06:26:58.217572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:51.030181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221387+0000","last_change":"2023-11-24T06:26:56.013312+0000","last_active":"2023-11-24T06:26:58.221387+0000","last_peered":"2023-11-24T06:26:58.221387+0000","last_clean":"2023-11-24T06:26:58.221387+0000","last_became_active":"2023-11-24T06:26:56.013018+0000","last_became_peered":"2023-11-24T06:26:56.013018+0000","last_unstale":"2023-11-24T06:26:58.221387+0000","last_undegraded":"2023-11-24T06:26:58.221387+0000","last_fullsized":"2023-11-24T06:26:58.221387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:03.493540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947707+0000","last_change":"2023-11-24T06:26:56.975118+0000","last_active":"2023-11-24T06:26:57.947707+0000","last_peered":"2023-11-24T06:26:57.947707+0000","last_clean":"2023-11-24T06:26:57.947707+0000","last_became_active":"2023-11-24T06:26:56.974734+0000","last_became_peered":"2023-11-24T06:26:56.974734+0000","last_unstale":"2023-11-24T06:26:57.947707+0000","last_undegraded":"2023-11-24T06:26:57.947707+0000","last_fullsized":"2023-11-24T06:26:57.947707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:15.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219402+0000","last_change":"2023-11-24T06:26:56.010113+0000","last_active":"2023-11-24T06:26:58.219402+0000","last_peered":"2023-11-24T06:26:58.219402+0000","last_clean":"2023-11-24T06:26:58.219402+0000","last_became_active":"2023-11-24T06:26:56.009550+0000","last_became_peered":"2023-11-24T06:26:56.009550+0000","last_unstale":"2023-11-24T06:26:58.219402+0000","last_undegraded":"2023-11-24T06:26:58.219402+0000","last_fullsized":"2023-11-24T06:26:58.219402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:13.157002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218000+0000","last_change":"2023-11-24T06:26:56.978061+0000","last_active":"2023-11-24T06:26:58.218000+0000","last_peered":"2023-11-24T06:26:58.218000+0000","last_clean":"2023-11-24T06:26:58.218000+0000","last_became_active":"2023-11-24T06:26:56.977760+0000","last_became_peered":"2023-11-24T06:26:56.977760+0000","last_unstale":"2023-11-24T06:26:58.218000+0000","last_undegraded":"2023-11-24T06:26:58.218000+0000","last_fullsized":"2023-11-24T06:26:58.218000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216616+0000","last_change":"2023-11-24T06:26:55.996462+0000","last_active":"2023-11-24T06:26:58.216616+0000","last_peered":"2023-11-24T06:26:58.216616+0000","last_clean":"2023-11-24T06:26:58.216616+0000","last_became_active":"2023-11-24T06:26:55.996178+0000","last_became_peered":"2023-11-24T06:26:55.996178+0000","last_unstale":"2023-11-24T06:26:58.216616+0000","last_undegraded":"2023-11-24T06:26:58.216616+0000","last_fullsized":"2023-11-24T06:26:58.216616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:59.647318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946852+0000","last_change":"2023-11-24T06:26:56.978947+0000","last_active":"2023-11-24T06:26:57.946852+0000","last_peered":"2023-11-24T06:26:57.946852+0000","last_clean":"2023-11-24T06:26:57.946852+0000","last_became_active":"2023-11-24T06:26:56.978643+0000","last_became_peered":"2023-11-24T06:26:56.978643+0000","last_unstale":"2023-11-24T06:26:57.946852+0000","last_undegraded":"2023-11-24T06:26:57.946852+0000","last_fullsized":"2023-11-24T06:26:57.946852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222582+0000","last_change":"2023-11-24T06:26:56.005504+0000","last_active":"2023-11-24T06:26:58.222582+0000","last_peered":"2023-11-24T06:26:58.222582+0000","last_clean":"2023-11-24T06:26:58.222582+0000","last_became_active":"2023-11-24T06:26:56.005287+0000","last_became_peered":"2023-11-24T06:26:56.005287+0000","last_unstale":"2023-11-24T06:26:58.222582+0000","last_undegraded":"2023-11-24T06:26:58.222582+0000","last_fullsized":"2023-11-24T06:26:58.222582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:24.763532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947843+0000","last_change":"2023-11-24T06:26:56.973083+0000","last_active":"2023-11-24T06:26:57.947843+0000","last_peered":"2023-11-24T06:26:57.947843+0000","last_clean":"2023-11-24T06:26:57.947843+0000","last_became_active":"2023-11-24T06:26:56.972750+0000","last_became_peered":"2023-11-24T06:26:56.972750+0000","last_unstale":"2023-11-24T06:26:57.947843+0000","last_undegraded":"2023-11-24T06:26:57.947843+0000","last_fullsized":"2023-11-24T06:26:57.947843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:08.969752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217141+0000","last_change":"2023-11-24T06:26:56.005050+0000","last_active":"2023-11-24T06:26:58.217141+0000","last_peered":"2023-11-24T06:26:58.217141+0000","last_clean":"2023-11-24T06:26:58.217141+0000","last_became_active":"2023-11-24T06:26:56.004658+0000","last_became_peered":"2023-11-24T06:26:56.004658+0000","last_unstale":"2023-11-24T06:26:58.217141+0000","last_undegraded":"2023-11-24T06:26:58.217141+0000","last_fullsized":"2023-11-24T06:26:58.217141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:27.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948405+0000","last_change":"2023-11-24T06:26:56.973292+0000","last_active":"2023-11-24T06:26:57.948405+0000","last_peered":"2023-11-24T06:26:57.948405+0000","last_clean":"2023-11-24T06:26:57.948405+0000","last_became_active":"2023-11-24T06:26:56.972959+0000","last_became_peered":"2023-11-24T06:26:56.972959+0000","last_unstale":"2023-11-24T06:26:57.948405+0000","last_undegraded":"2023-11-24T06:26:57.948405+0000","last_fullsized":"2023-11-24T06:26:57.948405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:48.721442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944407+0000","last_change":"2023-11-24T06:26:56.005932+0000","last_active":"2023-11-24T06:26:57.944407+0000","last_peered":"2023-11-24T06:26:57.944407+0000","last_clean":"2023-11-24T06:26:57.944407+0000","last_became_active":"2023-11-24T06:26:56.005603+0000","last_became_peered":"2023-11-24T06:26:56.005603+0000","last_unstale":"2023-11-24T06:26:57.944407+0000","last_undegraded":"2023-11-24T06:26:57.944407+0000","last_fullsized":"2023-11-24T06:26:57.944407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222520+0000","last_change":"2023-11-24T06:26:56.969503+0000","last_active":"2023-11-24T06:26:58.222520+0000","last_peered":"2023-11-24T06:26:58.222520+0000","last_clean":"2023-11-24T06:26:58.222520+0000","last_became_active":"2023-11-24T06:26:56.967704+0000","last_became_peered":"2023-11-24T06:26:56.967704+0000","last_unstale":"2023-11-24T06:26:58.222520+0000","last_undegraded":"2023-11-24T06:26:58.222520+0000","last_fullsized":"2023-11-24T06:26:58.222520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:17.662630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217140+0000","last_change":"2023-11-24T06:26:56.014819+0000","last_active":"2023-11-24T06:26:58.217140+0000","last_peered":"2023-11-24T06:26:58.217140+0000","last_clean":"2023-11-24T06:26:58.217140+0000","last_became_active":"2023-11-24T06:26:56.014287+0000","last_became_peered":"2023-11-24T06:26:56.014287+0000","last_unstale":"2023-11-24T06:26:58.217140+0000","last_undegraded":"2023-11-24T06:26:58.217140+0000","last_fullsized":"2023-11-24T06:26:58.217140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:13.157002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947402+0000","last_change":"2023-11-24T06:26:57.001053+0000","last_active":"2023-11-24T06:26:57.947402+0000","last_peered":"2023-11-24T06:26:57.947402+0000","last_clean":"2023-11-24T06:26:57.947402+0000","last_became_active":"2023-11-24T06:26:57.000802+0000","last_became_peered":"2023-11-24T06:26:57.000802+0000","last_unstale":"2023-11-24T06:26:57.947402+0000","last_undegraded":"2023-11-24T06:26:57.947402+0000","last_fullsized":"2023-11-24T06:26:57.947402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:26:56.946083+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:26:56.946083+0000","last_peered":"2023-11-24T06:26:56.946083+0000","last_clean":"2023-11-24T06:26:56.946083+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:26:56.946083+0000","last_undegraded":"2023-11-24T06:26:56.946083+0000","last_fullsized":"2023-11-24T06:26:56.946083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221515+0000","last_change":"2023-11-24T06:26:56.008553+0000","last_active":"2023-11-24T06:26:58.221515+0000","last_peered":"2023-11-24T06:26:58.221515+0000","last_clean":"2023-11-24T06:26:58.221515+0000","last_became_active":"2023-11-24T06:26:56.006284+0000","last_became_peered":"2023-11-24T06:26:56.006284+0000","last_unstale":"2023-11-24T06:26:58.221515+0000","last_undegraded":"2023-11-24T06:26:58.221515+0000","last_fullsized":"2023-11-24T06:26:58.221515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:24.763532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945070+0000","last_change":"2023-11-24T06:26:56.998859+0000","last_active":"2023-11-24T06:26:57.945070+0000","last_peered":"2023-11-24T06:26:57.945070+0000","last_clean":"2023-11-24T06:26:57.945070+0000","last_became_active":"2023-11-24T06:26:56.998524+0000","last_became_peered":"2023-11-24T06:26:56.998524+0000","last_unstale":"2023-11-24T06:26:57.945070+0000","last_undegraded":"2023-11-24T06:26:57.945070+0000","last_fullsized":"2023-11-24T06:26:57.945070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946597+0000","last_change":"2023-11-24T06:26:56.005300+0000","last_active":"2023-11-24T06:26:57.946597+0000","last_peered":"2023-11-24T06:26:57.946597+0000","last_clean":"2023-11-24T06:26:57.946597+0000","last_became_active":"2023-11-24T06:26:56.004984+0000","last_became_peered":"2023-11-24T06:26:56.004984+0000","last_unstale":"2023-11-24T06:26:57.946597+0000","last_undegraded":"2023-11-24T06:26:57.946597+0000","last_fullsized":"2023-11-24T06:26:57.946597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217825+0000","last_change":"2023-11-24T06:26:56.984062+0000","last_active":"2023-11-24T06:26:58.217825+0000","last_peered":"2023-11-24T06:26:58.217825+0000","last_clean":"2023-11-24T06:26:58.217825+0000","last_became_active":"2023-11-24T06:26:56.983679+0000","last_became_peered":"2023-11-24T06:26:56.983679+0000","last_unstale":"2023-11-24T06:26:58.217825+0000","last_undegraded":"2023-11-24T06:26:58.217825+0000","last_fullsized":"2023-11-24T06:26:58.217825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:58.210301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216848+0000","last_change":"2023-11-24T06:26:56.005001+0000","last_active":"2023-11-24T06:26:58.216848+0000","last_peered":"2023-11-24T06:26:58.216848+0000","last_clean":"2023-11-24T06:26:58.216848+0000","last_became_active":"2023-11-24T06:26:56.004612+0000","last_became_peered":"2023-11-24T06:26:56.004612+0000","last_unstale":"2023-11-24T06:26:58.216848+0000","last_undegraded":"2023-11-24T06:26:58.216848+0000","last_fullsized":"2023-11-24T06:26:58.216848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944532+0000","last_change":"2023-11-24T06:26:56.965796+0000","last_active":"2023-11-24T06:26:57.944532+0000","last_peered":"2023-11-24T06:26:57.944532+0000","last_clean":"2023-11-24T06:26:57.944532+0000","last_became_active":"2023-11-24T06:26:56.963344+0000","last_became_peered":"2023-11-24T06:26:56.963344+0000","last_unstale":"2023-11-24T06:26:57.944532+0000","last_undegraded":"2023-11-24T06:26:57.944532+0000","last_fullsized":"2023-11-24T06:26:57.944532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221498+0000","last_change":"2023-11-24T06:26:56.012192+0000","last_active":"2023-11-24T06:26:58.221498+0000","last_peered":"2023-11-24T06:26:58.221498+0000","last_clean":"2023-11-24T06:26:58.221498+0000","last_became_active":"2023-11-24T06:26:56.011796+0000","last_became_peered":"2023-11-24T06:26:56.011796+0000","last_unstale":"2023-11-24T06:26:58.221498+0000","last_undegraded":"2023-11-24T06:26:58.221498+0000","last_fullsized":"2023-11-24T06:26:58.221498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:50.399725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220548+0000","last_change":"2023-11-24T06:26:56.985067+0000","last_active":"2023-11-24T06:26:58.220548+0000","last_peered":"2023-11-24T06:26:58.220548+0000","last_clean":"2023-11-24T06:26:58.220548+0000","last_became_active":"2023-11-24T06:26:56.984360+0000","last_became_peered":"2023-11-24T06:26:56.984360+0000","last_unstale":"2023-11-24T06:26:58.220548+0000","last_undegraded":"2023-11-24T06:26:58.220548+0000","last_fullsized":"2023-11-24T06:26:58.220548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:50.371449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944321+0000","last_change":"2023-11-24T06:26:55.996919+0000","last_active":"2023-11-24T06:26:57.944321+0000","last_peered":"2023-11-24T06:26:57.944321+0000","last_clean":"2023-11-24T06:26:57.944321+0000","last_became_active":"2023-11-24T06:26:55.996431+0000","last_became_peered":"2023-11-24T06:26:55.996431+0000","last_unstale":"2023-11-24T06:26:57.944321+0000","last_undegraded":"2023-11-24T06:26:57.944321+0000","last_fullsized":"2023-11-24T06:26:57.944321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222871+0000","last_change":"2023-11-24T06:26:56.987025+0000","last_active":"2023-11-24T06:26:58.222871+0000","last_peered":"2023-11-24T06:26:58.222871+0000","last_clean":"2023-11-24T06:26:58.222871+0000","last_became_active":"2023-11-24T06:26:56.985920+0000","last_became_peered":"2023-11-24T06:26:56.985920+0000","last_unstale":"2023-11-24T06:26:58.222871+0000","last_undegraded":"2023-11-24T06:26:58.222871+0000","last_fullsized":"2023-11-24T06:26:58.222871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:08.969752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947843+0000","last_change":"2023-11-24T06:26:56.015006+0000","last_active":"2023-11-24T06:26:57.947843+0000","last_peered":"2023-11-24T06:26:57.947843+0000","last_clean":"2023-11-24T06:26:57.947843+0000","last_became_active":"2023-11-24T06:26:56.011264+0000","last_became_peered":"2023-11-24T06:26:56.011264+0000","last_unstale":"2023-11-24T06:26:57.947843+0000","last_undegraded":"2023-11-24T06:26:57.947843+0000","last_fullsized":"2023-11-24T06:26:57.947843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:51.788764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944165+0000","last_change":"2023-11-24T06:26:56.967275+0000","last_active":"2023-11-24T06:26:57.944165+0000","last_peered":"2023-11-24T06:26:57.944165+0000","last_clean":"2023-11-24T06:26:57.944165+0000","last_became_active":"2023-11-24T06:26:56.966261+0000","last_became_peered":"2023-11-24T06:26:56.966261+0000","last_unstale":"2023-11-24T06:26:57.944165+0000","last_undegraded":"2023-11-24T06:26:57.944165+0000","last_fullsized":"2023-11-24T06:26:57.944165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944646+0000","last_change":"2023-11-24T06:26:56.011770+0000","last_active":"2023-11-24T06:26:57.944646+0000","last_peered":"2023-11-24T06:26:57.944646+0000","last_clean":"2023-11-24T06:26:57.944646+0000","last_became_active":"2023-11-24T06:26:56.011386+0000","last_became_peered":"2023-11-24T06:26:56.011386+0000","last_unstale":"2023-11-24T06:26:57.944646+0000","last_undegraded":"2023-11-24T06:26:57.944646+0000","last_fullsized":"2023-11-24T06:26:57.944646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945877+0000","last_change":"2023-11-24T06:26:56.978788+0000","last_active":"2023-11-24T06:26:57.945877+0000","last_peered":"2023-11-24T06:26:57.945877+0000","last_clean":"2023-11-24T06:26:57.945877+0000","last_became_active":"2023-11-24T06:26:56.978112+0000","last_became_peered":"2023-11-24T06:26:56.978112+0000","last_unstale":"2023-11-24T06:26:57.945877+0000","last_undegraded":"2023-11-24T06:26:57.945877+0000","last_fullsized":"2023-11-24T06:26:57.945877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220478+0000","last_change":"2023-11-24T06:26:56.012367+0000","last_active":"2023-11-24T06:26:58.220478+0000","last_peered":"2023-11-24T06:26:58.220478+0000","last_clean":"2023-11-24T06:26:58.220478+0000","last_became_active":"2023-11-24T06:26:56.011973+0000","last_became_peered":"2023-11-24T06:26:56.011973+0000","last_unstale":"2023-11-24T06:26:58.220478+0000","last_undegraded":"2023-11-24T06:26:58.220478+0000","last_fullsized":"2023-11-24T06:26:58.220478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218351+0000","last_change":"2023-11-24T06:26:56.976256+0000","last_active":"2023-11-24T06:26:58.218351+0000","last_peered":"2023-11-24T06:26:58.218351+0000","last_clean":"2023-11-24T06:26:58.218351+0000","last_became_active":"2023-11-24T06:26:56.975889+0000","last_became_peered":"2023-11-24T06:26:56.975889+0000","last_unstale":"2023-11-24T06:26:58.218351+0000","last_undegraded":"2023-11-24T06:26:58.218351+0000","last_fullsized":"2023-11-24T06:26:58.218351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946577+0000","last_change":"2023-11-24T06:26:56.008104+0000","last_active":"2023-11-24T06:26:57.946577+0000","last_peered":"2023-11-24T06:26:57.946577+0000","last_clean":"2023-11-24T06:26:57.946577+0000","last_became_active":"2023-11-24T06:26:56.007669+0000","last_became_peered":"2023-11-24T06:26:56.007669+0000","last_unstale":"2023-11-24T06:26:57.946577+0000","last_undegraded":"2023-11-24T06:26:57.946577+0000","last_fullsized":"2023-11-24T06:26:57.946577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:27.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220486+0000","last_change":"2023-11-24T06:26:56.974769+0000","last_active":"2023-11-24T06:26:58.220486+0000","last_peered":"2023-11-24T06:26:58.220486+0000","last_clean":"2023-11-24T06:26:58.220486+0000","last_became_active":"2023-11-24T06:26:56.974530+0000","last_became_peered":"2023-11-24T06:26:56.974530+0000","last_unstale":"2023-11-24T06:26:58.220486+0000","last_undegraded":"2023-11-24T06:26:58.220486+0000","last_fullsized":"2023-11-24T06:26:58.220486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:02.036523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944166+0000","last_change":"2023-11-24T06:26:56.001637+0000","last_active":"2023-11-24T06:26:57.944166+0000","last_peered":"2023-11-24T06:26:57.944166+0000","last_clean":"2023-11-24T06:26:57.944166+0000","last_became_active":"2023-11-24T06:26:56.000662+0000","last_became_peered":"2023-11-24T06:26:56.000662+0000","last_unstale":"2023-11-24T06:26:57.944166+0000","last_undegraded":"2023-11-24T06:26:57.944166+0000","last_fullsized":"2023-11-24T06:26:57.944166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:49.146302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945138+0000","last_change":"2023-11-24T06:26:56.961879+0000","last_active":"2023-11-24T06:26:57.945138+0000","last_peered":"2023-11-24T06:26:57.945138+0000","last_clean":"2023-11-24T06:26:57.945138+0000","last_became_active":"2023-11-24T06:26:56.961593+0000","last_became_peered":"2023-11-24T06:26:56.961593+0000","last_unstale":"2023-11-24T06:26:57.945138+0000","last_undegraded":"2023-11-24T06:26:57.945138+0000","last_fullsized":"2023-11-24T06:26:57.945138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944425+0000","last_change":"2023-11-24T06:26:55.995684+0000","last_active":"2023-11-24T06:26:57.944425+0000","last_peered":"2023-11-24T06:26:57.944425+0000","last_clean":"2023-11-24T06:26:57.944425+0000","last_became_active":"2023-11-24T06:26:55.994739+0000","last_became_peered":"2023-11-24T06:26:55.994739+0000","last_unstale":"2023-11-24T06:26:57.944425+0000","last_undegraded":"2023-11-24T06:26:57.944425+0000","last_fullsized":"2023-11-24T06:26:57.944425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:59.647318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945536+0000","last_change":"2023-11-24T06:26:56.976564+0000","last_active":"2023-11-24T06:26:57.945536+0000","last_peered":"2023-11-24T06:26:57.945536+0000","last_clean":"2023-11-24T06:26:57.945536+0000","last_became_active":"2023-11-24T06:26:56.976081+0000","last_became_peered":"2023-11-24T06:26:56.976081+0000","last_unstale":"2023-11-24T06:26:57.945536+0000","last_undegraded":"2023-11-24T06:26:57.945536+0000","last_fullsized":"2023-11-24T06:26:57.945536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947699+0000","last_change":"2023-11-24T06:26:56.015851+0000","last_active":"2023-11-24T06:26:57.947699+0000","last_peered":"2023-11-24T06:26:57.947699+0000","last_clean":"2023-11-24T06:26:57.947699+0000","last_became_active":"2023-11-24T06:26:56.014016+0000","last_became_peered":"2023-11-24T06:26:56.014016+0000","last_unstale":"2023-11-24T06:26:57.947699+0000","last_undegraded":"2023-11-24T06:26:57.947699+0000","last_fullsized":"2023-11-24T06:26:57.947699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218077+0000","last_change":"2023-11-24T06:26:56.975486+0000","last_active":"2023-11-24T06:26:58.218077+0000","last_peered":"2023-11-24T06:26:58.218077+0000","last_clean":"2023-11-24T06:26:58.218077+0000","last_became_active":"2023-11-24T06:26:56.975147+0000","last_became_peered":"2023-11-24T06:26:56.975147+0000","last_unstale":"2023-11-24T06:26:58.218077+0000","last_undegraded":"2023-11-24T06:26:58.218077+0000","last_fullsized":"2023-11-24T06:26:58.218077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222273+0000","last_change":"2023-11-24T06:26:56.994982+0000","last_active":"2023-11-24T06:26:58.222273+0000","last_peered":"2023-11-24T06:26:58.222273+0000","last_clean":"2023-11-24T06:26:58.222273+0000","last_became_active":"2023-11-24T06:26:56.993765+0000","last_became_peered":"2023-11-24T06:26:56.993765+0000","last_unstale":"2023-11-24T06:26:58.222273+0000","last_undegraded":"2023-11-24T06:26:58.222273+0000","last_fullsized":"2023-11-24T06:26:58.222273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947612+0000","last_change":"2023-11-24T06:26:56.015039+0000","last_active":"2023-11-24T06:26:57.947612+0000","last_peered":"2023-11-24T06:26:57.947612+0000","last_clean":"2023-11-24T06:26:57.947612+0000","last_became_active":"2023-11-24T06:26:56.014562+0000","last_became_peered":"2023-11-24T06:26:56.014562+0000","last_unstale":"2023-11-24T06:26:57.947612+0000","last_undegraded":"2023-11-24T06:26:57.947612+0000","last_fullsized":"2023-11-24T06:26:57.947612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222503+0000","last_change":"2023-11-24T06:26:56.992781+0000","last_active":"2023-11-24T06:26:58.222503+0000","last_peered":"2023-11-24T06:26:58.222503+0000","last_clean":"2023-11-24T06:26:58.222503+0000","last_became_active":"2023-11-24T06:26:56.992337+0000","last_became_peered":"2023-11-24T06:26:56.992337+0000","last_unstale":"2023-11-24T06:26:58.222503+0000","last_undegraded":"2023-11-24T06:26:58.222503+0000","last_fullsized":"2023-11-24T06:26:58.222503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:15.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947644+0000","last_change":"2023-11-24T06:26:56.015616+0000","last_active":"2023-11-24T06:26:57.947644+0000","last_peered":"2023-11-24T06:26:57.947644+0000","last_clean":"2023-11-24T06:26:57.947644+0000","last_became_active":"2023-11-24T06:26:56.015317+0000","last_became_peered":"2023-11-24T06:26:56.015317+0000","last_unstale":"2023-11-24T06:26:57.947644+0000","last_undegraded":"2023-11-24T06:26:57.947644+0000","last_fullsized":"2023-11-24T06:26:57.947644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:10.844401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218457+0000","last_change":"2023-11-24T06:26:57.001097+0000","last_active":"2023-11-24T06:26:58.218457+0000","last_peered":"2023-11-24T06:26:58.218457+0000","last_clean":"2023-11-24T06:26:58.218457+0000","last_became_active":"2023-11-24T06:26:57.000751+0000","last_became_peered":"2023-11-24T06:26:57.000751+0000","last_unstale":"2023-11-24T06:26:58.218457+0000","last_undegraded":"2023-11-24T06:26:58.218457+0000","last_fullsized":"2023-11-24T06:26:58.218457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945720+0000","last_change":"2023-11-24T06:26:56.003621+0000","last_active":"2023-11-24T06:26:57.945720+0000","last_peered":"2023-11-24T06:26:57.945720+0000","last_clean":"2023-11-24T06:26:57.945720+0000","last_became_active":"2023-11-24T06:26:56.003232+0000","last_became_peered":"2023-11-24T06:26:56.003232+0000","last_unstale":"2023-11-24T06:26:57.945720+0000","last_undegraded":"2023-11-24T06:26:57.945720+0000","last_fullsized":"2023-11-24T06:26:57.945720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:10.844401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945648+0000","last_change":"2023-11-24T06:26:56.972822+0000","last_active":"2023-11-24T06:26:57.945648+0000","last_peered":"2023-11-24T06:26:57.945648+0000","last_clean":"2023-11-24T06:26:57.945648+0000","last_became_active":"2023-11-24T06:26:56.972529+0000","last_became_peered":"2023-11-24T06:26:56.972529+0000","last_unstale":"2023-11-24T06:26:57.945648+0000","last_undegraded":"2023-11-24T06:26:57.945648+0000","last_fullsized":"2023-11-24T06:26:57.945648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218623+0000","last_change":"2023-11-24T06:26:56.009185+0000","last_active":"2023-11-24T06:26:58.218623+0000","last_peered":"2023-11-24T06:26:58.218623+0000","last_clean":"2023-11-24T06:26:58.218623+0000","last_became_active":"2023-11-24T06:26:56.008855+0000","last_became_peered":"2023-11-24T06:26:56.008855+0000","last_unstale":"2023-11-24T06:26:58.218623+0000","last_undegraded":"2023-11-24T06:26:58.218623+0000","last_fullsized":"2023-11-24T06:26:58.218623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.657}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0939999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3519999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.571}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.788}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0960000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:01.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.762+0000 7fde808bb640 1 Processor -- start 2023-11-24T06:27:01.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.763+0000 7fde808bb640 1 -- start start 2023-11-24T06:27:01.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.764+0000 7fde808bb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fde7c143c40 0x7fde7c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.764+0000 7fde808bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde7c0ccc10 0x7fde7c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.764+0000 7fde808bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde7c070e50 con 0x7fde7c0717d0 2023-11-24T06:27:01.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.764+0000 7fde808bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7c0723c0 con 0x7fde7c0ccc10 2023-11-24T06:27:01.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.764+0000 7fde808bb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7c144020 con 0x7fde7c143c40 2023-11-24T06:27:01.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fde7c0717d0 0x7fde7c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35000/0 (socket says 172.21.15.16:35000) 2023-11-24T06:27:01.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7affd640 1 -- 172.21.15.16:0/3594175750 learned_addr learned my addr 172.21.15.16:0/3594175750 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:01.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7bfff640 1 --2- 172.21.15.16:0/3594175750 >> v2:172.21.15.16:3300/0 conn(0x7fde7c0ccc10 0x7fde7c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7b7fe640 1 --2- 172.21.15.16:0/3594175750 >> v2:172.21.15.173:3300/0 conn(0x7fde7c143c40 0x7fde7c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7a7fc640 1 -- 172.21.15.16:0/3594175750 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234490393 0 0) 0x7fde7c070e50 con 0x7fde7c0717d0 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7a7fc640 1 -- 172.21.15.16:0/3594175750 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde64003520 con 0x7fde7c0717d0 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7bfff640 1 -- 172.21.15.16:0/3594175750 >> v2:172.21.15.173:3300/0 conn(0x7fde7c143c40 msgr2=0x7fde7c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7bfff640 1 --2- 172.21.15.16:0/3594175750 >> v2:172.21.15.173:3300/0 conn(0x7fde7c143c40 0x7fde7c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7bfff640 1 -- 172.21.15.16:0/3594175750 >> v1:172.21.15.16:6789/0 conn(0x7fde7c0717d0 legacy=0x7fde7c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.765+0000 7fde7bfff640 1 -- 172.21.15.16:0/3594175750 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c071e00 con 0x7fde7c0ccc10 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.766+0000 7fde7bfff640 1 --2- 172.21.15.16:0/3594175750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 0x7fde7c06d160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fde7c06e410 tx=0x7fde7402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870af53624a7d9ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.767+0000 7fde7a7fc640 1 -- 172.21.15.16:0/3594175750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7402cbe0 con 0x7fde7c0ccc10 2023-11-24T06:27:01.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.767+0000 7fde7a7fc640 1 -- 172.21.15.16:0/3594175750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7402cd40 con 0x7fde7c0ccc10 2023-11-24T06:27:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.767+0000 7fde7a7fc640 1 -- 172.21.15.16:0/3594175750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74034660 con 0x7fde7c0ccc10 2023-11-24T06:27:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.767+0000 7fde808bb640 1 -- 172.21.15.16:0/3594175750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 msgr2=0x7fde7c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:01.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.767+0000 7fde808bb640 1 --2- 172.21.15.16:0/3594175750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 0x7fde7c06d160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fde7c06e410 tx=0x7fde7402be30 comp rx=0 tx=0).stop 2023-11-24T06:27:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.768+0000 7fde808bb640 1 -- 172.21.15.16:0/3594175750 shutdown_connections 2023-11-24T06:27:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.768+0000 7fde808bb640 1 --2- 172.21.15.16:0/3594175750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 0x7fde7c06d160 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.768+0000 7fde808bb640 1 --2- 172.21.15.16:0/3594175750 >> v2:172.21.15.173:3300/0 conn(0x7fde7c143c40 0x7fde7c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.768+0000 7fde808bb640 1 -- 172.21.15.16:0/3594175750 >> 172.21.15.16:0/3594175750 conn(0x7fde7c06a800 msgr2=0x7fde7c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.768+0000 7fde808bb640 1 -- 172.21.15.16:0/3594175750 shutdown_connections 2023-11-24T06:27:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.768+0000 7fde808bb640 1 -- 172.21.15.16:0/3594175750 wait complete. 2023-11-24T06:27:01.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.770+0000 7fde808bb640 1 Processor -- start 2023-11-24T06:27:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.770+0000 7fde808bb640 1 -- start start 2023-11-24T06:27:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde808bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde7c0717d0 0x7fde7c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde808bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 0x7fde7c142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde808bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde7c143c40 0x7fde7c15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde808bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c0cb300 con 0x7fde7c0ccc10 2023-11-24T06:27:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde808bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c0ca040 con 0x7fde7c143c40 2023-11-24T06:27:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde808bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c0c4b10 con 0x7fde7c0717d0 2023-11-24T06:27:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde7affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 0x7fde7c142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde7affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 0x7fde7c142a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40636/0 (socket says 172.21.15.16:40636) 2023-11-24T06:27:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde7affd640 1 -- 172.21.15.16:0/2824129857 learned_addr learned my addr 172.21.15.16:0/2824129857 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde7b7fe640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde7c0717d0 0x7fde7c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.771+0000 7fde7bfff640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde7c143c40 0x7fde7c15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde7affd640 1 -- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde7c0717d0 msgr2=0x7fde7c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde7affd640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde7c0717d0 0x7fde7c0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde7affd640 1 -- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde7c143c40 msgr2=0x7fde7c15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde7affd640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde7c143c40 0x7fde7c15a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde7affd640 1 -- 172.21.15.16:0/2824129857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde74009a70 con 0x7fde7c0ccc10 2023-11-24T06:27:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde7affd640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 0x7fde7c142a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde7000c410 tx=0x7fde7000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde63fff640 1 -- 172.21.15.16:0/2824129857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70005ce0 con 0x7fde7c0ccc10 2023-11-24T06:27:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde63fff640 1 -- 172.21.15.16:0/2824129857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70005e40 con 0x7fde7c0ccc10 2023-11-24T06:27:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde63fff640 1 -- 172.21.15.16:0/2824129857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde700072f0 con 0x7fde7c0ccc10 2023-11-24T06:27:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.772+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c0ca2e0 con 0x7fde7c0ccc10 2023-11-24T06:27:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.773+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c0ca8a0 con 0x7fde7c0ccc10 2023-11-24T06:27:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.774+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7c067450 con 0x7fde7c0ccc10 2023-11-24T06:27:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.775+0000 7fde63fff640 1 -- 172.21.15.16:0/2824129857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde70007450 con 0x7fde7c0ccc10 2023-11-24T06:27:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.775+0000 7fde63fff640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6806ae40 0x7fde6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.776+0000 7fde63fff640 1 -- 172.21.15.16:0/2824129857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde70088c50 con 0x7fde7c0ccc10 2023-11-24T06:27:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.776+0000 7fde7b7fe640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6806ae40 0x7fde6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:01.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.777+0000 7fde7b7fe640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6806ae40 0x7fde6806d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde6c006fd0 tx=0x7fde6c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:01.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.780+0000 7fde63fff640 1 -- 172.21.15.16:0/2824129857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde7008e020 con 0x7fde7c0ccc10 2023-11-24T06:27:01.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.970+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde7c06cc40 con 0x7fde6806ae40 2023-11-24T06:27:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.991+0000 7fde63fff640 1 -- 172.21.15.16:0/2824129857 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264501 (secure 0 0 0) 0x7fde6c00b200 con 0x7fde6806ae40 2023-11-24T06:27:01.996 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:01.999 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:27:02.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6806ae40 msgr2=0x7fde6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6806ae40 0x7fde6806d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde6c006fd0 tx=0x7fde6c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 msgr2=0x7fde7c142a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 0x7fde7c142a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde7000c410 tx=0x7fde7000c8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 shutdown_connections 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6806ae40 0x7fde6806d300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde7c143c40 0x7fde7c15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7c0ccc10 0x7fde7c142a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 --2- 172.21.15.16:0/2824129857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde7c0717d0 0x7fde7c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 >> 172.21.15.16:0/2824129857 conn(0x7fde7c06a800 msgr2=0x7fde7c0cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 shutdown_connections 2023-11-24T06:27:02.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:01.998+0000 7fde808bb640 1 -- 172.21.15.16:0/2824129857 wait complete. 2023-11-24T06:27:02.022 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-11-24T06:27:00.101266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235876,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738844,"statfs":{"total":773094113280,"available":772852576256,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.133694"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948224+0000","last_change":"2023-11-24T06:26:56.969865+0000","last_active":"2023-11-24T06:26:57.948224+0000","last_peered":"2023-11-24T06:26:57.948224+0000","last_clean":"2023-11-24T06:26:57.948224+0000","last_became_active":"2023-11-24T06:26:56.969570+0000","last_became_peered":"2023-11-24T06:26:56.969570+0000","last_unstale":"2023-11-24T06:26:57.948224+0000","last_undegraded":"2023-11-24T06:26:57.948224+0000","last_fullsized":"2023-11-24T06:26:57.948224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:56.199949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944608+0000","last_change":"2023-11-24T06:26:56.979241+0000","last_active":"2023-11-24T06:26:57.944608+0000","last_peered":"2023-11-24T06:26:57.944608+0000","last_clean":"2023-11-24T06:26:57.944608+0000","last_became_active":"2023-11-24T06:26:56.978864+0000","last_became_peered":"2023-11-24T06:26:56.978864+0000","last_unstale":"2023-11-24T06:26:57.944608+0000","last_undegraded":"2023-11-24T06:26:57.944608+0000","last_fullsized":"2023-11-24T06:26:57.944608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218032+0000","last_change":"2023-11-24T06:26:56.976436+0000","last_active":"2023-11-24T06:26:58.218032+0000","last_peered":"2023-11-24T06:26:58.218032+0000","last_clean":"2023-11-24T06:26:58.218032+0000","last_became_active":"2023-11-24T06:26:56.976114+0000","last_became_peered":"2023-11-24T06:26:56.976114+0000","last_unstale":"2023-11-24T06:26:58.218032+0000","last_undegraded":"2023-11-24T06:26:58.218032+0000","last_fullsized":"2023-11-24T06:26:58.218032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222085+0000","last_change":"2023-11-24T06:26:56.992564+0000","last_active":"2023-11-24T06:26:58.222085+0000","last_peered":"2023-11-24T06:26:58.222085+0000","last_clean":"2023-11-24T06:26:58.222085+0000","last_became_active":"2023-11-24T06:26:56.992045+0000","last_became_peered":"2023-11-24T06:26:56.992045+0000","last_unstale":"2023-11-24T06:26:58.222085+0000","last_undegraded":"2023-11-24T06:26:58.222085+0000","last_fullsized":"2023-11-24T06:26:58.222085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947424+0000","last_change":"2023-11-24T06:26:56.964941+0000","last_active":"2023-11-24T06:26:57.947424+0000","last_peered":"2023-11-24T06:26:57.947424+0000","last_clean":"2023-11-24T06:26:57.947424+0000","last_became_active":"2023-11-24T06:26:56.964244+0000","last_became_peered":"2023-11-24T06:26:56.964244+0000","last_unstale":"2023-11-24T06:26:57.947424+0000","last_undegraded":"2023-11-24T06:26:57.947424+0000","last_fullsized":"2023-11-24T06:26:57.947424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219175+0000","last_change":"2023-11-24T06:26:56.993038+0000","last_active":"2023-11-24T06:26:58.219175+0000","last_peered":"2023-11-24T06:26:58.219175+0000","last_clean":"2023-11-24T06:26:58.219175+0000","last_became_active":"2023-11-24T06:26:56.986387+0000","last_became_peered":"2023-11-24T06:26:56.986387+0000","last_unstale":"2023-11-24T06:26:58.219175+0000","last_undegraded":"2023-11-24T06:26:58.219175+0000","last_fullsized":"2023-11-24T06:26:58.219175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946794+0000","last_change":"2023-11-24T06:26:56.977591+0000","last_active":"2023-11-24T06:26:57.946794+0000","last_peered":"2023-11-24T06:26:57.946794+0000","last_clean":"2023-11-24T06:26:57.946794+0000","last_became_active":"2023-11-24T06:26:56.976851+0000","last_became_peered":"2023-11-24T06:26:56.976851+0000","last_unstale":"2023-11-24T06:26:57.946794+0000","last_undegraded":"2023-11-24T06:26:57.946794+0000","last_fullsized":"2023-11-24T06:26:57.946794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944925+0000","last_change":"2023-11-24T06:26:56.991219+0000","last_active":"2023-11-24T06:26:57.944925+0000","last_peered":"2023-11-24T06:26:57.944925+0000","last_clean":"2023-11-24T06:26:57.944925+0000","last_became_active":"2023-11-24T06:26:56.990836+0000","last_became_peered":"2023-11-24T06:26:56.990836+0000","last_unstale":"2023-11-24T06:26:57.944925+0000","last_undegraded":"2023-11-24T06:26:57.944925+0000","last_fullsized":"2023-11-24T06:26:57.944925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948378+0000","last_change":"2023-11-24T06:26:57.001238+0000","last_active":"2023-11-24T06:26:57.948378+0000","last_peered":"2023-11-24T06:26:57.948378+0000","last_clean":"2023-11-24T06:26:57.948378+0000","last_became_active":"2023-11-24T06:26:57.000919+0000","last_became_peered":"2023-11-24T06:26:57.000919+0000","last_unstale":"2023-11-24T06:26:57.948378+0000","last_undegraded":"2023-11-24T06:26:57.948378+0000","last_fullsized":"2023-11-24T06:26:57.948378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:38.348575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219671+0000","last_change":"2023-11-24T06:26:56.998442+0000","last_active":"2023-11-24T06:26:58.219671+0000","last_peered":"2023-11-24T06:26:58.219671+0000","last_clean":"2023-11-24T06:26:58.219671+0000","last_became_active":"2023-11-24T06:26:56.998097+0000","last_became_peered":"2023-11-24T06:26:56.998097+0000","last_unstale":"2023-11-24T06:26:58.219671+0000","last_undegraded":"2023-11-24T06:26:58.219671+0000","last_fullsized":"2023-11-24T06:26:58.219671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218262+0000","last_change":"2023-11-24T06:26:57.000349+0000","last_active":"2023-11-24T06:26:58.218262+0000","last_peered":"2023-11-24T06:26:58.218262+0000","last_clean":"2023-11-24T06:26:58.218262+0000","last_became_active":"2023-11-24T06:26:57.000122+0000","last_became_peered":"2023-11-24T06:26:57.000122+0000","last_unstale":"2023-11-24T06:26:58.218262+0000","last_undegraded":"2023-11-24T06:26:58.218262+0000","last_fullsized":"2023-11-24T06:26:58.218262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221702+0000","last_change":"2023-11-24T06:26:56.969362+0000","last_active":"2023-11-24T06:26:58.221702+0000","last_peered":"2023-11-24T06:26:58.221702+0000","last_clean":"2023-11-24T06:26:58.221702+0000","last_became_active":"2023-11-24T06:26:56.969057+0000","last_became_peered":"2023-11-24T06:26:56.969057+0000","last_unstale":"2023-11-24T06:26:58.221702+0000","last_undegraded":"2023-11-24T06:26:58.221702+0000","last_fullsized":"2023-11-24T06:26:58.221702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:04.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945262+0000","last_change":"2023-11-24T06:26:56.970657+0000","last_active":"2023-11-24T06:26:57.945262+0000","last_peered":"2023-11-24T06:26:57.945262+0000","last_clean":"2023-11-24T06:26:57.945262+0000","last_became_active":"2023-11-24T06:26:56.970348+0000","last_became_peered":"2023-11-24T06:26:56.970348+0000","last_unstale":"2023-11-24T06:26:57.945262+0000","last_undegraded":"2023-11-24T06:26:57.945262+0000","last_fullsized":"2023-11-24T06:26:57.945262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217421+0000","last_change":"2023-11-24T06:26:56.972241+0000","last_active":"2023-11-24T06:26:58.217421+0000","last_peered":"2023-11-24T06:26:58.217421+0000","last_clean":"2023-11-24T06:26:58.217421+0000","last_became_active":"2023-11-24T06:26:56.971982+0000","last_became_peered":"2023-11-24T06:26:56.971982+0000","last_unstale":"2023-11-24T06:26:58.217421+0000","last_undegraded":"2023-11-24T06:26:58.217421+0000","last_fullsized":"2023-11-24T06:26:58.217421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:19.450311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944041+0000","last_change":"2023-11-24T06:26:56.978171+0000","last_active":"2023-11-24T06:26:57.944041+0000","last_peered":"2023-11-24T06:26:57.944041+0000","last_clean":"2023-11-24T06:26:57.944041+0000","last_became_active":"2023-11-24T06:26:56.977851+0000","last_became_peered":"2023-11-24T06:26:56.977851+0000","last_unstale":"2023-11-24T06:26:57.944041+0000","last_undegraded":"2023-11-24T06:26:57.944041+0000","last_fullsized":"2023-11-24T06:26:57.944041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218965+0000","last_change":"2023-11-24T06:26:56.991619+0000","last_active":"2023-11-24T06:26:58.218965+0000","last_peered":"2023-11-24T06:26:58.218965+0000","last_clean":"2023-11-24T06:26:58.218965+0000","last_became_active":"2023-11-24T06:26:56.986865+0000","last_became_peered":"2023-11-24T06:26:56.986865+0000","last_unstale":"2023-11-24T06:26:58.218965+0000","last_undegraded":"2023-11-24T06:26:58.218965+0000","last_fullsized":"2023-11-24T06:26:58.218965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.223090+0000","last_change":"2023-11-24T06:26:56.997282+0000","last_active":"2023-11-24T06:26:58.223090+0000","last_peered":"2023-11-24T06:26:58.223090+0000","last_clean":"2023-11-24T06:26:58.223090+0000","last_became_active":"2023-11-24T06:26:56.995631+0000","last_became_peered":"2023-11-24T06:26:56.995631+0000","last_unstale":"2023-11-24T06:26:58.223090+0000","last_undegraded":"2023-11-24T06:26:58.223090+0000","last_fullsized":"2023-11-24T06:26:58.223090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:38.348575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217200+0000","last_change":"2023-11-24T06:26:56.969721+0000","last_active":"2023-11-24T06:26:58.217200+0000","last_peered":"2023-11-24T06:26:58.217200+0000","last_clean":"2023-11-24T06:26:58.217200+0000","last_became_active":"2023-11-24T06:26:56.969475+0000","last_became_peered":"2023-11-24T06:26:56.969475+0000","last_unstale":"2023-11-24T06:26:58.217200+0000","last_undegraded":"2023-11-24T06:26:58.217200+0000","last_fullsized":"2023-11-24T06:26:58.217200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:04.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947802+0000","last_change":"2023-11-24T06:26:56.964126+0000","last_active":"2023-11-24T06:26:57.947802+0000","last_peered":"2023-11-24T06:26:57.947802+0000","last_clean":"2023-11-24T06:26:57.947802+0000","last_became_active":"2023-11-24T06:26:56.963622+0000","last_became_peered":"2023-11-24T06:26:56.963622+0000","last_unstale":"2023-11-24T06:26:57.947802+0000","last_undegraded":"2023-11-24T06:26:57.947802+0000","last_fullsized":"2023-11-24T06:26:57.947802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:17.662630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218464+0000","last_change":"2023-11-24T06:26:56.975222+0000","last_active":"2023-11-24T06:26:58.218464+0000","last_peered":"2023-11-24T06:26:58.218464+0000","last_clean":"2023-11-24T06:26:58.218464+0000","last_became_active":"2023-11-24T06:26:56.974069+0000","last_became_peered":"2023-11-24T06:26:56.974069+0000","last_unstale":"2023-11-24T06:26:58.218464+0000","last_undegraded":"2023-11-24T06:26:58.218464+0000","last_fullsized":"2023-11-24T06:26:58.218464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218120+0000","last_change":"2023-11-24T06:26:56.975253+0000","last_active":"2023-11-24T06:26:58.218120+0000","last_peered":"2023-11-24T06:26:58.218120+0000","last_clean":"2023-11-24T06:26:58.218120+0000","last_became_active":"2023-11-24T06:26:56.974888+0000","last_became_peered":"2023-11-24T06:26:56.974888+0000","last_unstale":"2023-11-24T06:26:58.218120+0000","last_undegraded":"2023-11-24T06:26:58.218120+0000","last_fullsized":"2023-11-24T06:26:58.218120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221143+0000","last_change":"2023-11-24T06:26:56.985344+0000","last_active":"2023-11-24T06:26:58.221143+0000","last_peered":"2023-11-24T06:26:58.221143+0000","last_clean":"2023-11-24T06:26:58.221143+0000","last_became_active":"2023-11-24T06:26:56.984528+0000","last_became_peered":"2023-11-24T06:26:56.984528+0000","last_unstale":"2023-11-24T06:26:58.221143+0000","last_undegraded":"2023-11-24T06:26:58.221143+0000","last_fullsized":"2023-11-24T06:26:58.221143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944974+0000","last_change":"2023-11-24T06:26:56.979949+0000","last_active":"2023-11-24T06:26:57.944974+0000","last_peered":"2023-11-24T06:26:57.944974+0000","last_clean":"2023-11-24T06:26:57.944974+0000","last_became_active":"2023-11-24T06:26:56.979635+0000","last_became_peered":"2023-11-24T06:26:56.979635+0000","last_unstale":"2023-11-24T06:26:57.944974+0000","last_undegraded":"2023-11-24T06:26:57.944974+0000","last_fullsized":"2023-11-24T06:26:57.944974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219328+0000","last_change":"2023-11-24T06:26:56.968934+0000","last_active":"2023-11-24T06:26:58.219328+0000","last_peered":"2023-11-24T06:26:58.219328+0000","last_clean":"2023-11-24T06:26:58.219328+0000","last_became_active":"2023-11-24T06:26:56.967198+0000","last_became_peered":"2023-11-24T06:26:56.967198+0000","last_unstale":"2023-11-24T06:26:58.219328+0000","last_undegraded":"2023-11-24T06:26:58.219328+0000","last_fullsized":"2023-11-24T06:26:58.219328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945713+0000","last_change":"2023-11-24T06:26:56.974003+0000","last_active":"2023-11-24T06:26:57.945713+0000","last_peered":"2023-11-24T06:26:57.945713+0000","last_clean":"2023-11-24T06:26:57.945713+0000","last_became_active":"2023-11-24T06:26:56.973631+0000","last_became_peered":"2023-11-24T06:26:56.973631+0000","last_unstale":"2023-11-24T06:26:57.945713+0000","last_undegraded":"2023-11-24T06:26:57.945713+0000","last_fullsized":"2023-11-24T06:26:57.945713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219990+0000","last_change":"2023-11-24T06:26:56.972613+0000","last_active":"2023-11-24T06:26:58.219990+0000","last_peered":"2023-11-24T06:26:58.219990+0000","last_clean":"2023-11-24T06:26:58.219990+0000","last_became_active":"2023-11-24T06:26:56.972280+0000","last_became_peered":"2023-11-24T06:26:56.972280+0000","last_unstale":"2023-11-24T06:26:58.219990+0000","last_undegraded":"2023-11-24T06:26:58.219990+0000","last_fullsized":"2023-11-24T06:26:58.219990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944265+0000","last_change":"2023-11-24T06:26:56.966680+0000","last_active":"2023-11-24T06:26:57.944265+0000","last_peered":"2023-11-24T06:26:57.944265+0000","last_clean":"2023-11-24T06:26:57.944265+0000","last_became_active":"2023-11-24T06:26:56.966315+0000","last_became_peered":"2023-11-24T06:26:56.966315+0000","last_unstale":"2023-11-24T06:26:57.944265+0000","last_undegraded":"2023-11-24T06:26:57.944265+0000","last_fullsized":"2023-11-24T06:26:57.944265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945586+0000","last_change":"2023-11-24T06:26:56.974920+0000","last_active":"2023-11-24T06:26:57.945586+0000","last_peered":"2023-11-24T06:26:57.945586+0000","last_clean":"2023-11-24T06:26:57.945586+0000","last_became_active":"2023-11-24T06:26:56.974594+0000","last_became_peered":"2023-11-24T06:26:56.974594+0000","last_unstale":"2023-11-24T06:26:57.945586+0000","last_undegraded":"2023-11-24T06:26:57.945586+0000","last_fullsized":"2023-11-24T06:26:57.945586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217994+0000","last_change":"2023-11-24T06:26:56.976017+0000","last_active":"2023-11-24T06:26:58.217994+0000","last_peered":"2023-11-24T06:26:58.217994+0000","last_clean":"2023-11-24T06:26:58.217994+0000","last_became_active":"2023-11-24T06:26:56.975709+0000","last_became_peered":"2023-11-24T06:26:56.975709+0000","last_unstale":"2023-11-24T06:26:58.217994+0000","last_undegraded":"2023-11-24T06:26:58.217994+0000","last_fullsized":"2023-11-24T06:26:58.217994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221250+0000","last_change":"2023-11-24T06:26:56.980269+0000","last_active":"2023-11-24T06:26:58.221250+0000","last_peered":"2023-11-24T06:26:58.221250+0000","last_clean":"2023-11-24T06:26:58.221250+0000","last_became_active":"2023-11-24T06:26:56.979929+0000","last_became_peered":"2023-11-24T06:26:56.979929+0000","last_unstale":"2023-11-24T06:26:58.221250+0000","last_undegraded":"2023-11-24T06:26:58.221250+0000","last_fullsized":"2023-11-24T06:26:58.221250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216766+0000","last_change":"2023-11-24T06:26:56.986717+0000","last_active":"2023-11-24T06:26:58.216766+0000","last_peered":"2023-11-24T06:26:58.216766+0000","last_clean":"2023-11-24T06:26:58.216766+0000","last_became_active":"2023-11-24T06:26:56.986398+0000","last_became_peered":"2023-11-24T06:26:56.986398+0000","last_unstale":"2023-11-24T06:26:58.216766+0000","last_undegraded":"2023-11-24T06:26:58.216766+0000","last_fullsized":"2023-11-24T06:26:58.216766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:47.926009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217958+0000","last_change":"2023-11-24T06:26:56.978275+0000","last_active":"2023-11-24T06:26:58.217958+0000","last_peered":"2023-11-24T06:26:58.217958+0000","last_clean":"2023-11-24T06:26:58.217958+0000","last_became_active":"2023-11-24T06:26:56.977924+0000","last_became_peered":"2023-11-24T06:26:56.977924+0000","last_unstale":"2023-11-24T06:26:58.217958+0000","last_undegraded":"2023-11-24T06:26:58.217958+0000","last_fullsized":"2023-11-24T06:26:58.217958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:58.210301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220024+0000","last_change":"2023-11-24T06:26:56.970266+0000","last_active":"2023-11-24T06:26:58.220024+0000","last_peered":"2023-11-24T06:26:58.220024+0000","last_clean":"2023-11-24T06:26:58.220024+0000","last_became_active":"2023-11-24T06:26:56.969973+0000","last_became_peered":"2023-11-24T06:26:56.969973+0000","last_unstale":"2023-11-24T06:26:58.220024+0000","last_undegraded":"2023-11-24T06:26:58.220024+0000","last_fullsized":"2023-11-24T06:26:58.220024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:28.630873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945477+0000","last_change":"2023-11-24T06:26:56.008228+0000","last_active":"2023-11-24T06:26:57.945477+0000","last_peered":"2023-11-24T06:26:57.945477+0000","last_clean":"2023-11-24T06:26:57.945477+0000","last_became_active":"2023-11-24T06:26:56.007916+0000","last_became_peered":"2023-11-24T06:26:56.007916+0000","last_unstale":"2023-11-24T06:26:57.945477+0000","last_undegraded":"2023-11-24T06:26:57.945477+0000","last_fullsized":"2023-11-24T06:26:57.945477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218141+0000","last_change":"2023-11-24T06:26:56.982794+0000","last_active":"2023-11-24T06:26:58.218141+0000","last_peered":"2023-11-24T06:26:58.218141+0000","last_clean":"2023-11-24T06:26:58.218141+0000","last_became_active":"2023-11-24T06:26:56.982455+0000","last_became_peered":"2023-11-24T06:26:56.982455+0000","last_unstale":"2023-11-24T06:26:58.218141+0000","last_undegraded":"2023-11-24T06:26:58.218141+0000","last_fullsized":"2023-11-24T06:26:58.218141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218348+0000","last_change":"2023-11-24T06:26:56.010407+0000","last_active":"2023-11-24T06:26:58.218348+0000","last_peered":"2023-11-24T06:26:58.218348+0000","last_clean":"2023-11-24T06:26:58.218348+0000","last_became_active":"2023-11-24T06:26:56.009548+0000","last_became_peered":"2023-11-24T06:26:56.009548+0000","last_unstale":"2023-11-24T06:26:58.218348+0000","last_undegraded":"2023-11-24T06:26:58.218348+0000","last_fullsized":"2023-11-24T06:26:58.218348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:58.487660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945323+0000","last_change":"2023-11-24T06:26:56.005804+0000","last_active":"2023-11-24T06:26:57.945323+0000","last_peered":"2023-11-24T06:26:57.945323+0000","last_clean":"2023-11-24T06:26:57.945323+0000","last_became_active":"2023-11-24T06:26:56.005473+0000","last_became_peered":"2023-11-24T06:26:56.005473+0000","last_unstale":"2023-11-24T06:26:57.945323+0000","last_undegraded":"2023-11-24T06:26:57.945323+0000","last_fullsized":"2023-11-24T06:26:57.945323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:18.225163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944935+0000","last_change":"2023-11-24T06:26:56.977459+0000","last_active":"2023-11-24T06:26:57.944935+0000","last_peered":"2023-11-24T06:26:57.944935+0000","last_clean":"2023-11-24T06:26:57.944935+0000","last_became_active":"2023-11-24T06:26:56.977160+0000","last_became_peered":"2023-11-24T06:26:56.977160+0000","last_unstale":"2023-11-24T06:26:57.944935+0000","last_undegraded":"2023-11-24T06:26:57.944935+0000","last_fullsized":"2023-11-24T06:26:57.944935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219546+0000","last_change":"2023-11-24T06:26:56.010202+0000","last_active":"2023-11-24T06:26:58.219546+0000","last_peered":"2023-11-24T06:26:58.219546+0000","last_clean":"2023-11-24T06:26:58.219546+0000","last_became_active":"2023-11-24T06:26:56.009924+0000","last_became_peered":"2023-11-24T06:26:56.009924+0000","last_unstale":"2023-11-24T06:26:58.219546+0000","last_undegraded":"2023-11-24T06:26:58.219546+0000","last_fullsized":"2023-11-24T06:26:58.219546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:17.355207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944060+0000","last_change":"2023-11-24T06:26:56.962258+0000","last_active":"2023-11-24T06:26:57.944060+0000","last_peered":"2023-11-24T06:26:57.944060+0000","last_clean":"2023-11-24T06:26:57.944060+0000","last_became_active":"2023-11-24T06:26:56.961665+0000","last_became_peered":"2023-11-24T06:26:56.961665+0000","last_unstale":"2023-11-24T06:26:57.944060+0000","last_undegraded":"2023-11-24T06:26:57.944060+0000","last_fullsized":"2023-11-24T06:26:57.944060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221161+0000","last_change":"2023-11-24T06:26:56.017642+0000","last_active":"2023-11-24T06:26:58.221161+0000","last_peered":"2023-11-24T06:26:58.221161+0000","last_clean":"2023-11-24T06:26:58.221161+0000","last_became_active":"2023-11-24T06:26:56.017386+0000","last_became_peered":"2023-11-24T06:26:56.017386+0000","last_unstale":"2023-11-24T06:26:58.221161+0000","last_undegraded":"2023-11-24T06:26:58.221161+0000","last_fullsized":"2023-11-24T06:26:58.221161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:18.225163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222938+0000","last_change":"2023-11-24T06:26:56.985441+0000","last_active":"2023-11-24T06:26:58.222938+0000","last_peered":"2023-11-24T06:26:58.222938+0000","last_clean":"2023-11-24T06:26:58.222938+0000","last_became_active":"2023-11-24T06:26:56.984430+0000","last_became_peered":"2023-11-24T06:26:56.984430+0000","last_unstale":"2023-11-24T06:26:58.222938+0000","last_undegraded":"2023-11-24T06:26:58.222938+0000","last_fullsized":"2023-11-24T06:26:58.222938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:56.199949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222514+0000","last_change":"2023-11-24T06:26:56.017588+0000","last_active":"2023-11-24T06:26:58.222514+0000","last_peered":"2023-11-24T06:26:58.222514+0000","last_clean":"2023-11-24T06:26:58.222514+0000","last_became_active":"2023-11-24T06:26:56.017128+0000","last_became_peered":"2023-11-24T06:26:56.017128+0000","last_unstale":"2023-11-24T06:26:58.222514+0000","last_undegraded":"2023-11-24T06:26:58.222514+0000","last_fullsized":"2023-11-24T06:26:58.222514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218742+0000","last_change":"2023-11-24T06:26:57.002819+0000","last_active":"2023-11-24T06:26:58.218742+0000","last_peered":"2023-11-24T06:26:58.218742+0000","last_clean":"2023-11-24T06:26:58.218742+0000","last_became_active":"2023-11-24T06:26:57.002515+0000","last_became_peered":"2023-11-24T06:26:57.002515+0000","last_unstale":"2023-11-24T06:26:58.218742+0000","last_undegraded":"2023-11-24T06:26:58.218742+0000","last_fullsized":"2023-11-24T06:26:58.218742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944342+0000","last_change":"2023-11-24T06:26:56.014991+0000","last_active":"2023-11-24T06:26:57.944342+0000","last_peered":"2023-11-24T06:26:57.944342+0000","last_clean":"2023-11-24T06:26:57.944342+0000","last_became_active":"2023-11-24T06:26:56.014706+0000","last_became_peered":"2023-11-24T06:26:56.014706+0000","last_unstale":"2023-11-24T06:26:57.944342+0000","last_undegraded":"2023-11-24T06:26:57.944342+0000","last_fullsized":"2023-11-24T06:26:57.944342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:56.985153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221316+0000","last_change":"2023-11-24T06:26:56.970097+0000","last_active":"2023-11-24T06:26:58.221316+0000","last_peered":"2023-11-24T06:26:58.221316+0000","last_clean":"2023-11-24T06:26:58.221316+0000","last_became_active":"2023-11-24T06:26:56.969183+0000","last_became_peered":"2023-11-24T06:26:56.969183+0000","last_unstale":"2023-11-24T06:26:58.221316+0000","last_undegraded":"2023-11-24T06:26:58.221316+0000","last_fullsized":"2023-11-24T06:26:58.221316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:47.926009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221734+0000","last_change":"2023-11-24T06:26:56.002890+0000","last_active":"2023-11-24T06:26:58.221734+0000","last_peered":"2023-11-24T06:26:58.221734+0000","last_clean":"2023-11-24T06:26:58.221734+0000","last_became_active":"2023-11-24T06:26:56.002603+0000","last_became_peered":"2023-11-24T06:26:56.002603+0000","last_unstale":"2023-11-24T06:26:58.221734+0000","last_undegraded":"2023-11-24T06:26:58.221734+0000","last_fullsized":"2023-11-24T06:26:58.221734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947457+0000","last_change":"2023-11-24T06:26:56.961409+0000","last_active":"2023-11-24T06:26:57.947457+0000","last_peered":"2023-11-24T06:26:57.947457+0000","last_clean":"2023-11-24T06:26:57.947457+0000","last_became_active":"2023-11-24T06:26:56.961107+0000","last_became_peered":"2023-11-24T06:26:56.961107+0000","last_unstale":"2023-11-24T06:26:57.947457+0000","last_undegraded":"2023-11-24T06:26:57.947457+0000","last_fullsized":"2023-11-24T06:26:57.947457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218244+0000","last_change":"2023-11-24T06:26:56.006966+0000","last_active":"2023-11-24T06:26:58.218244+0000","last_peered":"2023-11-24T06:26:58.218244+0000","last_clean":"2023-11-24T06:26:58.218244+0000","last_became_active":"2023-11-24T06:26:56.006596+0000","last_became_peered":"2023-11-24T06:26:56.006596+0000","last_unstale":"2023-11-24T06:26:58.218244+0000","last_undegraded":"2023-11-24T06:26:58.218244+0000","last_fullsized":"2023-11-24T06:26:58.218244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217572+0000","last_change":"2023-11-24T06:26:56.974192+0000","last_active":"2023-11-24T06:26:58.217572+0000","last_peered":"2023-11-24T06:26:58.217572+0000","last_clean":"2023-11-24T06:26:58.217572+0000","last_became_active":"2023-11-24T06:26:56.973850+0000","last_became_peered":"2023-11-24T06:26:56.973850+0000","last_unstale":"2023-11-24T06:26:58.217572+0000","last_undegraded":"2023-11-24T06:26:58.217572+0000","last_fullsized":"2023-11-24T06:26:58.217572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:51.030181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221387+0000","last_change":"2023-11-24T06:26:56.013312+0000","last_active":"2023-11-24T06:26:58.221387+0000","last_peered":"2023-11-24T06:26:58.221387+0000","last_clean":"2023-11-24T06:26:58.221387+0000","last_became_active":"2023-11-24T06:26:56.013018+0000","last_became_peered":"2023-11-24T06:26:56.013018+0000","last_unstale":"2023-11-24T06:26:58.221387+0000","last_undegraded":"2023-11-24T06:26:58.221387+0000","last_fullsized":"2023-11-24T06:26:58.221387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:03.493540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947707+0000","last_change":"2023-11-24T06:26:56.975118+0000","last_active":"2023-11-24T06:26:57.947707+0000","last_peered":"2023-11-24T06:26:57.947707+0000","last_clean":"2023-11-24T06:26:57.947707+0000","last_became_active":"2023-11-24T06:26:56.974734+0000","last_became_peered":"2023-11-24T06:26:56.974734+0000","last_unstale":"2023-11-24T06:26:57.947707+0000","last_undegraded":"2023-11-24T06:26:57.947707+0000","last_fullsized":"2023-11-24T06:26:57.947707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:15.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219402+0000","last_change":"2023-11-24T06:26:56.010113+0000","last_active":"2023-11-24T06:26:58.219402+0000","last_peered":"2023-11-24T06:26:58.219402+0000","last_clean":"2023-11-24T06:26:58.219402+0000","last_became_active":"2023-11-24T06:26:56.009550+0000","last_became_peered":"2023-11-24T06:26:56.009550+0000","last_unstale":"2023-11-24T06:26:58.219402+0000","last_undegraded":"2023-11-24T06:26:58.219402+0000","last_fullsized":"2023-11-24T06:26:58.219402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:13.157002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218000+0000","last_change":"2023-11-24T06:26:56.978061+0000","last_active":"2023-11-24T06:26:58.218000+0000","last_peered":"2023-11-24T06:26:58.218000+0000","last_clean":"2023-11-24T06:26:58.218000+0000","last_became_active":"2023-11-24T06:26:56.977760+0000","last_became_peered":"2023-11-24T06:26:56.977760+0000","last_unstale":"2023-11-24T06:26:58.218000+0000","last_undegraded":"2023-11-24T06:26:58.218000+0000","last_fullsized":"2023-11-24T06:26:58.218000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216616+0000","last_change":"2023-11-24T06:26:55.996462+0000","last_active":"2023-11-24T06:26:58.216616+0000","last_peered":"2023-11-24T06:26:58.216616+0000","last_clean":"2023-11-24T06:26:58.216616+0000","last_became_active":"2023-11-24T06:26:55.996178+0000","last_became_peered":"2023-11-24T06:26:55.996178+0000","last_unstale":"2023-11-24T06:26:58.216616+0000","last_undegraded":"2023-11-24T06:26:58.216616+0000","last_fullsized":"2023-11-24T06:26:58.216616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:59.647318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946852+0000","last_change":"2023-11-24T06:26:56.978947+0000","last_active":"2023-11-24T06:26:57.946852+0000","last_peered":"2023-11-24T06:26:57.946852+0000","last_clean":"2023-11-24T06:26:57.946852+0000","last_became_active":"2023-11-24T06:26:56.978643+0000","last_became_peered":"2023-11-24T06:26:56.978643+0000","last_unstale":"2023-11-24T06:26:57.946852+0000","last_undegraded":"2023-11-24T06:26:57.946852+0000","last_fullsized":"2023-11-24T06:26:57.946852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222582+0000","last_change":"2023-11-24T06:26:56.005504+0000","last_active":"2023-11-24T06:26:58.222582+0000","last_peered":"2023-11-24T06:26:58.222582+0000","last_clean":"2023-11-24T06:26:58.222582+0000","last_became_active":"2023-11-24T06:26:56.005287+0000","last_became_peered":"2023-11-24T06:26:56.005287+0000","last_unstale":"2023-11-24T06:26:58.222582+0000","last_undegraded":"2023-11-24T06:26:58.222582+0000","last_fullsized":"2023-11-24T06:26:58.222582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:24.763532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947843+0000","last_change":"2023-11-24T06:26:56.973083+0000","last_active":"2023-11-24T06:26:57.947843+0000","last_peered":"2023-11-24T06:26:57.947843+0000","last_clean":"2023-11-24T06:26:57.947843+0000","last_became_active":"2023-11-24T06:26:56.972750+0000","last_became_peered":"2023-11-24T06:26:56.972750+0000","last_unstale":"2023-11-24T06:26:57.947843+0000","last_undegraded":"2023-11-24T06:26:57.947843+0000","last_fullsized":"2023-11-24T06:26:57.947843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:08.969752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217141+0000","last_change":"2023-11-24T06:26:56.005050+0000","last_active":"2023-11-24T06:26:58.217141+0000","last_peered":"2023-11-24T06:26:58.217141+0000","last_clean":"2023-11-24T06:26:58.217141+0000","last_became_active":"2023-11-24T06:26:56.004658+0000","last_became_peered":"2023-11-24T06:26:56.004658+0000","last_unstale":"2023-11-24T06:26:58.217141+0000","last_undegraded":"2023-11-24T06:26:58.217141+0000","last_fullsized":"2023-11-24T06:26:58.217141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:27.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948405+0000","last_change":"2023-11-24T06:26:56.973292+0000","last_active":"2023-11-24T06:26:57.948405+0000","last_peered":"2023-11-24T06:26:57.948405+0000","last_clean":"2023-11-24T06:26:57.948405+0000","last_became_active":"2023-11-24T06:26:56.972959+0000","last_became_peered":"2023-11-24T06:26:56.972959+0000","last_unstale":"2023-11-24T06:26:57.948405+0000","last_undegraded":"2023-11-24T06:26:57.948405+0000","last_fullsized":"2023-11-24T06:26:57.948405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:48.721442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944407+0000","last_change":"2023-11-24T06:26:56.005932+0000","last_active":"2023-11-24T06:26:57.944407+0000","last_peered":"2023-11-24T06:26:57.944407+0000","last_clean":"2023-11-24T06:26:57.944407+0000","last_became_active":"2023-11-24T06:26:56.005603+0000","last_became_peered":"2023-11-24T06:26:56.005603+0000","last_unstale":"2023-11-24T06:26:57.944407+0000","last_undegraded":"2023-11-24T06:26:57.944407+0000","last_fullsized":"2023-11-24T06:26:57.944407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222520+0000","last_change":"2023-11-24T06:26:56.969503+0000","last_active":"2023-11-24T06:26:58.222520+0000","last_peered":"2023-11-24T06:26:58.222520+0000","last_clean":"2023-11-24T06:26:58.222520+0000","last_became_active":"2023-11-24T06:26:56.967704+0000","last_became_peered":"2023-11-24T06:26:56.967704+0000","last_unstale":"2023-11-24T06:26:58.222520+0000","last_undegraded":"2023-11-24T06:26:58.222520+0000","last_fullsized":"2023-11-24T06:26:58.222520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:17.662630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217140+0000","last_change":"2023-11-24T06:26:56.014819+0000","last_active":"2023-11-24T06:26:58.217140+0000","last_peered":"2023-11-24T06:26:58.217140+0000","last_clean":"2023-11-24T06:26:58.217140+0000","last_became_active":"2023-11-24T06:26:56.014287+0000","last_became_peered":"2023-11-24T06:26:56.014287+0000","last_unstale":"2023-11-24T06:26:58.217140+0000","last_undegraded":"2023-11-24T06:26:58.217140+0000","last_fullsized":"2023-11-24T06:26:58.217140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:13.157002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947402+0000","last_change":"2023-11-24T06:26:57.001053+0000","last_active":"2023-11-24T06:26:57.947402+0000","last_peered":"2023-11-24T06:26:57.947402+0000","last_clean":"2023-11-24T06:26:57.947402+0000","last_became_active":"2023-11-24T06:26:57.000802+0000","last_became_peered":"2023-11-24T06:26:57.000802+0000","last_unstale":"2023-11-24T06:26:57.947402+0000","last_undegraded":"2023-11-24T06:26:57.947402+0000","last_fullsized":"2023-11-24T06:26:57.947402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:26:56.946083+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:26:56.946083+0000","last_peered":"2023-11-24T06:26:56.946083+0000","last_clean":"2023-11-24T06:26:56.946083+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:26:56.946083+0000","last_undegraded":"2023-11-24T06:26:56.946083+0000","last_fullsized":"2023-11-24T06:26:56.946083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221515+0000","last_change":"2023-11-24T06:26:56.008553+0000","last_active":"2023-11-24T06:26:58.221515+0000","last_peered":"2023-11-24T06:26:58.221515+0000","last_clean":"2023-11-24T06:26:58.221515+0000","last_became_active":"2023-11-24T06:26:56.006284+0000","last_became_peered":"2023-11-24T06:26:56.006284+0000","last_unstale":"2023-11-24T06:26:58.221515+0000","last_undegraded":"2023-11-24T06:26:58.221515+0000","last_fullsized":"2023-11-24T06:26:58.221515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:24.763532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945070+0000","last_change":"2023-11-24T06:26:56.998859+0000","last_active":"2023-11-24T06:26:57.945070+0000","last_peered":"2023-11-24T06:26:57.945070+0000","last_clean":"2023-11-24T06:26:57.945070+0000","last_became_active":"2023-11-24T06:26:56.998524+0000","last_became_peered":"2023-11-24T06:26:56.998524+0000","last_unstale":"2023-11-24T06:26:57.945070+0000","last_undegraded":"2023-11-24T06:26:57.945070+0000","last_fullsized":"2023-11-24T06:26:57.945070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946597+0000","last_change":"2023-11-24T06:26:56.005300+0000","last_active":"2023-11-24T06:26:57.946597+0000","last_peered":"2023-11-24T06:26:57.946597+0000","last_clean":"2023-11-24T06:26:57.946597+0000","last_became_active":"2023-11-24T06:26:56.004984+0000","last_became_peered":"2023-11-24T06:26:56.004984+0000","last_unstale":"2023-11-24T06:26:57.946597+0000","last_undegraded":"2023-11-24T06:26:57.946597+0000","last_fullsized":"2023-11-24T06:26:57.946597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217825+0000","last_change":"2023-11-24T06:26:56.984062+0000","last_active":"2023-11-24T06:26:58.217825+0000","last_peered":"2023-11-24T06:26:58.217825+0000","last_clean":"2023-11-24T06:26:58.217825+0000","last_became_active":"2023-11-24T06:26:56.983679+0000","last_became_peered":"2023-11-24T06:26:56.983679+0000","last_unstale":"2023-11-24T06:26:58.217825+0000","last_undegraded":"2023-11-24T06:26:58.217825+0000","last_fullsized":"2023-11-24T06:26:58.217825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:58.210301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216848+0000","last_change":"2023-11-24T06:26:56.005001+0000","last_active":"2023-11-24T06:26:58.216848+0000","last_peered":"2023-11-24T06:26:58.216848+0000","last_clean":"2023-11-24T06:26:58.216848+0000","last_became_active":"2023-11-24T06:26:56.004612+0000","last_became_peered":"2023-11-24T06:26:56.004612+0000","last_unstale":"2023-11-24T06:26:58.216848+0000","last_undegraded":"2023-11-24T06:26:58.216848+0000","last_fullsized":"2023-11-24T06:26:58.216848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944532+0000","last_change":"2023-11-24T06:26:56.965796+0000","last_active":"2023-11-24T06:26:57.944532+0000","last_peered":"2023-11-24T06:26:57.944532+0000","last_clean":"2023-11-24T06:26:57.944532+0000","last_became_active":"2023-11-24T06:26:56.963344+0000","last_became_peered":"2023-11-24T06:26:56.963344+0000","last_unstale":"2023-11-24T06:26:57.944532+0000","last_undegraded":"2023-11-24T06:26:57.944532+0000","last_fullsized":"2023-11-24T06:26:57.944532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221498+0000","last_change":"2023-11-24T06:26:56.012192+0000","last_active":"2023-11-24T06:26:58.221498+0000","last_peered":"2023-11-24T06:26:58.221498+0000","last_clean":"2023-11-24T06:26:58.221498+0000","last_became_active":"2023-11-24T06:26:56.011796+0000","last_became_peered":"2023-11-24T06:26:56.011796+0000","last_unstale":"2023-11-24T06:26:58.221498+0000","last_undegraded":"2023-11-24T06:26:58.221498+0000","last_fullsized":"2023-11-24T06:26:58.221498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:50.399725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220548+0000","last_change":"2023-11-24T06:26:56.985067+0000","last_active":"2023-11-24T06:26:58.220548+0000","last_peered":"2023-11-24T06:26:58.220548+0000","last_clean":"2023-11-24T06:26:58.220548+0000","last_became_active":"2023-11-24T06:26:56.984360+0000","last_became_peered":"2023-11-24T06:26:56.984360+0000","last_unstale":"2023-11-24T06:26:58.220548+0000","last_undegraded":"2023-11-24T06:26:58.220548+0000","last_fullsized":"2023-11-24T06:26:58.220548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:50.371449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944321+0000","last_change":"2023-11-24T06:26:55.996919+0000","last_active":"2023-11-24T06:26:57.944321+0000","last_peered":"2023-11-24T06:26:57.944321+0000","last_clean":"2023-11-24T06:26:57.944321+0000","last_became_active":"2023-11-24T06:26:55.996431+0000","last_became_peered":"2023-11-24T06:26:55.996431+0000","last_unstale":"2023-11-24T06:26:57.944321+0000","last_undegraded":"2023-11-24T06:26:57.944321+0000","last_fullsized":"2023-11-24T06:26:57.944321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222871+0000","last_change":"2023-11-24T06:26:56.987025+0000","last_active":"2023-11-24T06:26:58.222871+0000","last_peered":"2023-11-24T06:26:58.222871+0000","last_clean":"2023-11-24T06:26:58.222871+0000","last_became_active":"2023-11-24T06:26:56.985920+0000","last_became_peered":"2023-11-24T06:26:56.985920+0000","last_unstale":"2023-11-24T06:26:58.222871+0000","last_undegraded":"2023-11-24T06:26:58.222871+0000","last_fullsized":"2023-11-24T06:26:58.222871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:08.969752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947843+0000","last_change":"2023-11-24T06:26:56.015006+0000","last_active":"2023-11-24T06:26:57.947843+0000","last_peered":"2023-11-24T06:26:57.947843+0000","last_clean":"2023-11-24T06:26:57.947843+0000","last_became_active":"2023-11-24T06:26:56.011264+0000","last_became_peered":"2023-11-24T06:26:56.011264+0000","last_unstale":"2023-11-24T06:26:57.947843+0000","last_undegraded":"2023-11-24T06:26:57.947843+0000","last_fullsized":"2023-11-24T06:26:57.947843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:51.788764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944165+0000","last_change":"2023-11-24T06:26:56.967275+0000","last_active":"2023-11-24T06:26:57.944165+0000","last_peered":"2023-11-24T06:26:57.944165+0000","last_clean":"2023-11-24T06:26:57.944165+0000","last_became_active":"2023-11-24T06:26:56.966261+0000","last_became_peered":"2023-11-24T06:26:56.966261+0000","last_unstale":"2023-11-24T06:26:57.944165+0000","last_undegraded":"2023-11-24T06:26:57.944165+0000","last_fullsized":"2023-11-24T06:26:57.944165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944646+0000","last_change":"2023-11-24T06:26:56.011770+0000","last_active":"2023-11-24T06:26:57.944646+0000","last_peered":"2023-11-24T06:26:57.944646+0000","last_clean":"2023-11-24T06:26:57.944646+0000","last_became_active":"2023-11-24T06:26:56.011386+0000","last_became_peered":"2023-11-24T06:26:56.011386+0000","last_unstale":"2023-11-24T06:26:57.944646+0000","last_undegraded":"2023-11-24T06:26:57.944646+0000","last_fullsized":"2023-11-24T06:26:57.944646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945877+0000","last_change":"2023-11-24T06:26:56.978788+0000","last_active":"2023-11-24T06:26:57.945877+0000","last_peered":"2023-11-24T06:26:57.945877+0000","last_clean":"2023-11-24T06:26:57.945877+0000","last_became_active":"2023-11-24T06:26:56.978112+0000","last_became_peered":"2023-11-24T06:26:56.978112+0000","last_unstale":"2023-11-24T06:26:57.945877+0000","last_undegraded":"2023-11-24T06:26:57.945877+0000","last_fullsized":"2023-11-24T06:26:57.945877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220478+0000","last_change":"2023-11-24T06:26:56.012367+0000","last_active":"2023-11-24T06:26:58.220478+0000","last_peered":"2023-11-24T06:26:58.220478+0000","last_clean":"2023-11-24T06:26:58.220478+0000","last_became_active":"2023-11-24T06:26:56.011973+0000","last_became_peered":"2023-11-24T06:26:56.011973+0000","last_unstale":"2023-11-24T06:26:58.220478+0000","last_undegraded":"2023-11-24T06:26:58.220478+0000","last_fullsized":"2023-11-24T06:26:58.220478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218351+0000","last_change":"2023-11-24T06:26:56.976256+0000","last_active":"2023-11-24T06:26:58.218351+0000","last_peered":"2023-11-24T06:26:58.218351+0000","last_clean":"2023-11-24T06:26:58.218351+0000","last_became_active":"2023-11-24T06:26:56.975889+0000","last_became_peered":"2023-11-24T06:26:56.975889+0000","last_unstale":"2023-11-24T06:26:58.218351+0000","last_undegraded":"2023-11-24T06:26:58.218351+0000","last_fullsized":"2023-11-24T06:26:58.218351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946577+0000","last_change":"2023-11-24T06:26:56.008104+0000","last_active":"2023-11-24T06:26:57.946577+0000","last_peered":"2023-11-24T06:26:57.946577+0000","last_clean":"2023-11-24T06:26:57.946577+0000","last_became_active":"2023-11-24T06:26:56.007669+0000","last_became_peered":"2023-11-24T06:26:56.007669+0000","last_unstale":"2023-11-24T06:26:57.946577+0000","last_undegraded":"2023-11-24T06:26:57.946577+0000","last_fullsized":"2023-11-24T06:26:57.946577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:27.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220486+0000","last_change":"2023-11-24T06:26:56.974769+0000","last_active":"2023-11-24T06:26:58.220486+0000","last_peered":"2023-11-24T06:26:58.220486+0000","last_clean":"2023-11-24T06:26:58.220486+0000","last_became_active":"2023-11-24T06:26:56.974530+0000","last_became_peered":"2023-11-24T06:26:56.974530+0000","last_unstale":"2023-11-24T06:26:58.220486+0000","last_undegraded":"2023-11-24T06:26:58.220486+0000","last_fullsized":"2023-11-24T06:26:58.220486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:02.036523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944166+0000","last_change":"2023-11-24T06:26:56.001637+0000","last_active":"2023-11-24T06:26:57.944166+0000","last_peered":"2023-11-24T06:26:57.944166+0000","last_clean":"2023-11-24T06:26:57.944166+0000","last_became_active":"2023-11-24T06:26:56.000662+0000","last_became_peered":"2023-11-24T06:26:56.000662+0000","last_unstale":"2023-11-24T06:26:57.944166+0000","last_undegraded":"2023-11-24T06:26:57.944166+0000","last_fullsized":"2023-11-24T06:26:57.944166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:49.146302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945138+0000","last_change":"2023-11-24T06:26:56.961879+0000","last_active":"2023-11-24T06:26:57.945138+0000","last_peered":"2023-11-24T06:26:57.945138+0000","last_clean":"2023-11-24T06:26:57.945138+0000","last_became_active":"2023-11-24T06:26:56.961593+0000","last_became_peered":"2023-11-24T06:26:56.961593+0000","last_unstale":"2023-11-24T06:26:57.945138+0000","last_undegraded":"2023-11-24T06:26:57.945138+0000","last_fullsized":"2023-11-24T06:26:57.945138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944425+0000","last_change":"2023-11-24T06:26:55.995684+0000","last_active":"2023-11-24T06:26:57.944425+0000","last_peered":"2023-11-24T06:26:57.944425+0000","last_clean":"2023-11-24T06:26:57.944425+0000","last_became_active":"2023-11-24T06:26:55.994739+0000","last_became_peered":"2023-11-24T06:26:55.994739+0000","last_unstale":"2023-11-24T06:26:57.944425+0000","last_undegraded":"2023-11-24T06:26:57.944425+0000","last_fullsized":"2023-11-24T06:26:57.944425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:59.647318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945536+0000","last_change":"2023-11-24T06:26:56.976564+0000","last_active":"2023-11-24T06:26:57.945536+0000","last_peered":"2023-11-24T06:26:57.945536+0000","last_clean":"2023-11-24T06:26:57.945536+0000","last_became_active":"2023-11-24T06:26:56.976081+0000","last_became_peered":"2023-11-24T06:26:56.976081+0000","last_unstale":"2023-11-24T06:26:57.945536+0000","last_undegraded":"2023-11-24T06:26:57.945536+0000","last_fullsized":"2023-11-24T06:26:57.945536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947699+0000","last_change":"2023-11-24T06:26:56.015851+0000","last_active":"2023-11-24T06:26:57.947699+0000","last_peered":"2023-11-24T06:26:57.947699+0000","last_clean":"2023-11-24T06:26:57.947699+0000","last_became_active":"2023-11-24T06:26:56.014016+0000","last_became_peered":"2023-11-24T06:26:56.014016+0000","last_unstale":"2023-11-24T06:26:57.947699+0000","last_undegraded":"2023-11-24T06:26:57.947699+0000","last_fullsized":"2023-11-24T06:26:57.947699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218077+0000","last_change":"2023-11-24T06:26:56.975486+0000","last_active":"2023-11-24T06:26:58.218077+0000","last_peered":"2023-11-24T06:26:58.218077+0000","last_clean":"2023-11-24T06:26:58.218077+0000","last_became_active":"2023-11-24T06:26:56.975147+0000","last_became_peered":"2023-11-24T06:26:56.975147+0000","last_unstale":"2023-11-24T06:26:58.218077+0000","last_undegraded":"2023-11-24T06:26:58.218077+0000","last_fullsized":"2023-11-24T06:26:58.218077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222273+0000","last_change":"2023-11-24T06:26:56.994982+0000","last_active":"2023-11-24T06:26:58.222273+0000","last_peered":"2023-11-24T06:26:58.222273+0000","last_clean":"2023-11-24T06:26:58.222273+0000","last_became_active":"2023-11-24T06:26:56.993765+0000","last_became_peered":"2023-11-24T06:26:56.993765+0000","last_unstale":"2023-11-24T06:26:58.222273+0000","last_undegraded":"2023-11-24T06:26:58.222273+0000","last_fullsized":"2023-11-24T06:26:58.222273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947612+0000","last_change":"2023-11-24T06:26:56.015039+0000","last_active":"2023-11-24T06:26:57.947612+0000","last_peered":"2023-11-24T06:26:57.947612+0000","last_clean":"2023-11-24T06:26:57.947612+0000","last_became_active":"2023-11-24T06:26:56.014562+0000","last_became_peered":"2023-11-24T06:26:56.014562+0000","last_unstale":"2023-11-24T06:26:57.947612+0000","last_undegraded":"2023-11-24T06:26:57.947612+0000","last_fullsized":"2023-11-24T06:26:57.947612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222503+0000","last_change":"2023-11-24T06:26:56.992781+0000","last_active":"2023-11-24T06:26:58.222503+0000","last_peered":"2023-11-24T06:26:58.222503+0000","last_clean":"2023-11-24T06:26:58.222503+0000","last_became_active":"2023-11-24T06:26:56.992337+0000","last_became_peered":"2023-11-24T06:26:56.992337+0000","last_unstale":"2023-11-24T06:26:58.222503+0000","last_undegraded":"2023-11-24T06:26:58.222503+0000","last_fullsized":"2023-11-24T06:26:58.222503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:15.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947644+0000","last_change":"2023-11-24T06:26:56.015616+0000","last_active":"2023-11-24T06:26:57.947644+0000","last_peered":"2023-11-24T06:26:57.947644+0000","last_clean":"2023-11-24T06:26:57.947644+0000","last_became_active":"2023-11-24T06:26:56.015317+0000","last_became_peered":"2023-11-24T06:26:56.015317+0000","last_unstale":"2023-11-24T06:26:57.947644+0000","last_undegraded":"2023-11-24T06:26:57.947644+0000","last_fullsized":"2023-11-24T06:26:57.947644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:10.844401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218457+0000","last_change":"2023-11-24T06:26:57.001097+0000","last_active":"2023-11-24T06:26:58.218457+0000","last_peered":"2023-11-24T06:26:58.218457+0000","last_clean":"2023-11-24T06:26:58.218457+0000","last_became_active":"2023-11-24T06:26:57.000751+0000","last_became_peered":"2023-11-24T06:26:57.000751+0000","last_unstale":"2023-11-24T06:26:58.218457+0000","last_undegraded":"2023-11-24T06:26:58.218457+0000","last_fullsized":"2023-11-24T06:26:58.218457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945720+0000","last_change":"2023-11-24T06:26:56.003621+0000","last_active":"2023-11-24T06:26:57.945720+0000","last_peered":"2023-11-24T06:26:57.945720+0000","last_clean":"2023-11-24T06:26:57.945720+0000","last_became_active":"2023-11-24T06:26:56.003232+0000","last_became_peered":"2023-11-24T06:26:56.003232+0000","last_unstale":"2023-11-24T06:26:57.945720+0000","last_undegraded":"2023-11-24T06:26:57.945720+0000","last_fullsized":"2023-11-24T06:26:57.945720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:10.844401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945648+0000","last_change":"2023-11-24T06:26:56.972822+0000","last_active":"2023-11-24T06:26:57.945648+0000","last_peered":"2023-11-24T06:26:57.945648+0000","last_clean":"2023-11-24T06:26:57.945648+0000","last_became_active":"2023-11-24T06:26:56.972529+0000","last_became_peered":"2023-11-24T06:26:56.972529+0000","last_unstale":"2023-11-24T06:26:57.945648+0000","last_undegraded":"2023-11-24T06:26:57.945648+0000","last_fullsized":"2023-11-24T06:26:57.945648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218623+0000","last_change":"2023-11-24T06:26:56.009185+0000","last_active":"2023-11-24T06:26:58.218623+0000","last_peered":"2023-11-24T06:26:58.218623+0000","last_clean":"2023-11-24T06:26:58.218623+0000","last_became_active":"2023-11-24T06:26:56.008855+0000","last_became_peered":"2023-11-24T06:26:56.008855+0000","last_unstale":"2023-11-24T06:26:58.218623+0000","last_undegraded":"2023-11-24T06:26:58.218623+0000","last_fullsized":"2023-11-24T06:26:58.218623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.476}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.724}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.657}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0939999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3519999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.571}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0539999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.679}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.788}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0960000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6719999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:02.026 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:27:02.027 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T06:27:02.027 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T06:27:02.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T06:27:02.027 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T06:27:02.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.286+0000 7f08e384b640 1 Processor -- start 2023-11-24T06:27:02.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.286+0000 7f08e384b640 1 -- start start 2023-11-24T06:27:02.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.288+0000 7f08e384b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0ae5a0 0x7f08dc15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.288+0000 7f08e384b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f08dc0cf290 0x7f08dc0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.288+0000 7f08e384b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08dc070c40 con 0x7f08dc0b3ae0 2023-11-24T06:27:02.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.288+0000 7f08e384b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08dc071450 con 0x7f08dc0cf290 2023-11-24T06:27:02.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.288+0000 7f08e384b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0cfc60 con 0x7f08dc0ae5a0 2023-11-24T06:27:02.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.288+0000 7f08e0de6640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f08dc0b3ae0 0x7f08dc06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35004/0 (socket says 172.21.15.16:35004) 2023-11-24T06:27:02.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.288+0000 7f08e0de6640 1 -- 172.21.15.16:0/663300152 learned_addr learned my addr 172.21.15.16:0/663300152 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:02.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08e15e7640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0ae5a0 0x7f08dc15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08e1de8640 1 --2- 172.21.15.16:0/663300152 >> v2:172.21.15.173:3300/0 conn(0x7f08dc0cf290 0x7f08dc0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809003192 0 0) 0x7f08dc070c40 con 0x7f08dc0b3ae0 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08c8003520 con 0x7f08dc0b3ae0 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2974448484 0 0) 0x7f08c8003520 con 0x7f08dc0b3ae0 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0ae5a0 msgr2=0x7f08dc15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0ae5a0 0x7f08dc15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 >> v2:172.21.15.173:3300/0 conn(0x7f08dc0cf290 msgr2=0x7f08dc0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 --2- 172.21.15.16:0/663300152 >> v2:172.21.15.173:3300/0 conn(0x7f08dc0cf290 0x7f08dc0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc06c470 con 0x7f08dc0b3ae0 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f08cc001670 con 0x7f08dc0b3ae0 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 >> v1:172.21.15.16:6789/0 conn(0x7f08dc0b3ae0 legacy=0x7f08dc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c8003970 0x7f08c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.289+0000 7f08d3fff640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0ae5a0 0x7f08c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08d3fff640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08c8004d00 0x7f08c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc071450 con 0x7f08c8003970 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0cfc60 con 0x7f08c8004d00 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c800b9b0 con 0x7f08dc0ae5a0 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08e15e7640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c8003970 0x7f08c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08e15e7640 1 -- 172.21.15.16:0/663300152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0ae5a0 msgr2=0x7f08c80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08e15e7640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc0ae5a0 0x7f08c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08e15e7640 1 -- 172.21.15.16:0/663300152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08c8004d00 msgr2=0x7f08c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08e15e7640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08c8004d00 0x7f08c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.290+0000 7f08e15e7640 1 -- 172.21.15.16:0/663300152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8003520 con 0x7f08c8003970 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.291+0000 7f08e15e7640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c8003970 0x7f08c8003d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009ac0 tx=0x7f08c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee1921afaf9c6c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.291+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d802c710 con 0x7f08c8003970 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.291+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d802c870 con 0x7f08c8003970 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.291+0000 7f08d3fff640 1 -- 172.21.15.16:0/663300152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d802cc00 con 0x7f08c8003970 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.292+0000 7f08e384b640 1 -- 172.21.15.16:0/663300152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c8003970 msgr2=0x7f08c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.292+0000 7f08e384b640 1 --2- 172.21.15.16:0/663300152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08c8003970 0x7f08c8003d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009ac0 tx=0x7f08c80057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.292+0000 7f08e1de8640 1 -- 172.21.15.16:0/663300152 reap_dead start 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.292+0000 7f08e384b640 1 -- 172.21.15.16:0/663300152 shutdown_connections 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.292+0000 7f08e384b640 1 -- 172.21.15.16:0/663300152 >> 172.21.15.16:0/663300152 conn(0x7f08dc06a800 msgr2=0x7f08dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.292+0000 7f08e384b640 1 -- 172.21.15.16:0/663300152 shutdown_connections 2023-11-24T06:27:02.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.292+0000 7f08e384b640 1 -- 172.21.15.16:0/663300152 wait complete. 2023-11-24T06:27:02.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.294+0000 7f08e384b640 1 Processor -- start 2023-11-24T06:27:02.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.294+0000 7f08e384b640 1 -- start start 2023-11-24T06:27:02.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.295+0000 7f08e384b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c8003970 0x7f08dc1459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.295+0000 7f08e384b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc15c5f0 0x7f08dc15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.295+0000 7f08e384b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc0cc9b0 0x7f08dc0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.295+0000 7f08e384b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc15cfb0 con 0x7f08dc15c5f0 2023-11-24T06:27:02.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.295+0000 7f08e384b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc157f00 con 0x7f08dc0cc9b0 2023-11-24T06:27:02.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.295+0000 7f08e384b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0cd760 con 0x7f08c8003970 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.295+0000 7f08e0de6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc15c5f0 0x7f08dc15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.295+0000 7f08e0de6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc15c5f0 0x7f08dc15ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40646/0 (socket says 172.21.15.16:40646) 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.295+0000 7f08e0de6640 1 -- 172.21.15.16:0/662217801 learned_addr learned my addr 172.21.15.16:0/662217801 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08e15e7640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c8003970 0x7f08dc1459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08e1de8640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc0cc9b0 0x7f08dc0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08e0de6640 1 -- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c8003970 msgr2=0x7f08dc1459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08e0de6640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c8003970 0x7f08dc1459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08e0de6640 1 -- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc0cc9b0 msgr2=0x7f08dc0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08e0de6640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc0cc9b0 0x7f08dc0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08e0de6640 1 -- 172.21.15.16:0/662217801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc1575d0 con 0x7f08dc15c5f0 2023-11-24T06:27:02.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08e0de6640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc15c5f0 0x7f08dc15ca70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00c760 tx=0x7f08cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08d1ffb640 1 -- 172.21.15.16:0/662217801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0074e0 con 0x7f08dc15c5f0 2023-11-24T06:27:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08d1ffb640 1 -- 172.21.15.16:0/662217801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc007660 con 0x7f08dc15c5f0 2023-11-24T06:27:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08d1ffb640 1 -- 172.21.15.16:0/662217801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc004a20 con 0x7f08dc15c5f0 2023-11-24T06:27:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.296+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc1578b0 con 0x7f08dc15c5f0 2023-11-24T06:27:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.297+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc145f20 con 0x7f08dc15c5f0 2023-11-24T06:27:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.298+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc0675d0 con 0x7f08dc15c5f0 2023-11-24T06:27:02.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.299+0000 7f08d1ffb640 1 -- 172.21.15.16:0/662217801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08cc004b80 con 0x7f08dc15c5f0 2023-11-24T06:27:02.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.300+0000 7f08d1ffb640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f08c006ae40 0x7f08c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.300+0000 7f08d1ffb640 1 -- 172.21.15.16:0/662217801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08cc088ad0 con 0x7f08dc15c5f0 2023-11-24T06:27:02.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.301+0000 7f08e15e7640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f08c006ae40 0x7f08c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.302+0000 7f08e15e7640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f08c006ae40 0x7f08c006d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009ac0 tx=0x7f08d8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:02.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.305+0000 7f08d1ffb640 1 -- 172.21.15.16:0/662217801 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08cc053ae0 con 0x7f08dc15c5f0 2023-11-24T06:27:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.548+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f08dc072340 con 0x7f08dc15c5f0 2023-11-24T06:27:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.553+0000 7f08d1ffb640 1 -- 172.21.15.16:0/662217801 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171038 (secure 0 0 0) 0x7f08cc018020 con 0x7f08dc15c5f0 2023-11-24T06:27:02.557 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f08c006ae40 msgr2=0x7f08c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f08c006ae40 0x7f08c006d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009ac0 tx=0x7f08d8030000 comp rx=0 tx=0).stop 2023-11-24T06:27:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc15c5f0 msgr2=0x7f08dc15ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc15c5f0 0x7f08dc15ca70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00c760 tx=0x7f08cc00cc20 comp rx=0 tx=0).stop 2023-11-24T06:27:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 shutdown_connections 2023-11-24T06:27:02.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f08c006ae40 0x7f08c006d300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc0cc9b0 0x7f08dc0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08dc15c5f0 0x7f08dc15ca70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 --2- 172.21.15.16:0/662217801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08c8003970 0x7f08dc1459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.558+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 >> 172.21.15.16:0/662217801 conn(0x7f08dc06a800 msgr2=0x7f08dc160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:02.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.559+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 shutdown_connections 2023-11-24T06:27:02.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.559+0000 7f08e384b640 1 -- 172.21.15.16:0/662217801 wait complete. 2023-11-24T06:27:02.580 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"active_gid":4109,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2030233038},{"type":"v1","addr":"172.21.15.173:6833","nonce":2030233038}]},"active_addr":"172.21.15.173:6833/2030233038","active_change":"2023-11-24T06:26:42.084962+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":3494221154}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":345417589}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":1470240461}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":1841691838}]}]} 2023-11-24T06:27:02.583 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T06:27:02.583 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T06:27:02.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.806+0000 7f431b59e640 1 Processor -- start 2023-11-24T06:27:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.806+0000 7f431b59e640 1 -- start start 2023-11-24T06:27:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.808+0000 7f431b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f431c162810 0x7f431c071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.808+0000 7f431b59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c0702e0 con 0x7f431c071780 2023-11-24T06:27:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.808+0000 7f431b59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c0705c0 con 0x7f431c162430 2023-11-24T06:27:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.808+0000 7f431b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0708a0 con 0x7f431c162810 2023-11-24T06:27:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.808+0000 7f431ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f431c071780 0x7f431c06c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52742/0 (socket says 172.21.15.16:52742) 2023-11-24T06:27:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.808+0000 7f431ad9d640 1 -- 172.21.15.16:0/500126969 learned_addr learned my addr 172.21.15.16:0/500126969 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f4319d9b640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f431c162810 0x7f431c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424811576 0 0) 0x7f431c0702e0 con 0x7f431c071780 2023-11-24T06:27:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4304003520 con 0x7f431c071780 2023-11-24T06:27:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1522203994 0 0) 0x7f4304003520 con 0x7f431c071780 2023-11-24T06:27:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f431c162810 msgr2=0x7f431c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f431c162810 0x7f431c071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 >> v1:172.21.15.173:6789/0 conn(0x7f431c162430 legacy=0x7f431c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c070b80 con 0x7f431c071780 2023-11-24T06:27:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f43140019f0 con 0x7f431c071780 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 >> v1:172.21.15.16:6789/0 conn(0x7f431c071780 legacy=0x7f431c06c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43040039a0 0x7f4304003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43040047c0 0x7f4304004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4304005180 0x7f430400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0708a0 con 0x7f43040047c0 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430400bec0 con 0x7f43040039a0 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.809+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430400c030 con 0x7f4304005180 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.810+0000 7f4319d9b640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43040047c0 0x7f4304004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.810+0000 7f431a59c640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43040039a0 0x7f4304003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.810+0000 7f431ad9d640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4304005180 0x7f430400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.810+0000 7f4319d9b640 1 -- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4304005180 msgr2=0x7f430400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.810+0000 7f4319d9b640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4304005180 0x7f430400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.810+0000 7f4319d9b640 1 -- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43040039a0 msgr2=0x7f4304003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.810+0000 7f4319d9b640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43040039a0 0x7f4304003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.810+0000 7f4319d9b640 1 -- 172.21.15.16:0/500126969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4304003520 con 0x7f43040047c0 2023-11-24T06:27:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.811+0000 7f4319d9b640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43040047c0 0x7f4304004c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4310009ac0 tx=0x7f43040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422d3d0ac7514f03 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431002c6b0 con 0x7f43040047c0 2023-11-24T06:27:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431002c810 con 0x7f43040047c0 2023-11-24T06:27:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431959a640 1 -- 172.21.15.16:0/500126969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431002cba0 con 0x7f43040047c0 2023-11-24T06:27:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431b59e640 1 -- 172.21.15.16:0/500126969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43040047c0 msgr2=0x7f4304004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431b59e640 1 --2- 172.21.15.16:0/500126969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43040047c0 0x7f4304004c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4310009ac0 tx=0x7f43040057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431ad9d640 1 -- 172.21.15.16:0/500126969 reap_dead start 2023-11-24T06:27:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431b59e640 1 -- 172.21.15.16:0/500126969 shutdown_connections 2023-11-24T06:27:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431b59e640 1 -- 172.21.15.16:0/500126969 >> 172.21.15.16:0/500126969 conn(0x7f431c06a800 msgr2=0x7f431c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431b59e640 1 -- 172.21.15.16:0/500126969 shutdown_connections 2023-11-24T06:27:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.812+0000 7f431b59e640 1 -- 172.21.15.16:0/500126969 wait complete. 2023-11-24T06:27:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.814+0000 7f431b59e640 1 Processor -- start 2023-11-24T06:27:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.814+0000 7f431b59e640 1 -- start start 2023-11-24T06:27:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.815+0000 7f431b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43040047c0 0x7f431c156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.815+0000 7f431b59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f431c1521e0 0x7f431c152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.815+0000 7f431b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f431c13f880 0x7f431c13fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.815+0000 7f431b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c13e100 con 0x7f431c13f880 2023-11-24T06:27:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.815+0000 7f431b59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c13d7f0 con 0x7f431c1521e0 2023-11-24T06:27:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.815+0000 7f431b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c154710 con 0x7f43040047c0 2023-11-24T06:27:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f431c13f880 0x7f431c13fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f431c13f880 0x7f431c13fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38078/0 (socket says 172.21.15.16:38078) 2023-11-24T06:27:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431ad9d640 1 -- 172.21.15.16:0/2085627281 learned_addr learned my addr 172.21.15.16:0/2085627281 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f4319d9b640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f431c1521e0 0x7f431c152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431a59c640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43040047c0 0x7f431c156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431ad9d640 1 -- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43040047c0 msgr2=0x7f431c156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431ad9d640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43040047c0 0x7f431c156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431ad9d640 1 -- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f431c1521e0 msgr2=0x7f431c152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431ad9d640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f431c1521e0 0x7f431c152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431ad9d640 1 -- 172.21.15.16:0/2085627281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c152f10 con 0x7f431c13f880 2023-11-24T06:27:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.816+0000 7f431ad9d640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f431c13f880 0x7f431c13fcc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f431c072810 tx=0x7f431400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.817+0000 7f430affd640 1 -- 172.21.15.16:0/2085627281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400ec40 con 0x7f431c13f880 2023-11-24T06:27:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.817+0000 7f430affd640 1 -- 172.21.15.16:0/2085627281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431400eda0 con 0x7f431c13f880 2023-11-24T06:27:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.817+0000 7f430affd640 1 -- 172.21.15.16:0/2085627281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314014670 con 0x7f431c13f880 2023-11-24T06:27:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.817+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c13da40 con 0x7f431c13f880 2023-11-24T06:27:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.817+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c152b80 con 0x7f431c13f880 2023-11-24T06:27:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.818+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c13e240 con 0x7f431c13f880 2023-11-24T06:27:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.819+0000 7f430affd640 1 -- 172.21.15.16:0/2085627281 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43140147d0 con 0x7f431c13f880 2023-11-24T06:27:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.820+0000 7f430affd640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f42fc06ae40 0x7f42fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.820+0000 7f430affd640 1 -- 172.21.15.16:0/2085627281 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43140893f0 con 0x7f431c13f880 2023-11-24T06:27:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.821+0000 7f431a59c640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f42fc06ae40 0x7f42fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.822+0000 7f431a59c640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f42fc06ae40 0x7f42fc06d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f430c0035f0 tx=0x7f430c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:02.825+0000 7f430affd640 1 -- 172.21.15.16:0/2085627281 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f431c13e240 con 0x7f431c13f880 2023-11-24T06:27:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.023+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f431c13e240 con 0x7f431c13f880 2023-11-24T06:27:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.025+0000 7f430affd640 1 -- 172.21.15.16:0/2085627281 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f431c13e240 con 0x7f431c13f880 2023-11-24T06:27:03.028 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:03.028 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:03.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f42fc06ae40 msgr2=0x7f42fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f42fc06ae40 0x7f42fc06d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f430c0035f0 tx=0x7f430c009040 comp rx=0 tx=0).stop 2023-11-24T06:27:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f431c13f880 msgr2=0x7f431c13fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f431c13f880 0x7f431c13fcc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f431c072810 tx=0x7f431400e530 comp rx=0 tx=0).stop 2023-11-24T06:27:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 shutdown_connections 2023-11-24T06:27:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f42fc06ae40 0x7f42fc06d300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f431c13f880 0x7f431c13fcc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f431c1521e0 0x7f431c152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 --2- 172.21.15.16:0/2085627281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43040047c0 0x7f431c156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 >> 172.21.15.16:0/2085627281 conn(0x7f431c06a800 msgr2=0x7f431c0cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 shutdown_connections 2023-11-24T06:27:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.028+0000 7f431b59e640 1 -- 172.21.15.16:0/2085627281 wait complete. 2023-11-24T06:27:03.051 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T06:27:03.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.274+0000 7f3b0908a640 1 Processor -- start 2023-11-24T06:27:03.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.275+0000 7f3b0908a640 1 -- start start 2023-11-24T06:27:03.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.276+0000 7f3b0908a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b040d0100 0x7f3b040b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.276+0000 7f3b0908a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3b040d0f40 0x7f3b040d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.276+0000 7f3b0908a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b040d0a10 con 0x7f3b04160920 2023-11-24T06:27:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.276+0000 7f3b0908a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b040cf080 con 0x7f3b040d0f40 2023-11-24T06:27:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.276+0000 7f3b0908a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040c6be0 con 0x7f3b040d0100 2023-11-24T06:27:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b03fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3b04160920 0x7f3b040cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52758/0 (socket says 172.21.15.16:52758) 2023-11-24T06:27:03.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b03fff640 1 -- 172.21.15.16:0/1137632670 learned_addr learned my addr 172.21.15.16:0/1137632670 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032502877 0 0) 0x7f3b040d0a10 con 0x7f3b04160920 2023-11-24T06:27:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b08889640 1 --2- 172.21.15.16:0/1137632670 >> v2:172.21.15.173:3300/0 conn(0x7f3b040d0f40 0x7f3b040d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3af0003520 con 0x7f3b04160920 2023-11-24T06:27:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b037fe640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b040d0100 0x7f3b040b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3648648608 0 0) 0x7f3af0003520 con 0x7f3b04160920 2023-11-24T06:27:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b040d0100 msgr2=0x7f3b040b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b02ffd640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b040d0100 0x7f3b040b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 >> v2:172.21.15.173:3300/0 conn(0x7f3b040d0f40 msgr2=0x7f3b040d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b02ffd640 1 --2- 172.21.15.16:0/1137632670 >> v2:172.21.15.173:3300/0 conn(0x7f3b040d0f40 0x7f3b040d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b040ce7e0 con 0x7f3b04160920 2023-11-24T06:27:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.277+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f3af40019f0 con 0x7f3b04160920 2023-11-24T06:27:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 >> v1:172.21.15.16:6789/0 conn(0x7f3b04160920 legacy=0x7f3b040cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b02ffd640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b02ffd640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b040d0100 0x7f3af00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b02ffd640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3af0004d00 0x7f3af0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040cf080 con 0x7f3af0003970 2023-11-24T06:27:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040c6be0 con 0x7f3af0004d00 2023-11-24T06:27:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af000bab0 con 0x7f3b040d0100 2023-11-24T06:27:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b03fff640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b037fe640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b040d0100 0x7f3af00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b08889640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3af0004d00 0x7f3af0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b03fff640 1 -- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b040d0100 msgr2=0x7f3af00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b03fff640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b040d0100 0x7f3af00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b03fff640 1 -- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3af0004d00 msgr2=0x7f3af0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b03fff640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3af0004d00 0x7f3af0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.278+0000 7f3b03fff640 1 -- 172.21.15.16:0/1137632670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af0003520 con 0x7f3af0003970 2023-11-24T06:27:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.279+0000 7f3b03fff640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3af400a790 tx=0x7f3af00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26738be1bdfb7e6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.280+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4034630 con 0x7f3af0003970 2023-11-24T06:27:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.280+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af402f070 con 0x7f3af0003970 2023-11-24T06:27:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.280+0000 7f3b02ffd640 1 -- 172.21.15.16:0/1137632670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af40349f0 con 0x7f3af0003970 2023-11-24T06:27:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.280+0000 7f3b0908a640 1 -- 172.21.15.16:0/1137632670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 msgr2=0x7f3af0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.280+0000 7f3b0908a640 1 --2- 172.21.15.16:0/1137632670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3af0003d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3af400a790 tx=0x7f3af00057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.280+0000 7f3b08889640 1 -- 172.21.15.16:0/1137632670 reap_dead start 2023-11-24T06:27:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.280+0000 7f3b0908a640 1 -- 172.21.15.16:0/1137632670 shutdown_connections 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.280+0000 7f3b0908a640 1 -- 172.21.15.16:0/1137632670 >> 172.21.15.16:0/1137632670 conn(0x7f3b040d3090 msgr2=0x7f3b040d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.280+0000 7f3b0908a640 1 -- 172.21.15.16:0/1137632670 shutdown_connections 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.281+0000 7f3b0908a640 1 -- 172.21.15.16:0/1137632670 wait complete. 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.282+0000 7f3b0908a640 1 Processor -- start 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.282+0000 7f3b0908a640 1 -- start start 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.283+0000 7f3b0908a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3b040c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.283+0000 7f3b0908a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b040b7120 0x7f3b040b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.283+0000 7f3b0908a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b041f6b40 0x7f3b041f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.283+0000 7f3b0908a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b04154ea0 con 0x7f3af0003970 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.283+0000 7f3b0908a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b040b7ac0 con 0x7f3b040b7120 2023-11-24T06:27:03.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.283+0000 7f3b0908a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b04154a10 con 0x7f3b041f6b40 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3b040c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3b040c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38102/0 (socket says 172.21.15.16:38102) 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b03fff640 1 -- 172.21.15.16:0/1334261609 learned_addr learned my addr 172.21.15.16:0/1334261609 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b08889640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b041f6b40 0x7f3b041f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b037fe640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b040b7120 0x7f3b040b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b03fff640 1 -- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b041f6b40 msgr2=0x7f3b041f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b03fff640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b041f6b40 0x7f3b041f6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b03fff640 1 -- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b040b7120 msgr2=0x7f3b040b7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b03fff640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b040b7120 0x7f3b040b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b03fff640 1 -- 172.21.15.16:0/1334261609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04153e40 con 0x7f3af0003970 2023-11-24T06:27:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.284+0000 7f3b03fff640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3b040c6610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3af400a8b0 tx=0x7f3af4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.285+0000 7f3b00ff9640 1 -- 172.21.15.16:0/1334261609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4044030 con 0x7f3af0003970 2023-11-24T06:27:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.285+0000 7f3b00ff9640 1 -- 172.21.15.16:0/1334261609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af4004e30 con 0x7f3af0003970 2023-11-24T06:27:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.285+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b041540e0 con 0x7f3af0003970 2023-11-24T06:27:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.285+0000 7f3b00ff9640 1 -- 172.21.15.16:0/1334261609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af40346d0 con 0x7f3af0003970 2023-11-24T06:27:03.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.285+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b041546a0 con 0x7f3af0003970 2023-11-24T06:27:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.286+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b04067450 con 0x7f3af0003970 2023-11-24T06:27:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.287+0000 7f3b00ff9640 1 -- 172.21.15.16:0/1334261609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3af4047070 con 0x7f3af0003970 2023-11-24T06:27:03.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.288+0000 7f3b00ff9640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ae406ae40 0x7f3ae406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.288+0000 7f3b00ff9640 1 -- 172.21.15.16:0/1334261609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3af4045070 con 0x7f3af0003970 2023-11-24T06:27:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.289+0000 7f3b037fe640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ae406ae40 0x7f3ae406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.290+0000 7f3b037fe640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ae406ae40 0x7f3ae406d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3b041f6870 tx=0x7f3aec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.292+0000 7f3b00ff9640 1 -- 172.21.15.16:0/1334261609 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3af4076c60 con 0x7f3af0003970 2023-11-24T06:27:03.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.490+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b04067450 con 0x7f3af0003970 2023-11-24T06:27:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.491+0000 7f3b00ff9640 1 -- 172.21.15.16:0/1334261609 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f3af407a280 con 0x7f3af0003970 2023-11-24T06:27:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:03.494 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:03.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ae406ae40 msgr2=0x7f3ae406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ae406ae40 0x7f3ae406d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3b041f6870 tx=0x7f3aec008040 comp rx=0 tx=0).stop 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 msgr2=0x7f3b040c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3b040c6610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3af400a8b0 tx=0x7f3af4001cb0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 shutdown_connections 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ae406ae40 0x7f3ae406d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b041f6b40 0x7f3b041f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b040b7120 0x7f3b040b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 --2- 172.21.15.16:0/1334261609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af0003970 0x7f3b040c6610 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 >> 172.21.15.16:0/1334261609 conn(0x7f3b040d3090 msgr2=0x7f3b040d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 shutdown_connections 2023-11-24T06:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.495+0000 7f3b0908a640 1 -- 172.21.15.16:0/1334261609 wait complete. 2023-11-24T06:27:03.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T06:27:03.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T06:27:03.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T06:27:03.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T06:27:03.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T06:27:03.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T06:27:03.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T06:27:03.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T06:27:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.828+0000 7f6681a4b640 1 Processor -- start 2023-11-24T06:27:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.828+0000 7f6681a4b640 1 -- start start 2023-11-24T06:27:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.828+0000 7f4f07221640 1 Processor -- start 2023-11-24T06:27:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.828+0000 7f4f07221640 1 -- start start 2023-11-24T06:27:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.829+0000 7f45cc683640 1 Processor -- start 2023-11-24T06:27:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.830+0000 7f6681a4b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6674054390 0x7f66740e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.830+0000 7f6681a4b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6674007b40 con 0x7f66740e3130 2023-11-24T06:27:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.830+0000 7f6681a4b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66740587e0 con 0x7f667403c070 2023-11-24T06:27:03.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.830+0000 7f6681a4b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6674058ac0 con 0x7f6674054390 2023-11-24T06:27:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.830+0000 7f45cc683640 1 -- start start 2023-11-24T06:27:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.831+0000 7f668124a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6674054390 0x7f66740e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.831+0000 7f667bfff640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f667403c070 0x7f6674053c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:44414/0 (socket says 172.21.15.16:44414) 2023-11-24T06:27:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.831+0000 7f667bfff640 1 -- 172.21.15.16:0/3910067656 learned_addr learned my addr 172.21.15.16:0/3910067656 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.832+0000 7f4f07221640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f001536f0 0x7f4f00072290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.832+0000 7f667b7fe640 1 -- 172.21.15.16:0/3910067656 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197332545 0 0) 0x7f66740587e0 con 0x7f667403c070 2023-11-24T06:27:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.832+0000 7f667b7fe640 1 -- 172.21.15.16:0/3910067656 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f666c003520 con 0x7f667403c070 2023-11-24T06:27:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.832+0000 7f45cc683640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45c40b22b0 0x7f45c40d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.832+0000 7f45cc683640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45c40d0140 0x7f45c40b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.832+0000 7f4f07221640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f000b2f10 0x7f4f000b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.832+0000 7f4f07221640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f000b38a0 con 0x7f4f00159750 2023-11-24T06:27:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.832+0000 7f4f07221640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f00070160 con 0x7f4f001536f0 2023-11-24T06:27:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f4f07221640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0015a010 con 0x7f4f000b2f10 2023-11-24T06:27:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f4f06a20640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f000b2f10 0x7f4f000b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f4f06a20640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f000b2f10 0x7f4f000b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:40838/0 (socket says 172.21.15.16:40838) 2023-11-24T06:27:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f4f06a20640 1 -- 172.21.15.16:0/946977074 learned_addr learned my addr 172.21.15.16:0/946977074 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f45cc683640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c40d1970 con 0x7f45c415d560 2023-11-24T06:27:03.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f4f0621f640 1 --2- 172.21.15.16:0/946977074 >> v2:172.21.15.16:3300/0 conn(0x7f4f001536f0 0x7f4f00072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f45cc683640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c40d1c50 con 0x7f45c40b22b0 2023-11-24T06:27:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f45cc683640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c40d1f30 con 0x7f45c40d0140 2023-11-24T06:27:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f45c9c1e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45c40b22b0 0x7f45c40d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f45ca41f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f45c415d560 0x7f45c40cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52766/0 (socket says 172.21.15.16:52766) 2023-11-24T06:27:03.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f45ca41f640 1 -- 172.21.15.16:0/3267991771 learned_addr learned my addr 172.21.15.16:0/3267991771 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.834+0000 7f4f06a20640 1 -- 172.21.15.16:0/946977074 >> v2:172.21.15.16:3300/0 conn(0x7f4f001536f0 msgr2=0x7f4f00072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.834+0000 7f4f06a20640 1 --2- 172.21.15.16:0/946977074 >> v2:172.21.15.16:3300/0 conn(0x7f4f001536f0 0x7f4f00072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.834+0000 7f4f06a20640 1 -- 172.21.15.16:0/946977074 >> v1:172.21.15.16:6789/0 conn(0x7f4f00159750 legacy=0x7f4f000cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.834+0000 7f4f06a20640 1 -- 172.21.15.16:0/946977074 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0015a2f0 con 0x7f4f000b2f10 2023-11-24T06:27:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.834+0000 7f45cac20640 1 --2- 172.21.15.16:0/3267991771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45c40d0140 0x7f45c40b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.835+0000 7f45c9c1e640 1 -- 172.21.15.16:0/3267991771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45c40d0140 msgr2=0x7f45c40b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.835+0000 7f45c9c1e640 1 --2- 172.21.15.16:0/3267991771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45c40d0140 0x7f45c40b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.835+0000 7f45c9c1e640 1 -- 172.21.15.16:0/3267991771 >> v1:172.21.15.16:6789/0 conn(0x7f45c415d560 legacy=0x7f45c40cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.835+0000 7f45c9c1e640 1 -- 172.21.15.16:0/3267991771 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c40d2210 con 0x7f45c40b22b0 2023-11-24T06:27:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.834+0000 7f45c941d640 1 -- 172.21.15.16:0/3267991771 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910788030 0 0) 0x7f45c40d1970 con 0x7f45c415d560 2023-11-24T06:27:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f668124a640 1 -- 172.21.15.16:0/3910067656 >> v1:172.21.15.173:6789/0 conn(0x7f667403c070 legacy=0x7f6674053c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f668124a640 1 -- 172.21.15.16:0/3910067656 >> v1:172.21.15.16:6789/0 conn(0x7f66740e3130 legacy=0x7f6674058020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.833+0000 7f668124a640 1 -- 172.21.15.16:0/3910067656 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6674006330 con 0x7f6674054390 2023-11-24T06:27:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.836+0000 7f940c143640 1 Processor -- start 2023-11-24T06:27:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.836+0000 7f668124a640 1 --2- 172.21.15.16:0/3910067656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674054390 0x7f66740e7750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f667c04fab0 tx=0x7f667c0a5ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3046b8b5a4f6f9c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.836+0000 7f4f06a20640 1 --2- 172.21.15.16:0/946977074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f000b2f10 0x7f4f000b3360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009870 tx=0x7f4ef002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a6acb97a91e163 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.836+0000 7f667b7fe640 1 -- 172.21.15.16:0/3910067656 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0a6c70 con 0x7f6674054390 2023-11-24T06:27:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.837+0000 7f667b7fe640 1 -- 172.21.15.16:0/3910067656 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c0a6e50 con 0x7f6674054390 2023-11-24T06:27:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.836+0000 7f4f0521d640 1 -- 172.21.15.16:0/946977074 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0031640 con 0x7f4f000b2f10 2023-11-24T06:27:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.837+0000 7f4f0521d640 1 -- 172.21.15.16:0/946977074 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef00317a0 con 0x7f4f000b2f10 2023-11-24T06:27:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.837+0000 7f4f0521d640 1 -- 172.21.15.16:0/946977074 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0031b30 con 0x7f4f000b2f10 2023-11-24T06:27:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.837+0000 7f667b7fe640 1 -- 172.21.15.16:0/3910067656 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0ae640 con 0x7f6674054390 2023-11-24T06:27:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.837+0000 7f45c9c1e640 1 --2- 172.21.15.16:0/3267991771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c40b22b0 0x7f45c40d0f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009d90 tx=0x7f45bc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb958fbd61cce698 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.837+0000 7f45c941d640 1 -- 172.21.15.16:0/3267991771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc02cc50 con 0x7f45c40b22b0 2023-11-24T06:27:03.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.837+0000 7f45c941d640 1 -- 172.21.15.16:0/3267991771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc02cdb0 con 0x7f45c40b22b0 2023-11-24T06:27:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.837+0000 7f45c941d640 1 -- 172.21.15.16:0/3267991771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc02cc50 con 0x7f45c40b22b0 2023-11-24T06:27:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.836+0000 7f940c143640 1 -- start start 2023-11-24T06:27:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f4f07221640 1 -- 172.21.15.16:0/946977074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f000b2f10 msgr2=0x7f4f000b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f4f07221640 1 --2- 172.21.15.16:0/946977074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f000b2f10 0x7f4f000b3360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009870 tx=0x7f4ef002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f940c143640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93fc0e3080 0x7f93fc057d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f940c143640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f93fc0542c0 0x7f93fc006890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f940c143640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc058290 con 0x7f93fc03e800 2023-11-24T06:27:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f940c143640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc006160 con 0x7f93fc0e3080 2023-11-24T06:27:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f940c143640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc058530 con 0x7f93fc0542c0 2023-11-24T06:27:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f4f07221640 1 -- 172.21.15.16:0/946977074 shutdown_connections 2023-11-24T06:27:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f4f07221640 1 --2- 172.21.15.16:0/946977074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f000b2f10 0x7f4f000b3360 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f4f07221640 1 --2- 172.21.15.16:0/946977074 >> v2:172.21.15.16:3300/0 conn(0x7f4f001536f0 0x7f4f00072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f4f07221640 1 -- 172.21.15.16:0/946977074 >> 172.21.15.16:0/946977074 conn(0x7f4f000b0b00 msgr2=0x7f4f000b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f6681a4b640 1 -- 172.21.15.16:0/3910067656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674054390 msgr2=0x7f66740e7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f6681a4b640 1 --2- 172.21.15.16:0/3910067656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674054390 0x7f66740e7750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f667c04fab0 tx=0x7f667c0a5ec0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f6681a4b640 1 -- 172.21.15.16:0/3910067656 shutdown_connections 2023-11-24T06:27:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f6681a4b640 1 --2- 172.21.15.16:0/3910067656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6674054390 0x7f66740e7750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f6681a4b640 1 -- 172.21.15.16:0/3910067656 >> 172.21.15.16:0/3910067656 conn(0x7f6674058eb0 msgr2=0x7f6674059290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.838+0000 7f4f07221640 1 -- 172.21.15.16:0/946977074 shutdown_connections 2023-11-24T06:27:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f4f07221640 1 -- 172.21.15.16:0/946977074 wait complete. 2023-11-24T06:27:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f6681a4b640 1 -- 172.21.15.16:0/3910067656 shutdown_connections 2023-11-24T06:27:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f94096de640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f93fc03e800 0x7f93fc053bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52772/0 (socket says 172.21.15.16:52772) 2023-11-24T06:27:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f94096de640 1 -- 172.21.15.16:0/1269221738 learned_addr learned my addr 172.21.15.16:0/1269221738 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f9409edf640 1 --2- 172.21.15.16:0/1269221738 >> v2:172.21.15.16:3300/0 conn(0x7f93fc0e3080 0x7f93fc057d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f940a6e0640 1 --2- 172.21.15.16:0/1269221738 >> v2:172.21.15.173:3300/0 conn(0x7f93fc0542c0 0x7f93fc006890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.840+0000 7f9409edf640 1 -- 172.21.15.16:0/1269221738 >> v2:172.21.15.173:3300/0 conn(0x7f93fc0542c0 msgr2=0x7f93fc006890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.840+0000 7f6681a4b640 1 -- 172.21.15.16:0/3910067656 wait complete. 2023-11-24T06:27:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f45cc683640 1 -- 172.21.15.16:0/3267991771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c40b22b0 msgr2=0x7f45c40d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.839+0000 7f45cc683640 1 --2- 172.21.15.16:0/3267991771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c40b22b0 0x7f45c40d0f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009d90 tx=0x7f45bc02bea0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.840+0000 7f9408edd640 1 -- 172.21.15.16:0/1269221738 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106617838 0 0) 0x7f93fc058290 con 0x7f93fc03e800 2023-11-24T06:27:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.840+0000 7f9409edf640 1 --2- 172.21.15.16:0/1269221738 >> v2:172.21.15.173:3300/0 conn(0x7f93fc0542c0 0x7f93fc006890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.840+0000 7f9409edf640 1 -- 172.21.15.16:0/1269221738 >> v1:172.21.15.16:6789/0 conn(0x7f93fc03e800 legacy=0x7f93fc053bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.840+0000 7f9409edf640 1 -- 172.21.15.16:0/1269221738 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc058810 con 0x7f93fc0e3080 2023-11-24T06:27:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.841+0000 7f4f07221640 1 Processor -- start 2023-11-24T06:27:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.841+0000 7f940a6e0640 1 --2- 172.21.15.16:0/1269221738 >> v2:172.21.15.173:3300/0 conn(0x7f93fc0542c0 0x7f93fc006890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.841+0000 7f9409edf640 1 --2- 172.21.15.16:0/1269221738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc0e3080 0x7f93fc057d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f940404fab0 tx=0x7f94040a6160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84a8a79ca6aed6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.842+0000 7f9408edd640 1 -- 172.21.15.16:0/1269221738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94040a6dc0 con 0x7f93fc0e3080 2023-11-24T06:27:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.842+0000 7f45cc683640 1 -- 172.21.15.16:0/3267991771 shutdown_connections 2023-11-24T06:27:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.842+0000 7f9408edd640 1 -- 172.21.15.16:0/1269221738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94040ae440 con 0x7f93fc0e3080 2023-11-24T06:27:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.842+0000 7f9408edd640 1 -- 172.21.15.16:0/1269221738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94040ae830 con 0x7f93fc0e3080 2023-11-24T06:27:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.842+0000 7f45cc683640 1 --2- 172.21.15.16:0/3267991771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45c40d0140 0x7f45c40b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.842+0000 7f45cc683640 1 --2- 172.21.15.16:0/3267991771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c40b22b0 0x7f45c40d0f20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.842+0000 7f45cc683640 1 -- 172.21.15.16:0/3267991771 >> 172.21.15.16:0/3267991771 conn(0x7f45c40d3020 msgr2=0x7f45c40d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.842+0000 7f6681a4b640 1 Processor -- start 2023-11-24T06:27:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.841+0000 7f4f07221640 1 -- start start 2023-11-24T06:27:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.843+0000 7f6681a4b640 1 -- start start 2023-11-24T06:27:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.843+0000 7f4f07221640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f000b2f10 0x7f4f0013e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.843+0000 7f4f07221640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f001536f0 0x7f4f00158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.843+0000 7f4f07221640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f00159750 0x7f4f0015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.843+0000 7f4f07221640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f00158c70 con 0x7f4f000b2f10 2023-11-24T06:27:03.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.843+0000 7f4f07221640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000c4e30 con 0x7f4f00159750 2023-11-24T06:27:03.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.843+0000 7f940c143640 1 -- 172.21.15.16:0/1269221738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc0e3080 msgr2=0x7f93fc057d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.843+0000 7f940c143640 1 --2- 172.21.15.16:0/1269221738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc0e3080 0x7f93fc057d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f940404fab0 tx=0x7f94040a6160 comp rx=0 tx=0).stop 2023-11-24T06:27:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.843+0000 7f4f07221640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0013e790 con 0x7f4f001536f0 2023-11-24T06:27:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f4f05a1e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f001536f0 0x7f4f00158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f940c143640 1 -- 172.21.15.16:0/1269221738 shutdown_connections 2023-11-24T06:27:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f940c143640 1 --2- 172.21.15.16:0/1269221738 >> v2:172.21.15.173:3300/0 conn(0x7f93fc0542c0 0x7f93fc006890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f940c143640 1 --2- 172.21.15.16:0/1269221738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc0e3080 0x7f93fc057d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f940c143640 1 -- 172.21.15.16:0/1269221738 >> 172.21.15.16:0/1269221738 conn(0x7f93fc0559a0 msgr2=0x7f93fc055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f45cc683640 1 -- 172.21.15.16:0/3267991771 shutdown_connections 2023-11-24T06:27:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f4f0621f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f000b2f10 0x7f4f0013e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f4f0621f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f000b2f10 0x7f4f0013e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38134/0 (socket says 172.21.15.16:38134) 2023-11-24T06:27:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f4f0621f640 1 -- 172.21.15.16:0/3291244860 learned_addr learned my addr 172.21.15.16:0/3291244860 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f940c143640 1 -- 172.21.15.16:0/1269221738 shutdown_connections 2023-11-24T06:27:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f4f06a20640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f00159750 0x7f4f0015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f6681a4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f667403c070 0x7f66740e6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f6681a4b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6674054390 0x7f66740d9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f6681a4b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66740e3130 0x7f66740e3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f6681a4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66740d8960 con 0x7f667403c070 2023-11-24T06:27:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f6681a4b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66740e6550 con 0x7f66740e3130 2023-11-24T06:27:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f6681a4b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66740d8b70 con 0x7f6674054390 2023-11-24T06:27:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f45cc683640 1 -- 172.21.15.16:0/3267991771 wait complete. 2023-11-24T06:27:03.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f6680a49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f667403c070 0x7f66740e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f668124a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66740e3130 0x7f66740e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f667bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6674054390 0x7f66740d9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f668124a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66740e3130 0x7f66740e3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53062/0 (socket says 172.21.15.16:53062) 2023-11-24T06:27:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f668124a640 1 -- 172.21.15.16:0/1989800775 learned_addr learned my addr 172.21.15.16:0/1989800775 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f4f0621f640 1 -- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f001536f0 msgr2=0x7f4f00158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f4f0621f640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f001536f0 0x7f4f00158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f4f0621f640 1 -- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f00159750 msgr2=0x7f4f0015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f4f0621f640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f00159750 0x7f4f0015bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.845+0000 7f4f0621f640 1 -- 172.21.15.16:0/3291244860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef00094d0 con 0x7f4f000b2f10 2023-11-24T06:27:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.844+0000 7f940c143640 1 -- 172.21.15.16:0/1269221738 wait complete. 2023-11-24T06:27:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.846+0000 7f4f0621f640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f000b2f10 0x7f4f0013e250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d800 tx=0x7f4efc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.846+0000 7f78256b4640 1 Processor -- start 2023-11-24T06:27:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.847+0000 7f668124a640 1 -- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6674054390 msgr2=0x7f66740d9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.847+0000 7f668124a640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6674054390 0x7f66740d9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.847+0000 7f668124a640 1 -- 172.21.15.16:0/1989800775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f667403c070 msgr2=0x7f66740e6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.847+0000 7f668124a640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f667403c070 0x7f66740e6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.847+0000 7f668124a640 1 -- 172.21.15.16:0/1989800775 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c04edc0 con 0x7f66740e3130 2023-11-24T06:27:03.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.848+0000 7f45cc683640 1 Processor -- start 2023-11-24T06:27:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.848+0000 7f668124a640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66740e3130 0x7f66740e3690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f667c050020 tx=0x7f667c0a8070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.848+0000 7f4ef6ffd640 1 -- 172.21.15.16:0/3291244860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00b6c0 con 0x7f4f000b2f10 2023-11-24T06:27:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.848+0000 7f4ef6ffd640 1 -- 172.21.15.16:0/3291244860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc00b820 con 0x7f4f000b2f10 2023-11-24T06:27:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.848+0000 7f4ef6ffd640 1 -- 172.21.15.16:0/3291244860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00bbb0 con 0x7f4f000b2f10 2023-11-24T06:27:03.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.848+0000 7f45cc683640 1 -- start start 2023-11-24T06:27:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.848+0000 7f940c143640 1 Processor -- start 2023-11-24T06:27:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.848+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f001589a0 con 0x7f4f000b2f10 2023-11-24T06:27:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.848+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0013ee80 con 0x7f4f000b2f10 2023-11-24T06:27:03.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.849+0000 7f940c143640 1 -- start start 2023-11-24T06:27:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.846+0000 7f78256b4640 1 -- start start 2023-11-24T06:27:03.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.849+0000 7f78256b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7820160f80 0x7f7820161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.849+0000 7f78256b4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78200ce8f0 0x7f78200ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.849+0000 7f78256b4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78201618a0 con 0x7f78200cfbe0 2023-11-24T06:27:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.849+0000 7f78256b4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78200d0de0 con 0x7f7820160f80 2023-11-24T06:27:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.849+0000 7f78256b4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78200d10c0 con 0x7f78200ce8f0 2023-11-24T06:27:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f78167fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78200ce8f0 0x7f78200ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f781effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7820160f80 0x7f7820161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f78167fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78200ce8f0 0x7f78200ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:40870/0 (socket says 172.21.15.16:40870) 2023-11-24T06:27:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f78167fc640 1 -- 172.21.15.16:0/72770987 learned_addr learned my addr 172.21.15.16:0/72770987 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f66797fa640 1 -- 172.21.15.16:0/1989800775 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0ba650 con 0x7f66740e3130 2023-11-24T06:27:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f66797fa640 1 -- 172.21.15.16:0/1989800775 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c0ba7b0 con 0x7f66740e3130 2023-11-24T06:27:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f66797fa640 1 -- 172.21.15.16:0/1989800775 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0bab60 con 0x7f66740e3130 2023-11-24T06:27:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4f00070400 con 0x7f4f000b2f10 2023-11-24T06:27:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.851+0000 7f940c143640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93fc03e800 0x7f93fc03d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.851+0000 7f940c143640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93fc0542c0 0x7f93fc04ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.851+0000 7f781e7fc640 1 -- 172.21.15.16:0/72770987 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709613286 0 0) 0x7f78201618a0 con 0x7f78200cfbe0 2023-11-24T06:27:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66740d8e50 con 0x7f66740e3130 2023-11-24T06:27:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.850+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66740d9410 con 0x7f66740e3130 2023-11-24T06:27:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.851+0000 7f45cc683640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45c40b22b0 0x7f45c41fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.851+0000 7f45cc683640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45c40d0140 0x7f45c42029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.851+0000 7f45cc683640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c415d560 0x7f45c40b7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f940c143640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc0e3080 0x7f93fc052020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f45cc683640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c42038f0 con 0x7f45c415d560 2023-11-24T06:27:03.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f45cac20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c415d560 0x7f45c40b7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f9409edf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93fc03e800 0x7f93fc03d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.851+0000 7f781e7fc640 1 -- 172.21.15.16:0/72770987 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f780c003520 con 0x7f78200cfbe0 2023-11-24T06:27:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f78167fc640 1 -- 172.21.15.16:0/72770987 >> v2:172.21.15.16:3300/0 conn(0x7f7820160f80 msgr2=0x7f7820161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f78167fc640 1 --2- 172.21.15.16:0/72770987 >> v2:172.21.15.16:3300/0 conn(0x7f7820160f80 0x7f7820161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f78167fc640 1 -- 172.21.15.16:0/72770987 >> v1:172.21.15.16:6789/0 conn(0x7f78200cfbe0 legacy=0x7f78200d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f78167fc640 1 -- 172.21.15.16:0/72770987 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200d13a0 con 0x7f78200ce8f0 2023-11-24T06:27:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f781effd640 1 --2- 172.21.15.16:0/72770987 >> v2:172.21.15.16:3300/0 conn(0x7f7820160f80 0x7f7820161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f45cac20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c415d560 0x7f45c40b7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38148/0 (socket says 172.21.15.16:38148) 2023-11-24T06:27:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f45cac20640 1 -- 172.21.15.16:0/4225146360 learned_addr learned my addr 172.21.15.16:0/4225146360 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f45ca41f640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45c40b22b0 0x7f45c41fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4203330 con 0x7f45c40d0140 2023-11-24T06:27:03.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4203610 con 0x7f45c40b22b0 2023-11-24T06:27:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f9409edf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93fc03e800 0x7f93fc03d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53068/0 (socket says 172.21.15.16:53068) 2023-11-24T06:27:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f9409edf640 1 -- 172.21.15.16:0/1647591124 learned_addr learned my addr 172.21.15.16:0/1647591124 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.852+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66740e3510 con 0x7f66740e3130 2023-11-24T06:27:03.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc04efc0 con 0x7f93fc0e3080 2023-11-24T06:27:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0db590 con 0x7f93fc03e800 2023-11-24T06:27:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0db870 con 0x7f93fc0542c0 2023-11-24T06:27:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f94096de640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93fc0542c0 0x7f93fc04ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f940a6e0640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc0e3080 0x7f93fc052020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f45cac20640 1 -- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45c40b22b0 msgr2=0x7f45c41fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f45cac20640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45c40b22b0 0x7f45c41fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f45cac20640 1 -- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45c40d0140 msgr2=0x7f45c42029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f45cac20640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45c40d0140 0x7f45c42029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f45cac20640 1 -- 172.21.15.16:0/4225146360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc009a70 con 0x7f45c415d560 2023-11-24T06:27:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.854+0000 7f940a6e0640 1 -- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93fc0542c0 msgr2=0x7f93fc04ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.854+0000 7f940a6e0640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93fc0542c0 0x7f93fc04ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.854+0000 7f940a6e0640 1 -- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93fc03e800 msgr2=0x7f93fc03d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.854+0000 7f940a6e0640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93fc03e800 0x7f93fc03d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.853+0000 7f4ef6ffd640 1 -- 172.21.15.16:0/3291244860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4efc00bd10 con 0x7f4f000b2f10 2023-11-24T06:27:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.854+0000 7f940a6e0640 1 -- 172.21.15.16:0/1647591124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9400009400 con 0x7f93fc0e3080 2023-11-24T06:27:03.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.854+0000 7f45cac20640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c415d560 0x7f45c40b7580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45b400d5d0 tx=0x7f45b400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f940a6e0640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc0e3080 0x7f93fc052020 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f940000e830 tx=0x7f940000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.854+0000 7f78167fc640 1 --2- 172.21.15.16:0/72770987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78200ce8f0 0x7f78200ced50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7808009e50 tx=0x7f780802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6ee433dcd4af2c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f45baffd640 1 -- 172.21.15.16:0/4225146360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b40125f0 con 0x7f45c415d560 2023-11-24T06:27:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f781e7fc640 1 -- 172.21.15.16:0/72770987 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780802cb00 con 0x7f78200ce8f0 2023-11-24T06:27:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f93fa7fc640 1 -- 172.21.15.16:0/1647591124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940000c940 con 0x7f93fc0e3080 2023-11-24T06:27:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f93fa7fc640 1 -- 172.21.15.16:0/1647591124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940000cac0 con 0x7f93fc0e3080 2023-11-24T06:27:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f93fa7fc640 1 -- 172.21.15.16:0/1647591124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940000ce70 con 0x7f93fc0e3080 2023-11-24T06:27:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f781e7fc640 1 -- 172.21.15.16:0/72770987 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780802cc60 con 0x7f78200ce8f0 2023-11-24T06:27:03.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f781e7fc640 1 -- 172.21.15.16:0/72770987 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78080343f0 con 0x7f78200ce8f0 2023-11-24T06:27:03.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f4ef6ffd640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ee406ae90 0x7f4ee406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f4ef6ffd640 1 -- 172.21.15.16:0/3291244860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4efc0898e0 con 0x7f4f000b2f10 2023-11-24T06:27:03.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940404f090 con 0x7f93fc0e3080 2023-11-24T06:27:03.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc0dbe30 con 0x7f93fc0e3080 2023-11-24T06:27:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c415dac0 con 0x7f45c415d560 2023-11-24T06:27:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c40c6ed0 con 0x7f45c415d560 2023-11-24T06:27:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f78256b4640 1 -- 172.21.15.16:0/72770987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78200ce8f0 msgr2=0x7f78200ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.856+0000 7f68b2f7c640 1 Processor -- start 2023-11-24T06:27:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.856+0000 7f68b2f7c640 1 -- start start 2023-11-24T06:27:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.855+0000 7ff9b9a2b640 1 Processor -- start 2023-11-24T06:27:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.856+0000 7ff9b9a2b640 1 -- start start 2023-11-24T06:27:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f78256b4640 1 --2- 172.21.15.16:0/72770987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78200ce8f0 0x7f78200ced50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7808009e50 tx=0x7f780802bf60 comp rx=0 tx=0).stop 2023-11-24T06:27:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f66797fa640 1 -- 172.21.15.16:0/1989800775 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f667c0bacc0 con 0x7f66740e3130 2023-11-24T06:27:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f4ef6ffd640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7f4ee40730b0 0x7f4ee40754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f4ef6ffd640 1 -- 172.21.15.16:0/3291244860 --> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ee4075b80 con 0x7f4ee40730b0 2023-11-24T06:27:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f4ef6ffd640 1 -- 172.21.15.16:0/3291244860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4efc08e2c0 con 0x7f4f000b2f10 2023-11-24T06:27:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7f66797fa640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f666006af10 0x7f666006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7f66797fa640 1 -- 172.21.15.16:0/1989800775 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f667c1269b0 con 0x7f66740e3130 2023-11-24T06:27:03.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f45c415d940 con 0x7f45c415d560 2023-11-24T06:27:03.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f45baffd640 1 -- 172.21.15.16:0/4225146360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b4012750 con 0x7f45c415d560 2023-11-24T06:27:03.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f45baffd640 1 -- 172.21.15.16:0/4225146360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4012b00 con 0x7f45c415d560 2023-11-24T06:27:03.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f4f05a1e640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ee406ae90 0x7f4ee406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7f4f06a20640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7f4ee40730b0 0x7f4ee40754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f78256b4640 1 -- 172.21.15.16:0/72770987 shutdown_connections 2023-11-24T06:27:03.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f78256b4640 1 --2- 172.21.15.16:0/72770987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78200ce8f0 0x7f78200ced50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f78256b4640 1 --2- 172.21.15.16:0/72770987 >> v2:172.21.15.16:3300/0 conn(0x7f7820160f80 0x7f7820161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.857+0000 7f78256b4640 1 -- 172.21.15.16:0/72770987 >> 172.21.15.16:0/72770987 conn(0x7f782009c550 msgr2=0x7f78200d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7f68b2f7c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68ac0cf9a0 0x7f68ac070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7f68b2f7c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f68ac070ea0 0x7f68ac0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7f68b2f7c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac071690 con 0x7f68ac14b2c0 2023-11-24T06:27:03.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7f68b2f7c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac14b6a0 con 0x7f68ac070ea0 2023-11-24T06:27:03.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7f68b2f7c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac071970 con 0x7f68ac0cf9a0 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7ff9b9a2b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b4160f80 0x7ff9b4161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7ff9b9a2b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9b40ced50 0x7ff9b40d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7ff9b9a2b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40cf1b0 con 0x7ff9b40ce8f0 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7ff9b9a2b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b41618a0 con 0x7ff9b40ced50 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7ff9b9a2b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40d0de0 con 0x7ff9b4160f80 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7ff9b2ffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b4160f80 0x7ff9b4161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7ff9b2ffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b4160f80 0x7ff9b4161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:40900/0 (socket says 172.21.15.16:40900) 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7ff9b2ffd640 1 -- 172.21.15.16:0/706691665 learned_addr learned my addr 172.21.15.16:0/706691665 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.858+0000 7ff9b37fe640 1 --2- 172.21.15.16:0/706691665 >> v2:172.21.15.16:3300/0 conn(0x7ff9b40ced50 0x7ff9b40d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7ff9b2ffd640 1 -- 172.21.15.16:0/706691665 >> v2:172.21.15.16:3300/0 conn(0x7ff9b40ced50 msgr2=0x7ff9b40d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7ff9b2ffd640 1 --2- 172.21.15.16:0/706691665 >> v2:172.21.15.16:3300/0 conn(0x7ff9b40ced50 0x7ff9b40d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7ff9b2ffd640 1 -- 172.21.15.16:0/706691665 >> v1:172.21.15.16:6789/0 conn(0x7ff9b40ce8f0 legacy=0x7ff9b40cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7f66797fa640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7f6660073130 0x7f6660075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7f6680a49640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f666006af10 0x7f666006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7f66797fa640 1 -- 172.21.15.16:0/1989800775 --> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6660075c00 con 0x7f6660073130 2023-11-24T06:27:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7f66797fa640 1 -- 172.21.15.16:0/1989800775 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f667c126d60 con 0x7f66740e3130 2023-11-24T06:27:03.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7f68b0d18640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f68ac14b2c0 0x7f68ac0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:44426/0 (socket says 172.21.15.16:44426) 2023-11-24T06:27:03.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7f78256b4640 1 -- 172.21.15.16:0/72770987 shutdown_connections 2023-11-24T06:27:03.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7f78256b4640 1 -- 172.21.15.16:0/72770987 wait complete. 2023-11-24T06:27:03.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7f68b0d18640 1 -- 172.21.15.16:0/3579121847 learned_addr learned my addr 172.21.15.16:0/3579121847 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:03.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.859+0000 7ff9b2ffd640 1 -- 172.21.15.16:0/706691665 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40d10c0 con 0x7ff9b4160f80 2023-11-24T06:27:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.860+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f93ec000fc0 con 0x7f93fc0e3080 2023-11-24T06:27:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.860+0000 7f93fa7fc640 1 -- 172.21.15.16:0/1647591124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9400013470 con 0x7f93fc0e3080 2023-11-24T06:27:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.860+0000 7f68abfff640 1 --2- 172.21.15.16:0/3579121847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68ac0cf9a0 0x7f68ac070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.860+0000 7f4f05a1e640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ee406ae90 0x7f4ee406d350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f00158380 tx=0x7f4ef800d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.860+0000 7f68b1519640 1 --2- 172.21.15.16:0/3579121847 >> v2:172.21.15.173:3300/0 conn(0x7f68ac070ea0 0x7f68ac0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.860+0000 7f4f06a20640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7f4ee40730b0 0x7f4ee40754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7fddf8656640 1 Processor -- start 2023-11-24T06:27:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7f667bfff640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7f6660073130 0x7f6660075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7f6680a49640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f666006af10 0x7f666006d3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6670000f60 tx=0x7f6670009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7ff9b2ffd640 1 --2- 172.21.15.16:0/706691665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b4160f80 0x7ff9b4161360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009870 tx=0x7ff9a402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=574682f3cf6616a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7f68b1519640 1 -- 172.21.15.16:0/3579121847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68ac0cf9a0 msgr2=0x7f68ac070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7f68b1519640 1 --2- 172.21.15.16:0/3579121847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68ac0cf9a0 0x7f68ac070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7fddf8656640 1 -- start start 2023-11-24T06:27:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7f68b1519640 1 -- 172.21.15.16:0/3579121847 >> v1:172.21.15.173:6789/0 conn(0x7f68ac14b2c0 legacy=0x7f68ac0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7f68ab7fe640 1 -- 172.21.15.16:0/3579121847 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221160643 0 0) 0x7f68ac071690 con 0x7f68ac14b2c0 2023-11-24T06:27:03.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7f68b1519640 1 -- 172.21.15.16:0/3579121847 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac071c50 con 0x7f68ac070ea0 2023-11-24T06:27:03.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.861+0000 7ff9b1ffb640 1 -- 172.21.15.16:0/706691665 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4030840 con 0x7ff9b4160f80 2023-11-24T06:27:03.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.862+0000 7f667bfff640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7f6660073130 0x7f6660075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.862+0000 7f4ef6ffd640 1 -- 172.21.15.16:0/3291244860 <== osd.2 v2:172.21.15.16:6808/620212774 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4ee4075b80 con 0x7f4ee40730b0 2023-11-24T06:27:03.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.862+0000 7f78256b4640 1 Processor -- start 2023-11-24T06:27:03.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7f66797fa640 1 -- 172.21.15.16:0/1989800775 <== osd.0 v2:172.21.15.16:6800/4232398165 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6660075c00 con 0x7f6660073130 2023-11-24T06:27:03.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.862+0000 7ff9b1ffb640 1 -- 172.21.15.16:0/706691665 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a40309a0 con 0x7ff9b4160f80 2023-11-24T06:27:03.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.862+0000 7ff9b1ffb640 1 -- 172.21.15.16:0/706691665 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4030d30 con 0x7ff9b4160f80 2023-11-24T06:27:03.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.862+0000 7f93fa7fc640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93e806af40 0x7f93e806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.862+0000 7f93fa7fc640 1 -- 172.21.15.16:0/1647591124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94000896f0 con 0x7f93fc0e3080 2023-11-24T06:27:03.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7f93fa7fc640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f93e8073160 0x7f93e8075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7f93fa7fc640 1 -- 172.21.15.16:0/1647591124 --> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93e8075c30 con 0x7f93e8073160 2023-11-24T06:27:03.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7f93fa7fc640 1 -- 172.21.15.16:0/1647591124 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f940008bd10 con 0x7f93fc0e3080 2023-11-24T06:27:03.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7fddf8656640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fddf00cf9a0 0x7fddf0070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7f68b1519640 1 --2- 172.21.15.16:0/3579121847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68ac070ea0 0x7f68ac0cc160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68940097a0 tx=0x7f689402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317d70c9fefaae74 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7ff9b9a2b640 1 -- 172.21.15.16:0/706691665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b4160f80 msgr2=0x7ff9b4161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7ff9b9a2b640 1 --2- 172.21.15.16:0/706691665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b4160f80 0x7ff9b4161360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009870 tx=0x7ff9a402b980 comp rx=0 tx=0).stop 2023-11-24T06:27:03.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7f9409edf640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93e806af40 0x7f93e806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7f68ab7fe640 1 -- 172.21.15.16:0/3579121847 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894031830 con 0x7f68ac070ea0 2023-11-24T06:27:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7f68ab7fe640 1 -- 172.21.15.16:0/3579121847 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6894031990 con 0x7f68ac070ea0 2023-11-24T06:27:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7ff9b9a2b640 1 -- 172.21.15.16:0/706691665 shutdown_connections 2023-11-24T06:27:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7ff9b9a2b640 1 --2- 172.21.15.16:0/706691665 >> v2:172.21.15.16:3300/0 conn(0x7ff9b40ced50 0x7ff9b40d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7ff9b9a2b640 1 --2- 172.21.15.16:0/706691665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b4160f80 0x7ff9b4161360 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7ff9b9a2b640 1 -- 172.21.15.16:0/706691665 >> 172.21.15.16:0/706691665 conn(0x7ff9b409c550 msgr2=0x7ff9b40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7f68ab7fe640 1 -- 172.21.15.16:0/3579121847 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894031d20 con 0x7f68ac070ea0 2023-11-24T06:27:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7fddf8656640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf0071690 con 0x7fddf0070ea0 2023-11-24T06:27:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7fddf8656640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf0071970 con 0x7fddf014b2c0 2023-11-24T06:27:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7fddf8656640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf014b6a0 con 0x7fddf00cf9a0 2023-11-24T06:27:03.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7f78256b4640 1 -- start start 2023-11-24T06:27:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7fddf5bf1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fddf00cf9a0 0x7fddf0070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7fddf5bf1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fddf00cf9a0 0x7fddf0070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53098/0 (socket says 172.21.15.16:53098) 2023-11-24T06:27:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7fddf5bf1640 1 -- 172.21.15.16:0/1492885658 learned_addr learned my addr 172.21.15.16:0/1492885658 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7f94096de640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f93e8073160 0x7f93e8075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.863+0000 7f45baffd640 1 -- 172.21.15.16:0/4225146360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45b4012c60 con 0x7f45c415d560 2023-11-24T06:27:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7f45baffd640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f45a806ae40 0x7f45a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.865+0000 7f45baffd640 1 -- 172.21.15.16:0/4225146360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45b408a580 con 0x7f45c415d560 2023-11-24T06:27:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.865+0000 7fddf5bf1640 1 -- 172.21.15.16:0/1492885658 >> v1:172.21.15.173:6789/0 conn(0x7fddf014b2c0 legacy=0x7fddf00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.865+0000 7fddf5bf1640 1 -- 172.21.15.16:0/1492885658 >> v1:172.21.15.16:6789/0 conn(0x7fddf0070ea0 legacy=0x7fddf00cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.865+0000 7fddf5bf1640 1 -- 172.21.15.16:0/1492885658 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf0071c50 con 0x7fddf00cf9a0 2023-11-24T06:27:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.865+0000 7f68b2f7c640 1 -- 172.21.15.16:0/3579121847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68ac070ea0 msgr2=0x7f68ac0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.865+0000 7f68b2f7c640 1 --2- 172.21.15.16:0/3579121847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68ac070ea0 0x7f68ac0cc160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f68940097a0 tx=0x7f689402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.865+0000 7f45baffd640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f45a8073060 0x7f45a8075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.865+0000 7f45baffd640 1 -- 172.21.15.16:0/4225146360 --> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45a8075b30 con 0x7f45a8073060 2023-11-24T06:27:03.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.865+0000 7f45baffd640 1 -- 172.21.15.16:0/4225146360 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45b408a930 con 0x7f45c415d560 2023-11-24T06:27:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.864+0000 7ff9b9a2b640 1 -- 172.21.15.16:0/706691665 shutdown_connections 2023-11-24T06:27:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7f45ca41f640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f45a806ae40 0x7f45a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7f45c9c1e640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f45a8073060 0x7f45a8075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7f68b2f7c640 1 -- 172.21.15.16:0/3579121847 shutdown_connections 2023-11-24T06:27:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7ff9b9a2b640 1 -- 172.21.15.16:0/706691665 wait complete. 2023-11-24T06:27:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7f68b2f7c640 1 --2- 172.21.15.16:0/3579121847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68ac070ea0 0x7f68ac0cc160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7f68b2f7c640 1 --2- 172.21.15.16:0/3579121847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68ac0cf9a0 0x7f68ac070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7f68b2f7c640 1 -- 172.21.15.16:0/3579121847 >> 172.21.15.16:0/3579121847 conn(0x7f68ac06a800 msgr2=0x7f68ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7f68b2f7c640 1 -- 172.21.15.16:0/3579121847 shutdown_connections 2023-11-24T06:27:03.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7f68b2f7c640 1 -- 172.21.15.16:0/3579121847 wait complete. 2023-11-24T06:27:03.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.866+0000 7f45c9c1e640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f45a8073060 0x7f45a8075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.867+0000 7fddf5bf1640 1 --2- 172.21.15.16:0/1492885658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddf00cf9a0 0x7fddf0070960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009870 tx=0x7fdde002b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=554565c1a922cc7f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.867+0000 7f94096de640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f93e8073160 0x7f93e8075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.867+0000 7fddf53f0640 1 -- 172.21.15.16:0/1492885658 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0030880 con 0x7fddf00cf9a0 2023-11-24T06:27:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.867+0000 7f45ca41f640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f45a806ae40 0x7f45a806d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45c0007fd0 tx=0x7f45c0007de0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.867+0000 7f45baffd640 1 -- 172.21.15.16:0/4225146360 <== osd.1 v2:172.21.15.16:6810/3842150677 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f45a8075b30 con 0x7f45a8073060 2023-11-24T06:27:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.868+0000 7f78256b4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78200ce8f0 0x7f7820157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.868+0000 7f78256b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78200cfbe0 0x7f78200c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.868+0000 7f78256b4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7820160f80 0x7f78200c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.868+0000 7f78256b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200c9640 con 0x7f78200cfbe0 2023-11-24T06:27:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.868+0000 7f78256b4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200c8e10 con 0x7f78200ce8f0 2023-11-24T06:27:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.868+0000 7f78256b4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78200c9060 con 0x7f7820160f80 2023-11-24T06:27:03.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.868+0000 7ff9b9a2b640 1 Processor -- start 2023-11-24T06:27:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.868+0000 7f9409edf640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93e806af40 0x7f93e806d400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9404079ad0 tx=0x7f940406d030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7ff9b9a2b640 1 -- start start 2023-11-24T06:27:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7f78167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78200cfbe0 0x7f78200c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7f78167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78200cfbe0 0x7f78200c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38176/0 (socket says 172.21.15.16:38176) 2023-11-24T06:27:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7f78167fc640 1 -- 172.21.15.16:0/2241781157 learned_addr learned my addr 172.21.15.16:0/2241781157 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7f78167fc640 1 -- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7820160f80 msgr2=0x7f78200c88d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:27:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7f78167fc640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7820160f80 0x7f78200c88d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7f78167fc640 1 -- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78200ce8f0 msgr2=0x7f7820157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7fddf53f0640 1 -- 172.21.15.16:0/1492885658 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde00309e0 con 0x7fddf00cf9a0 2023-11-24T06:27:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7fddf53f0640 1 -- 172.21.15.16:0/1492885658 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0030880 con 0x7fddf00cf9a0 2023-11-24T06:27:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.869+0000 7f781effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78200ce8f0 0x7f7820157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7f78167fc640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78200ce8f0 0x7f7820157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7f93fa7fc640 1 -- 172.21.15.16:0/1647591124 <== osd.3 v2:172.21.15.16:6822/3227335215 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f93e8075c30 con 0x7f93e8073160 2023-11-24T06:27:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7f68b2f7c640 1 Processor -- start 2023-11-24T06:27:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7f78167fc640 1 -- 172.21.15.16:0/2241781157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7810009400 con 0x7f78200cfbe0 2023-11-24T06:27:03.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7f781effd640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78200ce8f0 0x7f7820157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:03.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7fddf8656640 1 -- 172.21.15.16:0/1492885658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddf00cf9a0 msgr2=0x7fddf0070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7fddf8656640 1 --2- 172.21.15.16:0/1492885658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddf00cf9a0 0x7fddf0070960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdde0009870 tx=0x7fdde002b9c0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7f78167fc640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78200cfbe0 0x7f78200c8390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f780802c570 tx=0x7f7808004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7f68b2f7c640 1 -- start start 2023-11-24T06:27:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7ff9b9a2b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b40ce8f0 0x7ff9b414f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7ff9b9a2b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b40ced50 0x7ff9b40c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.870+0000 7ff9b9a2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b4160f80 0x7ff9b41f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7fddf8656640 1 -- 172.21.15.16:0/1492885658 shutdown_connections 2023-11-24T06:27:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7fddf8656640 1 --2- 172.21.15.16:0/1492885658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddf00cf9a0 0x7fddf0070960 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7fddf8656640 1 -- 172.21.15.16:0/1492885658 >> 172.21.15.16:0/1492885658 conn(0x7fddf006a800 msgr2=0x7fddf006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7ff9b9a2b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40c8d40 con 0x7ff9b4160f80 2023-11-24T06:27:03.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7ff9b9a2b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40c8130 con 0x7ff9b40ce8f0 2023-11-24T06:27:03.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7f7817fff640 1 -- 172.21.15.16:0/2241781157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808004040 con 0x7f78200cfbe0 2023-11-24T06:27:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7f7817fff640 1 -- 172.21.15.16:0/2241781157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78080041a0 con 0x7f78200cfbe0 2023-11-24T06:27:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7f7817fff640 1 -- 172.21.15.16:0/2241781157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808004510 con 0x7f78200cfbe0 2023-11-24T06:27:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7f78256b4640 1 -- 172.21.15.16:0/2241781157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7808009b30 con 0x7f78200cfbe0 2023-11-24T06:27:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7ff9b9a2b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40c8410 con 0x7ff9b40ced50 2023-11-24T06:27:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7ff9b2ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b40ce8f0 0x7ff9b414f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7ff9b27fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b40ced50 0x7ff9b40c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7ff9b2ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b40ce8f0 0x7ff9b414f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53120/0 (socket says 172.21.15.16:53120) 2023-11-24T06:27:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7ff9b2ffd640 1 -- 172.21.15.16:0/3010772726 learned_addr learned my addr 172.21.15.16:0/3010772726 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7fddf8656640 1 -- 172.21.15.16:0/1492885658 shutdown_connections 2023-11-24T06:27:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7ff9b37fe640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b4160f80 0x7ff9b41f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.872+0000 7f68b2f7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac070ea0 0x7f68ac161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.872+0000 7ff9b37fe640 1 -- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b40ced50 msgr2=0x7ff9b40c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f68b2f7c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68ac0cf9a0 0x7f68ac162030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f68b2f7c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68ac14b2c0 0x7f68ac147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f68b2f7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac164770 con 0x7f68ac070ea0 2023-11-24T06:27:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f68b2f7c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac143f80 con 0x7f68ac14b2c0 2023-11-24T06:27:03.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f68b2f7c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac162730 con 0x7f68ac0cf9a0 2023-11-24T06:27:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.872+0000 7ff9b37fe640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9b40ced50 0x7ff9b40c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.872+0000 7ff9b37fe640 1 -- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b40ce8f0 msgr2=0x7ff9b414f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.872+0000 7ff9b37fe640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9b40ce8f0 0x7ff9b414f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7ff9b37fe640 1 -- 172.21.15.16:0/3010772726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a40094d0 con 0x7ff9b4160f80 2023-11-24T06:27:03.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f68b0d18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac070ea0 0x7f68ac161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.874+0000 7ff9b37fe640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b4160f80 0x7ff9b41f7700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00efc0 tx=0x7ff9ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f68b0d18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac070ea0 0x7f68ac161320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38192/0 (socket says 172.21.15.16:38192) 2023-11-24T06:27:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f68b0d18640 1 -- 172.21.15.16:0/1812659786 learned_addr learned my addr 172.21.15.16:0/1812659786 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f68abfff640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68ac0cf9a0 0x7f68ac162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.874+0000 7f68b1519640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68ac14b2c0 0x7f68ac147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7fddf8656640 1 -- 172.21.15.16:0/1492885658 wait complete. 2023-11-24T06:27:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.874+0000 7fddf8656640 1 Processor -- start 2023-11-24T06:27:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.871+0000 7f78256b4640 1 -- 172.21.15.16:0/2241781157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7820160160 con 0x7f78200cfbe0 2023-11-24T06:27:03.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.873+0000 7f78256b4640 1 -- 172.21.15.16:0/2241781157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78200d1c40 con 0x7f78200cfbe0 2023-11-24T06:27:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7ff99bfff640 1 -- 172.21.15.16:0/3010772726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac014480 con 0x7ff9b4160f80 2023-11-24T06:27:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7fddf8656640 1 -- start start 2023-11-24T06:27:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7f68b0d18640 1 -- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68ac0cf9a0 msgr2=0x7f68ac162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7f68b0d18640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68ac0cf9a0 0x7f68ac162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7f68b0d18640 1 -- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68ac14b2c0 msgr2=0x7f68ac147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7f68b0d18640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68ac14b2c0 0x7f68ac147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7f68b0d18640 1 -- 172.21.15.16:0/1812659786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894009400 con 0x7f68ac070ea0 2023-11-24T06:27:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7f68b0d18640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac070ea0 0x7f68ac161320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c460 tx=0x7f68a400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.876+0000 7f68a97fa640 1 -- 172.21.15.16:0/1812659786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a40074e0 con 0x7f68ac070ea0 2023-11-24T06:27:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.876+0000 7f68a97fa640 1 -- 172.21.15.16:0/1812659786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a4007640 con 0x7f68ac070ea0 2023-11-24T06:27:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.876+0000 7f68a97fa640 1 -- 172.21.15.16:0/1812659786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a40074e0 con 0x7f68ac070ea0 2023-11-24T06:27:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.876+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac162a10 con 0x7f68ac070ea0 2023-11-24T06:27:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.876+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac164020 con 0x7f68ac070ea0 2023-11-24T06:27:03.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7ff99bfff640 1 -- 172.21.15.16:0/3010772726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac00f040 con 0x7ff9b4160f80 2023-11-24T06:27:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7ff99bfff640 1 -- 172.21.15.16:0/3010772726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac014800 con 0x7ff9b4160f80 2023-11-24T06:27:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7ff9b9a2b640 1 -- 172.21.15.16:0/3010772726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b40c86f0 con 0x7ff9b4160f80 2023-11-24T06:27:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7ff9b9a2b640 1 -- 172.21.15.16:0/3010772726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b414fa80 con 0x7ff9b4160f80 2023-11-24T06:27:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.877+0000 7fddf8656640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddf0070ea0 0x7fddf013e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.875+0000 7f7817fff640 1 -- 172.21.15.16:0/2241781157 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7808004670 con 0x7f78200cfbe0 2023-11-24T06:27:03.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.877+0000 7fddf8656640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddf00cf9a0 0x7fddf0148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.876+0000 7f7817fff640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f780006ae90 0x7f780006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.876+0000 7f7817fff640 1 -- 172.21.15.16:0/2241781157 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78080ac9d0 con 0x7f78200cfbe0 2023-11-24T06:27:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.877+0000 7fddf8656640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf014b2c0 0x7fddf0148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.877+0000 7f7817fff640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f78000730b0 0x7f78000754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.877+0000 7f7817fff640 1 -- 172.21.15.16:0/2241781157 --> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7800075b80 con 0x7f78000730b0 2023-11-24T06:27:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.877+0000 7f7817fff640 1 -- 172.21.15.16:0/2241781157 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78080acd40 con 0x7f78200cfbe0 2023-11-24T06:27:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.877+0000 7fddf63f2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddf0070ea0 0x7fddf013e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.878+0000 7fddf6bf3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf014b2c0 0x7fddf0148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.878+0000 7fddf8656640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf013ed90 con 0x7fddf014b2c0 2023-11-24T06:27:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.878+0000 7fddf5bf1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddf00cf9a0 0x7fddf0148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.878+0000 7fddf5bf1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddf00cf9a0 0x7fddf0148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:40944/0 (socket says 172.21.15.16:40944) 2023-11-24T06:27:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.878+0000 7fddf5bf1640 1 -- 172.21.15.16:0/75075339 learned_addr learned my addr 172.21.15.16:0/75075339 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:03.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.878+0000 7f781effd640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f780006ae90 0x7f780006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.878+0000 7f781f7fe640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f78000730b0 0x7f78000754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.879+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68ac071330 con 0x7f68ac070ea0 2023-11-24T06:27:03.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.876+0000 7ff9b9a2b640 1 -- 172.21.15.16:0/3010772726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9b406c750 con 0x7ff9b4160f80 2023-11-24T06:27:03.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.879+0000 7ff99bfff640 1 -- 172.21.15.16:0/3010772726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9ac014960 con 0x7ff9b4160f80 2023-11-24T06:27:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.879+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0203790 con 0x7fddf0070ea0 2023-11-24T06:27:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.879+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf02006a0 con 0x7fddf00cf9a0 2023-11-24T06:27:03.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.880+0000 7f781f7fe640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f78000730b0 0x7f78000754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.880+0000 7fddf5bf1640 1 -- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddf0070ea0 msgr2=0x7fddf013e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.880+0000 7fddf5bf1640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddf0070ea0 0x7fddf013e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.880+0000 7fddf5bf1640 1 -- 172.21.15.16:0/75075339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf014b2c0 msgr2=0x7fddf0148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.880+0000 7fddf5bf1640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf014b2c0 0x7fddf0148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.880+0000 7fddf5bf1640 1 -- 172.21.15.16:0/75075339 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00094d0 con 0x7fddf00cf9a0 2023-11-24T06:27:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.880+0000 7fddf6bf3640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf014b2c0 0x7fddf0148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.881+0000 7ff99bfff640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9a006af10 0x7ff9a006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.881+0000 7fddf5bf1640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddf00cf9a0 0x7fddf0148850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdde00090b0 tx=0x7fdde00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.881+0000 7ff99bfff640 1 -- 172.21.15.16:0/3010772726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9ac08a420 con 0x7ff9b4160f80 2023-11-24T06:27:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7fdde6ffd640 1 -- 172.21.15.16:0/75075339 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde003e070 con 0x7fddf00cf9a0 2023-11-24T06:27:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7fdde6ffd640 1 -- 172.21.15.16:0/75075339 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde00037d0 con 0x7fddf00cf9a0 2023-11-24T06:27:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf0202e60 con 0x7fddf00cf9a0 2023-11-24T06:27:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7ff99bfff640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7ff9a0073130 0x7ff9a0075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7ff99bfff640 1 -- 172.21.15.16:0/3010772726 --> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9a0075c00 con 0x7ff9a0073130 2023-11-24T06:27:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7ff99bfff640 1 -- 172.21.15.16:0/3010772726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9ac08ca40 con 0x7ff9b4160f80 2023-11-24T06:27:03.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7f7817fff640 1 -- 172.21.15.16:0/2241781157 <== osd.4 v2:172.21.15.173:6800/3033116473 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7800075b80 con 0x7f78000730b0 2023-11-24T06:27:03.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7ff9b27fc640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7ff9a0073130 0x7ff9a0075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7f68a97fa640 1 -- 172.21.15.16:0/1812659786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68a40047e0 con 0x7f68ac070ea0 2023-11-24T06:27:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7ff9b2ffd640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9a006af10 0x7ff9a006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.883+0000 7f781effd640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f780006ae90 0x7f780006d350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7810009c50 tx=0x7f7810008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.882+0000 7fdde6ffd640 1 -- 172.21.15.16:0/75075339 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0003b40 con 0x7fddf00cf9a0 2023-11-24T06:27:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.883+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf0203420 con 0x7fddf00cf9a0 2023-11-24T06:27:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.885+0000 7ff9b27fc640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7ff9a0073130 0x7ff9a0075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.885+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fddf00cbc40 con 0x7fddf00cf9a0 2023-11-24T06:27:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.885+0000 7ff9b2ffd640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9a006af10 0x7ff9a006d3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40b16d0 tx=0x7ff9a4008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.886+0000 7f68a97fa640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f688c06aea0 0x7f688c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.886+0000 7f68a97fa640 1 -- 172.21.15.16:0/1812659786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68a40883f0 con 0x7f68ac070ea0 2023-11-24T06:27:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.886+0000 7f68a97fa640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f688c0730c0 0x7f688c0754e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.886+0000 7f68a97fa640 1 -- 172.21.15.16:0/1812659786 --> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f688c075b90 con 0x7f688c0730c0 2023-11-24T06:27:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.886+0000 7f68a97fa640 1 -- 172.21.15.16:0/1812659786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a408aa10 con 0x7f68ac070ea0 2023-11-24T06:27:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.886+0000 7f68abfff640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f688c06aea0 0x7f688c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.887+0000 7f68b1519640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f688c0730c0 0x7f688c0754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.888+0000 7f68abfff640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f688c06aea0 0x7f688c06d360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f68ac130380 tx=0x7f689c00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.888+0000 7ff99bfff640 1 -- 172.21.15.16:0/3010772726 <== osd.6 v2:172.21.15.173:6808/2309591130 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff9a0075c00 con 0x7ff9a0073130 2023-11-24T06:27:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.888+0000 7fdde6ffd640 1 -- 172.21.15.16:0/75075339 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdde0042070 con 0x7fddf00cf9a0 2023-11-24T06:27:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.889+0000 7f68b1519640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f688c0730c0 0x7f688c0754e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.890+0000 7fdde6ffd640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fddd406ae40 0x7fddd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.890+0000 7fdde6ffd640 1 -- 172.21.15.16:0/75075339 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdde00ac290 con 0x7fddf00cf9a0 2023-11-24T06:27:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.891+0000 7fdde6ffd640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7fddd4073060 0x7fddd4075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.891+0000 7fdde6ffd640 1 -- 172.21.15.16:0/75075339 --> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fddd4075b30 con 0x7fddd4073060 2023-11-24T06:27:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.891+0000 7fdde6ffd640 1 -- 172.21.15.16:0/75075339 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdde00ac640 con 0x7fddf00cf9a0 2023-11-24T06:27:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.891+0000 7fddf63f2640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fddd406ae40 0x7fddd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.891+0000 7f68a97fa640 1 -- 172.21.15.16:0/1812659786 <== osd.5 v2:172.21.15.173:6816/2053607116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f688c075b90 con 0x7f688c0730c0 2023-11-24T06:27:03.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.892+0000 7fddf6bf3640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7fddd4073060 0x7fddd4075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:03.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.892+0000 7fddf63f2640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fddd406ae40 0x7fddd406d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdde8006fd0 tx=0x7fdde8009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.893+0000 7fddf6bf3640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7fddd4073060 0x7fddd4075480 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.895+0000 7fdde6ffd640 1 -- 172.21.15.16:0/75075339 <== osd.7 v2:172.21.15.173:6820/2972943157 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fddd4075b30 con 0x7fddd4073060 2023-11-24T06:27:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.896+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 --> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6674005360 con 0x7f6660073130 2023-11-24T06:27:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.897+0000 7f66797fa640 1 -- 172.21.15.16:0/1989800775 <== osd.0 v2:172.21.15.16:6800/4232398165 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6674005360 con 0x7f6660073130 2023-11-24T06:27:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.897+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 --> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f45c4074700 con 0x7f45a8073060 2023-11-24T06:27:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.898+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7f6660073130 msgr2=0x7f6660075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.898+0000 7f6681a4b640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7f6660073130 0x7f6660075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.898+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 --> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4f00067450 con 0x7f4ee40730b0 2023-11-24T06:27:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.898+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f666006af10 msgr2=0x7f666006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.898+0000 7f6681a4b640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f666006af10 0x7f666006d3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6670000f60 tx=0x7f6670009040 comp rx=0 tx=0).stop 2023-11-24T06:27:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.898+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66740e3130 msgr2=0x7f66740e3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.898+0000 7f6681a4b640 1 --2- 172.21.15.16:0/1989800775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66740e3130 0x7f66740e3690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f667c050020 tx=0x7f667c0a8070 comp rx=0 tx=0).stop 2023-11-24T06:27:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.898+0000 7f668124a640 1 -- 172.21.15.16:0/1989800775 reap_dead start 2023-11-24T06:27:03.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.899+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 --> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f93ec002cb0 con 0x7f93e8073160 2023-11-24T06:27:03.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.900+0000 7f4ef6ffd640 1 -- 172.21.15.16:0/3291244860 <== osd.2 v2:172.21.15.16:6808/620212774 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4f00067450 con 0x7f4ee40730b0 2023-11-24T06:27:03.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.900+0000 7f45baffd640 1 -- 172.21.15.16:0/4225146360 <== osd.1 v2:172.21.15.16:6810/3842150677 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f45c4074700 con 0x7f45a8073060 2023-11-24T06:27:03.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.900+0000 7f93fa7fc640 1 -- 172.21.15.16:0/1647591124 <== osd.3 v2:172.21.15.16:6822/3227335215 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f93ec002cb0 con 0x7f93e8073160 2023-11-24T06:27:03.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.900+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7f4ee40730b0 msgr2=0x7f4ee40754d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.900+0000 7f4f07221640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7f4ee40730b0 0x7f4ee40754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 shutdown_connections 2023-11-24T06:27:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ee406ae90 msgr2=0x7f4ee406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f4f07221640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ee406ae90 0x7f4ee406d350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f00158380 tx=0x7f4ef800d040 comp rx=0 tx=0).stop 2023-11-24T06:27:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f000b2f10 msgr2=0x7f4f0013e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f4f07221640 1 --2- 172.21.15.16:0/3291244860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f000b2f10 0x7f4f0013e250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d800 tx=0x7f4efc00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f93e8073160 msgr2=0x7f93e8075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f940c143640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f93e8073160 0x7f93e8075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f4f06a20640 1 -- 172.21.15.16:0/3291244860 reap_dead start 2023-11-24T06:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 shutdown_connections 2023-11-24T06:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 >> 172.21.15.16:0/3291244860 conn(0x7f4f000b0b00 msgr2=0x7f4f0015acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 >> 172.21.15.16:0/1989800775 conn(0x7f6674058eb0 msgr2=0x7f6674006a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 shutdown_connections 2023-11-24T06:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f6681a4b640 1 -- 172.21.15.16:0/1989800775 wait complete. 2023-11-24T06:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f45a8073060 msgr2=0x7f45a8075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f45cc683640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f45a8073060 0x7f45a8075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93e806af40 msgr2=0x7f93e806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f940c143640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93e806af40 0x7f93e806d400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9404079ad0 tx=0x7f940406d030 comp rx=0 tx=0).stop 2023-11-24T06:27:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc0e3080 msgr2=0x7f93fc052020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.901+0000 7f940c143640 1 --2- 172.21.15.16:0/1647591124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc0e3080 0x7f93fc052020 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f940000e830 tx=0x7f940000ecf0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f45a806ae40 msgr2=0x7f45a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f45cc683640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f45a806ae40 0x7f45a806d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f45c0007fd0 tx=0x7f45c0007de0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c415d560 msgr2=0x7f45c40b7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f45cc683640 1 --2- 172.21.15.16:0/4225146360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c415d560 0x7f45c40b7580 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45b400d5d0 tx=0x7f45b400da90 comp rx=0 tx=0).stop 2023-11-24T06:27:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f45cac20640 1 -- 172.21.15.16:0/4225146360 reap_dead start 2023-11-24T06:27:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 shutdown_connections 2023-11-24T06:27:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 >> 172.21.15.16:0/4225146360 conn(0x7f45c40d3020 msgr2=0x7f45c415c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 shutdown_connections 2023-11-24T06:27:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f940a6e0640 1 -- 172.21.15.16:0/1647591124 reap_dead start 2023-11-24T06:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 shutdown_connections 2023-11-24T06:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f4f07221640 1 -- 172.21.15.16:0/3291244860 wait complete. 2023-11-24T06:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f45cc683640 1 -- 172.21.15.16:0/4225146360 wait complete. 2023-11-24T06:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 shutdown_connections 2023-11-24T06:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 >> 172.21.15.16:0/1647591124 conn(0x7f93fc0559a0 msgr2=0x7f93fc039320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 shutdown_connections 2023-11-24T06:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.902+0000 7f940c143640 1 -- 172.21.15.16:0/1647591124 wait complete. 2023-11-24T06:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.915+0000 7f78256b4640 1 -- 172.21.15.16:0/2241781157 --> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f78200d1960 con 0x7f78000730b0 2023-11-24T06:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.917+0000 7f7817fff640 1 -- 172.21.15.16:0/2241781157 <== osd.4 v2:172.21.15.173:6800/3033116473 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7818007df0 con 0x7f78000730b0 2023-11-24T06:27:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.918+0000 7f78157fa640 1 -- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f78000730b0 msgr2=0x7f78000754d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.918+0000 7f78157fa640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f78000730b0 0x7f78000754d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.918+0000 7f78157fa640 1 -- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f780006ae90 msgr2=0x7f780006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.918+0000 7f78157fa640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f780006ae90 0x7f780006d350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7810009c50 tx=0x7f7810008040 comp rx=0 tx=0).stop 2023-11-24T06:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.918+0000 7f78157fa640 1 -- 172.21.15.16:0/2241781157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78200cfbe0 msgr2=0x7f78200c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.918+0000 7f78157fa640 1 --2- 172.21.15.16:0/2241781157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78200cfbe0 0x7f78200c8390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f780802c570 tx=0x7f7808004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.919+0000 7f781f7fe640 1 -- 172.21.15.16:0/2241781157 reap_dead start 2023-11-24T06:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.919+0000 7f78157fa640 1 -- 172.21.15.16:0/2241781157 shutdown_connections 2023-11-24T06:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.919+0000 7f78157fa640 1 -- 172.21.15.16:0/2241781157 >> 172.21.15.16:0/2241781157 conn(0x7f782009c550 msgr2=0x7f78200cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.920+0000 7f78157fa640 1 -- 172.21.15.16:0/2241781157 shutdown_connections 2023-11-24T06:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.920+0000 7f78157fa640 1 -- 172.21.15.16:0/2241781157 wait complete. 2023-11-24T06:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.924+0000 7ff9b9a2b640 1 -- 172.21.15.16:0/3010772726 --> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9b40d1960 con 0x7ff9a0073130 2023-11-24T06:27:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.928+0000 7ff99bfff640 1 -- 172.21.15.16:0/3010772726 <== osd.6 v2:172.21.15.173:6808/2309591130 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff9a80012c0 con 0x7ff9a0073130 2023-11-24T06:27:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.925+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 --> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f68ac1628a0 con 0x7f688c0730c0 2023-11-24T06:27:03.948 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2023-11-24T06:27:03.949 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2023-11-24T06:27:03.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:27:03.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:27:03.951 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2023-11-24T06:27:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.929+0000 7ff999ffb640 1 -- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7ff9a0073130 msgr2=0x7ff9a0075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.929+0000 7ff999ffb640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7ff9a0073130 0x7ff9a0075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.929+0000 7ff999ffb640 1 -- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9a006af10 msgr2=0x7ff9a006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.929+0000 7ff999ffb640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9a006af10 0x7ff9a006d3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40b16d0 tx=0x7ff9a4008030 comp rx=0 tx=0).stop 2023-11-24T06:27:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.929+0000 7ff999ffb640 1 -- 172.21.15.16:0/3010772726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b4160f80 msgr2=0x7ff9b41f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.929+0000 7ff999ffb640 1 --2- 172.21.15.16:0/3010772726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b4160f80 0x7ff9b41f7700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00efc0 tx=0x7ff9ac00c490 comp rx=0 tx=0).stop 2023-11-24T06:27:03.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:27:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.929+0000 7f68a97fa640 1 -- 172.21.15.16:0/1812659786 <== osd.5 v2:172.21.15.173:6816/2053607116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f68ac1628a0 con 0x7f688c0730c0 2023-11-24T06:27:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.929+0000 7ff9b37fe640 1 -- 172.21.15.16:0/3010772726 reap_dead start 2023-11-24T06:27:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f688c0730c0 msgr2=0x7f688c0754e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7f68b2f7c640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f688c0730c0 0x7f688c0754e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7ff999ffb640 1 -- 172.21.15.16:0/3010772726 shutdown_connections 2023-11-24T06:27:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7ff999ffb640 1 -- 172.21.15.16:0/3010772726 >> 172.21.15.16:0/3010772726 conn(0x7ff9b409c550 msgr2=0x7ff9b40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7ff999ffb640 1 -- 172.21.15.16:0/3010772726 shutdown_connections 2023-11-24T06:27:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7ff999ffb640 1 -- 172.21.15.16:0/3010772726 wait complete. 2023-11-24T06:27:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f688c06aea0 msgr2=0x7f688c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7f68b2f7c640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f688c06aea0 0x7f688c06d360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f68ac130380 tx=0x7f689c00b040 comp rx=0 tx=0).stop 2023-11-24T06:27:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac070ea0 msgr2=0x7f68ac161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7f68b2f7c640 1 --2- 172.21.15.16:0/1812659786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68ac070ea0 0x7f68ac161320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c460 tx=0x7f68a400c920 comp rx=0 tx=0).stop 2023-11-24T06:27:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7f68b1519640 1 -- 172.21.15.16:0/1812659786 reap_dead start 2023-11-24T06:27:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 shutdown_connections 2023-11-24T06:27:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.930+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 >> 172.21.15.16:0/1812659786 conn(0x7f68ac06a800 msgr2=0x7f68ac149bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.956 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-11-24T06:27:03.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:27:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.931+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 shutdown_connections 2023-11-24T06:27:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.931+0000 7f68b2f7c640 1 -- 172.21.15.16:0/1812659786 wait complete. 2023-11-24T06:27:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.933+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 --> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fddf00675d0 con 0x7fddd4073060 2023-11-24T06:27:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.935+0000 7fdde6ffd640 1 -- 172.21.15.16:0/75075339 <== osd.7 v2:172.21.15.173:6820/2972943157 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fddf00675d0 con 0x7fddd4073060 2023-11-24T06:27:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.935+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7fddd4073060 msgr2=0x7fddd4075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.935+0000 7fddf8656640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7fddd4073060 0x7fddd4075480 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.935+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fddd406ae40 msgr2=0x7fddd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.935+0000 7fddf8656640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fddd406ae40 0x7fddd406d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdde8006fd0 tx=0x7fdde8009060 comp rx=0 tx=0).stop 2023-11-24T06:27:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.935+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddf00cf9a0 msgr2=0x7fddf0148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.935+0000 7fddf8656640 1 --2- 172.21.15.16:0/75075339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddf00cf9a0 0x7fddf0148850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdde00090b0 tx=0x7fdde00046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.936+0000 7fddf6bf3640 1 -- 172.21.15.16:0/75075339 reap_dead start 2023-11-24T06:27:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.936+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 shutdown_connections 2023-11-24T06:27:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.936+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 >> 172.21.15.16:0/75075339 conn(0x7fddf006a800 msgr2=0x7fddf006cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.936+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 shutdown_connections 2023-11-24T06:27:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:03.936+0000 7fddf8656640 1 -- 172.21.15.16:0/75075339 wait complete. 2023-11-24T06:27:03.960 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-11-24T06:27:03.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:27:03.961 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-11-24T06:27:03.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:27:03.962 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2023-11-24T06:27:03.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:27:03.963 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2023-11-24T06:27:03.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:27:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.301+0000 7fd92f631640 1 Processor -- start 2023-11-24T06:27:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.301+0000 7fd92f631640 1 -- start start 2023-11-24T06:27:04.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.303+0000 7efea19cb640 1 Processor -- start 2023-11-24T06:27:04.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.303+0000 7efea19cb640 1 -- start start 2023-11-24T06:27:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.303+0000 7fd92f631640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd92806c340 0x7fd92806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.303+0000 7fd92f631640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92806d0a0 con 0x7fd9280b3eb0 2023-11-24T06:27:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.303+0000 7fd92f631640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92806d380 con 0x7fd9280b0530 2023-11-24T06:27:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.303+0000 7fd92f631640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd928160040 con 0x7fd92806c340 2023-11-24T06:27:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.304+0000 7fd92dbce640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9280b3eb0 0x7fd9280cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52798/0 (socket says 172.21.15.16:52798) 2023-11-24T06:27:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.304+0000 7fd92dbce640 1 -- 172.21.15.16:0/1691691381 learned_addr learned my addr 172.21.15.16:0/1691691381 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.304+0000 7fd92cbcc640 1 --2- 172.21.15.16:0/1691691381 >> v2:172.21.15.173:3300/0 conn(0x7fd92806c340 0x7fd92806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.305+0000 7fd91ffff640 1 -- 172.21.15.16:0/1691691381 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125581405 0 0) 0x7fd92806d0a0 con 0x7fd9280b3eb0 2023-11-24T06:27:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.305+0000 7fd91ffff640 1 -- 172.21.15.16:0/1691691381 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd914003520 con 0x7fd9280b3eb0 2023-11-24T06:27:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7fd92cbcc640 1 -- 172.21.15.16:0/1691691381 >> v1:172.21.15.173:6789/0 conn(0x7fd9280b0530 legacy=0x7fd92815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7fd92cbcc640 1 -- 172.21.15.16:0/1691691381 >> v1:172.21.15.16:6789/0 conn(0x7fd9280b3eb0 legacy=0x7fd9280cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7fd92cbcc640 1 -- 172.21.15.16:0/1691691381 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92806d660 con 0x7fd92806c340 2023-11-24T06:27:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7efea19cb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7efe940078b0 0x7efe940edd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7efe9bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efe9403e810 0x7efe94053bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52804/0 (socket says 172.21.15.16:52804) 2023-11-24T06:27:04.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.304+0000 7f93ebfff640 1 Processor -- start 2023-11-24T06:27:04.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.304+0000 7f93ebfff640 1 -- start start 2023-11-24T06:27:04.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7f93ebfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93e40eb030 0x7f93e40552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7efe9bfff640 1 -- 172.21.15.16:0/1694456382 learned_addr learned my addr 172.21.15.16:0/1694456382 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:04.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7f93ebfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4006170 0x7f93e4006600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7f93ebfff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e4057e00 con 0x7f93e40eb410 2023-11-24T06:27:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7f93ebfff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e40ebfd0 con 0x7f93e40eb030 2023-11-24T06:27:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.306+0000 7f93ebfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4057970 con 0x7f93e4006170 2023-11-24T06:27:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.307+0000 7efea19cb640 1 -- 172.21.15.16:0/1694456382 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe94058540 con 0x7efe9403e810 2023-11-24T06:27:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.307+0000 7efea19cb640 1 -- 172.21.15.16:0/1694456382 --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe94058820 con 0x7efe940e3090 2023-11-24T06:27:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.307+0000 7efea19cb640 1 -- 172.21.15.16:0/1694456382 --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe94006170 con 0x7efe940078b0 2023-11-24T06:27:04.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.307+0000 7f93eaffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93e40eb030 0x7f93e40552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251687050 0 0) 0x7efe94058540 con 0x7efe9403e810 2023-11-24T06:27:04.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7f93ea7fc640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f93e40eb410 0x7f93e40eb8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:44470/0 (socket says 172.21.15.16:44470) 2023-11-24T06:27:04.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7f93ea7fc640 1 -- 172.21.15.16:0/2873363359 learned_addr learned my addr 172.21.15.16:0/2873363359 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe8c003520 con 0x7efe9403e810 2023-11-24T06:27:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7f93eb7fe640 1 --2- 172.21.15.16:0/2873363359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4006170 0x7f93e4006600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7f93eaffd640 1 -- 172.21.15.16:0/2873363359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4006170 msgr2=0x7f93e4006600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7f93eaffd640 1 --2- 172.21.15.16:0/2873363359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4006170 0x7f93e4006600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7f93eaffd640 1 -- 172.21.15.16:0/2873363359 >> v1:172.21.15.173:6789/0 conn(0x7f93e40eb410 legacy=0x7f93e40eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7f93eaffd640 1 -- 172.21.15.16:0/2873363359 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40387b0 con 0x7f93e40eb030 2023-11-24T06:27:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2454350608 0 0) 0x7efe8c003520 con 0x7efe9403e810 2023-11-24T06:27:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 >> v2:172.21.15.173:3300/0 conn(0x7efe940078b0 msgr2=0x7efe940edd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 --2- 172.21.15.16:0/1694456382 >> v2:172.21.15.173:3300/0 conn(0x7efe940078b0 0x7efe940edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 >> v1:172.21.15.173:6789/0 conn(0x7efe940e3090 legacy=0x7efe94057d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94058b00 con 0x7efe9403e810 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7efe9c04f090 con 0x7efe9403e810 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 >> v1:172.21.15.16:6789/0 conn(0x7efe9403e810 legacy=0x7efe94053bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c003970 0x7efe8c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c0047c0 0x7efe8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe8c005180 0x7efe8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe94006170 con 0x7efe8c003970 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c00c080 con 0x7efe8c005180 2023-11-24T06:27:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.308+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c00c240 con 0x7efe8c0047c0 2023-11-24T06:27:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.309+0000 7fd92cbcc640 1 --2- 172.21.15.16:0/1691691381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92806c340 0x7fd92806c7c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9240097a0 tx=0x7fd92402b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c58436223fee66eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.309+0000 7fd91ffff640 1 -- 172.21.15.16:0/1691691381 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92402f800 con 0x7fd92806c340 2023-11-24T06:27:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7fd91ffff640 1 -- 172.21.15.16:0/1691691381 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92402f960 con 0x7fd92806c340 2023-11-24T06:27:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7fd92f631640 1 -- 172.21.15.16:0/1691691381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92806c340 msgr2=0x7fd92806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7f93eaffd640 1 --2- 172.21.15.16:0/2873363359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e40eb030 0x7f93e40552d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0097a0 tx=0x7f93dc02b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcafc3847bee487 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7fd91ffff640 1 -- 172.21.15.16:0/1691691381 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92402fcd0 con 0x7fd92806c340 2023-11-24T06:27:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7fd92f631640 1 --2- 172.21.15.16:0/1691691381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92806c340 0x7fd92806c7c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9240097a0 tx=0x7fd92402b9e0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7f93e9ffb640 1 -- 172.21.15.16:0/2873363359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc030710 con 0x7f93e40eb030 2023-11-24T06:27:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7f93e9ffb640 1 -- 172.21.15.16:0/2873363359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc02e050 con 0x7f93e40eb030 2023-11-24T06:27:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7f93e9ffb640 1 -- 172.21.15.16:0/2873363359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc030710 con 0x7f93e40eb030 2023-11-24T06:27:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7fd92f631640 1 -- 172.21.15.16:0/1691691381 shutdown_connections 2023-11-24T06:27:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7fd92f631640 1 --2- 172.21.15.16:0/1691691381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92806c340 0x7fd92806c7c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7fd92f631640 1 -- 172.21.15.16:0/1691691381 >> 172.21.15.16:0/1691691381 conn(0x7fd9280aea00 msgr2=0x7fd9280aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7fd92f631640 1 -- 172.21.15.16:0/1691691381 shutdown_connections 2023-11-24T06:27:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.309+0000 7efea09c9640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c003970 0x7efe8c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7efe9bfff640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c0047c0 0x7efe8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7efea09c9640 1 -- 172.21.15.16:0/1694456382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c0047c0 msgr2=0x7efe8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7efea09c9640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c0047c0 0x7efe8c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7efea09c9640 1 -- 172.21.15.16:0/1694456382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe8c005180 msgr2=0x7efe8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7efea09c9640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe8c005180 0x7efe8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.310+0000 7efea09c9640 1 -- 172.21.15.16:0/1694456382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94058b00 con 0x7efe8c003970 2023-11-24T06:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7f93ebfff640 1 -- 172.21.15.16:0/2873363359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e40eb030 msgr2=0x7f93e40552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.311+0000 7f93ebfff640 1 --2- 172.21.15.16:0/2873363359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e40eb030 0x7f93e40552d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0097a0 tx=0x7f93dc02b6d0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.312+0000 7efea11ca640 1 -- 172.21.15.16:0/1694456382 reap_dead start 2023-11-24T06:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.312+0000 7efea09c9640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c003970 0x7efe8c003d70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe90001670 tx=0x7efe8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc46cdefe7ff26c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.312+0000 7fd92f631640 1 -- 172.21.15.16:0/1691691381 wait complete. 2023-11-24T06:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.312+0000 7f93ebfff640 1 -- 172.21.15.16:0/2873363359 shutdown_connections 2023-11-24T06:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.312+0000 7f93ebfff640 1 --2- 172.21.15.16:0/2873363359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e4006170 0x7f93e4006600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.312+0000 7f93ebfff640 1 --2- 172.21.15.16:0/2873363359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e40eb030 0x7f93e40552d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.312+0000 7f93ebfff640 1 -- 172.21.15.16:0/2873363359 >> 172.21.15.16:0/2873363359 conn(0x7f93e40559b0 msgr2=0x7f93e4055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.313+0000 7f93ebfff640 1 -- 172.21.15.16:0/2873363359 shutdown_connections 2023-11-24T06:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.313+0000 7f93ebfff640 1 -- 172.21.15.16:0/2873363359 wait complete. 2023-11-24T06:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.313+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90005270 con 0x7efe8c003970 2023-11-24T06:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.313+0000 7efe9b7fe640 1 -- 172.21.15.16:0/1694456382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe900053f0 con 0x7efe8c003970 2023-11-24T06:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.314+0000 7efea19cb640 1 -- 172.21.15.16:0/1694456382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c003970 msgr2=0x7efe8c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.314+0000 7efea19cb640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c003970 0x7efe8c003d70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe90001670 tx=0x7efe8c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.314+0000 7efea19cb640 1 -- 172.21.15.16:0/1694456382 shutdown_connections 2023-11-24T06:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.314+0000 7efea19cb640 1 --2- 172.21.15.16:0/1694456382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe8c003970 0x7efe8c003d70 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe90001670 tx=0x7efe8c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.314+0000 7efea19cb640 1 -- 172.21.15.16:0/1694456382 >> 172.21.15.16:0/1694456382 conn(0x7efe940559b0 msgr2=0x7efe94055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.315+0000 7fd92f631640 1 Processor -- start 2023-11-24T06:27:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.315+0000 7fd92f631640 1 -- start start 2023-11-24T06:27:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.315+0000 7f93ebfff640 1 Processor -- start 2023-11-24T06:27:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.316+0000 7efea19cb640 1 -- 172.21.15.16:0/1694456382 shutdown_connections 2023-11-24T06:27:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.316+0000 7f93ebfff640 1 -- start start 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.316+0000 7fd92f631640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92806c340 0x7fd928138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.316+0000 7fd92f631640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9280b0530 0x7fd9280ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.316+0000 7fd92f631640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9280b3eb0 0x7fd9281409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.316+0000 7fd92f631640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92815fa90 con 0x7fd92806c340 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.316+0000 7fd92f631640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928070100 con 0x7fd9280b0530 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.316+0000 7fd92f631640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280703e0 con 0x7fd9280b3eb0 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7fd92dbce640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9280b3eb0 0x7fd9281409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7fd92cbcc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9280b0530 0x7fd9280ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7fd92cbcc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9280b0530 0x7fd9280ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53168/0 (socket says 172.21.15.16:53168) 2023-11-24T06:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7fd92cbcc640 1 -- 172.21.15.16:0/466087929 learned_addr learned my addr 172.21.15.16:0/466087929 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7f93ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e4006170 0x7f93e40dcf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7f93ebfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e40eb030 0x7f93e40dd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7f93ebfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e40eb410 0x7f93e40df290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7f93ebfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40dee70 con 0x7f93e4006170 2023-11-24T06:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7f93ebfff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40de1f0 con 0x7f93e40eb410 2023-11-24T06:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.317+0000 7f93ebfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40de4d0 con 0x7f93e40eb030 2023-11-24T06:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.318+0000 7fd92d3cd640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92806c340 0x7fd928138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.316+0000 7efea19cb640 1 -- 172.21.15.16:0/1694456382 wait complete. 2023-11-24T06:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.318+0000 7efea19cb640 1 Processor -- start 2023-11-24T06:27:04.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.318+0000 7f93eb7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e40eb410 0x7f93e40df290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7f93eb7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e40eb410 0x7f93e40df290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53176/0 (socket says 172.21.15.16:53176) 2023-11-24T06:27:04.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7f93eaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e4006170 0x7f93e40dcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7f93eaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e4006170 0x7f93e40dcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38228/0 (socket says 172.21.15.16:38228) 2023-11-24T06:27:04.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7f93eaffd640 1 -- 172.21.15.16:0/3000119906 learned_addr learned my addr 172.21.15.16:0/3000119906 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7fd92cbcc640 1 -- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9280b3eb0 msgr2=0x7fd9281409a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7fd92cbcc640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9280b3eb0 0x7fd9281409a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7fd92cbcc640 1 -- 172.21.15.16:0/466087929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92806c340 msgr2=0x7fd928138ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7fd92cbcc640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92806c340 0x7fd928138ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7fd92cbcc640 1 -- 172.21.15.16:0/466087929 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd924009400 con 0x7fd9280b0530 2023-11-24T06:27:04.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7efea19cb640 1 -- start start 2023-11-24T06:27:04.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7f93ea7fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e40eb030 0x7f93e40dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.319+0000 7fd92dbce640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9280b3eb0 0x7fd9281409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:04.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.320+0000 7fd92cbcc640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9280b0530 0x7fd9280ce2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd9240098c0 tx=0x7fd9240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.320+0000 7fd91dffb640 1 -- 172.21.15.16:0/466087929 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92403e070 con 0x7fd9280b0530 2023-11-24T06:27:04.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.320+0000 7f93eaffd640 1 -- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e40eb030 msgr2=0x7f93e40dd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.320+0000 7f93eaffd640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e40eb030 0x7f93e40dd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.320+0000 7f93eaffd640 1 -- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e40eb410 msgr2=0x7f93e40df290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.320+0000 7f93eaffd640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e40eb410 0x7f93e40df290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.320+0000 7f93eaffd640 1 -- 172.21.15.16:0/3000119906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc009400 con 0x7f93e4006170 2023-11-24T06:27:04.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.321+0000 7f93eb7fe640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e40eb410 0x7f93e40df290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:04.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.321+0000 7fd91dffb640 1 -- 172.21.15.16:0/466087929 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd924006e10 con 0x7fd9280b0530 2023-11-24T06:27:04.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.321+0000 7fd91dffb640 1 -- 172.21.15.16:0/466087929 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924003a60 con 0x7fd9280b0530 2023-11-24T06:27:04.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.321+0000 7fd92f631640 1 -- 172.21.15.16:0/466087929 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9280706c0 con 0x7fd9280b0530 2023-11-24T06:27:04.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.321+0000 7fd92f631640 1 -- 172.21.15.16:0/466087929 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92815f5d0 con 0x7fd9280b0530 2023-11-24T06:27:04.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.321+0000 7f93eaffd640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e4006170 0x7f93e40dcf10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f93dc035040 tx=0x7f93dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.322+0000 7f93d3fff640 1 -- 172.21.15.16:0/3000119906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc030510 con 0x7f93e4006170 2023-11-24T06:27:04.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.322+0000 7f93d3fff640 1 -- 172.21.15.16:0/3000119906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc030670 con 0x7f93e4006170 2023-11-24T06:27:04.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.322+0000 7f93d3fff640 1 -- 172.21.15.16:0/3000119906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0309e0 con 0x7f93e4006170 2023-11-24T06:27:04.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.322+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e40de7b0 con 0x7f93e4006170 2023-11-24T06:27:04.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.322+0000 7efea19cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe940d59c0 0x7efe940d9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.323+0000 7efea19cb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe940d5da0 0x7efe940dea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.323+0000 7fd90b7fe640 1 -- 172.21.15.16:0/466087929 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd904005250 con 0x7fd9280b0530 2023-11-24T06:27:04.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.323+0000 7efea19cb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe940e17c0 0x7efe940e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.324+0000 7efea19cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940e1330 con 0x7efe940d59c0 2023-11-24T06:27:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.324+0000 7efea09c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe940d59c0 0x7efe940d9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.324+0000 7efea09c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe940d59c0 0x7efe940d9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38240/0 (socket says 172.21.15.16:38240) 2023-11-24T06:27:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.324+0000 7efea09c9640 1 -- 172.21.15.16:0/3626055543 learned_addr learned my addr 172.21.15.16:0/3626055543 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.324+0000 7efe9bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe940d5da0 0x7efe940dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.324+0000 7efea11ca640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe940e17c0 0x7efe940e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.325+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940e0720 con 0x7efe940e17c0 2023-11-24T06:27:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.325+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940e0a00 con 0x7efe940d5da0 2023-11-24T06:27:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.325+0000 7f807554c640 1 Processor -- start 2023-11-24T06:27:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.325+0000 7fde9f2bb640 1 Processor -- start 2023-11-24T06:27:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7f21b19b0640 1 Processor -- start 2023-11-24T06:27:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.324+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e4050ba0 con 0x7f93e4006170 2023-11-24T06:27:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.325+0000 7f807554c640 1 -- start start 2023-11-24T06:27:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.325+0000 7fd91dffb640 1 -- 172.21.15.16:0/466087929 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd924003bc0 con 0x7fd9280b0530 2023-11-24T06:27:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7f21b19b0640 1 -- start start 2023-11-24T06:27:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.327+0000 7fd91dffb640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd90c06ae40 0x7fd90c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.327+0000 7fd91dffb640 1 -- 172.21.15.16:0/466087929 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9240ab710 con 0x7fd9280b0530 2023-11-24T06:27:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7fd92d3cd640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd90c06ae40 0x7fd90c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7efea11ca640 1 -- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe940d5da0 msgr2=0x7efe940dea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7efea11ca640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe940d5da0 0x7efe940dea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7efea11ca640 1 -- 172.21.15.16:0/3626055543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe940d59c0 msgr2=0x7efe940d9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7efea11ca640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe940d59c0 0x7efe940d9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7efea11ca640 1 -- 172.21.15.16:0/3626055543 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940e0ce0 con 0x7efe940e17c0 2023-11-24T06:27:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7efea09c9640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe940d59c0 0x7efe940d9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.327+0000 7efea11ca640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe940e17c0 0x7efe940e1c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe94059f50 tx=0x7efe8800a800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7f93d3fff640 1 -- 172.21.15.16:0/3000119906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93dc030b40 con 0x7f93e4006170 2023-11-24T06:27:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f93d3fff640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93d806ae90 0x7f93d806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f93d3fff640 1 -- 172.21.15.16:0/3000119906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93dc0798e0 con 0x7f93e4006170 2023-11-24T06:27:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.327+0000 7f807554c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f807015d590 0x7f80700cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f807554c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 0x7f80700ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.326+0000 7fde9f2bb640 1 -- start start 2023-11-24T06:27:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f806effd640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f807015d590 0x7f80700cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f21b19b0640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f21ac06e380 0x7f21ac0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e40057b0 con 0x7f93e4006170 2023-11-24T06:27:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f21b19b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac15d840 con 0x7f21ac0b30b0 2023-11-24T06:27:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f806effd640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f807015d590 0x7f80700cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53198/0 (socket says 172.21.15.16:53198) 2023-11-24T06:27:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f21b19b0640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac072190 con 0x7f21ac15d460 2023-11-24T06:27:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.328+0000 7f21b19b0640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac071490 con 0x7f21ac06e380 2023-11-24T06:27:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.329+0000 7f8067fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 0x7f80700ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.329+0000 7f807554c640 1 -- 172.21.15.16:0/1732811482 --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80700b6750 con 0x7f80700ce6f0 2023-11-24T06:27:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.329+0000 7f807554c640 1 -- 172.21.15.16:0/1732811482 --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80700d0530 con 0x7f807015d590 2023-11-24T06:27:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.329+0000 7f807554c640 1 -- 172.21.15.16:0/1732811482 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80700cf1f0 con 0x7f80700d00b0 2023-11-24T06:27:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.329+0000 7f806effd640 1 -- 172.21.15.16:0/1732811482 learned_addr learned my addr 172.21.15.16:0/1732811482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.329+0000 7f21aa7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f21ac0b30b0 0x7f21ac06d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52812/0 (socket says 172.21.15.16:52812) 2023-11-24T06:27:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.329+0000 7f21aa7fc640 1 -- 172.21.15.16:0/1925727131 learned_addr learned my addr 172.21.15.16:0/1925727131 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7f21ab7fe640 1 --2- 172.21.15.16:0/1925727131 >> v2:172.21.15.173:3300/0 conn(0x7f21ac06e380 0x7f21ac0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7fd92d3cd640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd90c06ae40 0x7fd90c06d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd918000f60 tx=0x7fd918009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7fde9f2bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde980cf9a0 0x7fde98070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7f806effd640 1 -- 172.21.15.16:0/1732811482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 msgr2=0x7f80700ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7fde9f2bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde98070ea0 0x7fde980cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7fde9d057640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fde9814b2c0 0x7fde980cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52824/0 (socket says 172.21.15.16:52824) 2023-11-24T06:27:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7f806effd640 1 --2- 172.21.15.16:0/1732811482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 0x7f80700ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7f806effd640 1 -- 172.21.15.16:0/1732811482 >> v1:172.21.15.173:6789/0 conn(0x7f80700ce6f0 legacy=0x7f80700d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7f806effd640 1 -- 172.21.15.16:0/1732811482 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700cef10 con 0x7f807015d590 2023-11-24T06:27:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7fde9c856640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde980cf9a0 0x7fde98070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.330+0000 7fde9c856640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde980cf9a0 0x7fde98070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:40994/0 (socket says 172.21.15.16:40994) 2023-11-24T06:27:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7f21ab7fe640 1 -- 172.21.15.16:0/1925727131 >> v1:172.21.15.173:6789/0 conn(0x7f21ac15d460 legacy=0x7f21ac0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7f21ab7fe640 1 -- 172.21.15.16:0/1925727131 >> v1:172.21.15.16:6789/0 conn(0x7f21ac0b30b0 legacy=0x7f21ac06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7f21ab7fe640 1 -- 172.21.15.16:0/1925727131 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac072470 con 0x7f21ac06e380 2023-11-24T06:27:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7fde9d858640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde98070ea0 0x7fde980cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7fde9d858640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde98070ea0 0x7fde980cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38246/0 (socket says 172.21.15.16:38246) 2023-11-24T06:27:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7fde9f2bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde98071690 con 0x7fde9814b2c0 2023-11-24T06:27:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.329+0000 7f93ea7fc640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93d806ae90 0x7f93d806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7f93ea7fc640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93d806ae90 0x7f93d806d350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93e404f770 tx=0x7f93e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7fde9f2bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9814b6a0 con 0x7fde98070ea0 2023-11-24T06:27:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7fde9f2bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde98071970 con 0x7fde980cf9a0 2023-11-24T06:27:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.331+0000 7fde9d057640 1 -- 172.21.15.16:0/3708472797 learned_addr learned my addr 172.21.15.16:0/3708472797 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.332+0000 7fde9d858640 1 -- 172.21.15.16:0/3708472797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde980cf9a0 msgr2=0x7fde98070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.332+0000 7f21ab7fe640 1 --2- 172.21.15.16:0/1925727131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21ac06e380 0x7f21ac0ac8d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f21940097a0 tx=0x7f219402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f8ad20347993394 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.332+0000 7fde9d858640 1 --2- 172.21.15.16:0/3708472797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde980cf9a0 0x7fde98070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.332+0000 7fde9d858640 1 -- 172.21.15.16:0/3708472797 >> v1:172.21.15.16:6789/0 conn(0x7fde9814b2c0 legacy=0x7fde980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.332+0000 7fde9d858640 1 -- 172.21.15.16:0/3708472797 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98071c50 con 0x7fde98070ea0 2023-11-24T06:27:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.332+0000 7f806effd640 1 --2- 172.21.15.16:0/1732811482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f807015d590 0x7f80700cfb70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8060009870 tx=0x7f806002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450d86ac61d43f25 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.334+0000 7fde9d858640 1 --2- 172.21.15.16:0/3708472797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98070ea0 0x7fde980cc160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde90009d90 tx=0x7fde9002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27d9506db926d4d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.334+0000 7fe294cda640 1 Processor -- start 2023-11-24T06:27:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.334+0000 7fe294cda640 1 -- start start 2023-11-24T06:27:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.334+0000 7f20f5e9b640 1 Processor -- start 2023-11-24T06:27:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.335+0000 7f20f5e9b640 1 -- start start 2023-11-24T06:27:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.335+0000 7fe294cda640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe2900d0280 0x7fe2900ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.335+0000 7fe294cda640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900ce6f0 0x7fe2900b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.335+0000 7fe294cda640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2900b6750 con 0x7fe29015d590 2023-11-24T06:27:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.335+0000 7fe294cda640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900cf1f0 con 0x7fe2900d0280 2023-11-24T06:27:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7fe294cda640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900cef10 con 0x7fe2900ce6f0 2023-11-24T06:27:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7f20f5e9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20f00d0280 0x7f20f00ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7f20f5e9b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f00b6750 con 0x7f20f015d590 2023-11-24T06:27:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7f20f5e9b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f00cf1f0 con 0x7f20f00ce6f0 2023-11-24T06:27:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7f20f5e9b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00cef10 con 0x7f20f00d0280 2023-11-24T06:27:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7f20ef7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20f015d590 0x7f20f00cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52838/0 (socket says 172.21.15.16:52838) 2023-11-24T06:27:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7fe28e59c640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fe29015d590 0x7fe2900cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:44492/0 (socket says 172.21.15.16:44492) 2023-11-24T06:27:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7fe28e59c640 1 -- 172.21.15.16:0/1946015994 learned_addr learned my addr 172.21.15.16:0/1946015994 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7fde8bfff640 1 -- 172.21.15.16:0/3708472797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9002cc50 con 0x7fde98070ea0 2023-11-24T06:27:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7fde8bfff640 1 -- 172.21.15.16:0/3708472797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9002cdb0 con 0x7fde98070ea0 2023-11-24T06:27:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7fde8bfff640 1 -- 172.21.15.16:0/3708472797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90034660 con 0x7fde98070ea0 2023-11-24T06:27:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7fe28ed9d640 1 --2- 172.21.15.16:0/1946015994 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900ce6f0 0x7fe2900b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7f20eeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20f00d0280 0x7f20f00ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7f20eeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20f00d0280 0x7f20f00ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38252/0 (socket says 172.21.15.16:38252) 2023-11-24T06:27:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7f20eeffd640 1 -- 172.21.15.16:0/2221283505 learned_addr learned my addr 172.21.15.16:0/2221283505 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7f93d3fff640 1 -- 172.21.15.16:0/3000119906 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93dc0af060 con 0x7f93e4006170 2023-11-24T06:27:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.336+0000 7fe28dd9b640 1 --2- 172.21.15.16:0/1946015994 >> v2:172.21.15.173:3300/0 conn(0x7fe2900d0280 0x7fe2900ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7f20eeffd640 1 -- 172.21.15.16:0/2221283505 >> v1:172.21.15.173:6789/0 conn(0x7f20f00ce6f0 legacy=0x7f20f00b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7f20eeffd640 1 -- 172.21.15.16:0/2221283505 >> v1:172.21.15.16:6789/0 conn(0x7f20f015d590 legacy=0x7f20f00cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7f20eeffd640 1 -- 172.21.15.16:0/2221283505 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00b6100 con 0x7f20f00d0280 2023-11-24T06:27:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3708472797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98070ea0 msgr2=0x7fde980cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fde9f2bb640 1 --2- 172.21.15.16:0/3708472797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98070ea0 0x7fde980cc160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fde90009d90 tx=0x7fde9002bea0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fe28dd9b640 1 -- 172.21.15.16:0/1946015994 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900ce6f0 msgr2=0x7fe2900b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fe28dd9b640 1 --2- 172.21.15.16:0/1946015994 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900ce6f0 0x7fe2900b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fe28dd9b640 1 -- 172.21.15.16:0/1946015994 >> v1:172.21.15.173:6789/0 conn(0x7fe29015d590 legacy=0x7fe2900cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fe28dd9b640 1 -- 172.21.15.16:0/1946015994 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900b6100 con 0x7fe2900d0280 2023-11-24T06:27:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fe28ed9d640 1 --2- 172.21.15.16:0/1946015994 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900ce6f0 0x7fe2900b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3708472797 shutdown_connections 2023-11-24T06:27:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fde9f2bb640 1 --2- 172.21.15.16:0/3708472797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde98070ea0 0x7fde980cc160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fde9f2bb640 1 --2- 172.21.15.16:0/3708472797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde980cf9a0 0x7fde98070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.337+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3708472797 >> 172.21.15.16:0/3708472797 conn(0x7fde9806a800 msgr2=0x7fde9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.338+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3708472797 shutdown_connections 2023-11-24T06:27:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.338+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3708472797 wait complete. 2023-11-24T06:27:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.338+0000 7f20eeffd640 1 --2- 172.21.15.16:0/2221283505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00d0280 0x7f20f00ce1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009870 tx=0x7f20e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888f8f1e6fbe51f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.338+0000 7f20ee7fc640 1 -- 172.21.15.16:0/2221283505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4031870 con 0x7f20f00d0280 2023-11-24T06:27:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.338+0000 7f20ee7fc640 1 -- 172.21.15.16:0/2221283505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e40319d0 con 0x7f20f00d0280 2023-11-24T06:27:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.339+0000 7f20ee7fc640 1 -- 172.21.15.16:0/2221283505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4031d60 con 0x7f20f00d0280 2023-11-24T06:27:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.339+0000 7fe28dd9b640 1 --2- 172.21.15.16:0/1946015994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900d0280 0x7fe2900ce1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2840097a0 tx=0x7fe28402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd538f837c9a95e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.339+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2221283505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00d0280 msgr2=0x7f20f00ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.339+0000 7f20f5e9b640 1 --2- 172.21.15.16:0/2221283505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00d0280 0x7f20f00ce1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009870 tx=0x7f20e402b910 comp rx=0 tx=0).stop 2023-11-24T06:27:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.339+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2221283505 shutdown_connections 2023-11-24T06:27:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.339+0000 7f20f5e9b640 1 --2- 172.21.15.16:0/2221283505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00d0280 0x7f20f00ce1b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.339+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2221283505 >> 172.21.15.16:0/2221283505 conn(0x7f20f009c540 msgr2=0x7f20f00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.339+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2221283505 shutdown_connections 2023-11-24T06:27:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.339+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2221283505 wait complete. 2023-11-24T06:27:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.340+0000 7fde9f2bb640 1 Processor -- start 2023-11-24T06:27:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.340+0000 7fde9f2bb640 1 -- start start 2023-11-24T06:27:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7fde9f2bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde98070ea0 0x7fde98144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7f20f5e9b640 1 Processor -- start 2023-11-24T06:27:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7fde9f2bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde980cf9a0 0x7fde98147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7fde9f2bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9814b2c0 0x7fde9815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7fde9f2bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98148390 con 0x7fde9814b2c0 2023-11-24T06:27:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7fde9f2bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9815df20 con 0x7fde980cf9a0 2023-11-24T06:27:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7fde9f2bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde98143bf0 con 0x7fde98070ea0 2023-11-24T06:27:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7f20f5e9b640 1 -- start start 2023-11-24T06:27:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7fde9d858640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9814b2c0 0x7fde9815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7fde9d858640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9814b2c0 0x7fde9815d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38264/0 (socket says 172.21.15.16:38264) 2023-11-24T06:27:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.341+0000 7fde9d858640 1 -- 172.21.15.16:0/3353160825 learned_addr learned my addr 172.21.15.16:0/3353160825 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7fde9d057640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde98070ea0 0x7fde98144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7fde9c856640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde980cf9a0 0x7fde98147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7f20f5e9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00ce6f0 0x7f20f009db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7f20f5e9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f00d0280 0x7f20f00a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7f20f5e9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f015d590 0x7f20f01f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7f20f5e9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00a1050 con 0x7f20f00ce6f0 2023-11-24T06:27:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7f20f5e9b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f01f2d60 con 0x7f20f00d0280 2023-11-24T06:27:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7f20f5e9b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f01f6890 con 0x7f20f015d590 2023-11-24T06:27:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7fde9d858640 1 -- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde98070ea0 msgr2=0x7fde98144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7fde9d858640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde98070ea0 0x7fde98144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7fde9d858640 1 -- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde980cf9a0 msgr2=0x7fde98147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7fde9d858640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde980cf9a0 0x7fde98147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7fde9d858640 1 -- 172.21.15.16:0/3353160825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde90009a70 con 0x7fde9814b2c0 2023-11-24T06:27:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.342+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00ce6f0 0x7f20f009db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00ce6f0 0x7f20f009db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38278/0 (socket says 172.21.15.16:38278) 2023-11-24T06:27:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7f20ef7fe640 1 -- 172.21.15.16:0/2561519465 learned_addr learned my addr 172.21.15.16:0/2561519465 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7f806e7fc640 1 -- 172.21.15.16:0/1732811482 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060030840 con 0x7f807015d590 2023-11-24T06:27:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7f21a9ffb640 1 -- 172.21.15.16:0/1925727131 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219402e070 con 0x7f21ac06e380 2023-11-24T06:27:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7fde9d858640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9814b2c0 0x7fde9815d9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fde90009d90 tx=0x7fde90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7f21a9ffb640 1 -- 172.21.15.16:0/1925727131 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21940315a0 con 0x7f21ac06e380 2023-11-24T06:27:04.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7f806e7fc640 1 -- 172.21.15.16:0/1732811482 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80600309a0 con 0x7f807015d590 2023-11-24T06:27:04.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7f806e7fc640 1 -- 172.21.15.16:0/1732811482 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060030d30 con 0x7f807015d590 2023-11-24T06:27:04.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7f20effff640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f015d590 0x7f20f01f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7efe997fa640 1 -- 172.21.15.16:0/3626055543 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88005dd0 con 0x7efe940e17c0 2023-11-24T06:27:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7fde89ffb640 1 -- 172.21.15.16:0/3353160825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90040650 con 0x7fde9814b2c0 2023-11-24T06:27:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7fde89ffb640 1 -- 172.21.15.16:0/3353160825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde900407d0 con 0x7fde9814b2c0 2023-11-24T06:27:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7fde89ffb640 1 -- 172.21.15.16:0/3353160825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90040b80 con 0x7fde9814b2c0 2023-11-24T06:27:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7fe28d59a640 1 -- 172.21.15.16:0/1946015994 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28402d040 con 0x7fe2900d0280 2023-11-24T06:27:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7fe28d59a640 1 -- 172.21.15.16:0/1946015994 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe284031810 con 0x7fe2900d0280 2023-11-24T06:27:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.343+0000 7fe28d59a640 1 -- 172.21.15.16:0/1946015994 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284031ba0 con 0x7fe2900d0280 2023-11-24T06:27:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7efe997fa640 1 -- 172.21.15.16:0/3626055543 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8800cec0 con 0x7efe940e17c0 2023-11-24T06:27:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde98143ed0 con 0x7fde9814b2c0 2023-11-24T06:27:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde98144490 con 0x7fde9814b2c0 2023-11-24T06:27:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f21b19b0640 1 -- 172.21.15.16:0/1925727131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21ac06e380 msgr2=0x7f21ac0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f21a9ffb640 1 -- 172.21.15.16:0/1925727131 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194031930 con 0x7f21ac06e380 2023-11-24T06:27:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f807554c640 1 -- 172.21.15.16:0/1732811482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f807015d590 msgr2=0x7f80700cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f807554c640 1 --2- 172.21.15.16:0/1732811482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f807015d590 0x7f80700cfb70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8060009870 tx=0x7f806002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f20eeffd640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f00d0280 0x7f20f00a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f21b19b0640 1 --2- 172.21.15.16:0/1925727131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21ac06e380 0x7f21ac0ac8d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f21940097a0 tx=0x7f219402b840 comp rx=0 tx=0).stop 2023-11-24T06:27:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7fd91dffb640 1 -- 172.21.15.16:0/466087929 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd924076860 con 0x7fd9280b0530 2023-11-24T06:27:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f20effff640 1 -- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f00d0280 msgr2=0x7f20f00a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f20effff640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f00d0280 0x7f20f00a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f21b19b0640 1 -- 172.21.15.16:0/1925727131 shutdown_connections 2023-11-24T06:27:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f807554c640 1 -- 172.21.15.16:0/1732811482 shutdown_connections 2023-11-24T06:27:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f807554c640 1 --2- 172.21.15.16:0/1732811482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 0x7f80700ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f807554c640 1 --2- 172.21.15.16:0/1732811482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f807015d590 0x7f80700cfb70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f807554c640 1 -- 172.21.15.16:0/1732811482 >> 172.21.15.16:0/1732811482 conn(0x7f807009c540 msgr2=0x7f80700d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f21b19b0640 1 --2- 172.21.15.16:0/1925727131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21ac06e380 0x7f21ac0ac8d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f21b19b0640 1 -- 172.21.15.16:0/1925727131 >> 172.21.15.16:0/1925727131 conn(0x7f21ac0b0be0 msgr2=0x7f21ac0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f20effff640 1 -- 172.21.15.16:0/2561519465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00ce6f0 msgr2=0x7f20f009db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f20effff640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00ce6f0 0x7f20f009db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7f20effff640 1 -- 172.21.15.16:0/2561519465 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40094d0 con 0x7f20f015d590 2023-11-24T06:27:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7f807554c640 1 -- 172.21.15.16:0/1732811482 shutdown_connections 2023-11-24T06:27:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7f20eeffd640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f00d0280 0x7f20f00a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7f20effff640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f015d590 0x7f20f01f6350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c7b0 tx=0x7f20e800cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7f20d7fff640 1 -- 172.21.15.16:0/2561519465 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8003d80 con 0x7f20f015d590 2023-11-24T06:27:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7f20d7fff640 1 -- 172.21.15.16:0/2561519465 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e800ce60 con 0x7f20f015d590 2023-11-24T06:27:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7f20d7fff640 1 -- 172.21.15.16:0/2561519465 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80052b0 con 0x7f20f015d590 2023-11-24T06:27:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.344+0000 7efe997fa640 1 -- 172.21.15.16:0/3626055543 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe880072d0 con 0x7efe940e17c0 2023-11-24T06:27:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.346+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe940e0fc0 con 0x7efe940e17c0 2023-11-24T06:27:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.346+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f01f6b10 con 0x7f20f015d590 2023-11-24T06:27:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7fe294cda640 1 -- 172.21.15.16:0/1946015994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900d0280 msgr2=0x7fe2900ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7fe294cda640 1 --2- 172.21.15.16:0/1946015994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900d0280 0x7fe2900ce1b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe2840097a0 tx=0x7fe28402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7fe294cda640 1 -- 172.21.15.16:0/1946015994 shutdown_connections 2023-11-24T06:27:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7fe294cda640 1 --2- 172.21.15.16:0/1946015994 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900ce6f0 0x7fe2900b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7fe294cda640 1 --2- 172.21.15.16:0/1946015994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900d0280 0x7fe2900ce1b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7fe294cda640 1 -- 172.21.15.16:0/1946015994 >> 172.21.15.16:0/1946015994 conn(0x7fe29009c540 msgr2=0x7fe2900d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.346+0000 7f21b19b0640 1 -- 172.21.15.16:0/1925727131 shutdown_connections 2023-11-24T06:27:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde980675d0 con 0x7fde9814b2c0 2023-11-24T06:27:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.346+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f01f70d0 con 0x7f20f015d590 2023-11-24T06:27:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.346+0000 7fe294cda640 1 -- 172.21.15.16:0/1946015994 shutdown_connections 2023-11-24T06:27:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.346+0000 7f21b19b0640 1 -- 172.21.15.16:0/1925727131 wait complete. 2023-11-24T06:27:04.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.345+0000 7f807554c640 1 -- 172.21.15.16:0/1732811482 wait complete. 2023-11-24T06:27:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.347+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f01f72c0 con 0x7f20f015d590 2023-11-24T06:27:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7f21b19b0640 1 Processor -- start 2023-11-24T06:27:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.347+0000 7fe294cda640 1 -- 172.21.15.16:0/1946015994 wait complete. 2023-11-24T06:27:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.348+0000 7fe294cda640 1 Processor -- start 2023-11-24T06:27:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.349+0000 7fe294cda640 1 -- start start 2023-11-24T06:27:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.349+0000 7fe294cda640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900ce6f0 0x7fe2900a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.349+0000 7fe294cda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900d0280 0x7fe2901580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.349+0000 7fe294cda640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe29015d590 0x7fe2901585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7fe294cda640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290159480 con 0x7fe2900d0280 2023-11-24T06:27:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7fe294cda640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900cd170 con 0x7fe2900ce6f0 2023-11-24T06:27:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7fe294cda640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29014eff0 con 0x7fe29015d590 2023-11-24T06:27:04.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7f20d7fff640 1 -- 172.21.15.16:0/2561519465 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20e8006b10 con 0x7f20f015d590 2023-11-24T06:27:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7fe28e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900ce6f0 0x7fe2900a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7fe28e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900ce6f0 0x7fe2900a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53238/0 (socket says 172.21.15.16:53238) 2023-11-24T06:27:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7fe28e59c640 1 -- 172.21.15.16:0/3408066492 learned_addr learned my addr 172.21.15.16:0/3408066492 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7fe28ed9d640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe29015d590 0x7fe2901585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.351+0000 7fe28dd9b640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900d0280 0x7fe2901580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.348+0000 7fde89ffb640 1 -- 172.21.15.16:0/3353160825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde90040ce0 con 0x7fde9814b2c0 2023-11-24T06:27:04.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.351+0000 7fe28e59c640 1 -- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe29015d590 msgr2=0x7fe2901585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.351+0000 7fe28e59c640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe29015d590 0x7fe2901585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7fde89ffb640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde7c06ae40 0x7fde7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7fde89ffb640 1 -- 172.21.15.16:0/3353160825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde900ac860 con 0x7fde9814b2c0 2023-11-24T06:27:04.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.351+0000 7fe28e59c640 1 -- 172.21.15.16:0/3408066492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900d0280 msgr2=0x7fe2901580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.350+0000 7f21b19b0640 1 -- start start 2023-11-24T06:27:04.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.351+0000 7fe28e59c640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900d0280 0x7fe2901580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.351+0000 7f20d7fff640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f20dc06ae40 0x7f20dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe940e35f0 con 0x7efe940e17c0 2023-11-24T06:27:04.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7efe997fa640 1 -- 172.21.15.16:0/3626055543 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe88007430 con 0x7efe940e17c0 2023-11-24T06:27:04.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7f20d7fff640 1 -- 172.21.15.16:0/2561519465 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20e8088ce0 con 0x7f20f015d590 2023-11-24T06:27:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7f21b19b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06e380 0x7f21ac0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7f21b19b0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21ac0b30b0 0x7f21ac15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7f21b19b0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21ac15d460 0x7f21ac1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7f21b19b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac15f220 con 0x7f21ac06e380 2023-11-24T06:27:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7f21b19b0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac154be0 con 0x7f21ac15d460 2023-11-24T06:27:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7f21b19b0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac1fb2b0 con 0x7f21ac0b30b0 2023-11-24T06:27:04.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7f20ef7fe640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f20dc06ae40 0x7f20dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.351+0000 7fe28e59c640 1 -- 172.21.15.16:0/3408066492 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288009a70 con 0x7fe2900ce6f0 2023-11-24T06:27:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.353+0000 7f21ab7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21ac15d460 0x7f21ac1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.353+0000 7f21aa7fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21ac0b30b0 0x7f21ac15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.351+0000 7f807554c640 1 Processor -- start 2023-11-24T06:27:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.353+0000 7f807554c640 1 -- start start 2023-11-24T06:27:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.353+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe940e14a0 con 0x7efe940e17c0 2023-11-24T06:27:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7fde9d057640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde7c06ae40 0x7fde7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.354+0000 7fde9d057640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde7c06ae40 0x7fde7c06d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde8c005880 tx=0x7fde8c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.354+0000 7f21aaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06e380 0x7f21ac0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.354+0000 7f21aaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06e380 0x7f21ac0cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38298/0 (socket says 172.21.15.16:38298) 2023-11-24T06:27:04.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.354+0000 7f21aaffd640 1 -- 172.21.15.16:0/2716266443 learned_addr learned my addr 172.21.15.16:0/2716266443 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.355+0000 7efe997fa640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efe8006ae70 0x7efe8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.355+0000 7efe997fa640 1 -- 172.21.15.16:0/3626055543 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe88088160 con 0x7efe940e17c0 2023-11-24T06:27:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.351+0000 7fe28dd9b640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900d0280 0x7fe2901580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7fe28e59c640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900ce6f0 0x7fe2900a0720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe28000c480 tx=0x7fe28000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7fe27effd640 1 -- 172.21.15.16:0/3408066492 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280005ce0 con 0x7fe2900ce6f0 2023-11-24T06:27:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7fe27effd640 1 -- 172.21.15.16:0/3408066492 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe280005e40 con 0x7fe2900ce6f0 2023-11-24T06:27:04.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7fe27effd640 1 -- 172.21.15.16:0/3408066492 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2800072d0 con 0x7fe2900ce6f0 2023-11-24T06:27:04.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f807554c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80700ce6f0 0x7f80701572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7efea09c9640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efe8006ae70 0x7efe8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f807554c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 0x7f80701e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f807554c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807015d590 0x7f80700b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.352+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe284009400 con 0x7fe2900ce6f0 2023-11-24T06:27:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.353+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2900cc8d0 con 0x7fe2900ce6f0 2023-11-24T06:27:04.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.354+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2900675d0 con 0x7fe2900ce6f0 2023-11-24T06:27:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f807554c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701e8c20 con 0x7f807015d590 2023-11-24T06:27:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f806f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807015d590 0x7f80700b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f806f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807015d590 0x7f80700b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38314/0 (socket says 172.21.15.16:38314) 2023-11-24T06:27:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f807554c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701f7840 con 0x7f80700ce6f0 2023-11-24T06:27:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f807554c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80701f7b20 con 0x7f80700d00b0 2023-11-24T06:27:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f806f7fe640 1 -- 172.21.15.16:0/3920839540 learned_addr learned my addr 172.21.15.16:0/3920839540 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.356+0000 7f806effd640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80700ce6f0 0x7f80701572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.357+0000 7f8067fff640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 0x7f80701e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.357+0000 7f806f7fe640 1 -- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 msgr2=0x7f80701e81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.357+0000 7f806f7fe640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 0x7f80701e81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.357+0000 7f806f7fe640 1 -- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80700ce6f0 msgr2=0x7f80701572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.357+0000 7f806f7fe640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80700ce6f0 0x7f80701572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.357+0000 7f806f7fe640 1 -- 172.21.15.16:0/3920839540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80600094d0 con 0x7f807015d590 2023-11-24T06:27:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.357+0000 7f806f7fe640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807015d590 0x7f80700b4880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f806800c6b0 tx=0x7f806800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.358+0000 7f21aaffd640 1 -- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21ac0b30b0 msgr2=0x7f21ac15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.358+0000 7f21aaffd640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21ac0b30b0 0x7f21ac15ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.358+0000 7f21aaffd640 1 -- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21ac15d460 msgr2=0x7f21ac1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.358+0000 7f21aaffd640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21ac15d460 0x7f21ac1546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.358+0000 7f21aaffd640 1 -- 172.21.15.16:0/2716266443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2194009400 con 0x7f21ac06e380 2023-11-24T06:27:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.358+0000 7f21aaffd640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06e380 0x7f21ac0cb930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f21a400c730 tx=0x7f21a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.358+0000 7fe27effd640 1 -- 172.21.15.16:0/3408066492 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe280007430 con 0x7fe2900ce6f0 2023-11-24T06:27:04.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.358+0000 7f21aa7fc640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21ac0b30b0 0x7f21ac15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:04.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.359+0000 7f20ef7fe640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f20dc06ae40 0x7f20dc06d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006fd0 tx=0x7f20e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.359+0000 7fe27effd640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe270073720 0x7fe270075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.359+0000 7efea09c9640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efe8006ae70 0x7efe8006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe90034990 tx=0x7efe90005000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.359+0000 7fe27effd640 1 -- 172.21.15.16:0/3408066492 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe280088e20 con 0x7fe2900ce6f0 2023-11-24T06:27:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.359+0000 7fe28dd9b640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe270073720 0x7fe270075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.360+0000 7fe28dd9b640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe270073720 0x7fe270075be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe284036000 tx=0x7fe28403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.362+0000 7fde89ffb640 1 -- 172.21.15.16:0/3353160825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde90077980 con 0x7fde9814b2c0 2023-11-24T06:27:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.362+0000 7f21a3fff640 1 -- 172.21.15.16:0/2716266443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4003b70 con 0x7f21ac06e380 2023-11-24T06:27:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.362+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac1fb590 con 0x7f21ac06e380 2023-11-24T06:27:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.362+0000 7f80677fe640 1 -- 172.21.15.16:0/3920839540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068003b70 con 0x7f807015d590 2023-11-24T06:27:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.362+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac1fbb50 con 0x7f21ac06e380 2023-11-24T06:27:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.363+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80701f7e00 con 0x7f807015d590 2023-11-24T06:27:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.363+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80701577e0 con 0x7f807015d590 2023-11-24T06:27:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.363+0000 7f80677fe640 1 -- 172.21.15.16:0/3920839540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8068001790 con 0x7f807015d590 2023-11-24T06:27:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.363+0000 7f80677fe640 1 -- 172.21.15.16:0/3920839540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680050d0 con 0x7f807015d590 2023-11-24T06:27:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.364+0000 7f21a3fff640 1 -- 172.21.15.16:0/2716266443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a400ce40 con 0x7f21ac06e380 2023-11-24T06:27:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.364+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac0710c0 con 0x7f21ac06e380 2023-11-24T06:27:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.364+0000 7f21a3fff640 1 -- 172.21.15.16:0/2716266443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4006b50 con 0x7f21ac06e380 2023-11-24T06:27:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.365+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8054005250 con 0x7f807015d590 2023-11-24T06:27:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.367+0000 7f21a3fff640 1 -- 172.21.15.16:0/2716266443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21a400a850 con 0x7f21ac06e380 2023-11-24T06:27:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.367+0000 7f21a3fff640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218c06ae40 0x7f218c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.367+0000 7f21a3fff640 1 -- 172.21.15.16:0/2716266443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21a4089d10 con 0x7f21ac06e380 2023-11-24T06:27:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.368+0000 7f21aa7fc640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218c06ae40 0x7f218c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.368+0000 7f80677fe640 1 -- 172.21.15.16:0/3920839540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8068005230 con 0x7f807015d590 2023-11-24T06:27:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.369+0000 7efe997fa640 1 -- 172.21.15.16:0/3626055543 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe8808d060 con 0x7efe940e17c0 2023-11-24T06:27:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.369+0000 7f20d7fff640 1 -- 172.21.15.16:0/2561519465 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20e8053e00 con 0x7f20f015d590 2023-11-24T06:27:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.370+0000 7f80677fe640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f805006ae70 0x7f805006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.370+0000 7f80677fe640 1 -- 172.21.15.16:0/3920839540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8068089410 con 0x7f807015d590 2023-11-24T06:27:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.370+0000 7f21aa7fc640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218c06ae40 0x7f218c06d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f219c005160 tx=0x7f219c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.370+0000 7f806effd640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f805006ae70 0x7f805006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.371+0000 7f806effd640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f805006ae70 0x7f805006d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8060036000 tx=0x7f8060008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.375+0000 7f21a3fff640 1 -- 172.21.15.16:0/2716266443 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21a4054e30 con 0x7f21ac06e380 2023-11-24T06:27:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.378+0000 7f80677fe640 1 -- 172.21.15.16:0/3920839540 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8068058030 con 0x7f807015d590 2023-11-24T06:27:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.383+0000 7fe27effd640 1 -- 172.21.15.16:0/3408066492 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe280053ff0 con 0x7fe2900ce6f0 2023-11-24T06:27:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.756+0000 7fd90b7fe640 1 -- 172.21.15.16:0/466087929 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd9040057d0 con 0x7fd9280b0530 2023-11-24T06:27:04.762 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-11-24T06:27:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.757+0000 7fd91dffb640 1 -- 172.21.15.16:0/466087929 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd924047020 con 0x7fd9280b0530 2023-11-24T06:27:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 -- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd90c06ae40 msgr2=0x7fd90c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd90c06ae40 0x7fd90c06d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd918000f60 tx=0x7fd918009040 comp rx=0 tx=0).stop 2023-11-24T06:27:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 -- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9280b0530 msgr2=0x7fd9280ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9280b0530 0x7fd9280ce2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd9240098c0 tx=0x7fd9240046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 -- 172.21.15.16:0/466087929 shutdown_connections 2023-11-24T06:27:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd90c06ae40 0x7fd90c06d300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9280b3eb0 0x7fd9281409a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9280b0530 0x7fd9280ce2e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 --2- 172.21.15.16:0/466087929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd92806c340 0x7fd928138ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.761+0000 7fd90b7fe640 1 -- 172.21.15.16:0/466087929 >> 172.21.15.16:0/466087929 conn(0x7fd9280aea00 msgr2=0x7fd92815a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.762+0000 7fd90b7fe640 1 -- 172.21.15.16:0/466087929 shutdown_connections 2023-11-24T06:27:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.762+0000 7fd90b7fe640 1 -- 172.21.15.16:0/466087929 wait complete. 2023-11-24T06:27:04.769 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-11-24T06:27:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.765+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f93e4038260 con 0x7f93e4006170 2023-11-24T06:27:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.765+0000 7f93d3fff640 1 -- 172.21.15.16:0/3000119906 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93dc037030 con 0x7f93e4006170 2023-11-24T06:27:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93d806ae90 msgr2=0x7f93d806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93d806ae90 0x7f93d806d350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93e404f770 tx=0x7f93e0009040 comp rx=0 tx=0).stop 2023-11-24T06:27:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e4006170 msgr2=0x7f93e40dcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e4006170 0x7f93e40dcf10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f93dc035040 tx=0x7f93dc0046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 shutdown_connections 2023-11-24T06:27:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f93d806ae90 0x7f93d806d350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e40eb410 0x7f93e40df290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93e40eb030 0x7f93e40dd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 --2- 172.21.15.16:0/3000119906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93e4006170 0x7f93e40dcf10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.769+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 >> 172.21.15.16:0/3000119906 conn(0x7f93e40559b0 msgr2=0x7f93e403c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.770+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 shutdown_connections 2023-11-24T06:27:04.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.770+0000 7f93ebfff640 1 -- 172.21.15.16:0/3000119906 wait complete. 2023-11-24T06:27:04.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.3 2023-11-24T06:27:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.795+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efe940057b0 con 0x7efe940e17c0 2023-11-24T06:27:04.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2023-11-24T06:27:04.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.796+0000 7efe997fa640 1 -- 172.21.15.16:0/3626055543 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe880532a0 con 0x7efe940e17c0 2023-11-24T06:27:04.799 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-11-24T06:27:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe2900b70f0 con 0x7fe2900ce6f0 2023-11-24T06:27:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efe8006ae70 msgr2=0x7efe8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efe8006ae70 0x7efe8006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe90034990 tx=0x7efe90005000 comp rx=0 tx=0).stop 2023-11-24T06:27:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe940e17c0 msgr2=0x7efe940e1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe940e17c0 0x7efe940e1c40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe94059f50 tx=0x7efe8800a800 comp rx=0 tx=0).stop 2023-11-24T06:27:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 shutdown_connections 2023-11-24T06:27:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efe8006ae70 0x7efe8006d330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe940e17c0 0x7efe940e1c40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe940d5da0 0x7efe940dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 --2- 172.21.15.16:0/3626055543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe940d59c0 0x7efe940d9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 >> 172.21.15.16:0/3626055543 conn(0x7efe940559b0 msgr2=0x7efe94057560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 shutdown_connections 2023-11-24T06:27:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.799+0000 7efea19cb640 1 -- 172.21.15.16:0/3626055543 wait complete. 2023-11-24T06:27:04.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.800+0000 7fe27effd640 1 -- 172.21.15.16:0/3408066492 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe280017090 con 0x7fe2900ce6f0 2023-11-24T06:27:04.803 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-11-24T06:27:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f20f00675d0 con 0x7f20f015d590 2023-11-24T06:27:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fde980cb2a0 con 0x7fde9814b2c0 2023-11-24T06:27:04.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f21ac0675d0 con 0x7f21ac06e380 2023-11-24T06:27:04.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe270073720 msgr2=0x7fe270075be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe270073720 0x7fe270075be0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe284036000 tx=0x7fe28403c040 comp rx=0 tx=0).stop 2023-11-24T06:27:04.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900ce6f0 msgr2=0x7fe2900a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900ce6f0 0x7fe2900a0720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe28000c480 tx=0x7fe28000c880 comp rx=0 tx=0).stop 2023-11-24T06:27:04.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 shutdown_connections 2023-11-24T06:27:04.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe270073720 0x7fe270075be0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe284036000 tx=0x7fe28403c040 comp rx=0 tx=0).stop 2023-11-24T06:27:04.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe29015d590 0x7fe2901585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2900d0280 0x7fe2901580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 --2- 172.21.15.16:0/3408066492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900ce6f0 0x7fe2900a0720 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.802+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 >> 172.21.15.16:0/3408066492 conn(0x7fe29009c540 msgr2=0x7fe290160430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.803+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 shutdown_connections 2023-11-24T06:27:04.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.803+0000 7f21a3fff640 1 -- 172.21.15.16:0/2716266443 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f21a4058450 con 0x7f21ac06e380 2023-11-24T06:27:04.806 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-11-24T06:27:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.803+0000 7fe294cda640 1 -- 172.21.15.16:0/3408066492 wait complete. 2023-11-24T06:27:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.803+0000 7f20d7fff640 1 -- 172.21.15.16:0/2561519465 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20e8017070 con 0x7f20f015d590 2023-11-24T06:27:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.803+0000 7fde89ffb640 1 -- 172.21.15.16:0/3353160825 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde9007afa0 con 0x7fde9814b2c0 2023-11-24T06:27:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2023-11-24T06:27:04.808 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2023-11-24T06:27:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.806+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f80540057d0 con 0x7f807015d590 2023-11-24T06:27:04.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.806+0000 7f80677fe640 1 -- 172.21.15.16:0/3920839540 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8068057bc0 con 0x7f807015d590 2023-11-24T06:27:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218c06ae40 msgr2=0x7f218c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f21b19b0640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218c06ae40 0x7f218c06d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f219c005160 tx=0x7f219c009040 comp rx=0 tx=0).stop 2023-11-24T06:27:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06e380 msgr2=0x7f21ac0cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f21b19b0640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06e380 0x7f21ac0cb930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f21a400c730 tx=0x7f21a400cbf0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.810 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2023-11-24T06:27:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde7c06ae40 msgr2=0x7fde7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde7c06ae40 0x7fde7c06d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde8c005880 tx=0x7fde8c009040 comp rx=0 tx=0).stop 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9814b2c0 msgr2=0x7fde9815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9814b2c0 0x7fde9815d9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fde90009d90 tx=0x7fde90004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 shutdown_connections 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde7c06ae40 0x7fde7c06d300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9814b2c0 0x7fde9815d9e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde980cf9a0 0x7fde98147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 --2- 172.21.15.16:0/3353160825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde98070ea0 0x7fde98144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 >> 172.21.15.16:0/3353160825 conn(0x7fde9806a800 msgr2=0x7fde9806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 shutdown_connections 2023-11-24T06:27:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7fde9f2bb640 1 -- 172.21.15.16:0/3353160825 wait complete. 2023-11-24T06:27:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f20dc06ae40 msgr2=0x7f20dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f20dc06ae40 0x7f20dc06d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006fd0 tx=0x7f20e0008040 comp rx=0 tx=0).stop 2023-11-24T06:27:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f015d590 msgr2=0x7f20f01f6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f015d590 0x7f20f01f6350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c7b0 tx=0x7f20e800cc70 comp rx=0 tx=0).stop 2023-11-24T06:27:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 shutdown_connections 2023-11-24T06:27:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f20dc06ae40 0x7f20dc06d300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f015d590 0x7f20f01f6350 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f00d0280 0x7f20f00a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 --2- 172.21.15.16:0/2561519465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f00ce6f0 0x7f20f009db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 >> 172.21.15.16:0/2561519465 conn(0x7f20f009c540 msgr2=0x7f20f00b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 shutdown_connections 2023-11-24T06:27:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.807+0000 7f20f5e9b640 1 -- 172.21.15.16:0/2561519465 wait complete. 2023-11-24T06:27:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.808+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 shutdown_connections 2023-11-24T06:27:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.808+0000 7f21b19b0640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218c06ae40 0x7f218c06d300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.808+0000 7f21b19b0640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21ac15d460 0x7f21ac1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.808+0000 7f21b19b0640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21ac0b30b0 0x7f21ac15ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.808+0000 7f21b19b0640 1 --2- 172.21.15.16:0/2716266443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06e380 0x7f21ac0cb930 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.808+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 >> 172.21.15.16:0/2716266443 conn(0x7f21ac0b0be0 msgr2=0x7f21ac156cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.808+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 shutdown_connections 2023-11-24T06:27:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.808+0000 7f21b19b0640 1 -- 172.21.15.16:0/2716266443 wait complete. 2023-11-24T06:27:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.809+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f805006ae70 msgr2=0x7f805006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.809+0000 7f807554c640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f805006ae70 0x7f805006d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8060036000 tx=0x7f8060008030 comp rx=0 tx=0).stop 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.809+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807015d590 msgr2=0x7f80700b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.809+0000 7f807554c640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807015d590 0x7f80700b4880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f806800c6b0 tx=0x7f806800cb70 comp rx=0 tx=0).stop 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.810+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 shutdown_connections 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.810+0000 7f807554c640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f805006ae70 0x7f805006d330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.810+0000 7f807554c640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f807015d590 0x7f80700b4880 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.810+0000 7f807554c640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80700d00b0 0x7f80701e81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.810+0000 7f807554c640 1 --2- 172.21.15.16:0/3920839540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80700ce6f0 0x7f80701572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.810+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 >> 172.21.15.16:0/3920839540 conn(0x7f807009c540 msgr2=0x7f807015c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.810+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 shutdown_connections 2023-11-24T06:27:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:04.810+0000 7f807554c640 1 -- 172.21.15.16:0/3920839540 wait complete. 2023-11-24T06:27:04.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-11-24T06:27:04.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2023-11-24T06:27:04.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2023-11-24T06:27:04.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2023-11-24T06:27:04.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2023-11-24T06:27:04.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-11-24T06:27:05.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:27:05.799 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:27:05.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:27:05.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:27:05.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:27:05.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:27:05.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:27:05.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:27:06.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.092+0000 7f2dd26bb640 1 Processor -- start 2023-11-24T06:27:06.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.093+0000 7f2dd26bb640 1 -- start start 2023-11-24T06:27:06.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.095+0000 7f2dd26bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2dc403c060 0x7f2dc40e7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.096+0000 7f2dd26bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc4007b30 con 0x7f2dc4053c70 2023-11-24T06:27:06.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.096+0000 7f2dd26bb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc40587d0 con 0x7f2dc40e3120 2023-11-24T06:27:06.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.096+0000 7f2dd26bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc4058ab0 con 0x7f2dc403c060 2023-11-24T06:27:06.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.097+0000 7f2dcbfff640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f2dc40e3120 0x7f2dc4058010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:44512/0 (socket says 172.21.15.16:44512) 2023-11-24T06:27:06.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.097+0000 7f2dcbfff640 1 -- 172.21.15.16:0/2416801953 learned_addr learned my addr 172.21.15.16:0/2416801953 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:06.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.098+0000 7f2dcb7fe640 1 --2- 172.21.15.16:0/2416801953 >> v2:172.21.15.16:3300/0 conn(0x7f2dc403c060 0x7f2dc40e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.099+0000 7f2dcb7fe640 1 -- 172.21.15.16:0/2416801953 >> v1:172.21.15.173:6789/0 conn(0x7f2dc40e3120 legacy=0x7f2dc4058010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.099+0000 7f2dcb7fe640 1 -- 172.21.15.16:0/2416801953 >> v1:172.21.15.16:6789/0 conn(0x7f2dc4053c70 legacy=0x7f2dc40540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.100+0000 7f2dcb7fe640 1 -- 172.21.15.16:0/2416801953 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc4006320 con 0x7f2dc403c060 2023-11-24T06:27:06.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.102+0000 7f2dcb7fe640 1 --2- 172.21.15.16:0/2416801953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc403c060 0x7f2dc40e7740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc04fab0 tx=0x7f2dcc0a5e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fe74c1d34e7238 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.103+0000 7f2dcaffd640 1 -- 172.21.15.16:0/2416801953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0a6cf0 con 0x7f2dc403c060 2023-11-24T06:27:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.103+0000 7f2dcaffd640 1 -- 172.21.15.16:0/2416801953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc0a6e50 con 0x7f2dc403c060 2023-11-24T06:27:06.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.103+0000 7f2dcaffd640 1 -- 172.21.15.16:0/2416801953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0ae640 con 0x7f2dc403c060 2023-11-24T06:27:06.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.106+0000 7f2dd26bb640 1 -- 172.21.15.16:0/2416801953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc403c060 msgr2=0x7f2dc40e7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.106+0000 7f2dd26bb640 1 --2- 172.21.15.16:0/2416801953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc403c060 0x7f2dc40e7740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc04fab0 tx=0x7f2dcc0a5e80 comp rx=0 tx=0).stop 2023-11-24T06:27:06.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.107+0000 7f2dd26bb640 1 -- 172.21.15.16:0/2416801953 shutdown_connections 2023-11-24T06:27:06.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.107+0000 7f2dd26bb640 1 --2- 172.21.15.16:0/2416801953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc403c060 0x7f2dc40e7740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.107+0000 7f2dd26bb640 1 -- 172.21.15.16:0/2416801953 >> 172.21.15.16:0/2416801953 conn(0x7f2dc4058ea0 msgr2=0x7f2dc4059280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.107+0000 7f2dd26bb640 1 -- 172.21.15.16:0/2416801953 shutdown_connections 2023-11-24T06:27:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.108+0000 7f2dd26bb640 1 -- 172.21.15.16:0/2416801953 wait complete. 2023-11-24T06:27:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.110+0000 7f2dd26bb640 1 Processor -- start 2023-11-24T06:27:06.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.111+0000 7f2dd26bb640 1 -- start start 2023-11-24T06:27:06.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.113+0000 7f2dd26bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc403c060 0x7f2dc40d9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.113+0000 7f2dd26bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc4053c70 0x7f2dc40e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.114+0000 7f2dd26bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc40e3120 0x7f2dc40e7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.114+0000 7f2dd26bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40d8470 con 0x7f2dc4053c70 2023-11-24T06:27:06.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.114+0000 7f2dd26bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40d4610 con 0x7f2dc40e3120 2023-11-24T06:27:06.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.114+0000 7f2dd26bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40d86c0 con 0x7f2dc403c060 2023-11-24T06:27:06.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.115+0000 7f2dd0c58640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc40e3120 0x7f2dc40e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.115+0000 7f2dd0c58640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc40e3120 0x7f2dc40e7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53264/0 (socket says 172.21.15.16:53264) 2023-11-24T06:27:06.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.115+0000 7f2dd0c58640 1 -- 172.21.15.16:0/3864774869 learned_addr learned my addr 172.21.15.16:0/3864774869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.116+0000 7f2dcbfff640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc403c060 0x7f2dc40d9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.116+0000 7f2dcb7fe640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc4053c70 0x7f2dc40e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.117+0000 7f2dd0c58640 1 -- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc403c060 msgr2=0x7f2dc40d9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.117+0000 7f2dd0c58640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc403c060 0x7f2dc40d9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.117+0000 7f2dd0c58640 1 -- 172.21.15.16:0/3864774869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc4053c70 msgr2=0x7f2dc40e6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.117+0000 7f2dd0c58640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc4053c70 0x7f2dc40e6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.117+0000 7f2dd0c58640 1 -- 172.21.15.16:0/3864774869 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc04f090 con 0x7f2dc40e3120 2023-11-24T06:27:06.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.118+0000 7f2dd0c58640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc40e3120 0x7f2dc40e7150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2db800c400 tx=0x7f2db800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.119+0000 7f2dc8ff9640 1 -- 172.21.15.16:0/3864774869 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8015e30 con 0x7f2dc40e3120 2023-11-24T06:27:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.119+0000 7f2dc8ff9640 1 -- 172.21.15.16:0/3864774869 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db80050d0 con 0x7f2dc40e3120 2023-11-24T06:27:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.119+0000 7f2dc8ff9640 1 -- 172.21.15.16:0/3864774869 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8005460 con 0x7f2dc40e3120 2023-11-24T06:27:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.119+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc40d89a0 con 0x7f2dc40e3120 2023-11-24T06:27:06.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.119+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc40d8f60 con 0x7f2dc40e3120 2023-11-24T06:27:06.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.123+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dac005250 con 0x7f2dc40e3120 2023-11-24T06:27:06.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.125+0000 7f2dc8ff9640 1 -- 172.21.15.16:0/3864774869 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2db80035f0 con 0x7f2dc40e3120 2023-11-24T06:27:06.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.127+0000 7f2dc8ff9640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2db006ae90 0x7f2db006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.128+0000 7f2dc8ff9640 1 -- 172.21.15.16:0/3864774869 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2db8090f30 con 0x7f2dc40e3120 2023-11-24T06:27:06.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.128+0000 7f2dcbfff640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2db006ae90 0x7f2db006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.129+0000 7f2dcbfff640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2db006ae90 0x7f2db006d350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2dc403dd80 tx=0x7f2dc0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.143+0000 7f2dc8ff9640 1 -- 172.21.15.16:0/3864774869 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2db80c3e90 con 0x7f2dc40e3120 2023-11-24T06:27:06.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.148+0000 7f71af59e640 1 Processor -- start 2023-11-24T06:27:06.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.149+0000 7f71af59e640 1 -- start start 2023-11-24T06:27:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.151+0000 7f71af59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a80eacd0 0x7f71a804fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.151+0000 7f71af59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71a8006e70 0x7f71a8007300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.151+0000 7f71af59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a8054300 con 0x7f71a804ffa0 2023-11-24T06:27:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.151+0000 7f71af59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a8007840 con 0x7f71a8006e70 2023-11-24T06:27:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.151+0000 7f71af59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80536f0 con 0x7f71a80eacd0 2023-11-24T06:27:06.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.152+0000 7f71ae59c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a80eacd0 0x7f71a804fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.152+0000 7f71aed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71a8006e70 0x7f71a8007300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.152+0000 7f71aed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71a8006e70 0x7f71a8007300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38346/0 (socket says 172.21.15.16:38346) 2023-11-24T06:27:06.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.152+0000 7f71aed9d640 1 -- 172.21.15.16:0/1546190850 learned_addr learned my addr 172.21.15.16:0/1546190850 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.154+0000 7f71aed9d640 1 -- 172.21.15.16:0/1546190850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a80eacd0 msgr2=0x7f71a804fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.154+0000 7f71aed9d640 1 --2- 172.21.15.16:0/1546190850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a80eacd0 0x7f71a804fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.154+0000 7f71aed9d640 1 -- 172.21.15.16:0/1546190850 >> v1:172.21.15.16:6789/0 conn(0x7f71a804ffa0 legacy=0x7f71a8037ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.155+0000 7f71aed9d640 1 -- 172.21.15.16:0/1546190850 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a80539d0 con 0x7f71a8006e70 2023-11-24T06:27:06.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.156+0000 7f71aed9d640 1 --2- 172.21.15.16:0/1546190850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 0x7f71a8007300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71b00695b0 tx=0x7f71b00a5ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380f2b4e778c6cfe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.156+0000 7f71ad59a640 1 -- 172.21.15.16:0/1546190850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b00a6d50 con 0x7f71a8006e70 2023-11-24T06:27:06.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.157+0000 7f71ad59a640 1 -- 172.21.15.16:0/1546190850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b00ae440 con 0x7f71a8006e70 2023-11-24T06:27:06.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.157+0000 7f71ad59a640 1 -- 172.21.15.16:0/1546190850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b00ae8b0 con 0x7f71a8006e70 2023-11-24T06:27:06.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.158+0000 7f71af59e640 1 -- 172.21.15.16:0/1546190850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 msgr2=0x7f71a8007300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.158+0000 7f71af59e640 1 --2- 172.21.15.16:0/1546190850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 0x7f71a8007300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f71b00695b0 tx=0x7f71b00a5ee0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.158+0000 7fb690051640 1 Processor -- start 2023-11-24T06:27:06.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.159+0000 7f71af59e640 1 -- 172.21.15.16:0/1546190850 shutdown_connections 2023-11-24T06:27:06.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.159+0000 7f71af59e640 1 --2- 172.21.15.16:0/1546190850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 0x7f71a8007300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.159+0000 7f71af59e640 1 --2- 172.21.15.16:0/1546190850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a80eacd0 0x7f71a804fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.159+0000 7f71af59e640 1 -- 172.21.15.16:0/1546190850 >> 172.21.15.16:0/1546190850 conn(0x7f71a8009f80 msgr2=0x7f71a800a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.159+0000 7fb690051640 1 -- start start 2023-11-24T06:27:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.159+0000 7f71af59e640 1 -- 172.21.15.16:0/1546190850 shutdown_connections 2023-11-24T06:27:06.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.159+0000 7f71af59e640 1 -- 172.21.15.16:0/1546190850 wait complete. 2023-11-24T06:27:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.162+0000 7fb68dded640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb68814b2c0 0x7fb6880cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52856/0 (socket says 172.21.15.16:52856) 2023-11-24T06:27:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.162+0000 7f71af59e640 1 Processor -- start 2023-11-24T06:27:06.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.162+0000 7f71af59e640 1 -- start start 2023-11-24T06:27:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.164+0000 7f71af59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 0x7f71a817cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.164+0000 7fb690051640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6880cd170 0x7fb688071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.164+0000 7fb68e5ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6880cd170 0x7fb688071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.164+0000 7fb68e5ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6880cd170 0x7fb688071070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38360/0 (socket says 172.21.15.16:38360) 2023-11-24T06:27:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.164+0000 7fb690051640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb688071690 con 0x7fb68814b2c0 2023-11-24T06:27:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.165+0000 7f71af59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a804ffa0 0x7f71a8047400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.165+0000 7f71af59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a80eacd0 0x7f71a8045240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.164+0000 7fb68dded640 1 -- 172.21.15.16:0/764870964 learned_addr learned my addr 172.21.15.16:0/764870964 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.165+0000 7fb690051640 1 -- 172.21.15.16:0/764870964 --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb688071970 con 0x7fb6880cf9a0 2023-11-24T06:27:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.165+0000 7fb690051640 1 -- 172.21.15.16:0/764870964 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68814b6a0 con 0x7fb6880cd170 2023-11-24T06:27:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.165+0000 7f71af59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a8180330 con 0x7f71a8006e70 2023-11-24T06:27:06.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.165+0000 7f1934d0c640 1 Processor -- start 2023-11-24T06:27:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.165+0000 7f71af59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a817da60 con 0x7f71a80eacd0 2023-11-24T06:27:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7f71af59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a8047940 con 0x7f71a804ffa0 2023-11-24T06:27:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7f1934d0c640 1 -- start start 2023-11-24T06:27:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7fc04bb1d640 1 Processor -- start 2023-11-24T06:27:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7fb68cdeb640 1 -- 172.21.15.16:0/764870964 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582197282 0 0) 0x7fb688071690 con 0x7fb68814b2c0 2023-11-24T06:27:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7fb68cdeb640 1 -- 172.21.15.16:0/764870964 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb674003520 con 0x7fb68814b2c0 2023-11-24T06:27:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7f71aed9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a80eacd0 0x7f71a8045240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7f71aed9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a80eacd0 0x7f71a8045240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53276/0 (socket says 172.21.15.16:53276) 2023-11-24T06:27:06.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7f71aed9d640 1 -- 172.21.15.16:0/1789302865 learned_addr learned my addr 172.21.15.16:0/1789302865 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7fb68e5ee640 1 -- 172.21.15.16:0/764870964 >> v1:172.21.15.173:6789/0 conn(0x7fb6880cf9a0 legacy=0x7fb688070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.167+0000 7fb68cdeb640 1 -- 172.21.15.16:0/764870964 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 540403603 0 0) 0x7fb674003520 con 0x7fb68814b2c0 2023-11-24T06:27:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.167+0000 7fb68e5ee640 1 -- 172.21.15.16:0/764870964 >> v1:172.21.15.16:6789/0 conn(0x7fb68814b2c0 legacy=0x7fb6880cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.167+0000 7fb68e5ee640 1 -- 172.21.15.16:0/764870964 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb688071c50 con 0x7fb6880cd170 2023-11-24T06:27:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.167+0000 7f7054a09640 1 Processor -- start 2023-11-24T06:27:06.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7f71add9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a804ffa0 0x7f71a8047400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.166+0000 7f71ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 0x7f71a817cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.167+0000 7f7054a09640 1 -- start start 2023-11-24T06:27:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.168+0000 7f1934d0c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19300b7bf0 0x7f19300ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.168+0000 7f1934d0c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19300b6750 con 0x7f19300d0280 2023-11-24T06:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.168+0000 7f1934d0c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19300cf1f0 con 0x7f193015d590 2023-11-24T06:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.168+0000 7f1934d0c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19300cef10 con 0x7f19300b7bf0 2023-11-24T06:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.168+0000 7f71ae59c640 1 -- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a804ffa0 msgr2=0x7f71a8047400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.169+0000 7f192ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19300b7bf0 0x7f19300ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.167+0000 7fc04bb1d640 1 -- start start 2023-11-24T06:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.168+0000 7f71ae59c640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a804ffa0 0x7f71a8047400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.168+0000 7f71ae59c640 1 -- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a80eacd0 msgr2=0x7f71a8045240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.168+0000 7f71ae59c640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a80eacd0 0x7f71a8045240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.168+0000 7f71ae59c640 1 -- 172.21.15.16:0/1789302865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b004ee20 con 0x7f71a8006e70 2023-11-24T06:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.169+0000 7f192ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19300b7bf0 0x7f19300ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38370/0 (socket says 172.21.15.16:38370) 2023-11-24T06:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.169+0000 7f71aed9d640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a80eacd0 0x7f71a8045240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.169+0000 7f192ed9d640 1 -- 172.21.15.16:0/3783676995 learned_addr learned my addr 172.21.15.16:0/3783676995 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.169+0000 7fc04bb1d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc044153950 0x7fc0440cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.169+0000 7f71ae59c640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 0x7f71a817cac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f71a400d5f0 tx=0x7f71a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.169+0000 7fb68e5ee640 1 --2- 172.21.15.16:0/764870964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cd170 0x7fb688071070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb680009d90 tx=0x7fb68002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2092b48d7cccfc77 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.169+0000 7fc04bb1d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc04415ce90 0x7fc04415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f7054a09640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70501598d0 0x7f70500cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7fb68cdeb640 1 -- 172.21.15.16:0/764870964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678001670 con 0x7fb6880cd170 2023-11-24T06:27:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f7054a09640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f70500cf900 0x7f7050070e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f7054a09640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7050159e20 con 0x7f7050153a50 2023-11-24T06:27:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7fb68cdeb640 1 -- 172.21.15.16:0/764870964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68002c940 con 0x7fb6880cd170 2023-11-24T06:27:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379671025 0 0) 0x7f19300cf1f0 con 0x7f193015d590 2023-11-24T06:27:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f719effd640 1 -- 172.21.15.16:0/1789302865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a40125f0 con 0x7f71a8006e70 2023-11-24T06:27:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7fb68cdeb640 1 -- 172.21.15.16:0/764870964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678001670 con 0x7fb6880cd170 2023-11-24T06:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1914003520 con 0x7f193015d590 2023-11-24T06:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f719effd640 1 -- 172.21.15.16:0/1789302865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a4012750 con 0x7f71a8006e70 2023-11-24T06:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f719effd640 1 -- 172.21.15.16:0/1789302865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4012b00 con 0x7f71a8006e70 2023-11-24T06:27:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7fc0498b9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc044153950 0x7fc0440cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7fc04bb1d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04415d830 con 0x7fc0441598a0 2023-11-24T06:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7fc04bb1d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc044153d30 con 0x7fc04415ce90 2023-11-24T06:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7fc04bb1d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0440b2b70 con 0x7fc044153950 2023-11-24T06:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7fc0498b9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc044153950 0x7fc0440cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:41066/0 (socket says 172.21.15.16:41066) 2023-11-24T06:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a817d1c0 con 0x7f71a8006e70 2023-11-24T06:27:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a817d780 con 0x7f71a8006e70 2023-11-24T06:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f7054a09640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7050153e30 con 0x7f70501598d0 2023-11-24T06:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f7054a09640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70500b31c0 con 0x7f70500cf900 2023-11-24T06:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f704e59c640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f7050153a50 0x7f70500b2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:44538/0 (socket says 172.21.15.16:44538) 2023-11-24T06:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.170+0000 7f704e59c640 1 -- 172.21.15.16:0/3576896827 learned_addr learned my addr 172.21.15.16:0/3576896827 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.171+0000 7fc0498b9640 1 -- 172.21.15.16:0/4209488416 learned_addr learned my addr 172.21.15.16:0/4209488416 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.171+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761073020 0 0) 0x7f7050159e20 con 0x7f7050153a50 2023-11-24T06:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.171+0000 7fb690051640 1 -- 172.21.15.16:0/764870964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cd170 msgr2=0x7fb688071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.171+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f703c003520 con 0x7f7050153a50 2023-11-24T06:27:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.171+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 <== mon.1 v1:172.21.15.173:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867621884 0 0) 0x7f1914003520 con 0x7f193015d590 2023-11-24T06:27:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.171+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 >> v2:172.21.15.16:3300/0 conn(0x7f19300b7bf0 msgr2=0x7f19300ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.171+0000 7f192d59a640 1 --2- 172.21.15.16:0/3783676995 >> v2:172.21.15.16:3300/0 conn(0x7f19300b7bf0 0x7f19300ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.171+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 >> v1:172.21.15.16:6789/0 conn(0x7f19300d0280 legacy=0x7f19300ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.171+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 --> v1:172.21.15.173:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19300b6100 con 0x7f193015d590 2023-11-24T06:27:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7fb690051640 1 --2- 172.21.15.16:0/764870964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cd170 0x7fb688071070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb680009d90 tx=0x7fb68002be10 comp rx=0 tx=0).stop 2023-11-24T06:27:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 <== mon.1 v1:172.21.15.173:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 133337476 0 0) 0x7f703c003520 con 0x7f7050153a50 2023-11-24T06:27:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 >> v2:172.21.15.173:3300/0 conn(0x7f70500cf900 msgr2=0x7f7050070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 <== mon.1 v1:172.21.15.173:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f1920001670 con 0x7f193015d590 2023-11-24T06:27:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 >> v1:172.21.15.173:6789/0 conn(0x7f193015d590 legacy=0x7f19300cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f192d59a640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19140039c0 0x7f1914003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f192d59a640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19140047c0 0x7f1914004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7fc0498b9640 1 -- 172.21.15.16:0/4209488416 >> v2:172.21.15.16:3300/0 conn(0x7fc04415ce90 msgr2=0x7fc04415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7fc0498b9640 1 --2- 172.21.15.16:0/4209488416 >> v2:172.21.15.16:3300/0 conn(0x7fc04415ce90 0x7fc04415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f192d59a640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1914005180 0x7f191400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19300cef10 con 0x7f1914005180 2023-11-24T06:27:06.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1914014860 con 0x7f19140039c0 2023-11-24T06:27:06.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1914014a20 con 0x7f19140047c0 2023-11-24T06:27:06.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7fc0498b9640 1 -- 172.21.15.16:0/4209488416 >> v1:172.21.15.16:6789/0 conn(0x7fc0441598a0 legacy=0x7fc044159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7fc0498b9640 1 -- 172.21.15.16:0/4209488416 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0440b2e50 con 0x7fc044153950 2023-11-24T06:27:06.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f704d59a640 1 --2- 172.21.15.16:0/3576896827 >> v2:172.21.15.173:3300/0 conn(0x7f70500cf900 0x7f7050070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 >> v2:172.21.15.16:3300/0 conn(0x7f70501598d0 msgr2=0x7f70500cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f704d59a640 1 --2- 172.21.15.16:0/3576896827 >> v2:172.21.15.16:3300/0 conn(0x7f70501598d0 0x7f70500cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 --> v1:172.21.15.173:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705015a100 con 0x7f7050153a50 2023-11-24T06:27:06.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.172+0000 7fb690051640 1 -- 172.21.15.16:0/764870964 shutdown_connections 2023-11-24T06:27:06.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7fb690051640 1 --2- 172.21.15.16:0/764870964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cd170 0x7fb688071070 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7fb690051640 1 -- 172.21.15.16:0/764870964 >> 172.21.15.16:0/764870964 conn(0x7fb68806a800 msgr2=0x7fb68806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 <== mon.1 v1:172.21.15.173:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f7048004df0 con 0x7f7050153a50 2023-11-24T06:27:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 >> v1:172.21.15.173:6789/0 conn(0x7f7050153a50 legacy=0x7f70500b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7fb690051640 1 -- 172.21.15.16:0/764870964 shutdown_connections 2023-11-24T06:27:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7fb690051640 1 -- 172.21.15.16:0/764870964 wait complete. 2023-11-24T06:27:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f192e59c640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19140039c0 0x7f1914003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f192ed9d640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1914005180 0x7f191400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f192dd9b640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19140047c0 0x7f1914004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f704d59a640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 0x7f703c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f704d59a640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f703c0047c0 0x7f703c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f192ed9d640 1 -- 172.21.15.16:0/3783676995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19140047c0 msgr2=0x7f1914004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f704d59a640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703c0052a0 0x7f703c007960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050153e30 con 0x7f703c0052a0 2023-11-24T06:27:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70500b31c0 con 0x7f703c0047c0 2023-11-24T06:27:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0081e0 con 0x7f703c003970 2023-11-24T06:27:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.174+0000 7f704dd9b640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f703c0047c0 0x7f703c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.174+0000 7f704ed9d640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703c0052a0 0x7f703c007960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.174+0000 7f704e59c640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 0x7f703c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a8180090 con 0x7f71a8006e70 2023-11-24T06:27:06.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f704e59c640 1 -- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f703c0047c0 msgr2=0x7f703c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f192ed9d640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19140047c0 0x7f1914004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f192ed9d640 1 -- 172.21.15.16:0/3783676995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19140039c0 msgr2=0x7f1914003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f192ed9d640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19140039c0 0x7f1914003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.173+0000 7f192ed9d640 1 -- 172.21.15.16:0/3783676995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1914003520 con 0x7f1914005180 2023-11-24T06:27:06.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.174+0000 7f192e59c640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19140039c0 0x7f1914003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:06.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f192ed9d640 1 -- 172.21.15.16:0/3783676995 reap_dead start 2023-11-24T06:27:06.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f192ed9d640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1914005180 0x7f191400b980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f192800e190 tx=0x7f19140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21553f76424afe7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7fc0498b9640 1 --2- 172.21.15.16:0/4209488416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc044153950 0x7fc0440cb0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0440b1c10 tx=0x7fc03802ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8736c8589e5d8952 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f704e59c640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f703c0047c0 0x7f703c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f704e59c640 1 -- 172.21.15.16:0/3576896827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703c0052a0 msgr2=0x7f703c007960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f704e59c640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703c0052a0 0x7f703c007960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f704e59c640 1 -- 172.21.15.16:0/3576896827 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7050159e20 con 0x7f703c003970 2023-11-24T06:27:06.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f704ed9d640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703c0052a0 0x7f703c007960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:06.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f704ed9d640 1 -- 172.21.15.16:0/3576896827 reap_dead start 2023-11-24T06:27:06.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f704dd9b640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f703c0047c0 0x7f703c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:06.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f550c246640 1 Processor -- start 2023-11-24T06:27:06.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.176+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192802cac0 con 0x7f1914005180 2023-11-24T06:27:06.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.176+0000 7f192d59a640 1 -- 172.21.15.16:0/3783676995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192800f040 con 0x7f1914005180 2023-11-24T06:27:06.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.176+0000 7fc0488b7640 1 -- 172.21.15.16:0/4209488416 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038030cd0 con 0x7fc044153950 2023-11-24T06:27:06.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.176+0000 7fc0488b7640 1 -- 172.21.15.16:0/4209488416 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc038030e30 con 0x7fc044153950 2023-11-24T06:27:06.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.176+0000 7fc0488b7640 1 -- 172.21.15.16:0/4209488416 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038038660 con 0x7fc044153950 2023-11-24T06:27:06.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.176+0000 7f550c246640 1 -- start start 2023-11-24T06:27:06.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.175+0000 7f719effd640 1 -- 172.21.15.16:0/1789302865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f71a4012c60 con 0x7f71a8006e70 2023-11-24T06:27:06.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.176+0000 7f719effd640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f719006af10 0x7f719006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.176+0000 7f719effd640 1 -- 172.21.15.16:0/1789302865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71a40898e0 con 0x7f71a8006e70 2023-11-24T06:27:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.177+0000 7f704e59c640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 0x7f703c003d70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70480089f0 tx=0x7f703c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=320584cfc88f8abc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.177+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048008270 con 0x7f703c003970 2023-11-24T06:27:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.177+0000 7f1934d0c640 1 -- 172.21.15.16:0/3783676995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1914005180 msgr2=0x7f191400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.177+0000 7f1934d0c640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1914005180 0x7f191400b980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f192800e190 tx=0x7f19140057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7fb690051640 1 Processor -- start 2023-11-24T06:27:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7f71add9b640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f719006af10 0x7f719006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7f1934d0c640 1 -- 172.21.15.16:0/3783676995 shutdown_connections 2023-11-24T06:27:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7f1934d0c640 1 --2- 172.21.15.16:0/3783676995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1914005180 0x7f191400b980 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7f1934d0c640 1 -- 172.21.15.16:0/3783676995 >> 172.21.15.16:0/3783676995 conn(0x7f193009c540 msgr2=0x7f19300d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70480083d0 con 0x7f703c003970 2023-11-24T06:27:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7f704d59a640 1 -- 172.21.15.16:0/3576896827 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048006700 con 0x7f703c003970 2023-11-24T06:27:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f71add9b640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f719006af10 0x7f719006d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71a8047c40 tx=0x7f71a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7f1934d0c640 1 -- 172.21.15.16:0/3783676995 shutdown_connections 2023-11-24T06:27:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.177+0000 7fc04bb1d640 1 -- 172.21.15.16:0/4209488416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc044153950 msgr2=0x7fc0440cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.177+0000 7fc04bb1d640 1 --2- 172.21.15.16:0/4209488416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc044153950 0x7fc0440cb0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc0440b1c10 tx=0x7fc03802ff20 comp rx=0 tx=0).stop 2023-11-24T06:27:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7fc04bb1d640 1 -- 172.21.15.16:0/4209488416 shutdown_connections 2023-11-24T06:27:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7fc04bb1d640 1 --2- 172.21.15.16:0/4209488416 >> v2:172.21.15.16:3300/0 conn(0x7fc04415ce90 0x7fc04415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7fc04bb1d640 1 --2- 172.21.15.16:0/4209488416 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc044153950 0x7fc0440cb0d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7fc04bb1d640 1 -- 172.21.15.16:0/4209488416 >> 172.21.15.16:0/4209488416 conn(0x7fc0440b0be0 msgr2=0x7fc0440b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7fc04bb1d640 1 -- 172.21.15.16:0/4209488416 shutdown_connections 2023-11-24T06:27:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7fc04bb1d640 1 -- 172.21.15.16:0/4209488416 wait complete. 2023-11-24T06:27:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f7054a09640 1 -- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 msgr2=0x7f703c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f7054a09640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 0x7f703c003d70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70480089f0 tx=0x7f703c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f1934d0c640 1 -- 172.21.15.16:0/3783676995 wait complete. 2023-11-24T06:27:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f550c246640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f54fc048350 0x7f54fc006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f7054a09640 1 -- 172.21.15.16:0/3576896827 shutdown_connections 2023-11-24T06:27:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f7054a09640 1 --2- 172.21.15.16:0/3576896827 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 0x7f703c003d70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f7054a09640 1 -- 172.21.15.16:0/3576896827 >> 172.21.15.16:0/3576896827 conn(0x7f705006a800 msgr2=0x7f705006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f550c246640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54fc006030 0x7f54fc04f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f550c246640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc04fa00 0x7f54fc052d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f550c246640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc007460 con 0x7f54fc006030 2023-11-24T06:27:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f550c246640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc04fe90 con 0x7f54fc048350 2023-11-24T06:27:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.179+0000 7f550c246640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc054aa0 con 0x7f54fc04fa00 2023-11-24T06:27:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f55097e1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54fc006030 0x7f54fc04f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f5509fe2640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f54fc048350 0x7f54fc006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f5509fe2640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f54fc048350 0x7f54fc006f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53298/0 (socket says 172.21.15.16:53298) 2023-11-24T06:27:06.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f5509fe2640 1 -- 172.21.15.16:0/531740367 learned_addr learned my addr 172.21.15.16:0/531740367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.178+0000 7fb690051640 1 -- start start 2023-11-24T06:27:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f7054a09640 1 -- 172.21.15.16:0/3576896827 shutdown_connections 2023-11-24T06:27:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f55097e1640 1 -- 172.21.15.16:0/531740367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc04fa00 msgr2=0x7f54fc052d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f550afe4640 1 --2- 172.21.15.16:0/531740367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc04fa00 0x7f54fc052d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f55097e1640 1 --2- 172.21.15.16:0/531740367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc04fa00 0x7f54fc052d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f55097e1640 1 -- 172.21.15.16:0/531740367 >> v2:172.21.15.173:3300/0 conn(0x7f54fc048350 msgr2=0x7f54fc006f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f55097e1640 1 --2- 172.21.15.16:0/531740367 >> v2:172.21.15.173:3300/0 conn(0x7f54fc048350 0x7f54fc006f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.180+0000 7f55097e1640 1 -- 172.21.15.16:0/531740367 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0384e0 con 0x7f54fc006030 2023-11-24T06:27:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.181+0000 7fc04bb1d640 1 Processor -- start 2023-11-24T06:27:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.181+0000 7fb690051640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880cd170 0x7fb68814a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.181+0000 7fc04bb1d640 1 -- start start 2023-11-24T06:27:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.181+0000 7fb690051640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cf9a0 0x7fb68815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.182+0000 7fb690051640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb68814b2c0 0x7fb68813ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.182+0000 7fb690051640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68815fb40 con 0x7fb6880cf9a0 2023-11-24T06:27:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.182+0000 7fb690051640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6881648b0 con 0x7fb6880cd170 2023-11-24T06:27:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.182+0000 7fb690051640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb688164b90 con 0x7fb68814b2c0 2023-11-24T06:27:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.182+0000 7fb68dded640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880cd170 0x7fb68814a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.182+0000 7fb68dded640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880cd170 0x7fb68814a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53314/0 (socket says 172.21.15.16:53314) 2023-11-24T06:27:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.182+0000 7fb68dded640 1 -- 172.21.15.16:0/1161979031 learned_addr learned my addr 172.21.15.16:0/1161979031 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.181+0000 7f7054a09640 1 -- 172.21.15.16:0/3576896827 wait complete. 2023-11-24T06:27:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.182+0000 7f55097e1640 1 --2- 172.21.15.16:0/531740367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc006030 0x7f54fc04f4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00bec0 tx=0x7f54f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938fc1ce37bd1419 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.182+0000 7fb68d5ec640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cf9a0 0x7fb68815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7f1934d0c640 1 Processor -- start 2023-11-24T06:27:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fb68e5ee640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb68814b2c0 0x7fb68813ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7f1934d0c640 1 -- start start 2023-11-24T06:27:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fb68d5ec640 1 -- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb68814b2c0 msgr2=0x7fb68813ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fb68d5ec640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb68814b2c0 0x7fb68813ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fb68d5ec640 1 -- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880cd170 msgr2=0x7fb68814a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fb68d5ec640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880cd170 0x7fb68814a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fc04bb1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044153950 0x7fc044070530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fc04bb1d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0441598a0 0x7fc0441f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fc04bb1d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc04415ce90 0x7fc0440cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fc04bb1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0441f85c0 con 0x7fc044153950 2023-11-24T06:27:06.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fc04bb1d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0441f88a0 con 0x7fc0441598a0 2023-11-24T06:27:06.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fc04bb1d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0441f8b80 con 0x7fc04415ce90 2023-11-24T06:27:06.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.183+0000 7fb68d5ec640 1 -- 172.21.15.16:0/1161979031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb680009a70 con 0x7fb6880cf9a0 2023-11-24T06:27:06.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.184+0000 7f7054a09640 1 Processor -- start 2023-11-24T06:27:06.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.184+0000 7f7054a09640 1 -- start start 2023-11-24T06:27:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.184+0000 7fb68d5ec640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cf9a0 0x7fb68815eef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c340 tx=0x7fb68400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.184+0000 7fc04a0ba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc04415ce90 0x7fc0440cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.184+0000 7fc04a0ba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc04415ce90 0x7fc0440cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:41118/0 (socket says 172.21.15.16:41118) 2023-11-24T06:27:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.184+0000 7fc04a0ba640 1 -- 172.21.15.16:0/3920271717 learned_addr learned my addr 172.21.15.16:0/3920271717 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.184+0000 7fc0498b9640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044153950 0x7fc044070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.184+0000 7fc0490b8640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0441598a0 0x7fc0441f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7fc0498b9640 1 -- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc04415ce90 msgr2=0x7fc0440cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7fc0498b9640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc04415ce90 0x7fc0440cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7fc0498b9640 1 -- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0441598a0 msgr2=0x7fc0441f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7fc0498b9640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0441598a0 0x7fc0441f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7fc0498b9640 1 -- 172.21.15.16:0/3920271717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc038009d60 con 0x7fc044153950 2023-11-24T06:27:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f4948c75640 1 Processor -- start 2023-11-24T06:27:06.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f1934d0c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1914005180 0x7f19300c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f7054a09640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 0x7f70500cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f7054a09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7050149d30 0x7f70500cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f1934d0c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193015fb00 0x7f193015ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7fc0498b9640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044153950 0x7fc044070530 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc038037000 tx=0x7fc038004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f7054a09640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705014a1b0 0x7f70500ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f7054a09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70500ce640 con 0x7f7050149d30 2023-11-24T06:27:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f7054a09640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705014a710 con 0x7f705014a1b0 2023-11-24T06:27:06.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f7054a09640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705014a9f0 con 0x7f703c003970 2023-11-24T06:27:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7050149d30 0x7f70500cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7050149d30 0x7f70500cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38440/0 (socket says 172.21.15.16:38440) 2023-11-24T06:27:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704dd9b640 1 -- 172.21.15.16:0/813648157 learned_addr learned my addr 172.21.15.16:0/813648157 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f1934d0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19300c6ad0 0x7f19300c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704e59c640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 0x7f70500cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f5508fe0640 1 -- 172.21.15.16:0/531740367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f402e070 con 0x7f54fc006030 2023-11-24T06:27:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7fb67e7fc640 1 -- 172.21.15.16:0/1161979031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840052d0 con 0x7fb6880cf9a0 2023-11-24T06:27:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f192e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1914005180 0x7f19300c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f5508fe0640 1 -- 172.21.15.16:0/531740367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f40315a0 con 0x7f54fc006030 2023-11-24T06:27:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f5508fe0640 1 -- 172.21.15.16:0/531740367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4031930 con 0x7f54fc006030 2023-11-24T06:27:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6881f8b20 con 0x7fb6880cf9a0 2023-11-24T06:27:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7fb67e7fc640 1 -- 172.21.15.16:0/1161979031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb684005430 con 0x7fb6880cf9a0 2023-11-24T06:27:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704ed9d640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705014a1b0 0x7f70500ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704dd9b640 1 -- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 msgr2=0x7f70500cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704dd9b640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 0x7f70500cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704dd9b640 1 -- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705014a1b0 msgr2=0x7f70500ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704dd9b640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705014a1b0 0x7f70500ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f704dd9b640 1 -- 172.21.15.16:0/813648157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705014acd0 con 0x7f7050149d30 2023-11-24T06:27:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7f192e59c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1914005180 0x7f19300c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:53340/0 (socket says 172.21.15.16:53340) 2023-11-24T06:27:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.186+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68815f430 con 0x7fb6880cf9a0 2023-11-24T06:27:06.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7fb67e7fc640 1 -- 172.21.15.16:0/1161979031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684003830 con 0x7fb6880cf9a0 2023-11-24T06:27:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7f192dd9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193015fb00 0x7f193015ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7f192e59c640 1 -- 172.21.15.16:0/1239638340 learned_addr learned my addr 172.21.15.16:0/1239638340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19300c83d0 con 0x7f19300c6ad0 2023-11-24T06:27:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19301604c0 con 0x7f1914005180 2023-11-24T06:27:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19300c77c0 con 0x7f193015fb00 2023-11-24T06:27:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7fc0367fc640 1 -- 172.21.15.16:0/3920271717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038013390 con 0x7fc044153950 2023-11-24T06:27:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7fc0367fc640 1 -- 172.21.15.16:0/3920271717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0380134f0 con 0x7fc044153950 2023-11-24T06:27:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7fc0367fc640 1 -- 172.21.15.16:0/3920271717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038043440 con 0x7fc044153950 2023-11-24T06:27:06.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0440cbb70 con 0x7fc044153950 2023-11-24T06:27:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7f192ed9d640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19300c6ad0 0x7f19300c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc044070a70 con 0x7fc044153950 2023-11-24T06:27:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.187+0000 7f704dd9b640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7050149d30 0x7f70500cdbc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7040006c90 tx=0x7f704000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.185+0000 7f4948c75640 1 -- start start 2023-11-24T06:27:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7f192dd9b640 1 -- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1914005180 msgr2=0x7f19300c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7f192dd9b640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1914005180 0x7f19300c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7f192dd9b640 1 -- 172.21.15.16:0/1239638340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19300c6ad0 msgr2=0x7f19300c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7f192dd9b640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19300c6ad0 0x7f19300c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7f192dd9b640 1 -- 172.21.15.16:0/1239638340 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19300c7aa0 con 0x7f193015fb00 2023-11-24T06:27:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7f192e59c640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1914005180 0x7f19300c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:06.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7f192ed9d640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19300c6ad0 0x7f19300c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:06.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6881f8980 con 0x7fb6880cf9a0 2023-11-24T06:27:06.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7f192dd9b640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193015fb00 0x7f193015ff80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f191c00d480 tx=0x7f191c00d940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.188+0000 7f4948c75640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f4948c75640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f494409a170 0x7f494406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f719effd640 1 -- 172.21.15.16:0/1789302865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71a8180090 con 0x7f71a8006e70 2023-11-24T06:27:06.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f1926ffd640 1 -- 172.21.15.16:0/1239638340 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c012800 con 0x7f193015fb00 2023-11-24T06:27:06.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f1926ffd640 1 -- 172.21.15.16:0/1239638340 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c012960 con 0x7f193015fb00 2023-11-24T06:27:06.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f1926ffd640 1 -- 172.21.15.16:0/1239638340 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c012d10 con 0x7f193015fb00 2023-11-24T06:27:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f4948c75640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440cfa70 con 0x7f4944070490 2023-11-24T06:27:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f4948c75640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49440cff00 con 0x7f494409a170 2023-11-24T06:27:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f4948c75640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494406fc70 con 0x7f494415e880 2023-11-24T06:27:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f494259c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19300c7d80 con 0x7f193015fb00 2023-11-24T06:27:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19300c7490 con 0x7f193015fb00 2023-11-24T06:27:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0440675d0 con 0x7fc044153950 2023-11-24T06:27:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f4941d9b640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f4944070490 0x7f4944099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:44554/0 (socket says 172.21.15.16:44554) 2023-11-24T06:27:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.189+0000 7f494259c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494406ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:41140/0 (socket says 172.21.15.16:41140) 2023-11-24T06:27:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.190+0000 7f494259c640 1 -- 172.21.15.16:0/2439828592 learned_addr learned my addr 172.21.15.16:0/2439828592 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.190+0000 7f4942d9d640 1 --2- 172.21.15.16:0/2439828592 >> v2:172.21.15.16:3300/0 conn(0x7f494409a170 0x7f494406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.190+0000 7f494259c640 1 -- 172.21.15.16:0/2439828592 >> v2:172.21.15.16:3300/0 conn(0x7f494409a170 msgr2=0x7f494406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.190+0000 7f494259c640 1 --2- 172.21.15.16:0/2439828592 >> v2:172.21.15.16:3300/0 conn(0x7f494409a170 0x7f494406e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.190+0000 7f494259c640 1 -- 172.21.15.16:0/2439828592 >> v1:172.21.15.173:6789/0 conn(0x7f4944070490 legacy=0x7f4944099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.190+0000 7f494259c640 1 -- 172.21.15.16:0/2439828592 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440cf420 con 0x7f494415e880 2023-11-24T06:27:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.190+0000 7f4942d9d640 1 --2- 172.21.15.16:0/2439828592 >> v2:172.21.15.16:3300/0 conn(0x7f494409a170 0x7f494406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.191+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19300675d0 con 0x7f193015fb00 2023-11-24T06:27:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.191+0000 7f7046ffd640 1 -- 172.21.15.16:0/813648157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000eea0 con 0x7f7050149d30 2023-11-24T06:27:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.191+0000 7f7046ffd640 1 -- 172.21.15.16:0/813648157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7040004260 con 0x7f7050149d30 2023-11-24T06:27:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.191+0000 7f7046ffd640 1 -- 172.21.15.16:0/813648157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000c730 con 0x7f7050149d30 2023-11-24T06:27:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.191+0000 7f7054a09640 1 -- 172.21.15.16:0/813648157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705014afb0 con 0x7f7050149d30 2023-11-24T06:27:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.191+0000 7f7054a09640 1 -- 172.21.15.16:0/813648157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705014c150 con 0x7f7050149d30 2023-11-24T06:27:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.191+0000 7f550c246640 1 -- 172.21.15.16:0/531740367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc006030 msgr2=0x7f54fc04f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.191+0000 7f550c246640 1 --2- 172.21.15.16:0/531740367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc006030 0x7f54fc04f4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00bec0 tx=0x7f54f402b840 comp rx=0 tx=0).stop 2023-11-24T06:27:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.192+0000 7f550c246640 1 -- 172.21.15.16:0/531740367 shutdown_connections 2023-11-24T06:27:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.192+0000 7f550c246640 1 --2- 172.21.15.16:0/531740367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc04fa00 0x7f54fc052d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.192+0000 7f550c246640 1 --2- 172.21.15.16:0/531740367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc006030 0x7f54fc04f4c0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00bec0 tx=0x7f54f402b840 comp rx=0 tx=0).stop 2023-11-24T06:27:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.192+0000 7f550c246640 1 --2- 172.21.15.16:0/531740367 >> v2:172.21.15.173:3300/0 conn(0x7f54fc048350 0x7f54fc006f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.192+0000 7f550c246640 1 -- 172.21.15.16:0/531740367 >> 172.21.15.16:0/531740367 conn(0x7f54fc054c60 msgr2=0x7f54fc055070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.192+0000 7fb67e7fc640 1 -- 172.21.15.16:0/1161979031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb684003990 con 0x7fb6880cf9a0 2023-11-24T06:27:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.193+0000 7fb67e7fc640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb66c06af10 0x7fb66c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.193+0000 7fb67e7fc640 1 -- 172.21.15.16:0/1161979031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6840890e0 con 0x7fb6880cf9a0 2023-11-24T06:27:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.193+0000 7f7044ff9640 1 -- 172.21.15.16:0/813648157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7028005250 con 0x7f7050149d30 2023-11-24T06:27:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.194+0000 7fb68dded640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb66c06af10 0x7fb66c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.195+0000 7fc0367fc640 1 -- 172.21.15.16:0/3920271717 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc038038440 con 0x7fc044153950 2023-11-24T06:27:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.195+0000 7f494259c640 1 --2- 172.21.15.16:0/2439828592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494406ff50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4934009870 tx=0x7f493402ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2364921831d52a8a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.195+0000 7f1926ffd640 1 -- 172.21.15.16:0/1239638340 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f191c025430 con 0x7f193015fb00 2023-11-24T06:27:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.195+0000 7f550c246640 1 -- 172.21.15.16:0/531740367 shutdown_connections 2023-11-24T06:27:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.196+0000 7fb68dded640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb66c06af10 0x7fb66c06d3d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb678001500 tx=0x7fb678001490 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.196+0000 7f1926ffd640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f191806ae90 0x7f191806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.197+0000 7fc0367fc640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc02806af10 0x7fc02806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.197+0000 7fc0367fc640 1 -- 172.21.15.16:0/3920271717 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0380abaa0 con 0x7fc044153950 2023-11-24T06:27:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.197+0000 7f1926ffd640 1 -- 172.21.15.16:0/1239638340 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f191c0899e0 con 0x7f193015fb00 2023-11-24T06:27:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.196+0000 7f550c246640 1 -- 172.21.15.16:0/531740367 wait complete. 2023-11-24T06:27:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.198+0000 7f192e59c640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f191806ae90 0x7f191806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.198+0000 7fc0490b8640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc02806af10 0x7fc02806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.198+0000 7f550c246640 1 Processor -- start 2023-11-24T06:27:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.199+0000 7fb67e7fc640 1 -- 172.21.15.16:0/1161979031 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb684054220 con 0x7fb6880cf9a0 2023-11-24T06:27:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.199+0000 7f192e59c640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f191806ae90 0x7f191806d350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1920001b50 tx=0x7f1920009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.199+0000 7fc0490b8640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc02806af10 0x7fc02806d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc044157380 tx=0x7fc040009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.198+0000 7f550c246640 1 -- start start 2023-11-24T06:27:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.202+0000 7f550c246640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc048350 0x7f54fc0d4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.202+0000 7f550c246640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc04fa00 0x7f54fc17ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.202+0000 7f550c246640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc0de5e0 0x7f54fc0ea300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.202+0000 7f550c246640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc17f1e0 con 0x7f54fc0de5e0 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.202+0000 7f550c246640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc17e300 con 0x7f54fc04fa00 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.202+0000 7f550c246640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc17e5d0 con 0x7f54fc048350 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f550afe4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc0de5e0 0x7f54fc0ea300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f550afe4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc0de5e0 0x7f54fc0ea300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38462/0 (socket says 172.21.15.16:38462) 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f550afe4640 1 -- 172.21.15.16:0/1616337290 learned_addr learned my addr 172.21.15.16:0/1616337290 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f5509fe2640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc048350 0x7f54fc0d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f55097e1640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc04fa00 0x7f54fc17ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f550afe4640 1 -- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc048350 msgr2=0x7f54fc0d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f550afe4640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc048350 0x7f54fc0d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f550afe4640 1 -- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc04fa00 msgr2=0x7f54fc17ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f550afe4640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc04fa00 0x7f54fc17ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f550afe4640 1 -- 172.21.15.16:0/1616337290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f4009400 con 0x7f54fc0de5e0 2023-11-24T06:27:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.203+0000 7f550afe4640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc0de5e0 0x7f54fc0ea300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f550404f5f0 tx=0x7f55040748f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.207+0000 7f494159a640 1 -- 172.21.15.16:0/2439828592 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493402f830 con 0x7f494415e880 2023-11-24T06:27:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.207+0000 7f494159a640 1 -- 172.21.15.16:0/2439828592 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493402f990 con 0x7f494415e880 2023-11-24T06:27:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.207+0000 7f494159a640 1 -- 172.21.15.16:0/2439828592 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493402fd20 con 0x7f494415e880 2023-11-24T06:27:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.208+0000 7fc0367fc640 1 -- 172.21.15.16:0/3920271717 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc038076be0 con 0x7fc044153950 2023-11-24T06:27:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.208+0000 7f4948c75640 1 -- 172.21.15.16:0/2439828592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 msgr2=0x7f494406ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.208+0000 7f4948c75640 1 --2- 172.21.15.16:0/2439828592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494406ff50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4934009870 tx=0x7f493402ba10 comp rx=0 tx=0).stop 2023-11-24T06:27:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.208+0000 7f1926ffd640 1 -- 172.21.15.16:0/1239638340 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f191c054bb0 con 0x7f193015fb00 2023-11-24T06:27:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.208+0000 7f4948c75640 1 -- 172.21.15.16:0/2439828592 shutdown_connections 2023-11-24T06:27:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.209+0000 7f4948c75640 1 --2- 172.21.15.16:0/2439828592 >> v2:172.21.15.16:3300/0 conn(0x7f494409a170 0x7f494406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.209+0000 7f4948c75640 1 --2- 172.21.15.16:0/2439828592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494406ff50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.209+0000 7f4948c75640 1 -- 172.21.15.16:0/2439828592 >> 172.21.15.16:0/2439828592 conn(0x7f49440b2d00 msgr2=0x7f49440b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.209+0000 7f54fa7fc640 1 -- 172.21.15.16:0/1616337290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504072b20 con 0x7f54fc0de5e0 2023-11-24T06:27:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.209+0000 7f54fa7fc640 1 -- 172.21.15.16:0/1616337290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504071070 con 0x7f54fc0de5e0 2023-11-24T06:27:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.209+0000 7f54fa7fc640 1 -- 172.21.15.16:0/1616337290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550406b660 con 0x7f54fc0de5e0 2023-11-24T06:27:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.211+0000 7f4948c75640 1 -- 172.21.15.16:0/2439828592 shutdown_connections 2023-11-24T06:27:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.211+0000 7f7046ffd640 1 -- 172.21.15.16:0/813648157 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f704000c890 con 0x7f7050149d30 2023-11-24T06:27:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.209+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc17e8b0 con 0x7f54fc0de5e0 2023-11-24T06:27:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.210+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc17ee70 con 0x7f54fc0de5e0 2023-11-24T06:27:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.211+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec005250 con 0x7f54fc0de5e0 2023-11-24T06:27:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.212+0000 7f7046ffd640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006af10 0x7f703006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.213+0000 7f7046ffd640 1 -- 172.21.15.16:0/813648157 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7040089900 con 0x7f7050149d30 2023-11-24T06:27:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.211+0000 7f4948c75640 1 -- 172.21.15.16:0/2439828592 wait complete. 2023-11-24T06:27:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.213+0000 7f704e59c640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006af10 0x7f703006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.214+0000 7f4948c75640 1 Processor -- start 2023-11-24T06:27:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.214+0000 7f4948c75640 1 -- start start 2023-11-24T06:27:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.215+0000 7f704e59c640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006af10 0x7f703006d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7048005450 tx=0x7f7048030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.216+0000 7f4948c75640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4944070490 0x7f4944158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.216+0000 7f4948c75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f494409a170 0x7f49440ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.216+0000 7f4948c75640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494414a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.216+0000 7f4948c75640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494414b0c0 con 0x7f494409a170 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.216+0000 7f4948c75640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49441589a0 con 0x7f4944070490 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.216+0000 7f4948c75640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4944159370 con 0x7f494415e880 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.217+0000 7f4941d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f494409a170 0x7f49440ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.217+0000 7f4941d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f494409a170 0x7f49440ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38472/0 (socket says 172.21.15.16:38472) 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.217+0000 7f4942d9d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494414a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.217+0000 7f4942d9d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494414a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:41152/0 (socket says 172.21.15.16:41152) 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.217+0000 7f4942d9d640 1 -- 172.21.15.16:0/197564849 learned_addr learned my addr 172.21.15.16:0/197564849 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.217+0000 7f494259c640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4944070490 0x7f4944158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.219+0000 7f4942d9d640 1 -- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4944070490 msgr2=0x7f4944158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.219+0000 7f4942d9d640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4944070490 0x7f4944158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.219+0000 7f4942d9d640 1 -- 172.21.15.16:0/197564849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f494409a170 msgr2=0x7f49440ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.219+0000 7f4942d9d640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f494409a170 0x7f49440ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.219+0000 7f4942d9d640 1 -- 172.21.15.16:0/197564849 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c009a70 con 0x7f494415e880 2023-11-24T06:27:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.219+0000 7f4941d9b640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f494409a170 0x7f49440ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.220+0000 7f7046ffd640 1 -- 172.21.15.16:0/813648157 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7040054ad0 con 0x7f7050149d30 2023-11-24T06:27:06.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.220+0000 7f4942d9d640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494414a0f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f493c009eb0 tx=0x7f493c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.221+0000 7f54fa7fc640 1 -- 172.21.15.16:0/1616337290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f550406b880 con 0x7f54fc0de5e0 2023-11-24T06:27:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.221+0000 7f54fa7fc640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54e806ae90 0x7f54e806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.223+0000 7f5509fe2640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54e806ae90 0x7f54e806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.224+0000 7f54fa7fc640 1 -- 172.21.15.16:0/1616337290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55040d2a20 con 0x7f54fc0de5e0 2023-11-24T06:27:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.225+0000 7f5509fe2640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54e806ae90 0x7f54e806d350 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f550000ba10 tx=0x7f5500008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.225+0000 7f4932ffd640 1 -- 172.21.15.16:0/197564849 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c009d50 con 0x7f494415e880 2023-11-24T06:27:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.225+0000 7f4932ffd640 1 -- 172.21.15.16:0/197564849 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c00f040 con 0x7f494415e880 2023-11-24T06:27:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.225+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49340094d0 con 0x7f494415e880 2023-11-24T06:27:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.226+0000 7f4932ffd640 1 -- 172.21.15.16:0/197564849 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c0146a0 con 0x7f494415e880 2023-11-24T06:27:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.226+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494414ade0 con 0x7f494415e880 2023-11-24T06:27:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.228+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49441581f0 con 0x7f494415e880 2023-11-24T06:27:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.230+0000 7f4932ffd640 1 -- 172.21.15.16:0/197564849 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f493c014800 con 0x7f494415e880 2023-11-24T06:27:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.232+0000 7f54fa7fc640 1 -- 172.21.15.16:0/1616337290 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55040ced60 con 0x7f54fc0de5e0 2023-11-24T06:27:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.232+0000 7f4932ffd640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492406ae90 0x7f492406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.233+0000 7f4932ffd640 1 -- 172.21.15.16:0/197564849 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f493c08a4a0 con 0x7f494415e880 2023-11-24T06:27:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.235+0000 7f494259c640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492406ae90 0x7f492406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.236+0000 7f494259c640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492406ae90 0x7f492406d350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4934036000 tx=0x7f4934008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.245+0000 7f4932ffd640 1 -- 172.21.15.16:0/197564849 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49441581f0 con 0x7f494415e880 2023-11-24T06:27:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.552+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2dac005500 con 0x7f2dc40e3120 2023-11-24T06:27:06.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.553+0000 7f2dc8ff9640 1 -- 172.21.15.16:0/3864774869 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2db80c3e90 con 0x7f2dc40e3120 2023-11-24T06:27:06.558 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-11-24T06:27:06.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.557+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2db006ae90 msgr2=0x7f2db006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.557+0000 7f2dd26bb640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2db006ae90 0x7f2db006d350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2dc403dd80 tx=0x7f2dc0009040 comp rx=0 tx=0).stop 2023-11-24T06:27:06.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.558+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc40e3120 msgr2=0x7f2dc40e7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.558+0000 7f2dd26bb640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc40e3120 0x7f2dc40e7150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2db800c400 tx=0x7f2db800c8c0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.558+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 shutdown_connections 2023-11-24T06:27:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.558+0000 7f2dd26bb640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2db006ae90 0x7f2db006d350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.559+0000 7f2dd26bb640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc40e3120 0x7f2dc40e7150 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.559+0000 7f2dd26bb640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc4053c70 0x7f2dc40e6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.559+0000 7f2dd26bb640 1 --2- 172.21.15.16:0/3864774869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc403c060 0x7f2dc40d9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.559+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 >> 172.21.15.16:0/3864774869 conn(0x7f2dc4058ea0 msgr2=0x7f2dc40e23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.559+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 shutdown_connections 2023-11-24T06:27:06.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.560+0000 7f2dd26bb640 1 -- 172.21.15.16:0/3864774869 wait complete. 2023-11-24T06:27:06.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.3 2023-11-24T06:27:06.590 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:06.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.595+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f71a8180090 con 0x7f71a8006e70 2023-11-24T06:27:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.596+0000 7f719effd640 1 -- 172.21.15.16:0/1789302865 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71a8180090 con 0x7f71a8006e70 2023-11-24T06:27:06.598 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-11-24T06:27:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.598+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f719006af10 msgr2=0x7f719006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.598+0000 7f71af59e640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f719006af10 0x7f719006d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71a8047c40 tx=0x7f71a0009040 comp rx=0 tx=0).stop 2023-11-24T06:27:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.598+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 msgr2=0x7f71a817cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.598+0000 7f71af59e640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 0x7f71a817cac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f71a400d5f0 tx=0x7f71a400dab0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.599+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 shutdown_connections 2023-11-24T06:27:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.599+0000 7f71af59e640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f719006af10 0x7f719006d3d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.599+0000 7f71af59e640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71a80eacd0 0x7f71a8045240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.599+0000 7f71af59e640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71a804ffa0 0x7f71a8047400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.599+0000 7f71af59e640 1 --2- 172.21.15.16:0/1789302865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71a8006e70 0x7f71a817cac0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.599+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 >> 172.21.15.16:0/1789302865 conn(0x7f71a8009f80 msgr2=0x7f71a803a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.599+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 shutdown_connections 2023-11-24T06:27:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.599+0000 7f71af59e640 1 -- 172.21.15.16:0/1789302865 wait complete. 2023-11-24T06:27:06.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.1 2023-11-24T06:27:06.618 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:06.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.622+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f19300b53b0 con 0x7f193015fb00 2023-11-24T06:27:06.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.623+0000 7f1926ffd640 1 -- 172.21.15.16:0/1239638340 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f191c0581d0 con 0x7f193015fb00 2023-11-24T06:27:06.625 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f191806ae90 msgr2=0x7f191806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f191806ae90 0x7f191806d350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1920001b50 tx=0x7f1920009040 comp rx=0 tx=0).stop 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193015fb00 msgr2=0x7f193015ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193015fb00 0x7f193015ff80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f191c00d480 tx=0x7f191c00d940 comp rx=0 tx=0).stop 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 shutdown_connections 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f191806ae90 0x7f191806d350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19300c6ad0 0x7f19300c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f193015fb00 0x7f193015ff80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 --2- 172.21.15.16:0/1239638340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1914005180 0x7f19300c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.626+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 >> 172.21.15.16:0/1239638340 conn(0x7f193009c540 msgr2=0x7f19300b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.627+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 shutdown_connections 2023-11-24T06:27:06.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.627+0000 7f1934d0c640 1 -- 172.21.15.16:0/1239638340 wait complete. 2023-11-24T06:27:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.631+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb6881f9040 con 0x7fb6880cf9a0 2023-11-24T06:27:06.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.631+0000 7fb67e7fc640 1 -- 172.21.15.16:0/1161979031 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6881f9040 con 0x7fb6880cf9a0 2023-11-24T06:27:06.633 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-11-24T06:27:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb66c06af10 msgr2=0x7fb66c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb66c06af10 0x7fb66c06d3d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb678001500 tx=0x7fb678001490 comp rx=0 tx=0).stop 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cf9a0 msgr2=0x7fb68815eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cf9a0 0x7fb68815eef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c340 tx=0x7fb68400c800 comp rx=0 tx=0).stop 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 shutdown_connections 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb66c06af10 0x7fb66c06d3d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb68814b2c0 0x7fb68813ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6880cf9a0 0x7fb68815eef0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 --2- 172.21.15.16:0/1161979031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880cd170 0x7fb68814a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 >> 172.21.15.16:0/1161979031 conn(0x7fb68806a800 msgr2=0x7fb6881612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 shutdown_connections 2023-11-24T06:27:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.635+0000 7fb690051640 1 -- 172.21.15.16:0/1161979031 wait complete. 2023-11-24T06:27:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.639+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc0440b43c0 con 0x7fc044153950 2023-11-24T06:27:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.640+0000 7fc0367fc640 1 -- 172.21.15.16:0/3920271717 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc038048020 con 0x7fc044153950 2023-11-24T06:27:06.642 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-11-24T06:27:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.643+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc02806af10 msgr2=0x7fc02806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.643+0000 7fc04bb1d640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc02806af10 0x7fc02806d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc044157380 tx=0x7fc040009060 comp rx=0 tx=0).stop 2023-11-24T06:27:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.643+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044153950 msgr2=0x7fc044070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.643+0000 7fc04bb1d640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044153950 0x7fc044070530 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc038037000 tx=0x7fc038004730 comp rx=0 tx=0).stop 2023-11-24T06:27:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.644+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 shutdown_connections 2023-11-24T06:27:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.644+0000 7fc04bb1d640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc02806af10 0x7fc02806d3d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.644+0000 7fc04bb1d640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc04415ce90 0x7fc0440cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.644+0000 7fc04bb1d640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0441598a0 0x7fc0441f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.644+0000 7fc04bb1d640 1 --2- 172.21.15.16:0/3920271717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc044153950 0x7fc044070530 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.644+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 >> 172.21.15.16:0/3920271717 conn(0x7fc0440b0be0 msgr2=0x7fc044158200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.644+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 shutdown_connections 2023-11-24T06:27:06.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.644+0000 7fc04bb1d640 1 -- 172.21.15.16:0/3920271717 wait complete. 2023-11-24T06:27:06.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.2 2023-11-24T06:27:06.648 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:06.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.650+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f49441581f0 con 0x7f494415e880 2023-11-24T06:27:06.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.651+0000 7f7044ff9640 1 -- 172.21.15.16:0/813648157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f70280057d0 con 0x7f7050149d30 2023-11-24T06:27:06.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.651+0000 7f7046ffd640 1 -- 172.21.15.16:0/813648157 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7040016020 con 0x7f7050149d30 2023-11-24T06:27:06.653 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-11-24T06:27:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.651+0000 7f4932ffd640 1 -- 172.21.15.16:0/197564849 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f493c018020 con 0x7f494415e880 2023-11-24T06:27:06.655 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-11-24T06:27:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.653+0000 7f7044ff9640 1 -- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006af10 msgr2=0x7f703006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.653+0000 7f7044ff9640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006af10 0x7f703006d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7048005450 tx=0x7f7048030000 comp rx=0 tx=0).stop 2023-11-24T06:27:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.653+0000 7f7044ff9640 1 -- 172.21.15.16:0/813648157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7050149d30 msgr2=0x7f70500cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.653+0000 7f7044ff9640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7050149d30 0x7f70500cdbc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7040006c90 tx=0x7f704000ec50 comp rx=0 tx=0).stop 2023-11-24T06:27:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.654+0000 7f7044ff9640 1 -- 172.21.15.16:0/813648157 shutdown_connections 2023-11-24T06:27:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.654+0000 7f7044ff9640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006af10 0x7f703006d3d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.654+0000 7f7044ff9640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f705014a1b0 0x7f70500ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.654+0000 7f7044ff9640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7050149d30 0x7f70500cdbc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.654+0000 7f7044ff9640 1 --2- 172.21.15.16:0/813648157 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f703c003970 0x7f70500cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.654+0000 7f7044ff9640 1 -- 172.21.15.16:0/813648157 >> 172.21.15.16:0/813648157 conn(0x7f705006a800 msgr2=0x7f70500b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.654+0000 7f7044ff9640 1 -- 172.21.15.16:0/813648157 shutdown_connections 2023-11-24T06:27:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.654+0000 7f7044ff9640 1 -- 172.21.15.16:0/813648157 wait complete. 2023-11-24T06:27:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492406ae90 msgr2=0x7f492406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492406ae90 0x7f492406d350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4934036000 tx=0x7f4934008030 comp rx=0 tx=0).stop 2023-11-24T06:27:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 msgr2=0x7f494414a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494414a0f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f493c009eb0 tx=0x7f493c00c490 comp rx=0 tx=0).stop 2023-11-24T06:27:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 shutdown_connections 2023-11-24T06:27:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492406ae90 0x7f492406d350 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494415e880 0x7f494414a0f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f494409a170 0x7f49440ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 --2- 172.21.15.16:0/197564849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4944070490 0x7f4944158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 >> 172.21.15.16:0/197564849 conn(0x7f49440b2d00 msgr2=0x7f494415dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 shutdown_connections 2023-11-24T06:27:06.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.655+0000 7f4948c75640 1 -- 172.21.15.16:0/197564849 wait complete. 2023-11-24T06:27:06.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.659+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f54ec0050c0 con 0x7f54fc0de5e0 2023-11-24T06:27:06.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.659+0000 7f54fa7fc640 1 -- 172.21.15.16:0/1616337290 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55040d2380 con 0x7f54fc0de5e0 2023-11-24T06:27:06.662 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-11-24T06:27:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54e806ae90 msgr2=0x7f54e806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54e806ae90 0x7f54e806d350 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f550000ba10 tx=0x7f5500008040 comp rx=0 tx=0).stop 2023-11-24T06:27:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc0de5e0 msgr2=0x7f54fc0ea300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc0de5e0 0x7f54fc0ea300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f550404f5f0 tx=0x7f55040748f0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 shutdown_connections 2023-11-24T06:27:06.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54e806ae90 0x7f54e806d350 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54fc0de5e0 0x7f54fc0ea300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc04fa00 0x7f54fc17ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 --2- 172.21.15.16:0/1616337290 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54fc048350 0x7f54fc0d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 >> 172.21.15.16:0/1616337290 conn(0x7f54fc054c60 msgr2=0x7f54fc039e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 shutdown_connections 2023-11-24T06:27:06.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.662+0000 7f550c246640 1 -- 172.21.15.16:0/1616337290 wait complete. 2023-11-24T06:27:06.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.6 2023-11-24T06:27:06.666 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:06.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.5 2023-11-24T06:27:06.670 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:06.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.4 2023-11-24T06:27:06.675 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:06.677 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.0 2023-11-24T06:27:06.677 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:06.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.7 2023-11-24T06:27:06.681 DEBUG:teuthology.parallel:result is None 2023-11-24T06:27:06.681 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:27:06.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.939+0000 7f071f567640 1 Processor -- start 2023-11-24T06:27:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.939+0000 7f071f567640 1 -- start start 2023-11-24T06:27:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.941+0000 7f071f567640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f071806a850 0x7f07180ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.941+0000 7f071f567640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07180b3970 con 0x7f071806a3e0 2023-11-24T06:27:06.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.941+0000 7f071f567640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07180b3c50 con 0x7f0718146a40 2023-11-24T06:27:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.941+0000 7f071f567640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07180b3f30 con 0x7f071806a850 2023-11-24T06:27:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.941+0000 7f071dd64640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f071806a3e0 0x7f07180acb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52872/0 (socket says 172.21.15.16:52872) 2023-11-24T06:27:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.941+0000 7f071dd64640 1 -- 172.21.15.16:0/2976722390 learned_addr learned my addr 172.21.15.16:0/2976722390 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.941+0000 7f071ed66640 1 --2- 172.21.15.16:0/2976722390 >> v2:172.21.15.16:3300/0 conn(0x7f071806a850 0x7f07180ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.942+0000 7f071ed66640 1 -- 172.21.15.16:0/2976722390 >> v1:172.21.15.173:6789/0 conn(0x7f0718146a40 legacy=0x7f071806db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.942+0000 7f071ed66640 1 -- 172.21.15.16:0/2976722390 >> v1:172.21.15.16:6789/0 conn(0x7f071806a3e0 legacy=0x7f07180acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.942+0000 7f071ed66640 1 -- 172.21.15.16:0/2976722390 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07180b4210 con 0x7f071806a850 2023-11-24T06:27:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.943+0000 7f071ed66640 1 --2- 172.21.15.16:0/2976722390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071806a850 0x7f07180ad220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f07100097a0 tx=0x7f071002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae84e9a876328c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.943+0000 7f071d563640 1 -- 172.21.15.16:0/2976722390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710031830 con 0x7f071806a850 2023-11-24T06:27:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.943+0000 7f071d563640 1 -- 172.21.15.16:0/2976722390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0710031990 con 0x7f071806a850 2023-11-24T06:27:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.943+0000 7f071d563640 1 -- 172.21.15.16:0/2976722390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710031d20 con 0x7f071806a850 2023-11-24T06:27:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.944+0000 7f071f567640 1 -- 172.21.15.16:0/2976722390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071806a850 msgr2=0x7f07180ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.944+0000 7f071f567640 1 --2- 172.21.15.16:0/2976722390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071806a850 0x7f07180ad220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f07100097a0 tx=0x7f071002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.944+0000 7f071f567640 1 -- 172.21.15.16:0/2976722390 shutdown_connections 2023-11-24T06:27:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.944+0000 7f071f567640 1 --2- 172.21.15.16:0/2976722390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071806a850 0x7f07180ad220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.944+0000 7f071f567640 1 -- 172.21.15.16:0/2976722390 >> 172.21.15.16:0/2976722390 conn(0x7f07180cffb0 msgr2=0x7f07180d0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.944+0000 7f071f567640 1 -- 172.21.15.16:0/2976722390 shutdown_connections 2023-11-24T06:27:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.944+0000 7f071f567640 1 -- 172.21.15.16:0/2976722390 wait complete. 2023-11-24T06:27:06.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.946+0000 7f071f567640 1 Processor -- start 2023-11-24T06:27:06.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.946+0000 7f071f567640 1 -- start start 2023-11-24T06:27:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.947+0000 7f071f567640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f071806a3e0 0x7f07181e7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.947+0000 7f071f567640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071806a850 0x7f07181419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.947+0000 7f071f567640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0718146a40 0x7f0718141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.947+0000 7f071f567640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071815d670 con 0x7f0718146a40 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.947+0000 7f071f567640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071812d7b0 con 0x7f071806a850 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.947+0000 7f071f567640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07181e4640 con 0x7f071806a3e0 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.947+0000 7f071ed66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0718146a40 0x7f0718141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.947+0000 7f071ed66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0718146a40 0x7f0718141ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38490/0 (socket says 172.21.15.16:38490) 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.947+0000 7f071ed66640 1 -- 172.21.15.16:0/1199794974 learned_addr learned my addr 172.21.15.16:0/1199794974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071ed66640 1 -- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f071806a3e0 msgr2=0x7f07181e7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071ed66640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f071806a3e0 0x7f07181e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071ed66640 1 -- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071806a850 msgr2=0x7f07181419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071dd64640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071806a850 0x7f07181419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071ed66640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071806a850 0x7f07181419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071ed66640 1 -- 172.21.15.16:0/1199794974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0710009400 con 0x7f0718146a40 2023-11-24T06:27:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071dd64640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071806a850 0x7f07181419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071ed66640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0718146a40 0x7f0718141ee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f07100097a0 tx=0x7f07100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f0716ffd640 1 -- 172.21.15.16:0/1199794974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071003a9e0 con 0x7f0718146a40 2023-11-24T06:27:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f0716ffd640 1 -- 172.21.15.16:0/1199794974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071003ab40 con 0x7f0718146a40 2023-11-24T06:27:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f0716ffd640 1 -- 172.21.15.16:0/1199794974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07100443f0 con 0x7f0718146a40 2023-11-24T06:27:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0718130620 con 0x7f0718146a40 2023-11-24T06:27:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.948+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071812dce0 con 0x7f0718146a40 2023-11-24T06:27:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.950+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0718067b60 con 0x7f0718146a40 2023-11-24T06:27:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.950+0000 7f0716ffd640 1 -- 172.21.15.16:0/1199794974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f071004d450 con 0x7f0718146a40 2023-11-24T06:27:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.952+0000 7f0716ffd640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f06fc06ae40 0x7f06fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.952+0000 7f0716ffd640 1 -- 172.21.15.16:0/1199794974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07100ab7f0 con 0x7f0718146a40 2023-11-24T06:27:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.953+0000 7f071e565640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f06fc06ae40 0x7f06fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.954+0000 7f071e565640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f06fc06ae40 0x7f06fc06d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07181f5ef0 tx=0x7f070c0079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:06.957+0000 7f0716ffd640 1 -- 172.21.15.16:0/1199794974 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0710076890 con 0x7f0718146a40 2023-11-24T06:27:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.139+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f071806ac60 con 0x7f06fc06ae40 2023-11-24T06:27:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.159+0000 7f0716ffd640 1 -- 172.21.15.16:0/1199794974 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264380 (secure 0 0 0) 0x7f070c005030 con 0x7f06fc06ae40 2023-11-24T06:27:07.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:27:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f06fc06ae40 msgr2=0x7f06fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f06fc06ae40 0x7f06fc06d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07181f5ef0 tx=0x7f070c0079b0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0718146a40 msgr2=0x7f0718141ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0718146a40 0x7f0718141ee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f07100097a0 tx=0x7f07100046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 shutdown_connections 2023-11-24T06:27:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f06fc06ae40 0x7f06fc06d300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0718146a40 0x7f0718141ee0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f071806a850 0x7f07181419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 --2- 172.21.15.16:0/1199794974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f071806a3e0 0x7f07181e7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.164+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 >> 172.21.15.16:0/1199794974 conn(0x7f07180cffb0 msgr2=0x7f07180b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.165+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 shutdown_connections 2023-11-24T06:27:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.165+0000 7f071f567640 1 -- 172.21.15.16:0/1199794974 wait complete. 2023-11-24T06:27:07.187 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T06:27:06.103821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235876,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738844,"statfs":{"total":773094113280,"available":772852576256,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.163291"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948224+0000","last_change":"2023-11-24T06:26:56.969865+0000","last_active":"2023-11-24T06:26:57.948224+0000","last_peered":"2023-11-24T06:26:57.948224+0000","last_clean":"2023-11-24T06:26:57.948224+0000","last_became_active":"2023-11-24T06:26:56.969570+0000","last_became_peered":"2023-11-24T06:26:56.969570+0000","last_unstale":"2023-11-24T06:26:57.948224+0000","last_undegraded":"2023-11-24T06:26:57.948224+0000","last_fullsized":"2023-11-24T06:26:57.948224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:56.199949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944608+0000","last_change":"2023-11-24T06:26:56.979241+0000","last_active":"2023-11-24T06:26:57.944608+0000","last_peered":"2023-11-24T06:26:57.944608+0000","last_clean":"2023-11-24T06:26:57.944608+0000","last_became_active":"2023-11-24T06:26:56.978864+0000","last_became_peered":"2023-11-24T06:26:56.978864+0000","last_unstale":"2023-11-24T06:26:57.944608+0000","last_undegraded":"2023-11-24T06:26:57.944608+0000","last_fullsized":"2023-11-24T06:26:57.944608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218032+0000","last_change":"2023-11-24T06:26:56.976436+0000","last_active":"2023-11-24T06:26:58.218032+0000","last_peered":"2023-11-24T06:26:58.218032+0000","last_clean":"2023-11-24T06:26:58.218032+0000","last_became_active":"2023-11-24T06:26:56.976114+0000","last_became_peered":"2023-11-24T06:26:56.976114+0000","last_unstale":"2023-11-24T06:26:58.218032+0000","last_undegraded":"2023-11-24T06:26:58.218032+0000","last_fullsized":"2023-11-24T06:26:58.218032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222085+0000","last_change":"2023-11-24T06:26:56.992564+0000","last_active":"2023-11-24T06:26:58.222085+0000","last_peered":"2023-11-24T06:26:58.222085+0000","last_clean":"2023-11-24T06:26:58.222085+0000","last_became_active":"2023-11-24T06:26:56.992045+0000","last_became_peered":"2023-11-24T06:26:56.992045+0000","last_unstale":"2023-11-24T06:26:58.222085+0000","last_undegraded":"2023-11-24T06:26:58.222085+0000","last_fullsized":"2023-11-24T06:26:58.222085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947424+0000","last_change":"2023-11-24T06:26:56.964941+0000","last_active":"2023-11-24T06:26:57.947424+0000","last_peered":"2023-11-24T06:26:57.947424+0000","last_clean":"2023-11-24T06:26:57.947424+0000","last_became_active":"2023-11-24T06:26:56.964244+0000","last_became_peered":"2023-11-24T06:26:56.964244+0000","last_unstale":"2023-11-24T06:26:57.947424+0000","last_undegraded":"2023-11-24T06:26:57.947424+0000","last_fullsized":"2023-11-24T06:26:57.947424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219175+0000","last_change":"2023-11-24T06:26:56.993038+0000","last_active":"2023-11-24T06:26:58.219175+0000","last_peered":"2023-11-24T06:26:58.219175+0000","last_clean":"2023-11-24T06:26:58.219175+0000","last_became_active":"2023-11-24T06:26:56.986387+0000","last_became_peered":"2023-11-24T06:26:56.986387+0000","last_unstale":"2023-11-24T06:26:58.219175+0000","last_undegraded":"2023-11-24T06:26:58.219175+0000","last_fullsized":"2023-11-24T06:26:58.219175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946794+0000","last_change":"2023-11-24T06:26:56.977591+0000","last_active":"2023-11-24T06:26:57.946794+0000","last_peered":"2023-11-24T06:26:57.946794+0000","last_clean":"2023-11-24T06:26:57.946794+0000","last_became_active":"2023-11-24T06:26:56.976851+0000","last_became_peered":"2023-11-24T06:26:56.976851+0000","last_unstale":"2023-11-24T06:26:57.946794+0000","last_undegraded":"2023-11-24T06:26:57.946794+0000","last_fullsized":"2023-11-24T06:26:57.946794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944925+0000","last_change":"2023-11-24T06:26:56.991219+0000","last_active":"2023-11-24T06:26:57.944925+0000","last_peered":"2023-11-24T06:26:57.944925+0000","last_clean":"2023-11-24T06:26:57.944925+0000","last_became_active":"2023-11-24T06:26:56.990836+0000","last_became_peered":"2023-11-24T06:26:56.990836+0000","last_unstale":"2023-11-24T06:26:57.944925+0000","last_undegraded":"2023-11-24T06:26:57.944925+0000","last_fullsized":"2023-11-24T06:26:57.944925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948378+0000","last_change":"2023-11-24T06:26:57.001238+0000","last_active":"2023-11-24T06:26:57.948378+0000","last_peered":"2023-11-24T06:26:57.948378+0000","last_clean":"2023-11-24T06:26:57.948378+0000","last_became_active":"2023-11-24T06:26:57.000919+0000","last_became_peered":"2023-11-24T06:26:57.000919+0000","last_unstale":"2023-11-24T06:26:57.948378+0000","last_undegraded":"2023-11-24T06:26:57.948378+0000","last_fullsized":"2023-11-24T06:26:57.948378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:38.348575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219671+0000","last_change":"2023-11-24T06:26:56.998442+0000","last_active":"2023-11-24T06:26:58.219671+0000","last_peered":"2023-11-24T06:26:58.219671+0000","last_clean":"2023-11-24T06:26:58.219671+0000","last_became_active":"2023-11-24T06:26:56.998097+0000","last_became_peered":"2023-11-24T06:26:56.998097+0000","last_unstale":"2023-11-24T06:26:58.219671+0000","last_undegraded":"2023-11-24T06:26:58.219671+0000","last_fullsized":"2023-11-24T06:26:58.219671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218262+0000","last_change":"2023-11-24T06:26:57.000349+0000","last_active":"2023-11-24T06:26:58.218262+0000","last_peered":"2023-11-24T06:26:58.218262+0000","last_clean":"2023-11-24T06:26:58.218262+0000","last_became_active":"2023-11-24T06:26:57.000122+0000","last_became_peered":"2023-11-24T06:26:57.000122+0000","last_unstale":"2023-11-24T06:26:58.218262+0000","last_undegraded":"2023-11-24T06:26:58.218262+0000","last_fullsized":"2023-11-24T06:26:58.218262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221702+0000","last_change":"2023-11-24T06:26:56.969362+0000","last_active":"2023-11-24T06:26:58.221702+0000","last_peered":"2023-11-24T06:26:58.221702+0000","last_clean":"2023-11-24T06:26:58.221702+0000","last_became_active":"2023-11-24T06:26:56.969057+0000","last_became_peered":"2023-11-24T06:26:56.969057+0000","last_unstale":"2023-11-24T06:26:58.221702+0000","last_undegraded":"2023-11-24T06:26:58.221702+0000","last_fullsized":"2023-11-24T06:26:58.221702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:04.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945262+0000","last_change":"2023-11-24T06:26:56.970657+0000","last_active":"2023-11-24T06:26:57.945262+0000","last_peered":"2023-11-24T06:26:57.945262+0000","last_clean":"2023-11-24T06:26:57.945262+0000","last_became_active":"2023-11-24T06:26:56.970348+0000","last_became_peered":"2023-11-24T06:26:56.970348+0000","last_unstale":"2023-11-24T06:26:57.945262+0000","last_undegraded":"2023-11-24T06:26:57.945262+0000","last_fullsized":"2023-11-24T06:26:57.945262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217421+0000","last_change":"2023-11-24T06:26:56.972241+0000","last_active":"2023-11-24T06:26:58.217421+0000","last_peered":"2023-11-24T06:26:58.217421+0000","last_clean":"2023-11-24T06:26:58.217421+0000","last_became_active":"2023-11-24T06:26:56.971982+0000","last_became_peered":"2023-11-24T06:26:56.971982+0000","last_unstale":"2023-11-24T06:26:58.217421+0000","last_undegraded":"2023-11-24T06:26:58.217421+0000","last_fullsized":"2023-11-24T06:26:58.217421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:19.450311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944041+0000","last_change":"2023-11-24T06:26:56.978171+0000","last_active":"2023-11-24T06:26:57.944041+0000","last_peered":"2023-11-24T06:26:57.944041+0000","last_clean":"2023-11-24T06:26:57.944041+0000","last_became_active":"2023-11-24T06:26:56.977851+0000","last_became_peered":"2023-11-24T06:26:56.977851+0000","last_unstale":"2023-11-24T06:26:57.944041+0000","last_undegraded":"2023-11-24T06:26:57.944041+0000","last_fullsized":"2023-11-24T06:26:57.944041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218965+0000","last_change":"2023-11-24T06:26:56.991619+0000","last_active":"2023-11-24T06:26:58.218965+0000","last_peered":"2023-11-24T06:26:58.218965+0000","last_clean":"2023-11-24T06:26:58.218965+0000","last_became_active":"2023-11-24T06:26:56.986865+0000","last_became_peered":"2023-11-24T06:26:56.986865+0000","last_unstale":"2023-11-24T06:26:58.218965+0000","last_undegraded":"2023-11-24T06:26:58.218965+0000","last_fullsized":"2023-11-24T06:26:58.218965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.223090+0000","last_change":"2023-11-24T06:26:56.997282+0000","last_active":"2023-11-24T06:26:58.223090+0000","last_peered":"2023-11-24T06:26:58.223090+0000","last_clean":"2023-11-24T06:26:58.223090+0000","last_became_active":"2023-11-24T06:26:56.995631+0000","last_became_peered":"2023-11-24T06:26:56.995631+0000","last_unstale":"2023-11-24T06:26:58.223090+0000","last_undegraded":"2023-11-24T06:26:58.223090+0000","last_fullsized":"2023-11-24T06:26:58.223090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:38.348575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217200+0000","last_change":"2023-11-24T06:26:56.969721+0000","last_active":"2023-11-24T06:26:58.217200+0000","last_peered":"2023-11-24T06:26:58.217200+0000","last_clean":"2023-11-24T06:26:58.217200+0000","last_became_active":"2023-11-24T06:26:56.969475+0000","last_became_peered":"2023-11-24T06:26:56.969475+0000","last_unstale":"2023-11-24T06:26:58.217200+0000","last_undegraded":"2023-11-24T06:26:58.217200+0000","last_fullsized":"2023-11-24T06:26:58.217200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:04.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947802+0000","last_change":"2023-11-24T06:26:56.964126+0000","last_active":"2023-11-24T06:26:57.947802+0000","last_peered":"2023-11-24T06:26:57.947802+0000","last_clean":"2023-11-24T06:26:57.947802+0000","last_became_active":"2023-11-24T06:26:56.963622+0000","last_became_peered":"2023-11-24T06:26:56.963622+0000","last_unstale":"2023-11-24T06:26:57.947802+0000","last_undegraded":"2023-11-24T06:26:57.947802+0000","last_fullsized":"2023-11-24T06:26:57.947802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:17.662630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218464+0000","last_change":"2023-11-24T06:26:56.975222+0000","last_active":"2023-11-24T06:26:58.218464+0000","last_peered":"2023-11-24T06:26:58.218464+0000","last_clean":"2023-11-24T06:26:58.218464+0000","last_became_active":"2023-11-24T06:26:56.974069+0000","last_became_peered":"2023-11-24T06:26:56.974069+0000","last_unstale":"2023-11-24T06:26:58.218464+0000","last_undegraded":"2023-11-24T06:26:58.218464+0000","last_fullsized":"2023-11-24T06:26:58.218464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218120+0000","last_change":"2023-11-24T06:26:56.975253+0000","last_active":"2023-11-24T06:26:58.218120+0000","last_peered":"2023-11-24T06:26:58.218120+0000","last_clean":"2023-11-24T06:26:58.218120+0000","last_became_active":"2023-11-24T06:26:56.974888+0000","last_became_peered":"2023-11-24T06:26:56.974888+0000","last_unstale":"2023-11-24T06:26:58.218120+0000","last_undegraded":"2023-11-24T06:26:58.218120+0000","last_fullsized":"2023-11-24T06:26:58.218120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221143+0000","last_change":"2023-11-24T06:26:56.985344+0000","last_active":"2023-11-24T06:26:58.221143+0000","last_peered":"2023-11-24T06:26:58.221143+0000","last_clean":"2023-11-24T06:26:58.221143+0000","last_became_active":"2023-11-24T06:26:56.984528+0000","last_became_peered":"2023-11-24T06:26:56.984528+0000","last_unstale":"2023-11-24T06:26:58.221143+0000","last_undegraded":"2023-11-24T06:26:58.221143+0000","last_fullsized":"2023-11-24T06:26:58.221143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944974+0000","last_change":"2023-11-24T06:26:56.979949+0000","last_active":"2023-11-24T06:26:57.944974+0000","last_peered":"2023-11-24T06:26:57.944974+0000","last_clean":"2023-11-24T06:26:57.944974+0000","last_became_active":"2023-11-24T06:26:56.979635+0000","last_became_peered":"2023-11-24T06:26:56.979635+0000","last_unstale":"2023-11-24T06:26:57.944974+0000","last_undegraded":"2023-11-24T06:26:57.944974+0000","last_fullsized":"2023-11-24T06:26:57.944974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219328+0000","last_change":"2023-11-24T06:26:56.968934+0000","last_active":"2023-11-24T06:26:58.219328+0000","last_peered":"2023-11-24T06:26:58.219328+0000","last_clean":"2023-11-24T06:26:58.219328+0000","last_became_active":"2023-11-24T06:26:56.967198+0000","last_became_peered":"2023-11-24T06:26:56.967198+0000","last_unstale":"2023-11-24T06:26:58.219328+0000","last_undegraded":"2023-11-24T06:26:58.219328+0000","last_fullsized":"2023-11-24T06:26:58.219328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945713+0000","last_change":"2023-11-24T06:26:56.974003+0000","last_active":"2023-11-24T06:26:57.945713+0000","last_peered":"2023-11-24T06:26:57.945713+0000","last_clean":"2023-11-24T06:26:57.945713+0000","last_became_active":"2023-11-24T06:26:56.973631+0000","last_became_peered":"2023-11-24T06:26:56.973631+0000","last_unstale":"2023-11-24T06:26:57.945713+0000","last_undegraded":"2023-11-24T06:26:57.945713+0000","last_fullsized":"2023-11-24T06:26:57.945713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219990+0000","last_change":"2023-11-24T06:26:56.972613+0000","last_active":"2023-11-24T06:26:58.219990+0000","last_peered":"2023-11-24T06:26:58.219990+0000","last_clean":"2023-11-24T06:26:58.219990+0000","last_became_active":"2023-11-24T06:26:56.972280+0000","last_became_peered":"2023-11-24T06:26:56.972280+0000","last_unstale":"2023-11-24T06:26:58.219990+0000","last_undegraded":"2023-11-24T06:26:58.219990+0000","last_fullsized":"2023-11-24T06:26:58.219990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944265+0000","last_change":"2023-11-24T06:26:56.966680+0000","last_active":"2023-11-24T06:26:57.944265+0000","last_peered":"2023-11-24T06:26:57.944265+0000","last_clean":"2023-11-24T06:26:57.944265+0000","last_became_active":"2023-11-24T06:26:56.966315+0000","last_became_peered":"2023-11-24T06:26:56.966315+0000","last_unstale":"2023-11-24T06:26:57.944265+0000","last_undegraded":"2023-11-24T06:26:57.944265+0000","last_fullsized":"2023-11-24T06:26:57.944265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945586+0000","last_change":"2023-11-24T06:26:56.974920+0000","last_active":"2023-11-24T06:26:57.945586+0000","last_peered":"2023-11-24T06:26:57.945586+0000","last_clean":"2023-11-24T06:26:57.945586+0000","last_became_active":"2023-11-24T06:26:56.974594+0000","last_became_peered":"2023-11-24T06:26:56.974594+0000","last_unstale":"2023-11-24T06:26:57.945586+0000","last_undegraded":"2023-11-24T06:26:57.945586+0000","last_fullsized":"2023-11-24T06:26:57.945586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217994+0000","last_change":"2023-11-24T06:26:56.976017+0000","last_active":"2023-11-24T06:26:58.217994+0000","last_peered":"2023-11-24T06:26:58.217994+0000","last_clean":"2023-11-24T06:26:58.217994+0000","last_became_active":"2023-11-24T06:26:56.975709+0000","last_became_peered":"2023-11-24T06:26:56.975709+0000","last_unstale":"2023-11-24T06:26:58.217994+0000","last_undegraded":"2023-11-24T06:26:58.217994+0000","last_fullsized":"2023-11-24T06:26:58.217994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221250+0000","last_change":"2023-11-24T06:26:56.980269+0000","last_active":"2023-11-24T06:26:58.221250+0000","last_peered":"2023-11-24T06:26:58.221250+0000","last_clean":"2023-11-24T06:26:58.221250+0000","last_became_active":"2023-11-24T06:26:56.979929+0000","last_became_peered":"2023-11-24T06:26:56.979929+0000","last_unstale":"2023-11-24T06:26:58.221250+0000","last_undegraded":"2023-11-24T06:26:58.221250+0000","last_fullsized":"2023-11-24T06:26:58.221250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216766+0000","last_change":"2023-11-24T06:26:56.986717+0000","last_active":"2023-11-24T06:26:58.216766+0000","last_peered":"2023-11-24T06:26:58.216766+0000","last_clean":"2023-11-24T06:26:58.216766+0000","last_became_active":"2023-11-24T06:26:56.986398+0000","last_became_peered":"2023-11-24T06:26:56.986398+0000","last_unstale":"2023-11-24T06:26:58.216766+0000","last_undegraded":"2023-11-24T06:26:58.216766+0000","last_fullsized":"2023-11-24T06:26:58.216766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:47.926009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217958+0000","last_change":"2023-11-24T06:26:56.978275+0000","last_active":"2023-11-24T06:26:58.217958+0000","last_peered":"2023-11-24T06:26:58.217958+0000","last_clean":"2023-11-24T06:26:58.217958+0000","last_became_active":"2023-11-24T06:26:56.977924+0000","last_became_peered":"2023-11-24T06:26:56.977924+0000","last_unstale":"2023-11-24T06:26:58.217958+0000","last_undegraded":"2023-11-24T06:26:58.217958+0000","last_fullsized":"2023-11-24T06:26:58.217958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:58.210301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220024+0000","last_change":"2023-11-24T06:26:56.970266+0000","last_active":"2023-11-24T06:26:58.220024+0000","last_peered":"2023-11-24T06:26:58.220024+0000","last_clean":"2023-11-24T06:26:58.220024+0000","last_became_active":"2023-11-24T06:26:56.969973+0000","last_became_peered":"2023-11-24T06:26:56.969973+0000","last_unstale":"2023-11-24T06:26:58.220024+0000","last_undegraded":"2023-11-24T06:26:58.220024+0000","last_fullsized":"2023-11-24T06:26:58.220024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:28.630873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945477+0000","last_change":"2023-11-24T06:26:56.008228+0000","last_active":"2023-11-24T06:26:57.945477+0000","last_peered":"2023-11-24T06:26:57.945477+0000","last_clean":"2023-11-24T06:26:57.945477+0000","last_became_active":"2023-11-24T06:26:56.007916+0000","last_became_peered":"2023-11-24T06:26:56.007916+0000","last_unstale":"2023-11-24T06:26:57.945477+0000","last_undegraded":"2023-11-24T06:26:57.945477+0000","last_fullsized":"2023-11-24T06:26:57.945477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218141+0000","last_change":"2023-11-24T06:26:56.982794+0000","last_active":"2023-11-24T06:26:58.218141+0000","last_peered":"2023-11-24T06:26:58.218141+0000","last_clean":"2023-11-24T06:26:58.218141+0000","last_became_active":"2023-11-24T06:26:56.982455+0000","last_became_peered":"2023-11-24T06:26:56.982455+0000","last_unstale":"2023-11-24T06:26:58.218141+0000","last_undegraded":"2023-11-24T06:26:58.218141+0000","last_fullsized":"2023-11-24T06:26:58.218141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218348+0000","last_change":"2023-11-24T06:26:56.010407+0000","last_active":"2023-11-24T06:26:58.218348+0000","last_peered":"2023-11-24T06:26:58.218348+0000","last_clean":"2023-11-24T06:26:58.218348+0000","last_became_active":"2023-11-24T06:26:56.009548+0000","last_became_peered":"2023-11-24T06:26:56.009548+0000","last_unstale":"2023-11-24T06:26:58.218348+0000","last_undegraded":"2023-11-24T06:26:58.218348+0000","last_fullsized":"2023-11-24T06:26:58.218348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:58.487660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945323+0000","last_change":"2023-11-24T06:26:56.005804+0000","last_active":"2023-11-24T06:26:57.945323+0000","last_peered":"2023-11-24T06:26:57.945323+0000","last_clean":"2023-11-24T06:26:57.945323+0000","last_became_active":"2023-11-24T06:26:56.005473+0000","last_became_peered":"2023-11-24T06:26:56.005473+0000","last_unstale":"2023-11-24T06:26:57.945323+0000","last_undegraded":"2023-11-24T06:26:57.945323+0000","last_fullsized":"2023-11-24T06:26:57.945323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:18.225163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944935+0000","last_change":"2023-11-24T06:26:56.977459+0000","last_active":"2023-11-24T06:26:57.944935+0000","last_peered":"2023-11-24T06:26:57.944935+0000","last_clean":"2023-11-24T06:26:57.944935+0000","last_became_active":"2023-11-24T06:26:56.977160+0000","last_became_peered":"2023-11-24T06:26:56.977160+0000","last_unstale":"2023-11-24T06:26:57.944935+0000","last_undegraded":"2023-11-24T06:26:57.944935+0000","last_fullsized":"2023-11-24T06:26:57.944935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219546+0000","last_change":"2023-11-24T06:26:56.010202+0000","last_active":"2023-11-24T06:26:58.219546+0000","last_peered":"2023-11-24T06:26:58.219546+0000","last_clean":"2023-11-24T06:26:58.219546+0000","last_became_active":"2023-11-24T06:26:56.009924+0000","last_became_peered":"2023-11-24T06:26:56.009924+0000","last_unstale":"2023-11-24T06:26:58.219546+0000","last_undegraded":"2023-11-24T06:26:58.219546+0000","last_fullsized":"2023-11-24T06:26:58.219546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:17.355207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944060+0000","last_change":"2023-11-24T06:26:56.962258+0000","last_active":"2023-11-24T06:26:57.944060+0000","last_peered":"2023-11-24T06:26:57.944060+0000","last_clean":"2023-11-24T06:26:57.944060+0000","last_became_active":"2023-11-24T06:26:56.961665+0000","last_became_peered":"2023-11-24T06:26:56.961665+0000","last_unstale":"2023-11-24T06:26:57.944060+0000","last_undegraded":"2023-11-24T06:26:57.944060+0000","last_fullsized":"2023-11-24T06:26:57.944060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221161+0000","last_change":"2023-11-24T06:26:56.017642+0000","last_active":"2023-11-24T06:26:58.221161+0000","last_peered":"2023-11-24T06:26:58.221161+0000","last_clean":"2023-11-24T06:26:58.221161+0000","last_became_active":"2023-11-24T06:26:56.017386+0000","last_became_peered":"2023-11-24T06:26:56.017386+0000","last_unstale":"2023-11-24T06:26:58.221161+0000","last_undegraded":"2023-11-24T06:26:58.221161+0000","last_fullsized":"2023-11-24T06:26:58.221161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:18.225163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222938+0000","last_change":"2023-11-24T06:26:56.985441+0000","last_active":"2023-11-24T06:26:58.222938+0000","last_peered":"2023-11-24T06:26:58.222938+0000","last_clean":"2023-11-24T06:26:58.222938+0000","last_became_active":"2023-11-24T06:26:56.984430+0000","last_became_peered":"2023-11-24T06:26:56.984430+0000","last_unstale":"2023-11-24T06:26:58.222938+0000","last_undegraded":"2023-11-24T06:26:58.222938+0000","last_fullsized":"2023-11-24T06:26:58.222938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:56.199949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222514+0000","last_change":"2023-11-24T06:26:56.017588+0000","last_active":"2023-11-24T06:26:58.222514+0000","last_peered":"2023-11-24T06:26:58.222514+0000","last_clean":"2023-11-24T06:26:58.222514+0000","last_became_active":"2023-11-24T06:26:56.017128+0000","last_became_peered":"2023-11-24T06:26:56.017128+0000","last_unstale":"2023-11-24T06:26:58.222514+0000","last_undegraded":"2023-11-24T06:26:58.222514+0000","last_fullsized":"2023-11-24T06:26:58.222514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218742+0000","last_change":"2023-11-24T06:26:57.002819+0000","last_active":"2023-11-24T06:26:58.218742+0000","last_peered":"2023-11-24T06:26:58.218742+0000","last_clean":"2023-11-24T06:26:58.218742+0000","last_became_active":"2023-11-24T06:26:57.002515+0000","last_became_peered":"2023-11-24T06:26:57.002515+0000","last_unstale":"2023-11-24T06:26:58.218742+0000","last_undegraded":"2023-11-24T06:26:58.218742+0000","last_fullsized":"2023-11-24T06:26:58.218742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944342+0000","last_change":"2023-11-24T06:26:56.014991+0000","last_active":"2023-11-24T06:26:57.944342+0000","last_peered":"2023-11-24T06:26:57.944342+0000","last_clean":"2023-11-24T06:26:57.944342+0000","last_became_active":"2023-11-24T06:26:56.014706+0000","last_became_peered":"2023-11-24T06:26:56.014706+0000","last_unstale":"2023-11-24T06:26:57.944342+0000","last_undegraded":"2023-11-24T06:26:57.944342+0000","last_fullsized":"2023-11-24T06:26:57.944342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:56.985153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221316+0000","last_change":"2023-11-24T06:26:56.970097+0000","last_active":"2023-11-24T06:26:58.221316+0000","last_peered":"2023-11-24T06:26:58.221316+0000","last_clean":"2023-11-24T06:26:58.221316+0000","last_became_active":"2023-11-24T06:26:56.969183+0000","last_became_peered":"2023-11-24T06:26:56.969183+0000","last_unstale":"2023-11-24T06:26:58.221316+0000","last_undegraded":"2023-11-24T06:26:58.221316+0000","last_fullsized":"2023-11-24T06:26:58.221316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:47.926009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221734+0000","last_change":"2023-11-24T06:26:56.002890+0000","last_active":"2023-11-24T06:26:58.221734+0000","last_peered":"2023-11-24T06:26:58.221734+0000","last_clean":"2023-11-24T06:26:58.221734+0000","last_became_active":"2023-11-24T06:26:56.002603+0000","last_became_peered":"2023-11-24T06:26:56.002603+0000","last_unstale":"2023-11-24T06:26:58.221734+0000","last_undegraded":"2023-11-24T06:26:58.221734+0000","last_fullsized":"2023-11-24T06:26:58.221734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947457+0000","last_change":"2023-11-24T06:26:56.961409+0000","last_active":"2023-11-24T06:26:57.947457+0000","last_peered":"2023-11-24T06:26:57.947457+0000","last_clean":"2023-11-24T06:26:57.947457+0000","last_became_active":"2023-11-24T06:26:56.961107+0000","last_became_peered":"2023-11-24T06:26:56.961107+0000","last_unstale":"2023-11-24T06:26:57.947457+0000","last_undegraded":"2023-11-24T06:26:57.947457+0000","last_fullsized":"2023-11-24T06:26:57.947457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218244+0000","last_change":"2023-11-24T06:26:56.006966+0000","last_active":"2023-11-24T06:26:58.218244+0000","last_peered":"2023-11-24T06:26:58.218244+0000","last_clean":"2023-11-24T06:26:58.218244+0000","last_became_active":"2023-11-24T06:26:56.006596+0000","last_became_peered":"2023-11-24T06:26:56.006596+0000","last_unstale":"2023-11-24T06:26:58.218244+0000","last_undegraded":"2023-11-24T06:26:58.218244+0000","last_fullsized":"2023-11-24T06:26:58.218244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217572+0000","last_change":"2023-11-24T06:26:56.974192+0000","last_active":"2023-11-24T06:26:58.217572+0000","last_peered":"2023-11-24T06:26:58.217572+0000","last_clean":"2023-11-24T06:26:58.217572+0000","last_became_active":"2023-11-24T06:26:56.973850+0000","last_became_peered":"2023-11-24T06:26:56.973850+0000","last_unstale":"2023-11-24T06:26:58.217572+0000","last_undegraded":"2023-11-24T06:26:58.217572+0000","last_fullsized":"2023-11-24T06:26:58.217572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:51.030181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221387+0000","last_change":"2023-11-24T06:26:56.013312+0000","last_active":"2023-11-24T06:26:58.221387+0000","last_peered":"2023-11-24T06:26:58.221387+0000","last_clean":"2023-11-24T06:26:58.221387+0000","last_became_active":"2023-11-24T06:26:56.013018+0000","last_became_peered":"2023-11-24T06:26:56.013018+0000","last_unstale":"2023-11-24T06:26:58.221387+0000","last_undegraded":"2023-11-24T06:26:58.221387+0000","last_fullsized":"2023-11-24T06:26:58.221387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:03.493540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947707+0000","last_change":"2023-11-24T06:26:56.975118+0000","last_active":"2023-11-24T06:26:57.947707+0000","last_peered":"2023-11-24T06:26:57.947707+0000","last_clean":"2023-11-24T06:26:57.947707+0000","last_became_active":"2023-11-24T06:26:56.974734+0000","last_became_peered":"2023-11-24T06:26:56.974734+0000","last_unstale":"2023-11-24T06:26:57.947707+0000","last_undegraded":"2023-11-24T06:26:57.947707+0000","last_fullsized":"2023-11-24T06:26:57.947707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:15.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219402+0000","last_change":"2023-11-24T06:26:56.010113+0000","last_active":"2023-11-24T06:26:58.219402+0000","last_peered":"2023-11-24T06:26:58.219402+0000","last_clean":"2023-11-24T06:26:58.219402+0000","last_became_active":"2023-11-24T06:26:56.009550+0000","last_became_peered":"2023-11-24T06:26:56.009550+0000","last_unstale":"2023-11-24T06:26:58.219402+0000","last_undegraded":"2023-11-24T06:26:58.219402+0000","last_fullsized":"2023-11-24T06:26:58.219402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:13.157002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218000+0000","last_change":"2023-11-24T06:26:56.978061+0000","last_active":"2023-11-24T06:26:58.218000+0000","last_peered":"2023-11-24T06:26:58.218000+0000","last_clean":"2023-11-24T06:26:58.218000+0000","last_became_active":"2023-11-24T06:26:56.977760+0000","last_became_peered":"2023-11-24T06:26:56.977760+0000","last_unstale":"2023-11-24T06:26:58.218000+0000","last_undegraded":"2023-11-24T06:26:58.218000+0000","last_fullsized":"2023-11-24T06:26:58.218000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216616+0000","last_change":"2023-11-24T06:26:55.996462+0000","last_active":"2023-11-24T06:26:58.216616+0000","last_peered":"2023-11-24T06:26:58.216616+0000","last_clean":"2023-11-24T06:26:58.216616+0000","last_became_active":"2023-11-24T06:26:55.996178+0000","last_became_peered":"2023-11-24T06:26:55.996178+0000","last_unstale":"2023-11-24T06:26:58.216616+0000","last_undegraded":"2023-11-24T06:26:58.216616+0000","last_fullsized":"2023-11-24T06:26:58.216616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:59.647318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946852+0000","last_change":"2023-11-24T06:26:56.978947+0000","last_active":"2023-11-24T06:26:57.946852+0000","last_peered":"2023-11-24T06:26:57.946852+0000","last_clean":"2023-11-24T06:26:57.946852+0000","last_became_active":"2023-11-24T06:26:56.978643+0000","last_became_peered":"2023-11-24T06:26:56.978643+0000","last_unstale":"2023-11-24T06:26:57.946852+0000","last_undegraded":"2023-11-24T06:26:57.946852+0000","last_fullsized":"2023-11-24T06:26:57.946852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222582+0000","last_change":"2023-11-24T06:26:56.005504+0000","last_active":"2023-11-24T06:26:58.222582+0000","last_peered":"2023-11-24T06:26:58.222582+0000","last_clean":"2023-11-24T06:26:58.222582+0000","last_became_active":"2023-11-24T06:26:56.005287+0000","last_became_peered":"2023-11-24T06:26:56.005287+0000","last_unstale":"2023-11-24T06:26:58.222582+0000","last_undegraded":"2023-11-24T06:26:58.222582+0000","last_fullsized":"2023-11-24T06:26:58.222582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:24.763532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947843+0000","last_change":"2023-11-24T06:26:56.973083+0000","last_active":"2023-11-24T06:26:57.947843+0000","last_peered":"2023-11-24T06:26:57.947843+0000","last_clean":"2023-11-24T06:26:57.947843+0000","last_became_active":"2023-11-24T06:26:56.972750+0000","last_became_peered":"2023-11-24T06:26:56.972750+0000","last_unstale":"2023-11-24T06:26:57.947843+0000","last_undegraded":"2023-11-24T06:26:57.947843+0000","last_fullsized":"2023-11-24T06:26:57.947843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:08.969752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217141+0000","last_change":"2023-11-24T06:26:56.005050+0000","last_active":"2023-11-24T06:26:58.217141+0000","last_peered":"2023-11-24T06:26:58.217141+0000","last_clean":"2023-11-24T06:26:58.217141+0000","last_became_active":"2023-11-24T06:26:56.004658+0000","last_became_peered":"2023-11-24T06:26:56.004658+0000","last_unstale":"2023-11-24T06:26:58.217141+0000","last_undegraded":"2023-11-24T06:26:58.217141+0000","last_fullsized":"2023-11-24T06:26:58.217141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:27.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948405+0000","last_change":"2023-11-24T06:26:56.973292+0000","last_active":"2023-11-24T06:26:57.948405+0000","last_peered":"2023-11-24T06:26:57.948405+0000","last_clean":"2023-11-24T06:26:57.948405+0000","last_became_active":"2023-11-24T06:26:56.972959+0000","last_became_peered":"2023-11-24T06:26:56.972959+0000","last_unstale":"2023-11-24T06:26:57.948405+0000","last_undegraded":"2023-11-24T06:26:57.948405+0000","last_fullsized":"2023-11-24T06:26:57.948405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:48.721442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944407+0000","last_change":"2023-11-24T06:26:56.005932+0000","last_active":"2023-11-24T06:26:57.944407+0000","last_peered":"2023-11-24T06:26:57.944407+0000","last_clean":"2023-11-24T06:26:57.944407+0000","last_became_active":"2023-11-24T06:26:56.005603+0000","last_became_peered":"2023-11-24T06:26:56.005603+0000","last_unstale":"2023-11-24T06:26:57.944407+0000","last_undegraded":"2023-11-24T06:26:57.944407+0000","last_fullsized":"2023-11-24T06:26:57.944407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222520+0000","last_change":"2023-11-24T06:26:56.969503+0000","last_active":"2023-11-24T06:26:58.222520+0000","last_peered":"2023-11-24T06:26:58.222520+0000","last_clean":"2023-11-24T06:26:58.222520+0000","last_became_active":"2023-11-24T06:26:56.967704+0000","last_became_peered":"2023-11-24T06:26:56.967704+0000","last_unstale":"2023-11-24T06:26:58.222520+0000","last_undegraded":"2023-11-24T06:26:58.222520+0000","last_fullsized":"2023-11-24T06:26:58.222520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:17.662630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217140+0000","last_change":"2023-11-24T06:26:56.014819+0000","last_active":"2023-11-24T06:26:58.217140+0000","last_peered":"2023-11-24T06:26:58.217140+0000","last_clean":"2023-11-24T06:26:58.217140+0000","last_became_active":"2023-11-24T06:26:56.014287+0000","last_became_peered":"2023-11-24T06:26:56.014287+0000","last_unstale":"2023-11-24T06:26:58.217140+0000","last_undegraded":"2023-11-24T06:26:58.217140+0000","last_fullsized":"2023-11-24T06:26:58.217140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:13.157002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947402+0000","last_change":"2023-11-24T06:26:57.001053+0000","last_active":"2023-11-24T06:26:57.947402+0000","last_peered":"2023-11-24T06:26:57.947402+0000","last_clean":"2023-11-24T06:26:57.947402+0000","last_became_active":"2023-11-24T06:26:57.000802+0000","last_became_peered":"2023-11-24T06:26:57.000802+0000","last_unstale":"2023-11-24T06:26:57.947402+0000","last_undegraded":"2023-11-24T06:26:57.947402+0000","last_fullsized":"2023-11-24T06:26:57.947402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:26:56.946083+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:26:56.946083+0000","last_peered":"2023-11-24T06:26:56.946083+0000","last_clean":"2023-11-24T06:26:56.946083+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:26:56.946083+0000","last_undegraded":"2023-11-24T06:26:56.946083+0000","last_fullsized":"2023-11-24T06:26:56.946083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221515+0000","last_change":"2023-11-24T06:26:56.008553+0000","last_active":"2023-11-24T06:26:58.221515+0000","last_peered":"2023-11-24T06:26:58.221515+0000","last_clean":"2023-11-24T06:26:58.221515+0000","last_became_active":"2023-11-24T06:26:56.006284+0000","last_became_peered":"2023-11-24T06:26:56.006284+0000","last_unstale":"2023-11-24T06:26:58.221515+0000","last_undegraded":"2023-11-24T06:26:58.221515+0000","last_fullsized":"2023-11-24T06:26:58.221515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:24.763532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945070+0000","last_change":"2023-11-24T06:26:56.998859+0000","last_active":"2023-11-24T06:26:57.945070+0000","last_peered":"2023-11-24T06:26:57.945070+0000","last_clean":"2023-11-24T06:26:57.945070+0000","last_became_active":"2023-11-24T06:26:56.998524+0000","last_became_peered":"2023-11-24T06:26:56.998524+0000","last_unstale":"2023-11-24T06:26:57.945070+0000","last_undegraded":"2023-11-24T06:26:57.945070+0000","last_fullsized":"2023-11-24T06:26:57.945070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946597+0000","last_change":"2023-11-24T06:26:56.005300+0000","last_active":"2023-11-24T06:26:57.946597+0000","last_peered":"2023-11-24T06:26:57.946597+0000","last_clean":"2023-11-24T06:26:57.946597+0000","last_became_active":"2023-11-24T06:26:56.004984+0000","last_became_peered":"2023-11-24T06:26:56.004984+0000","last_unstale":"2023-11-24T06:26:57.946597+0000","last_undegraded":"2023-11-24T06:26:57.946597+0000","last_fullsized":"2023-11-24T06:26:57.946597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217825+0000","last_change":"2023-11-24T06:26:56.984062+0000","last_active":"2023-11-24T06:26:58.217825+0000","last_peered":"2023-11-24T06:26:58.217825+0000","last_clean":"2023-11-24T06:26:58.217825+0000","last_became_active":"2023-11-24T06:26:56.983679+0000","last_became_peered":"2023-11-24T06:26:56.983679+0000","last_unstale":"2023-11-24T06:26:58.217825+0000","last_undegraded":"2023-11-24T06:26:58.217825+0000","last_fullsized":"2023-11-24T06:26:58.217825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:58.210301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216848+0000","last_change":"2023-11-24T06:26:56.005001+0000","last_active":"2023-11-24T06:26:58.216848+0000","last_peered":"2023-11-24T06:26:58.216848+0000","last_clean":"2023-11-24T06:26:58.216848+0000","last_became_active":"2023-11-24T06:26:56.004612+0000","last_became_peered":"2023-11-24T06:26:56.004612+0000","last_unstale":"2023-11-24T06:26:58.216848+0000","last_undegraded":"2023-11-24T06:26:58.216848+0000","last_fullsized":"2023-11-24T06:26:58.216848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944532+0000","last_change":"2023-11-24T06:26:56.965796+0000","last_active":"2023-11-24T06:26:57.944532+0000","last_peered":"2023-11-24T06:26:57.944532+0000","last_clean":"2023-11-24T06:26:57.944532+0000","last_became_active":"2023-11-24T06:26:56.963344+0000","last_became_peered":"2023-11-24T06:26:56.963344+0000","last_unstale":"2023-11-24T06:26:57.944532+0000","last_undegraded":"2023-11-24T06:26:57.944532+0000","last_fullsized":"2023-11-24T06:26:57.944532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221498+0000","last_change":"2023-11-24T06:26:56.012192+0000","last_active":"2023-11-24T06:26:58.221498+0000","last_peered":"2023-11-24T06:26:58.221498+0000","last_clean":"2023-11-24T06:26:58.221498+0000","last_became_active":"2023-11-24T06:26:56.011796+0000","last_became_peered":"2023-11-24T06:26:56.011796+0000","last_unstale":"2023-11-24T06:26:58.221498+0000","last_undegraded":"2023-11-24T06:26:58.221498+0000","last_fullsized":"2023-11-24T06:26:58.221498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:50.399725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220548+0000","last_change":"2023-11-24T06:26:56.985067+0000","last_active":"2023-11-24T06:26:58.220548+0000","last_peered":"2023-11-24T06:26:58.220548+0000","last_clean":"2023-11-24T06:26:58.220548+0000","last_became_active":"2023-11-24T06:26:56.984360+0000","last_became_peered":"2023-11-24T06:26:56.984360+0000","last_unstale":"2023-11-24T06:26:58.220548+0000","last_undegraded":"2023-11-24T06:26:58.220548+0000","last_fullsized":"2023-11-24T06:26:58.220548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:50.371449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944321+0000","last_change":"2023-11-24T06:26:55.996919+0000","last_active":"2023-11-24T06:26:57.944321+0000","last_peered":"2023-11-24T06:26:57.944321+0000","last_clean":"2023-11-24T06:26:57.944321+0000","last_became_active":"2023-11-24T06:26:55.996431+0000","last_became_peered":"2023-11-24T06:26:55.996431+0000","last_unstale":"2023-11-24T06:26:57.944321+0000","last_undegraded":"2023-11-24T06:26:57.944321+0000","last_fullsized":"2023-11-24T06:26:57.944321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222871+0000","last_change":"2023-11-24T06:26:56.987025+0000","last_active":"2023-11-24T06:26:58.222871+0000","last_peered":"2023-11-24T06:26:58.222871+0000","last_clean":"2023-11-24T06:26:58.222871+0000","last_became_active":"2023-11-24T06:26:56.985920+0000","last_became_peered":"2023-11-24T06:26:56.985920+0000","last_unstale":"2023-11-24T06:26:58.222871+0000","last_undegraded":"2023-11-24T06:26:58.222871+0000","last_fullsized":"2023-11-24T06:26:58.222871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:08.969752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947843+0000","last_change":"2023-11-24T06:26:56.015006+0000","last_active":"2023-11-24T06:26:57.947843+0000","last_peered":"2023-11-24T06:26:57.947843+0000","last_clean":"2023-11-24T06:26:57.947843+0000","last_became_active":"2023-11-24T06:26:56.011264+0000","last_became_peered":"2023-11-24T06:26:56.011264+0000","last_unstale":"2023-11-24T06:26:57.947843+0000","last_undegraded":"2023-11-24T06:26:57.947843+0000","last_fullsized":"2023-11-24T06:26:57.947843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:51.788764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944165+0000","last_change":"2023-11-24T06:26:56.967275+0000","last_active":"2023-11-24T06:26:57.944165+0000","last_peered":"2023-11-24T06:26:57.944165+0000","last_clean":"2023-11-24T06:26:57.944165+0000","last_became_active":"2023-11-24T06:26:56.966261+0000","last_became_peered":"2023-11-24T06:26:56.966261+0000","last_unstale":"2023-11-24T06:26:57.944165+0000","last_undegraded":"2023-11-24T06:26:57.944165+0000","last_fullsized":"2023-11-24T06:26:57.944165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:59.156742+0000","last_change":"2023-11-24T06:26:56.011770+0000","last_active":"2023-11-24T06:26:59.156742+0000","last_peered":"2023-11-24T06:26:59.156742+0000","last_clean":"2023-11-24T06:26:59.156742+0000","last_became_active":"2023-11-24T06:26:56.011386+0000","last_became_peered":"2023-11-24T06:26:56.011386+0000","last_unstale":"2023-11-24T06:26:59.156742+0000","last_undegraded":"2023-11-24T06:26:59.156742+0000","last_fullsized":"2023-11-24T06:26:59.156742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945877+0000","last_change":"2023-11-24T06:26:56.978788+0000","last_active":"2023-11-24T06:26:57.945877+0000","last_peered":"2023-11-24T06:26:57.945877+0000","last_clean":"2023-11-24T06:26:57.945877+0000","last_became_active":"2023-11-24T06:26:56.978112+0000","last_became_peered":"2023-11-24T06:26:56.978112+0000","last_unstale":"2023-11-24T06:26:57.945877+0000","last_undegraded":"2023-11-24T06:26:57.945877+0000","last_fullsized":"2023-11-24T06:26:57.945877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220478+0000","last_change":"2023-11-24T06:26:56.012367+0000","last_active":"2023-11-24T06:26:58.220478+0000","last_peered":"2023-11-24T06:26:58.220478+0000","last_clean":"2023-11-24T06:26:58.220478+0000","last_became_active":"2023-11-24T06:26:56.011973+0000","last_became_peered":"2023-11-24T06:26:56.011973+0000","last_unstale":"2023-11-24T06:26:58.220478+0000","last_undegraded":"2023-11-24T06:26:58.220478+0000","last_fullsized":"2023-11-24T06:26:58.220478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218351+0000","last_change":"2023-11-24T06:26:56.976256+0000","last_active":"2023-11-24T06:26:58.218351+0000","last_peered":"2023-11-24T06:26:58.218351+0000","last_clean":"2023-11-24T06:26:58.218351+0000","last_became_active":"2023-11-24T06:26:56.975889+0000","last_became_peered":"2023-11-24T06:26:56.975889+0000","last_unstale":"2023-11-24T06:26:58.218351+0000","last_undegraded":"2023-11-24T06:26:58.218351+0000","last_fullsized":"2023-11-24T06:26:58.218351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946577+0000","last_change":"2023-11-24T06:26:56.008104+0000","last_active":"2023-11-24T06:26:57.946577+0000","last_peered":"2023-11-24T06:26:57.946577+0000","last_clean":"2023-11-24T06:26:57.946577+0000","last_became_active":"2023-11-24T06:26:56.007669+0000","last_became_peered":"2023-11-24T06:26:56.007669+0000","last_unstale":"2023-11-24T06:26:57.946577+0000","last_undegraded":"2023-11-24T06:26:57.946577+0000","last_fullsized":"2023-11-24T06:26:57.946577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:27.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220486+0000","last_change":"2023-11-24T06:26:56.974769+0000","last_active":"2023-11-24T06:26:58.220486+0000","last_peered":"2023-11-24T06:26:58.220486+0000","last_clean":"2023-11-24T06:26:58.220486+0000","last_became_active":"2023-11-24T06:26:56.974530+0000","last_became_peered":"2023-11-24T06:26:56.974530+0000","last_unstale":"2023-11-24T06:26:58.220486+0000","last_undegraded":"2023-11-24T06:26:58.220486+0000","last_fullsized":"2023-11-24T06:26:58.220486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:02.036523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944166+0000","last_change":"2023-11-24T06:26:56.001637+0000","last_active":"2023-11-24T06:26:57.944166+0000","last_peered":"2023-11-24T06:26:57.944166+0000","last_clean":"2023-11-24T06:26:57.944166+0000","last_became_active":"2023-11-24T06:26:56.000662+0000","last_became_peered":"2023-11-24T06:26:56.000662+0000","last_unstale":"2023-11-24T06:26:57.944166+0000","last_undegraded":"2023-11-24T06:26:57.944166+0000","last_fullsized":"2023-11-24T06:26:57.944166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:49.146302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945138+0000","last_change":"2023-11-24T06:26:56.961879+0000","last_active":"2023-11-24T06:26:57.945138+0000","last_peered":"2023-11-24T06:26:57.945138+0000","last_clean":"2023-11-24T06:26:57.945138+0000","last_became_active":"2023-11-24T06:26:56.961593+0000","last_became_peered":"2023-11-24T06:26:56.961593+0000","last_unstale":"2023-11-24T06:26:57.945138+0000","last_undegraded":"2023-11-24T06:26:57.945138+0000","last_fullsized":"2023-11-24T06:26:57.945138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944425+0000","last_change":"2023-11-24T06:26:55.995684+0000","last_active":"2023-11-24T06:26:57.944425+0000","last_peered":"2023-11-24T06:26:57.944425+0000","last_clean":"2023-11-24T06:26:57.944425+0000","last_became_active":"2023-11-24T06:26:55.994739+0000","last_became_peered":"2023-11-24T06:26:55.994739+0000","last_unstale":"2023-11-24T06:26:57.944425+0000","last_undegraded":"2023-11-24T06:26:57.944425+0000","last_fullsized":"2023-11-24T06:26:57.944425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:59.647318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945536+0000","last_change":"2023-11-24T06:26:56.976564+0000","last_active":"2023-11-24T06:26:57.945536+0000","last_peered":"2023-11-24T06:26:57.945536+0000","last_clean":"2023-11-24T06:26:57.945536+0000","last_became_active":"2023-11-24T06:26:56.976081+0000","last_became_peered":"2023-11-24T06:26:56.976081+0000","last_unstale":"2023-11-24T06:26:57.945536+0000","last_undegraded":"2023-11-24T06:26:57.945536+0000","last_fullsized":"2023-11-24T06:26:57.945536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947699+0000","last_change":"2023-11-24T06:26:56.015851+0000","last_active":"2023-11-24T06:26:57.947699+0000","last_peered":"2023-11-24T06:26:57.947699+0000","last_clean":"2023-11-24T06:26:57.947699+0000","last_became_active":"2023-11-24T06:26:56.014016+0000","last_became_peered":"2023-11-24T06:26:56.014016+0000","last_unstale":"2023-11-24T06:26:57.947699+0000","last_undegraded":"2023-11-24T06:26:57.947699+0000","last_fullsized":"2023-11-24T06:26:57.947699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218077+0000","last_change":"2023-11-24T06:26:56.975486+0000","last_active":"2023-11-24T06:26:58.218077+0000","last_peered":"2023-11-24T06:26:58.218077+0000","last_clean":"2023-11-24T06:26:58.218077+0000","last_became_active":"2023-11-24T06:26:56.975147+0000","last_became_peered":"2023-11-24T06:26:56.975147+0000","last_unstale":"2023-11-24T06:26:58.218077+0000","last_undegraded":"2023-11-24T06:26:58.218077+0000","last_fullsized":"2023-11-24T06:26:58.218077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222273+0000","last_change":"2023-11-24T06:26:56.994982+0000","last_active":"2023-11-24T06:26:58.222273+0000","last_peered":"2023-11-24T06:26:58.222273+0000","last_clean":"2023-11-24T06:26:58.222273+0000","last_became_active":"2023-11-24T06:26:56.993765+0000","last_became_peered":"2023-11-24T06:26:56.993765+0000","last_unstale":"2023-11-24T06:26:58.222273+0000","last_undegraded":"2023-11-24T06:26:58.222273+0000","last_fullsized":"2023-11-24T06:26:58.222273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947612+0000","last_change":"2023-11-24T06:26:56.015039+0000","last_active":"2023-11-24T06:26:57.947612+0000","last_peered":"2023-11-24T06:26:57.947612+0000","last_clean":"2023-11-24T06:26:57.947612+0000","last_became_active":"2023-11-24T06:26:56.014562+0000","last_became_peered":"2023-11-24T06:26:56.014562+0000","last_unstale":"2023-11-24T06:26:57.947612+0000","last_undegraded":"2023-11-24T06:26:57.947612+0000","last_fullsized":"2023-11-24T06:26:57.947612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222503+0000","last_change":"2023-11-24T06:26:56.992781+0000","last_active":"2023-11-24T06:26:58.222503+0000","last_peered":"2023-11-24T06:26:58.222503+0000","last_clean":"2023-11-24T06:26:58.222503+0000","last_became_active":"2023-11-24T06:26:56.992337+0000","last_became_peered":"2023-11-24T06:26:56.992337+0000","last_unstale":"2023-11-24T06:26:58.222503+0000","last_undegraded":"2023-11-24T06:26:58.222503+0000","last_fullsized":"2023-11-24T06:26:58.222503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:15.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947644+0000","last_change":"2023-11-24T06:26:56.015616+0000","last_active":"2023-11-24T06:26:57.947644+0000","last_peered":"2023-11-24T06:26:57.947644+0000","last_clean":"2023-11-24T06:26:57.947644+0000","last_became_active":"2023-11-24T06:26:56.015317+0000","last_became_peered":"2023-11-24T06:26:56.015317+0000","last_unstale":"2023-11-24T06:26:57.947644+0000","last_undegraded":"2023-11-24T06:26:57.947644+0000","last_fullsized":"2023-11-24T06:26:57.947644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:10.844401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218457+0000","last_change":"2023-11-24T06:26:57.001097+0000","last_active":"2023-11-24T06:26:58.218457+0000","last_peered":"2023-11-24T06:26:58.218457+0000","last_clean":"2023-11-24T06:26:58.218457+0000","last_became_active":"2023-11-24T06:26:57.000751+0000","last_became_peered":"2023-11-24T06:26:57.000751+0000","last_unstale":"2023-11-24T06:26:58.218457+0000","last_undegraded":"2023-11-24T06:26:58.218457+0000","last_fullsized":"2023-11-24T06:26:58.218457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945720+0000","last_change":"2023-11-24T06:26:56.003621+0000","last_active":"2023-11-24T06:26:57.945720+0000","last_peered":"2023-11-24T06:26:57.945720+0000","last_clean":"2023-11-24T06:26:57.945720+0000","last_became_active":"2023-11-24T06:26:56.003232+0000","last_became_peered":"2023-11-24T06:26:56.003232+0000","last_unstale":"2023-11-24T06:26:57.945720+0000","last_undegraded":"2023-11-24T06:26:57.945720+0000","last_fullsized":"2023-11-24T06:26:57.945720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:10.844401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945648+0000","last_change":"2023-11-24T06:26:56.972822+0000","last_active":"2023-11-24T06:26:57.945648+0000","last_peered":"2023-11-24T06:26:57.945648+0000","last_clean":"2023-11-24T06:26:57.945648+0000","last_became_active":"2023-11-24T06:26:56.972529+0000","last_became_peered":"2023-11-24T06:26:56.972529+0000","last_unstale":"2023-11-24T06:26:57.945648+0000","last_undegraded":"2023-11-24T06:26:57.945648+0000","last_fullsized":"2023-11-24T06:26:57.945648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218623+0000","last_change":"2023-11-24T06:26:56.009185+0000","last_active":"2023-11-24T06:26:58.218623+0000","last_peered":"2023-11-24T06:26:58.218623+0000","last_clean":"2023-11-24T06:26:58.218623+0000","last_became_active":"2023-11-24T06:26:56.008855+0000","last_became_peered":"2023-11-24T06:26:56.008855+0000","last_unstale":"2023-11-24T06:26:58.218623+0000","last_undegraded":"2023-11-24T06:26:58.218623+0000","last_fullsized":"2023-11-24T06:26:58.218623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3090000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.464}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.706}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.117}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2349999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4180000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2639999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.528}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0089999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1349999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8460000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.722}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2440000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3959999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:07.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:27:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.452+0000 7f6194dde640 1 Processor -- start 2023-11-24T06:27:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.453+0000 7f6194dde640 1 -- start start 2023-11-24T06:27:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.454+0000 7f6194dde640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619015c360 0x7f61900b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.454+0000 7f6194dde640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f619006ebc0 0x7f619006f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.454+0000 7f6194dde640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61900af150 con 0x7f6190152250 2023-11-24T06:27:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.454+0000 7f6194dde640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619006f550 con 0x7f619006ebc0 2023-11-24T06:27:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.454+0000 7f6194dde640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61900ce050 con 0x7f619015c360 2023-11-24T06:27:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f618e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6190152250 0x7f619006b3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52880/0 (socket says 172.21.15.16:52880) 2023-11-24T06:27:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f618e59c640 1 -- 172.21.15.16:0/968703142 learned_addr learned my addr 172.21.15.16:0/968703142 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f618ed9d640 1 --2- 172.21.15.16:0/968703142 >> v2:172.21.15.16:3300/0 conn(0x7f619006ebc0 0x7f619006f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f6187fff640 1 --2- 172.21.15.16:0/968703142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619015c360 0x7f61900b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f618dd9b640 1 -- 172.21.15.16:0/968703142 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123677713 0 0) 0x7f61900af150 con 0x7f6190152250 2023-11-24T06:27:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f618dd9b640 1 -- 172.21.15.16:0/968703142 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f617c003520 con 0x7f6190152250 2023-11-24T06:27:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f618ed9d640 1 -- 172.21.15.16:0/968703142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619015c360 msgr2=0x7f61900b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f618ed9d640 1 --2- 172.21.15.16:0/968703142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619015c360 0x7f61900b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f618ed9d640 1 -- 172.21.15.16:0/968703142 >> v1:172.21.15.16:6789/0 conn(0x7f6190152250 legacy=0x7f619006b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.455+0000 7f618ed9d640 1 -- 172.21.15.16:0/968703142 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6190152630 con 0x7f619006ebc0 2023-11-24T06:27:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.456+0000 7f618ed9d640 1 --2- 172.21.15.16:0/968703142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619006ebc0 0x7f619006f010 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61880097a0 tx=0x7f618802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27f971a03375383 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f618dd9b640 1 -- 172.21.15.16:0/968703142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618802e070 con 0x7f619006ebc0 2023-11-24T06:27:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f618dd9b640 1 -- 172.21.15.16:0/968703142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61880315a0 con 0x7f619006ebc0 2023-11-24T06:27:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f618dd9b640 1 -- 172.21.15.16:0/968703142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188031930 con 0x7f619006ebc0 2023-11-24T06:27:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f6194dde640 1 -- 172.21.15.16:0/968703142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619006ebc0 msgr2=0x7f619006f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f6194dde640 1 --2- 172.21.15.16:0/968703142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619006ebc0 0x7f619006f010 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61880097a0 tx=0x7f618802b840 comp rx=0 tx=0).stop 2023-11-24T06:27:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f6194dde640 1 -- 172.21.15.16:0/968703142 shutdown_connections 2023-11-24T06:27:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f6194dde640 1 --2- 172.21.15.16:0/968703142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619006ebc0 0x7f619006f010 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f6194dde640 1 --2- 172.21.15.16:0/968703142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619015c360 0x7f61900b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f6194dde640 1 -- 172.21.15.16:0/968703142 >> 172.21.15.16:0/968703142 conn(0x7f6190070b10 msgr2=0x7f6190070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.457+0000 7f6194dde640 1 -- 172.21.15.16:0/968703142 shutdown_connections 2023-11-24T06:27:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.458+0000 7f6194dde640 1 -- 172.21.15.16:0/968703142 wait complete. 2023-11-24T06:27:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.459+0000 7f6194dde640 1 Processor -- start 2023-11-24T06:27:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.460+0000 7f6194dde640 1 -- start start 2023-11-24T06:27:07.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f6194dde640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619006ebc0 0x7f619006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f6194dde640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6190152250 0x7f6190155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f6194dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619015c360 0x7f619015a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f6194dde640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619006da50 con 0x7f619015c360 2023-11-24T06:27:07.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f6194dde640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619006ce40 con 0x7f6190152250 2023-11-24T06:27:07.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f6194dde640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619006d120 con 0x7f619006ebc0 2023-11-24T06:27:07.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f618ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619015c360 0x7f619015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f618ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619015c360 0x7f619015a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38502/0 (socket says 172.21.15.16:38502) 2023-11-24T06:27:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f618ed9d640 1 -- 172.21.15.16:0/833591295 learned_addr learned my addr 172.21.15.16:0/833591295 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f618e59c640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619006ebc0 0x7f619006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f618ed9d640 1 -- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619006ebc0 msgr2=0x7f619006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f6187fff640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6190152250 0x7f6190155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f618ed9d640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619006ebc0 0x7f619006dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f618ed9d640 1 -- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6190152250 msgr2=0x7f6190155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f618ed9d640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6190152250 0x7f6190155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.461+0000 7f618ed9d640 1 -- 172.21.15.16:0/833591295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188009400 con 0x7f619015c360 2023-11-24T06:27:07.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.462+0000 7f618ed9d640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619015c360 0x7f619015a490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6188034040 tx=0x7f61880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.462+0000 7f6186ffd640 1 -- 172.21.15.16:0/833591295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188032840 con 0x7f619015c360 2023-11-24T06:27:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.462+0000 7f6186ffd640 1 -- 172.21.15.16:0/833591295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61880329a0 con 0x7f619015c360 2023-11-24T06:27:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.462+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619006d400 con 0x7f619015c360 2023-11-24T06:27:07.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.462+0000 7f6186ffd640 1 -- 172.21.15.16:0/833591295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6188032d50 con 0x7f619015c360 2023-11-24T06:27:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.462+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6190155680 con 0x7f619015c360 2023-11-24T06:27:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.463+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619006dbc0 con 0x7f619015c360 2023-11-24T06:27:07.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.464+0000 7f6186ffd640 1 -- 172.21.15.16:0/833591295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f618804e400 con 0x7f619015c360 2023-11-24T06:27:07.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.465+0000 7f6186ffd640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f617006ae90 0x7f617006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.465+0000 7f6186ffd640 1 -- 172.21.15.16:0/833591295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61880ab320 con 0x7f619015c360 2023-11-24T06:27:07.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.466+0000 7f618e59c640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f617006ae90 0x7f617006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.467+0000 7f618e59c640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f617006ae90 0x7f617006d350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6180001560 tx=0x7f6180009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:07.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.469+0000 7f6186ffd640 1 -- 172.21.15.16:0/833591295 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6188076440 con 0x7f619015c360 2023-11-24T06:27:07.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.654+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f619015eb60 con 0x7f617006ae90 2023-11-24T06:27:07.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.675+0000 7f6186ffd640 1 -- 172.21.15.16:0/833591295 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264380 (secure 0 0 0) 0x7f619015eb60 con 0x7f617006ae90 2023-11-24T06:27:07.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:07.682 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:27:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.679+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f617006ae90 msgr2=0x7f617006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.679+0000 7f6194dde640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f617006ae90 0x7f617006d350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6180001560 tx=0x7f6180009040 comp rx=0 tx=0).stop 2023-11-24T06:27:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619015c360 msgr2=0x7f619015a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619015c360 0x7f619015a490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6188034040 tx=0x7f61880046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 shutdown_connections 2023-11-24T06:27:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f617006ae90 0x7f617006d350 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f619015c360 0x7f619015a490 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6190152250 0x7f6190155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 --2- 172.21.15.16:0/833591295 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f619006ebc0 0x7f619006dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 >> 172.21.15.16:0/833591295 conn(0x7f6190070b10 msgr2=0x7f61900b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 shutdown_connections 2023-11-24T06:27:07.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.680+0000 7f6194dde640 1 -- 172.21.15.16:0/833591295 wait complete. 2023-11-24T06:27:07.702 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T06:27:06.103821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235876,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738844,"statfs":{"total":773094113280,"available":772852576256,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.163291"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948224+0000","last_change":"2023-11-24T06:26:56.969865+0000","last_active":"2023-11-24T06:26:57.948224+0000","last_peered":"2023-11-24T06:26:57.948224+0000","last_clean":"2023-11-24T06:26:57.948224+0000","last_became_active":"2023-11-24T06:26:56.969570+0000","last_became_peered":"2023-11-24T06:26:56.969570+0000","last_unstale":"2023-11-24T06:26:57.948224+0000","last_undegraded":"2023-11-24T06:26:57.948224+0000","last_fullsized":"2023-11-24T06:26:57.948224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:56.199949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944608+0000","last_change":"2023-11-24T06:26:56.979241+0000","last_active":"2023-11-24T06:26:57.944608+0000","last_peered":"2023-11-24T06:26:57.944608+0000","last_clean":"2023-11-24T06:26:57.944608+0000","last_became_active":"2023-11-24T06:26:56.978864+0000","last_became_peered":"2023-11-24T06:26:56.978864+0000","last_unstale":"2023-11-24T06:26:57.944608+0000","last_undegraded":"2023-11-24T06:26:57.944608+0000","last_fullsized":"2023-11-24T06:26:57.944608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218032+0000","last_change":"2023-11-24T06:26:56.976436+0000","last_active":"2023-11-24T06:26:58.218032+0000","last_peered":"2023-11-24T06:26:58.218032+0000","last_clean":"2023-11-24T06:26:58.218032+0000","last_became_active":"2023-11-24T06:26:56.976114+0000","last_became_peered":"2023-11-24T06:26:56.976114+0000","last_unstale":"2023-11-24T06:26:58.218032+0000","last_undegraded":"2023-11-24T06:26:58.218032+0000","last_fullsized":"2023-11-24T06:26:58.218032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222085+0000","last_change":"2023-11-24T06:26:56.992564+0000","last_active":"2023-11-24T06:26:58.222085+0000","last_peered":"2023-11-24T06:26:58.222085+0000","last_clean":"2023-11-24T06:26:58.222085+0000","last_became_active":"2023-11-24T06:26:56.992045+0000","last_became_peered":"2023-11-24T06:26:56.992045+0000","last_unstale":"2023-11-24T06:26:58.222085+0000","last_undegraded":"2023-11-24T06:26:58.222085+0000","last_fullsized":"2023-11-24T06:26:58.222085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947424+0000","last_change":"2023-11-24T06:26:56.964941+0000","last_active":"2023-11-24T06:26:57.947424+0000","last_peered":"2023-11-24T06:26:57.947424+0000","last_clean":"2023-11-24T06:26:57.947424+0000","last_became_active":"2023-11-24T06:26:56.964244+0000","last_became_peered":"2023-11-24T06:26:56.964244+0000","last_unstale":"2023-11-24T06:26:57.947424+0000","last_undegraded":"2023-11-24T06:26:57.947424+0000","last_fullsized":"2023-11-24T06:26:57.947424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219175+0000","last_change":"2023-11-24T06:26:56.993038+0000","last_active":"2023-11-24T06:26:58.219175+0000","last_peered":"2023-11-24T06:26:58.219175+0000","last_clean":"2023-11-24T06:26:58.219175+0000","last_became_active":"2023-11-24T06:26:56.986387+0000","last_became_peered":"2023-11-24T06:26:56.986387+0000","last_unstale":"2023-11-24T06:26:58.219175+0000","last_undegraded":"2023-11-24T06:26:58.219175+0000","last_fullsized":"2023-11-24T06:26:58.219175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946794+0000","last_change":"2023-11-24T06:26:56.977591+0000","last_active":"2023-11-24T06:26:57.946794+0000","last_peered":"2023-11-24T06:26:57.946794+0000","last_clean":"2023-11-24T06:26:57.946794+0000","last_became_active":"2023-11-24T06:26:56.976851+0000","last_became_peered":"2023-11-24T06:26:56.976851+0000","last_unstale":"2023-11-24T06:26:57.946794+0000","last_undegraded":"2023-11-24T06:26:57.946794+0000","last_fullsized":"2023-11-24T06:26:57.946794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944925+0000","last_change":"2023-11-24T06:26:56.991219+0000","last_active":"2023-11-24T06:26:57.944925+0000","last_peered":"2023-11-24T06:26:57.944925+0000","last_clean":"2023-11-24T06:26:57.944925+0000","last_became_active":"2023-11-24T06:26:56.990836+0000","last_became_peered":"2023-11-24T06:26:56.990836+0000","last_unstale":"2023-11-24T06:26:57.944925+0000","last_undegraded":"2023-11-24T06:26:57.944925+0000","last_fullsized":"2023-11-24T06:26:57.944925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948378+0000","last_change":"2023-11-24T06:26:57.001238+0000","last_active":"2023-11-24T06:26:57.948378+0000","last_peered":"2023-11-24T06:26:57.948378+0000","last_clean":"2023-11-24T06:26:57.948378+0000","last_became_active":"2023-11-24T06:26:57.000919+0000","last_became_peered":"2023-11-24T06:26:57.000919+0000","last_unstale":"2023-11-24T06:26:57.948378+0000","last_undegraded":"2023-11-24T06:26:57.948378+0000","last_fullsized":"2023-11-24T06:26:57.948378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:38.348575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219671+0000","last_change":"2023-11-24T06:26:56.998442+0000","last_active":"2023-11-24T06:26:58.219671+0000","last_peered":"2023-11-24T06:26:58.219671+0000","last_clean":"2023-11-24T06:26:58.219671+0000","last_became_active":"2023-11-24T06:26:56.998097+0000","last_became_peered":"2023-11-24T06:26:56.998097+0000","last_unstale":"2023-11-24T06:26:58.219671+0000","last_undegraded":"2023-11-24T06:26:58.219671+0000","last_fullsized":"2023-11-24T06:26:58.219671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218262+0000","last_change":"2023-11-24T06:26:57.000349+0000","last_active":"2023-11-24T06:26:58.218262+0000","last_peered":"2023-11-24T06:26:58.218262+0000","last_clean":"2023-11-24T06:26:58.218262+0000","last_became_active":"2023-11-24T06:26:57.000122+0000","last_became_peered":"2023-11-24T06:26:57.000122+0000","last_unstale":"2023-11-24T06:26:58.218262+0000","last_undegraded":"2023-11-24T06:26:58.218262+0000","last_fullsized":"2023-11-24T06:26:58.218262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221702+0000","last_change":"2023-11-24T06:26:56.969362+0000","last_active":"2023-11-24T06:26:58.221702+0000","last_peered":"2023-11-24T06:26:58.221702+0000","last_clean":"2023-11-24T06:26:58.221702+0000","last_became_active":"2023-11-24T06:26:56.969057+0000","last_became_peered":"2023-11-24T06:26:56.969057+0000","last_unstale":"2023-11-24T06:26:58.221702+0000","last_undegraded":"2023-11-24T06:26:58.221702+0000","last_fullsized":"2023-11-24T06:26:58.221702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:04.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945262+0000","last_change":"2023-11-24T06:26:56.970657+0000","last_active":"2023-11-24T06:26:57.945262+0000","last_peered":"2023-11-24T06:26:57.945262+0000","last_clean":"2023-11-24T06:26:57.945262+0000","last_became_active":"2023-11-24T06:26:56.970348+0000","last_became_peered":"2023-11-24T06:26:56.970348+0000","last_unstale":"2023-11-24T06:26:57.945262+0000","last_undegraded":"2023-11-24T06:26:57.945262+0000","last_fullsized":"2023-11-24T06:26:57.945262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217421+0000","last_change":"2023-11-24T06:26:56.972241+0000","last_active":"2023-11-24T06:26:58.217421+0000","last_peered":"2023-11-24T06:26:58.217421+0000","last_clean":"2023-11-24T06:26:58.217421+0000","last_became_active":"2023-11-24T06:26:56.971982+0000","last_became_peered":"2023-11-24T06:26:56.971982+0000","last_unstale":"2023-11-24T06:26:58.217421+0000","last_undegraded":"2023-11-24T06:26:58.217421+0000","last_fullsized":"2023-11-24T06:26:58.217421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:19.450311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944041+0000","last_change":"2023-11-24T06:26:56.978171+0000","last_active":"2023-11-24T06:26:57.944041+0000","last_peered":"2023-11-24T06:26:57.944041+0000","last_clean":"2023-11-24T06:26:57.944041+0000","last_became_active":"2023-11-24T06:26:56.977851+0000","last_became_peered":"2023-11-24T06:26:56.977851+0000","last_unstale":"2023-11-24T06:26:57.944041+0000","last_undegraded":"2023-11-24T06:26:57.944041+0000","last_fullsized":"2023-11-24T06:26:57.944041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218965+0000","last_change":"2023-11-24T06:26:56.991619+0000","last_active":"2023-11-24T06:26:58.218965+0000","last_peered":"2023-11-24T06:26:58.218965+0000","last_clean":"2023-11-24T06:26:58.218965+0000","last_became_active":"2023-11-24T06:26:56.986865+0000","last_became_peered":"2023-11-24T06:26:56.986865+0000","last_unstale":"2023-11-24T06:26:58.218965+0000","last_undegraded":"2023-11-24T06:26:58.218965+0000","last_fullsized":"2023-11-24T06:26:58.218965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.223090+0000","last_change":"2023-11-24T06:26:56.997282+0000","last_active":"2023-11-24T06:26:58.223090+0000","last_peered":"2023-11-24T06:26:58.223090+0000","last_clean":"2023-11-24T06:26:58.223090+0000","last_became_active":"2023-11-24T06:26:56.995631+0000","last_became_peered":"2023-11-24T06:26:56.995631+0000","last_unstale":"2023-11-24T06:26:58.223090+0000","last_undegraded":"2023-11-24T06:26:58.223090+0000","last_fullsized":"2023-11-24T06:26:58.223090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:09:38.348575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217200+0000","last_change":"2023-11-24T06:26:56.969721+0000","last_active":"2023-11-24T06:26:58.217200+0000","last_peered":"2023-11-24T06:26:58.217200+0000","last_clean":"2023-11-24T06:26:58.217200+0000","last_became_active":"2023-11-24T06:26:56.969475+0000","last_became_peered":"2023-11-24T06:26:56.969475+0000","last_unstale":"2023-11-24T06:26:58.217200+0000","last_undegraded":"2023-11-24T06:26:58.217200+0000","last_fullsized":"2023-11-24T06:26:58.217200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:04.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947802+0000","last_change":"2023-11-24T06:26:56.964126+0000","last_active":"2023-11-24T06:26:57.947802+0000","last_peered":"2023-11-24T06:26:57.947802+0000","last_clean":"2023-11-24T06:26:57.947802+0000","last_became_active":"2023-11-24T06:26:56.963622+0000","last_became_peered":"2023-11-24T06:26:56.963622+0000","last_unstale":"2023-11-24T06:26:57.947802+0000","last_undegraded":"2023-11-24T06:26:57.947802+0000","last_fullsized":"2023-11-24T06:26:57.947802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:17.662630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218464+0000","last_change":"2023-11-24T06:26:56.975222+0000","last_active":"2023-11-24T06:26:58.218464+0000","last_peered":"2023-11-24T06:26:58.218464+0000","last_clean":"2023-11-24T06:26:58.218464+0000","last_became_active":"2023-11-24T06:26:56.974069+0000","last_became_peered":"2023-11-24T06:26:56.974069+0000","last_unstale":"2023-11-24T06:26:58.218464+0000","last_undegraded":"2023-11-24T06:26:58.218464+0000","last_fullsized":"2023-11-24T06:26:58.218464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218120+0000","last_change":"2023-11-24T06:26:56.975253+0000","last_active":"2023-11-24T06:26:58.218120+0000","last_peered":"2023-11-24T06:26:58.218120+0000","last_clean":"2023-11-24T06:26:58.218120+0000","last_became_active":"2023-11-24T06:26:56.974888+0000","last_became_peered":"2023-11-24T06:26:56.974888+0000","last_unstale":"2023-11-24T06:26:58.218120+0000","last_undegraded":"2023-11-24T06:26:58.218120+0000","last_fullsized":"2023-11-24T06:26:58.218120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221143+0000","last_change":"2023-11-24T06:26:56.985344+0000","last_active":"2023-11-24T06:26:58.221143+0000","last_peered":"2023-11-24T06:26:58.221143+0000","last_clean":"2023-11-24T06:26:58.221143+0000","last_became_active":"2023-11-24T06:26:56.984528+0000","last_became_peered":"2023-11-24T06:26:56.984528+0000","last_unstale":"2023-11-24T06:26:58.221143+0000","last_undegraded":"2023-11-24T06:26:58.221143+0000","last_fullsized":"2023-11-24T06:26:58.221143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944974+0000","last_change":"2023-11-24T06:26:56.979949+0000","last_active":"2023-11-24T06:26:57.944974+0000","last_peered":"2023-11-24T06:26:57.944974+0000","last_clean":"2023-11-24T06:26:57.944974+0000","last_became_active":"2023-11-24T06:26:56.979635+0000","last_became_peered":"2023-11-24T06:26:56.979635+0000","last_unstale":"2023-11-24T06:26:57.944974+0000","last_undegraded":"2023-11-24T06:26:57.944974+0000","last_fullsized":"2023-11-24T06:26:57.944974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219328+0000","last_change":"2023-11-24T06:26:56.968934+0000","last_active":"2023-11-24T06:26:58.219328+0000","last_peered":"2023-11-24T06:26:58.219328+0000","last_clean":"2023-11-24T06:26:58.219328+0000","last_became_active":"2023-11-24T06:26:56.967198+0000","last_became_peered":"2023-11-24T06:26:56.967198+0000","last_unstale":"2023-11-24T06:26:58.219328+0000","last_undegraded":"2023-11-24T06:26:58.219328+0000","last_fullsized":"2023-11-24T06:26:58.219328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945713+0000","last_change":"2023-11-24T06:26:56.974003+0000","last_active":"2023-11-24T06:26:57.945713+0000","last_peered":"2023-11-24T06:26:57.945713+0000","last_clean":"2023-11-24T06:26:57.945713+0000","last_became_active":"2023-11-24T06:26:56.973631+0000","last_became_peered":"2023-11-24T06:26:56.973631+0000","last_unstale":"2023-11-24T06:26:57.945713+0000","last_undegraded":"2023-11-24T06:26:57.945713+0000","last_fullsized":"2023-11-24T06:26:57.945713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:28.699614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219990+0000","last_change":"2023-11-24T06:26:56.972613+0000","last_active":"2023-11-24T06:26:58.219990+0000","last_peered":"2023-11-24T06:26:58.219990+0000","last_clean":"2023-11-24T06:26:58.219990+0000","last_became_active":"2023-11-24T06:26:56.972280+0000","last_became_peered":"2023-11-24T06:26:56.972280+0000","last_unstale":"2023-11-24T06:26:58.219990+0000","last_undegraded":"2023-11-24T06:26:58.219990+0000","last_fullsized":"2023-11-24T06:26:58.219990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944265+0000","last_change":"2023-11-24T06:26:56.966680+0000","last_active":"2023-11-24T06:26:57.944265+0000","last_peered":"2023-11-24T06:26:57.944265+0000","last_clean":"2023-11-24T06:26:57.944265+0000","last_became_active":"2023-11-24T06:26:56.966315+0000","last_became_peered":"2023-11-24T06:26:56.966315+0000","last_unstale":"2023-11-24T06:26:57.944265+0000","last_undegraded":"2023-11-24T06:26:57.944265+0000","last_fullsized":"2023-11-24T06:26:57.944265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945586+0000","last_change":"2023-11-24T06:26:56.974920+0000","last_active":"2023-11-24T06:26:57.945586+0000","last_peered":"2023-11-24T06:26:57.945586+0000","last_clean":"2023-11-24T06:26:57.945586+0000","last_became_active":"2023-11-24T06:26:56.974594+0000","last_became_peered":"2023-11-24T06:26:56.974594+0000","last_unstale":"2023-11-24T06:26:57.945586+0000","last_undegraded":"2023-11-24T06:26:57.945586+0000","last_fullsized":"2023-11-24T06:26:57.945586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217994+0000","last_change":"2023-11-24T06:26:56.976017+0000","last_active":"2023-11-24T06:26:58.217994+0000","last_peered":"2023-11-24T06:26:58.217994+0000","last_clean":"2023-11-24T06:26:58.217994+0000","last_became_active":"2023-11-24T06:26:56.975709+0000","last_became_peered":"2023-11-24T06:26:56.975709+0000","last_unstale":"2023-11-24T06:26:58.217994+0000","last_undegraded":"2023-11-24T06:26:58.217994+0000","last_fullsized":"2023-11-24T06:26:58.217994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221250+0000","last_change":"2023-11-24T06:26:56.980269+0000","last_active":"2023-11-24T06:26:58.221250+0000","last_peered":"2023-11-24T06:26:58.221250+0000","last_clean":"2023-11-24T06:26:58.221250+0000","last_became_active":"2023-11-24T06:26:56.979929+0000","last_became_peered":"2023-11-24T06:26:56.979929+0000","last_unstale":"2023-11-24T06:26:58.221250+0000","last_undegraded":"2023-11-24T06:26:58.221250+0000","last_fullsized":"2023-11-24T06:26:58.221250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216766+0000","last_change":"2023-11-24T06:26:56.986717+0000","last_active":"2023-11-24T06:26:58.216766+0000","last_peered":"2023-11-24T06:26:58.216766+0000","last_clean":"2023-11-24T06:26:58.216766+0000","last_became_active":"2023-11-24T06:26:56.986398+0000","last_became_peered":"2023-11-24T06:26:56.986398+0000","last_unstale":"2023-11-24T06:26:58.216766+0000","last_undegraded":"2023-11-24T06:26:58.216766+0000","last_fullsized":"2023-11-24T06:26:58.216766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:47.926009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217958+0000","last_change":"2023-11-24T06:26:56.978275+0000","last_active":"2023-11-24T06:26:58.217958+0000","last_peered":"2023-11-24T06:26:58.217958+0000","last_clean":"2023-11-24T06:26:58.217958+0000","last_became_active":"2023-11-24T06:26:56.977924+0000","last_became_peered":"2023-11-24T06:26:56.977924+0000","last_unstale":"2023-11-24T06:26:58.217958+0000","last_undegraded":"2023-11-24T06:26:58.217958+0000","last_fullsized":"2023-11-24T06:26:58.217958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:58.210301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220024+0000","last_change":"2023-11-24T06:26:56.970266+0000","last_active":"2023-11-24T06:26:58.220024+0000","last_peered":"2023-11-24T06:26:58.220024+0000","last_clean":"2023-11-24T06:26:58.220024+0000","last_became_active":"2023-11-24T06:26:56.969973+0000","last_became_peered":"2023-11-24T06:26:56.969973+0000","last_unstale":"2023-11-24T06:26:58.220024+0000","last_undegraded":"2023-11-24T06:26:58.220024+0000","last_fullsized":"2023-11-24T06:26:58.220024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:28.630873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945477+0000","last_change":"2023-11-24T06:26:56.008228+0000","last_active":"2023-11-24T06:26:57.945477+0000","last_peered":"2023-11-24T06:26:57.945477+0000","last_clean":"2023-11-24T06:26:57.945477+0000","last_became_active":"2023-11-24T06:26:56.007916+0000","last_became_peered":"2023-11-24T06:26:56.007916+0000","last_unstale":"2023-11-24T06:26:57.945477+0000","last_undegraded":"2023-11-24T06:26:57.945477+0000","last_fullsized":"2023-11-24T06:26:57.945477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218141+0000","last_change":"2023-11-24T06:26:56.982794+0000","last_active":"2023-11-24T06:26:58.218141+0000","last_peered":"2023-11-24T06:26:58.218141+0000","last_clean":"2023-11-24T06:26:58.218141+0000","last_became_active":"2023-11-24T06:26:56.982455+0000","last_became_peered":"2023-11-24T06:26:56.982455+0000","last_unstale":"2023-11-24T06:26:58.218141+0000","last_undegraded":"2023-11-24T06:26:58.218141+0000","last_fullsized":"2023-11-24T06:26:58.218141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.624873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218348+0000","last_change":"2023-11-24T06:26:56.010407+0000","last_active":"2023-11-24T06:26:58.218348+0000","last_peered":"2023-11-24T06:26:58.218348+0000","last_clean":"2023-11-24T06:26:58.218348+0000","last_became_active":"2023-11-24T06:26:56.009548+0000","last_became_peered":"2023-11-24T06:26:56.009548+0000","last_unstale":"2023-11-24T06:26:58.218348+0000","last_undegraded":"2023-11-24T06:26:58.218348+0000","last_fullsized":"2023-11-24T06:26:58.218348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:58.487660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945323+0000","last_change":"2023-11-24T06:26:56.005804+0000","last_active":"2023-11-24T06:26:57.945323+0000","last_peered":"2023-11-24T06:26:57.945323+0000","last_clean":"2023-11-24T06:26:57.945323+0000","last_became_active":"2023-11-24T06:26:56.005473+0000","last_became_peered":"2023-11-24T06:26:56.005473+0000","last_unstale":"2023-11-24T06:26:57.945323+0000","last_undegraded":"2023-11-24T06:26:57.945323+0000","last_fullsized":"2023-11-24T06:26:57.945323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:18.225163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944935+0000","last_change":"2023-11-24T06:26:56.977459+0000","last_active":"2023-11-24T06:26:57.944935+0000","last_peered":"2023-11-24T06:26:57.944935+0000","last_clean":"2023-11-24T06:26:57.944935+0000","last_became_active":"2023-11-24T06:26:56.977160+0000","last_became_peered":"2023-11-24T06:26:56.977160+0000","last_unstale":"2023-11-24T06:26:57.944935+0000","last_undegraded":"2023-11-24T06:26:57.944935+0000","last_fullsized":"2023-11-24T06:26:57.944935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219546+0000","last_change":"2023-11-24T06:26:56.010202+0000","last_active":"2023-11-24T06:26:58.219546+0000","last_peered":"2023-11-24T06:26:58.219546+0000","last_clean":"2023-11-24T06:26:58.219546+0000","last_became_active":"2023-11-24T06:26:56.009924+0000","last_became_peered":"2023-11-24T06:26:56.009924+0000","last_unstale":"2023-11-24T06:26:58.219546+0000","last_undegraded":"2023-11-24T06:26:58.219546+0000","last_fullsized":"2023-11-24T06:26:58.219546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:17.355207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944060+0000","last_change":"2023-11-24T06:26:56.962258+0000","last_active":"2023-11-24T06:26:57.944060+0000","last_peered":"2023-11-24T06:26:57.944060+0000","last_clean":"2023-11-24T06:26:57.944060+0000","last_became_active":"2023-11-24T06:26:56.961665+0000","last_became_peered":"2023-11-24T06:26:56.961665+0000","last_unstale":"2023-11-24T06:26:57.944060+0000","last_undegraded":"2023-11-24T06:26:57.944060+0000","last_fullsized":"2023-11-24T06:26:57.944060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221161+0000","last_change":"2023-11-24T06:26:56.017642+0000","last_active":"2023-11-24T06:26:58.221161+0000","last_peered":"2023-11-24T06:26:58.221161+0000","last_clean":"2023-11-24T06:26:58.221161+0000","last_became_active":"2023-11-24T06:26:56.017386+0000","last_became_peered":"2023-11-24T06:26:56.017386+0000","last_unstale":"2023-11-24T06:26:58.221161+0000","last_undegraded":"2023-11-24T06:26:58.221161+0000","last_fullsized":"2023-11-24T06:26:58.221161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:18.225163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222938+0000","last_change":"2023-11-24T06:26:56.985441+0000","last_active":"2023-11-24T06:26:58.222938+0000","last_peered":"2023-11-24T06:26:58.222938+0000","last_clean":"2023-11-24T06:26:58.222938+0000","last_became_active":"2023-11-24T06:26:56.984430+0000","last_became_peered":"2023-11-24T06:26:56.984430+0000","last_unstale":"2023-11-24T06:26:58.222938+0000","last_undegraded":"2023-11-24T06:26:58.222938+0000","last_fullsized":"2023-11-24T06:26:58.222938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:56.199949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222514+0000","last_change":"2023-11-24T06:26:56.017588+0000","last_active":"2023-11-24T06:26:58.222514+0000","last_peered":"2023-11-24T06:26:58.222514+0000","last_clean":"2023-11-24T06:26:58.222514+0000","last_became_active":"2023-11-24T06:26:56.017128+0000","last_became_peered":"2023-11-24T06:26:56.017128+0000","last_unstale":"2023-11-24T06:26:58.222514+0000","last_undegraded":"2023-11-24T06:26:58.222514+0000","last_fullsized":"2023-11-24T06:26:58.222514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218742+0000","last_change":"2023-11-24T06:26:57.002819+0000","last_active":"2023-11-24T06:26:58.218742+0000","last_peered":"2023-11-24T06:26:58.218742+0000","last_clean":"2023-11-24T06:26:58.218742+0000","last_became_active":"2023-11-24T06:26:57.002515+0000","last_became_peered":"2023-11-24T06:26:57.002515+0000","last_unstale":"2023-11-24T06:26:58.218742+0000","last_undegraded":"2023-11-24T06:26:58.218742+0000","last_fullsized":"2023-11-24T06:26:58.218742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:40.491868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944342+0000","last_change":"2023-11-24T06:26:56.014991+0000","last_active":"2023-11-24T06:26:57.944342+0000","last_peered":"2023-11-24T06:26:57.944342+0000","last_clean":"2023-11-24T06:26:57.944342+0000","last_became_active":"2023-11-24T06:26:56.014706+0000","last_became_peered":"2023-11-24T06:26:56.014706+0000","last_unstale":"2023-11-24T06:26:57.944342+0000","last_undegraded":"2023-11-24T06:26:57.944342+0000","last_fullsized":"2023-11-24T06:26:57.944342+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:56.985153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221316+0000","last_change":"2023-11-24T06:26:56.970097+0000","last_active":"2023-11-24T06:26:58.221316+0000","last_peered":"2023-11-24T06:26:58.221316+0000","last_clean":"2023-11-24T06:26:58.221316+0000","last_became_active":"2023-11-24T06:26:56.969183+0000","last_became_peered":"2023-11-24T06:26:56.969183+0000","last_unstale":"2023-11-24T06:26:58.221316+0000","last_undegraded":"2023-11-24T06:26:58.221316+0000","last_fullsized":"2023-11-24T06:26:58.221316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:47.926009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221734+0000","last_change":"2023-11-24T06:26:56.002890+0000","last_active":"2023-11-24T06:26:58.221734+0000","last_peered":"2023-11-24T06:26:58.221734+0000","last_clean":"2023-11-24T06:26:58.221734+0000","last_became_active":"2023-11-24T06:26:56.002603+0000","last_became_peered":"2023-11-24T06:26:56.002603+0000","last_unstale":"2023-11-24T06:26:58.221734+0000","last_undegraded":"2023-11-24T06:26:58.221734+0000","last_fullsized":"2023-11-24T06:26:58.221734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947457+0000","last_change":"2023-11-24T06:26:56.961409+0000","last_active":"2023-11-24T06:26:57.947457+0000","last_peered":"2023-11-24T06:26:57.947457+0000","last_clean":"2023-11-24T06:26:57.947457+0000","last_became_active":"2023-11-24T06:26:56.961107+0000","last_became_peered":"2023-11-24T06:26:56.961107+0000","last_unstale":"2023-11-24T06:26:57.947457+0000","last_undegraded":"2023-11-24T06:26:57.947457+0000","last_fullsized":"2023-11-24T06:26:57.947457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218244+0000","last_change":"2023-11-24T06:26:56.006966+0000","last_active":"2023-11-24T06:26:58.218244+0000","last_peered":"2023-11-24T06:26:58.218244+0000","last_clean":"2023-11-24T06:26:58.218244+0000","last_became_active":"2023-11-24T06:26:56.006596+0000","last_became_peered":"2023-11-24T06:26:56.006596+0000","last_unstale":"2023-11-24T06:26:58.218244+0000","last_undegraded":"2023-11-24T06:26:58.218244+0000","last_fullsized":"2023-11-24T06:26:58.218244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217572+0000","last_change":"2023-11-24T06:26:56.974192+0000","last_active":"2023-11-24T06:26:58.217572+0000","last_peered":"2023-11-24T06:26:58.217572+0000","last_clean":"2023-11-24T06:26:58.217572+0000","last_became_active":"2023-11-24T06:26:56.973850+0000","last_became_peered":"2023-11-24T06:26:56.973850+0000","last_unstale":"2023-11-24T06:26:58.217572+0000","last_undegraded":"2023-11-24T06:26:58.217572+0000","last_fullsized":"2023-11-24T06:26:58.217572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:51.030181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221387+0000","last_change":"2023-11-24T06:26:56.013312+0000","last_active":"2023-11-24T06:26:58.221387+0000","last_peered":"2023-11-24T06:26:58.221387+0000","last_clean":"2023-11-24T06:26:58.221387+0000","last_became_active":"2023-11-24T06:26:56.013018+0000","last_became_peered":"2023-11-24T06:26:56.013018+0000","last_unstale":"2023-11-24T06:26:58.221387+0000","last_undegraded":"2023-11-24T06:26:58.221387+0000","last_fullsized":"2023-11-24T06:26:58.221387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:03.493540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947707+0000","last_change":"2023-11-24T06:26:56.975118+0000","last_active":"2023-11-24T06:26:57.947707+0000","last_peered":"2023-11-24T06:26:57.947707+0000","last_clean":"2023-11-24T06:26:57.947707+0000","last_became_active":"2023-11-24T06:26:56.974734+0000","last_became_peered":"2023-11-24T06:26:56.974734+0000","last_unstale":"2023-11-24T06:26:57.947707+0000","last_undegraded":"2023-11-24T06:26:57.947707+0000","last_fullsized":"2023-11-24T06:26:57.947707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:15.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.219402+0000","last_change":"2023-11-24T06:26:56.010113+0000","last_active":"2023-11-24T06:26:58.219402+0000","last_peered":"2023-11-24T06:26:58.219402+0000","last_clean":"2023-11-24T06:26:58.219402+0000","last_became_active":"2023-11-24T06:26:56.009550+0000","last_became_peered":"2023-11-24T06:26:56.009550+0000","last_unstale":"2023-11-24T06:26:58.219402+0000","last_undegraded":"2023-11-24T06:26:58.219402+0000","last_fullsized":"2023-11-24T06:26:58.219402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:13.157002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218000+0000","last_change":"2023-11-24T06:26:56.978061+0000","last_active":"2023-11-24T06:26:58.218000+0000","last_peered":"2023-11-24T06:26:58.218000+0000","last_clean":"2023-11-24T06:26:58.218000+0000","last_became_active":"2023-11-24T06:26:56.977760+0000","last_became_peered":"2023-11-24T06:26:56.977760+0000","last_unstale":"2023-11-24T06:26:58.218000+0000","last_undegraded":"2023-11-24T06:26:58.218000+0000","last_fullsized":"2023-11-24T06:26:58.218000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:12.069548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216616+0000","last_change":"2023-11-24T06:26:55.996462+0000","last_active":"2023-11-24T06:26:58.216616+0000","last_peered":"2023-11-24T06:26:58.216616+0000","last_clean":"2023-11-24T06:26:58.216616+0000","last_became_active":"2023-11-24T06:26:55.996178+0000","last_became_peered":"2023-11-24T06:26:55.996178+0000","last_unstale":"2023-11-24T06:26:58.216616+0000","last_undegraded":"2023-11-24T06:26:58.216616+0000","last_fullsized":"2023-11-24T06:26:58.216616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:59.647318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946852+0000","last_change":"2023-11-24T06:26:56.978947+0000","last_active":"2023-11-24T06:26:57.946852+0000","last_peered":"2023-11-24T06:26:57.946852+0000","last_clean":"2023-11-24T06:26:57.946852+0000","last_became_active":"2023-11-24T06:26:56.978643+0000","last_became_peered":"2023-11-24T06:26:56.978643+0000","last_unstale":"2023-11-24T06:26:57.946852+0000","last_undegraded":"2023-11-24T06:26:57.946852+0000","last_fullsized":"2023-11-24T06:26:57.946852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:32.752617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222582+0000","last_change":"2023-11-24T06:26:56.005504+0000","last_active":"2023-11-24T06:26:58.222582+0000","last_peered":"2023-11-24T06:26:58.222582+0000","last_clean":"2023-11-24T06:26:58.222582+0000","last_became_active":"2023-11-24T06:26:56.005287+0000","last_became_peered":"2023-11-24T06:26:56.005287+0000","last_unstale":"2023-11-24T06:26:58.222582+0000","last_undegraded":"2023-11-24T06:26:58.222582+0000","last_fullsized":"2023-11-24T06:26:58.222582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:24.763532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947843+0000","last_change":"2023-11-24T06:26:56.973083+0000","last_active":"2023-11-24T06:26:57.947843+0000","last_peered":"2023-11-24T06:26:57.947843+0000","last_clean":"2023-11-24T06:26:57.947843+0000","last_became_active":"2023-11-24T06:26:56.972750+0000","last_became_peered":"2023-11-24T06:26:56.972750+0000","last_unstale":"2023-11-24T06:26:57.947843+0000","last_undegraded":"2023-11-24T06:26:57.947843+0000","last_fullsized":"2023-11-24T06:26:57.947843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:08.969752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217141+0000","last_change":"2023-11-24T06:26:56.005050+0000","last_active":"2023-11-24T06:26:58.217141+0000","last_peered":"2023-11-24T06:26:58.217141+0000","last_clean":"2023-11-24T06:26:58.217141+0000","last_became_active":"2023-11-24T06:26:56.004658+0000","last_became_peered":"2023-11-24T06:26:56.004658+0000","last_unstale":"2023-11-24T06:26:58.217141+0000","last_undegraded":"2023-11-24T06:26:58.217141+0000","last_fullsized":"2023-11-24T06:26:58.217141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:27.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.948405+0000","last_change":"2023-11-24T06:26:56.973292+0000","last_active":"2023-11-24T06:26:57.948405+0000","last_peered":"2023-11-24T06:26:57.948405+0000","last_clean":"2023-11-24T06:26:57.948405+0000","last_became_active":"2023-11-24T06:26:56.972959+0000","last_became_peered":"2023-11-24T06:26:56.972959+0000","last_unstale":"2023-11-24T06:26:57.948405+0000","last_undegraded":"2023-11-24T06:26:57.948405+0000","last_fullsized":"2023-11-24T06:26:57.948405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:48.721442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944407+0000","last_change":"2023-11-24T06:26:56.005932+0000","last_active":"2023-11-24T06:26:57.944407+0000","last_peered":"2023-11-24T06:26:57.944407+0000","last_clean":"2023-11-24T06:26:57.944407+0000","last_became_active":"2023-11-24T06:26:56.005603+0000","last_became_peered":"2023-11-24T06:26:56.005603+0000","last_unstale":"2023-11-24T06:26:57.944407+0000","last_undegraded":"2023-11-24T06:26:57.944407+0000","last_fullsized":"2023-11-24T06:26:57.944407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222520+0000","last_change":"2023-11-24T06:26:56.969503+0000","last_active":"2023-11-24T06:26:58.222520+0000","last_peered":"2023-11-24T06:26:58.222520+0000","last_clean":"2023-11-24T06:26:58.222520+0000","last_became_active":"2023-11-24T06:26:56.967704+0000","last_became_peered":"2023-11-24T06:26:56.967704+0000","last_unstale":"2023-11-24T06:26:58.222520+0000","last_undegraded":"2023-11-24T06:26:58.222520+0000","last_fullsized":"2023-11-24T06:26:58.222520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:17.662630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217140+0000","last_change":"2023-11-24T06:26:56.014819+0000","last_active":"2023-11-24T06:26:58.217140+0000","last_peered":"2023-11-24T06:26:58.217140+0000","last_clean":"2023-11-24T06:26:58.217140+0000","last_became_active":"2023-11-24T06:26:56.014287+0000","last_became_peered":"2023-11-24T06:26:56.014287+0000","last_unstale":"2023-11-24T06:26:58.217140+0000","last_undegraded":"2023-11-24T06:26:58.217140+0000","last_fullsized":"2023-11-24T06:26:58.217140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:13.157002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947402+0000","last_change":"2023-11-24T06:26:57.001053+0000","last_active":"2023-11-24T06:26:57.947402+0000","last_peered":"2023-11-24T06:26:57.947402+0000","last_clean":"2023-11-24T06:26:57.947402+0000","last_became_active":"2023-11-24T06:26:57.000802+0000","last_became_peered":"2023-11-24T06:26:57.000802+0000","last_unstale":"2023-11-24T06:26:57.947402+0000","last_undegraded":"2023-11-24T06:26:57.947402+0000","last_fullsized":"2023-11-24T06:26:57.947402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:26:56.946083+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:26:56.946083+0000","last_peered":"2023-11-24T06:26:56.946083+0000","last_clean":"2023-11-24T06:26:56.946083+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:26:56.946083+0000","last_undegraded":"2023-11-24T06:26:56.946083+0000","last_fullsized":"2023-11-24T06:26:56.946083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221515+0000","last_change":"2023-11-24T06:26:56.008553+0000","last_active":"2023-11-24T06:26:58.221515+0000","last_peered":"2023-11-24T06:26:58.221515+0000","last_clean":"2023-11-24T06:26:58.221515+0000","last_became_active":"2023-11-24T06:26:56.006284+0000","last_became_peered":"2023-11-24T06:26:56.006284+0000","last_unstale":"2023-11-24T06:26:58.221515+0000","last_undegraded":"2023-11-24T06:26:58.221515+0000","last_fullsized":"2023-11-24T06:26:58.221515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:30:24.763532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945070+0000","last_change":"2023-11-24T06:26:56.998859+0000","last_active":"2023-11-24T06:26:57.945070+0000","last_peered":"2023-11-24T06:26:57.945070+0000","last_clean":"2023-11-24T06:26:57.945070+0000","last_became_active":"2023-11-24T06:26:56.998524+0000","last_became_peered":"2023-11-24T06:26:56.998524+0000","last_unstale":"2023-11-24T06:26:57.945070+0000","last_undegraded":"2023-11-24T06:26:57.945070+0000","last_fullsized":"2023-11-24T06:26:57.945070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946597+0000","last_change":"2023-11-24T06:26:56.005300+0000","last_active":"2023-11-24T06:26:57.946597+0000","last_peered":"2023-11-24T06:26:57.946597+0000","last_clean":"2023-11-24T06:26:57.946597+0000","last_became_active":"2023-11-24T06:26:56.004984+0000","last_became_peered":"2023-11-24T06:26:56.004984+0000","last_unstale":"2023-11-24T06:26:57.946597+0000","last_undegraded":"2023-11-24T06:26:57.946597+0000","last_fullsized":"2023-11-24T06:26:57.946597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.217825+0000","last_change":"2023-11-24T06:26:56.984062+0000","last_active":"2023-11-24T06:26:58.217825+0000","last_peered":"2023-11-24T06:26:58.217825+0000","last_clean":"2023-11-24T06:26:58.217825+0000","last_became_active":"2023-11-24T06:26:56.983679+0000","last_became_peered":"2023-11-24T06:26:56.983679+0000","last_unstale":"2023-11-24T06:26:58.217825+0000","last_undegraded":"2023-11-24T06:26:58.217825+0000","last_fullsized":"2023-11-24T06:26:58.217825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:58.210301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.216848+0000","last_change":"2023-11-24T06:26:56.005001+0000","last_active":"2023-11-24T06:26:58.216848+0000","last_peered":"2023-11-24T06:26:58.216848+0000","last_clean":"2023-11-24T06:26:58.216848+0000","last_became_active":"2023-11-24T06:26:56.004612+0000","last_became_peered":"2023-11-24T06:26:56.004612+0000","last_unstale":"2023-11-24T06:26:58.216848+0000","last_undegraded":"2023-11-24T06:26:58.216848+0000","last_fullsized":"2023-11-24T06:26:58.216848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:00.811375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944532+0000","last_change":"2023-11-24T06:26:56.965796+0000","last_active":"2023-11-24T06:26:57.944532+0000","last_peered":"2023-11-24T06:26:57.944532+0000","last_clean":"2023-11-24T06:26:57.944532+0000","last_became_active":"2023-11-24T06:26:56.963344+0000","last_became_peered":"2023-11-24T06:26:56.963344+0000","last_unstale":"2023-11-24T06:26:57.944532+0000","last_undegraded":"2023-11-24T06:26:57.944532+0000","last_fullsized":"2023-11-24T06:26:57.944532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.221498+0000","last_change":"2023-11-24T06:26:56.012192+0000","last_active":"2023-11-24T06:26:58.221498+0000","last_peered":"2023-11-24T06:26:58.221498+0000","last_clean":"2023-11-24T06:26:58.221498+0000","last_became_active":"2023-11-24T06:26:56.011796+0000","last_became_peered":"2023-11-24T06:26:56.011796+0000","last_unstale":"2023-11-24T06:26:58.221498+0000","last_undegraded":"2023-11-24T06:26:58.221498+0000","last_fullsized":"2023-11-24T06:26:58.221498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:50.399725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220548+0000","last_change":"2023-11-24T06:26:56.985067+0000","last_active":"2023-11-24T06:26:58.220548+0000","last_peered":"2023-11-24T06:26:58.220548+0000","last_clean":"2023-11-24T06:26:58.220548+0000","last_became_active":"2023-11-24T06:26:56.984360+0000","last_became_peered":"2023-11-24T06:26:56.984360+0000","last_unstale":"2023-11-24T06:26:58.220548+0000","last_undegraded":"2023-11-24T06:26:58.220548+0000","last_fullsized":"2023-11-24T06:26:58.220548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:50.371449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944321+0000","last_change":"2023-11-24T06:26:55.996919+0000","last_active":"2023-11-24T06:26:57.944321+0000","last_peered":"2023-11-24T06:26:57.944321+0000","last_clean":"2023-11-24T06:26:57.944321+0000","last_became_active":"2023-11-24T06:26:55.996431+0000","last_became_peered":"2023-11-24T06:26:55.996431+0000","last_unstale":"2023-11-24T06:26:57.944321+0000","last_undegraded":"2023-11-24T06:26:57.944321+0000","last_fullsized":"2023-11-24T06:26:57.944321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:03.070359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222871+0000","last_change":"2023-11-24T06:26:56.987025+0000","last_active":"2023-11-24T06:26:58.222871+0000","last_peered":"2023-11-24T06:26:58.222871+0000","last_clean":"2023-11-24T06:26:58.222871+0000","last_became_active":"2023-11-24T06:26:56.985920+0000","last_became_peered":"2023-11-24T06:26:56.985920+0000","last_unstale":"2023-11-24T06:26:58.222871+0000","last_undegraded":"2023-11-24T06:26:58.222871+0000","last_fullsized":"2023-11-24T06:26:58.222871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:08.969752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947843+0000","last_change":"2023-11-24T06:26:56.015006+0000","last_active":"2023-11-24T06:26:57.947843+0000","last_peered":"2023-11-24T06:26:57.947843+0000","last_clean":"2023-11-24T06:26:57.947843+0000","last_became_active":"2023-11-24T06:26:56.011264+0000","last_became_peered":"2023-11-24T06:26:56.011264+0000","last_unstale":"2023-11-24T06:26:57.947843+0000","last_undegraded":"2023-11-24T06:26:57.947843+0000","last_fullsized":"2023-11-24T06:26:57.947843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:51.788764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944165+0000","last_change":"2023-11-24T06:26:56.967275+0000","last_active":"2023-11-24T06:26:57.944165+0000","last_peered":"2023-11-24T06:26:57.944165+0000","last_clean":"2023-11-24T06:26:57.944165+0000","last_became_active":"2023-11-24T06:26:56.966261+0000","last_became_peered":"2023-11-24T06:26:56.966261+0000","last_unstale":"2023-11-24T06:26:57.944165+0000","last_undegraded":"2023-11-24T06:26:57.944165+0000","last_fullsized":"2023-11-24T06:26:57.944165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:43:04.718688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:59.156742+0000","last_change":"2023-11-24T06:26:56.011770+0000","last_active":"2023-11-24T06:26:59.156742+0000","last_peered":"2023-11-24T06:26:59.156742+0000","last_clean":"2023-11-24T06:26:59.156742+0000","last_became_active":"2023-11-24T06:26:56.011386+0000","last_became_peered":"2023-11-24T06:26:56.011386+0000","last_unstale":"2023-11-24T06:26:59.156742+0000","last_undegraded":"2023-11-24T06:26:59.156742+0000","last_fullsized":"2023-11-24T06:26:59.156742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945877+0000","last_change":"2023-11-24T06:26:56.978788+0000","last_active":"2023-11-24T06:26:57.945877+0000","last_peered":"2023-11-24T06:26:57.945877+0000","last_clean":"2023-11-24T06:26:57.945877+0000","last_became_active":"2023-11-24T06:26:56.978112+0000","last_became_peered":"2023-11-24T06:26:56.978112+0000","last_unstale":"2023-11-24T06:26:57.945877+0000","last_undegraded":"2023-11-24T06:26:57.945877+0000","last_fullsized":"2023-11-24T06:26:57.945877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220478+0000","last_change":"2023-11-24T06:26:56.012367+0000","last_active":"2023-11-24T06:26:58.220478+0000","last_peered":"2023-11-24T06:26:58.220478+0000","last_clean":"2023-11-24T06:26:58.220478+0000","last_became_active":"2023-11-24T06:26:56.011973+0000","last_became_peered":"2023-11-24T06:26:56.011973+0000","last_unstale":"2023-11-24T06:26:58.220478+0000","last_undegraded":"2023-11-24T06:26:58.220478+0000","last_fullsized":"2023-11-24T06:26:58.220478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218351+0000","last_change":"2023-11-24T06:26:56.976256+0000","last_active":"2023-11-24T06:26:58.218351+0000","last_peered":"2023-11-24T06:26:58.218351+0000","last_clean":"2023-11-24T06:26:58.218351+0000","last_became_active":"2023-11-24T06:26:56.975889+0000","last_became_peered":"2023-11-24T06:26:56.975889+0000","last_unstale":"2023-11-24T06:26:58.218351+0000","last_undegraded":"2023-11-24T06:26:58.218351+0000","last_fullsized":"2023-11-24T06:26:58.218351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:22.193391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.946577+0000","last_change":"2023-11-24T06:26:56.008104+0000","last_active":"2023-11-24T06:26:57.946577+0000","last_peered":"2023-11-24T06:26:57.946577+0000","last_clean":"2023-11-24T06:26:57.946577+0000","last_became_active":"2023-11-24T06:26:56.007669+0000","last_became_peered":"2023-11-24T06:26:56.007669+0000","last_unstale":"2023-11-24T06:26:57.946577+0000","last_undegraded":"2023-11-24T06:26:57.946577+0000","last_fullsized":"2023-11-24T06:26:57.946577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:27.405726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.220486+0000","last_change":"2023-11-24T06:26:56.974769+0000","last_active":"2023-11-24T06:26:58.220486+0000","last_peered":"2023-11-24T06:26:58.220486+0000","last_clean":"2023-11-24T06:26:58.220486+0000","last_became_active":"2023-11-24T06:26:56.974530+0000","last_became_peered":"2023-11-24T06:26:56.974530+0000","last_unstale":"2023-11-24T06:26:58.220486+0000","last_undegraded":"2023-11-24T06:26:58.220486+0000","last_fullsized":"2023-11-24T06:26:58.220486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:02.036523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944166+0000","last_change":"2023-11-24T06:26:56.001637+0000","last_active":"2023-11-24T06:26:57.944166+0000","last_peered":"2023-11-24T06:26:57.944166+0000","last_clean":"2023-11-24T06:26:57.944166+0000","last_became_active":"2023-11-24T06:26:56.000662+0000","last_became_peered":"2023-11-24T06:26:56.000662+0000","last_unstale":"2023-11-24T06:26:57.944166+0000","last_undegraded":"2023-11-24T06:26:57.944166+0000","last_fullsized":"2023-11-24T06:26:57.944166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:49.146302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945138+0000","last_change":"2023-11-24T06:26:56.961879+0000","last_active":"2023-11-24T06:26:57.945138+0000","last_peered":"2023-11-24T06:26:57.945138+0000","last_clean":"2023-11-24T06:26:57.945138+0000","last_became_active":"2023-11-24T06:26:56.961593+0000","last_became_peered":"2023-11-24T06:26:56.961593+0000","last_unstale":"2023-11-24T06:26:57.945138+0000","last_undegraded":"2023-11-24T06:26:57.945138+0000","last_fullsized":"2023-11-24T06:26:57.945138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.944425+0000","last_change":"2023-11-24T06:26:55.995684+0000","last_active":"2023-11-24T06:26:57.944425+0000","last_peered":"2023-11-24T06:26:57.944425+0000","last_clean":"2023-11-24T06:26:57.944425+0000","last_became_active":"2023-11-24T06:26:55.994739+0000","last_became_peered":"2023-11-24T06:26:55.994739+0000","last_unstale":"2023-11-24T06:26:57.944425+0000","last_undegraded":"2023-11-24T06:26:57.944425+0000","last_fullsized":"2023-11-24T06:26:57.944425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:59.647318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945536+0000","last_change":"2023-11-24T06:26:56.976564+0000","last_active":"2023-11-24T06:26:57.945536+0000","last_peered":"2023-11-24T06:26:57.945536+0000","last_clean":"2023-11-24T06:26:57.945536+0000","last_became_active":"2023-11-24T06:26:56.976081+0000","last_became_peered":"2023-11-24T06:26:56.976081+0000","last_unstale":"2023-11-24T06:26:57.945536+0000","last_undegraded":"2023-11-24T06:26:57.945536+0000","last_fullsized":"2023-11-24T06:26:57.945536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:08:37.797585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947699+0000","last_change":"2023-11-24T06:26:56.015851+0000","last_active":"2023-11-24T06:26:57.947699+0000","last_peered":"2023-11-24T06:26:57.947699+0000","last_clean":"2023-11-24T06:26:57.947699+0000","last_became_active":"2023-11-24T06:26:56.014016+0000","last_became_peered":"2023-11-24T06:26:56.014016+0000","last_unstale":"2023-11-24T06:26:57.947699+0000","last_undegraded":"2023-11-24T06:26:57.947699+0000","last_fullsized":"2023-11-24T06:26:57.947699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:11:49.805033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218077+0000","last_change":"2023-11-24T06:26:56.975486+0000","last_active":"2023-11-24T06:26:58.218077+0000","last_peered":"2023-11-24T06:26:58.218077+0000","last_clean":"2023-11-24T06:26:58.218077+0000","last_became_active":"2023-11-24T06:26:56.975147+0000","last_became_peered":"2023-11-24T06:26:56.975147+0000","last_unstale":"2023-11-24T06:26:58.218077+0000","last_undegraded":"2023-11-24T06:26:58.218077+0000","last_fullsized":"2023-11-24T06:26:58.218077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:18.580355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222273+0000","last_change":"2023-11-24T06:26:56.994982+0000","last_active":"2023-11-24T06:26:58.222273+0000","last_peered":"2023-11-24T06:26:58.222273+0000","last_clean":"2023-11-24T06:26:58.222273+0000","last_became_active":"2023-11-24T06:26:56.993765+0000","last_became_peered":"2023-11-24T06:26:56.993765+0000","last_unstale":"2023-11-24T06:26:58.222273+0000","last_undegraded":"2023-11-24T06:26:58.222273+0000","last_fullsized":"2023-11-24T06:26:58.222273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:59.724290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947612+0000","last_change":"2023-11-24T06:26:56.015039+0000","last_active":"2023-11-24T06:26:57.947612+0000","last_peered":"2023-11-24T06:26:57.947612+0000","last_clean":"2023-11-24T06:26:57.947612+0000","last_became_active":"2023-11-24T06:26:56.014562+0000","last_became_peered":"2023-11-24T06:26:56.014562+0000","last_unstale":"2023-11-24T06:26:57.947612+0000","last_undegraded":"2023-11-24T06:26:57.947612+0000","last_fullsized":"2023-11-24T06:26:57.947612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.222503+0000","last_change":"2023-11-24T06:26:56.992781+0000","last_active":"2023-11-24T06:26:58.222503+0000","last_peered":"2023-11-24T06:26:58.222503+0000","last_clean":"2023-11-24T06:26:58.222503+0000","last_became_active":"2023-11-24T06:26:56.992337+0000","last_became_peered":"2023-11-24T06:26:56.992337+0000","last_unstale":"2023-11-24T06:26:58.222503+0000","last_undegraded":"2023-11-24T06:26:58.222503+0000","last_fullsized":"2023-11-24T06:26:58.222503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:15.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.947644+0000","last_change":"2023-11-24T06:26:56.015616+0000","last_active":"2023-11-24T06:26:57.947644+0000","last_peered":"2023-11-24T06:26:57.947644+0000","last_clean":"2023-11-24T06:26:57.947644+0000","last_became_active":"2023-11-24T06:26:56.015317+0000","last_became_peered":"2023-11-24T06:26:56.015317+0000","last_unstale":"2023-11-24T06:26:57.947644+0000","last_undegraded":"2023-11-24T06:26:57.947644+0000","last_fullsized":"2023-11-24T06:26:57.947644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:10.844401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218457+0000","last_change":"2023-11-24T06:26:57.001097+0000","last_active":"2023-11-24T06:26:58.218457+0000","last_peered":"2023-11-24T06:26:58.218457+0000","last_clean":"2023-11-24T06:26:58.218457+0000","last_became_active":"2023-11-24T06:26:57.000751+0000","last_became_peered":"2023-11-24T06:26:57.000751+0000","last_unstale":"2023-11-24T06:26:58.218457+0000","last_undegraded":"2023-11-24T06:26:58.218457+0000","last_fullsized":"2023-11-24T06:26:58.218457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:53.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945720+0000","last_change":"2023-11-24T06:26:56.003621+0000","last_active":"2023-11-24T06:26:57.945720+0000","last_peered":"2023-11-24T06:26:57.945720+0000","last_clean":"2023-11-24T06:26:57.945720+0000","last_became_active":"2023-11-24T06:26:56.003232+0000","last_became_peered":"2023-11-24T06:26:56.003232+0000","last_unstale":"2023-11-24T06:26:57.945720+0000","last_undegraded":"2023-11-24T06:26:57.945720+0000","last_fullsized":"2023-11-24T06:26:57.945720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:10.844401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:57.945648+0000","last_change":"2023-11-24T06:26:56.972822+0000","last_active":"2023-11-24T06:26:57.945648+0000","last_peered":"2023-11-24T06:26:57.945648+0000","last_clean":"2023-11-24T06:26:57.945648+0000","last_became_active":"2023-11-24T06:26:56.972529+0000","last_became_peered":"2023-11-24T06:26:56.972529+0000","last_unstale":"2023-11-24T06:26:57.945648+0000","last_undegraded":"2023-11-24T06:26:57.945648+0000","last_fullsized":"2023-11-24T06:26:57.945648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:55.963679+0000","last_clean_scrub_stamp":"2023-11-24T06:26:55.963679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:04.134143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:26:58.218623+0000","last_change":"2023-11-24T06:26:56.009185+0000","last_active":"2023-11-24T06:26:58.218623+0000","last_peered":"2023-11-24T06:26:58.218623+0000","last_clean":"2023-11-24T06:26:58.218623+0000","last_became_active":"2023-11-24T06:26:56.008855+0000","last_became_peered":"2023-11-24T06:26:56.008855+0000","last_unstale":"2023-11-24T06:26:58.218623+0000","last_undegraded":"2023-11-24T06:26:58.218623+0000","last_fullsized":"2023-11-24T06:26:58.218623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:54.738531+0000","last_clean_scrub_stamp":"2023-11-24T06:26:54.738531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:01.188689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3090000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.464}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5880000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.706}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.117}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2349999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4180000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2639999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.528}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0089999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1349999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.659}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8460000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.722}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2440000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3959999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:27:07.707 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:27:07.707 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T06:27:07.708 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T06:27:07.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T06:27:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.966+0000 7f49ec74c640 1 Processor -- start 2023-11-24T06:27:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.966+0000 7f49ec74c640 1 -- start start 2023-11-24T06:27:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.968+0000 7f49ec74c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49e413e360 0x7f49e40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.968+0000 7f49ec74c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e415f9c0 0x7f49e40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.968+0000 7f49ec74c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e415c5f0 con 0x7f49e413df80 2023-11-24T06:27:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.968+0000 7f49ec74c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e406c470 con 0x7f49e413e360 2023-11-24T06:27:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.968+0000 7f49ec74c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e406c750 con 0x7f49e415f9c0 2023-11-24T06:27:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.968+0000 7f49e9ce7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49e413e360 0x7f49e40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.968+0000 7f49e9ce7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49e413e360 0x7f49e40cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38512/0 (socket says 172.21.15.16:38512) 2023-11-24T06:27:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.968+0000 7f49e9ce7640 1 -- 172.21.15.16:0/3675881364 learned_addr learned my addr 172.21.15.16:0/3675881364 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.969+0000 7f49e9ce7640 1 -- 172.21.15.16:0/3675881364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e415f9c0 msgr2=0x7f49e40cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.969+0000 7f49eace9640 1 --2- 172.21.15.16:0/3675881364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e415f9c0 0x7f49e40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.969+0000 7f49e9ce7640 1 --2- 172.21.15.16:0/3675881364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e415f9c0 0x7f49e40cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.969+0000 7f49e9ce7640 1 -- 172.21.15.16:0/3675881364 >> v1:172.21.15.173:6789/0 conn(0x7f49e413df80 legacy=0x7f49e415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.969+0000 7f49e9ce7640 1 -- 172.21.15.16:0/3675881364 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e406ca30 con 0x7f49e413e360 2023-11-24T06:27:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.969+0000 7f49eace9640 1 --2- 172.21.15.16:0/3675881364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e415f9c0 0x7f49e40cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.970+0000 7f49e9ce7640 1 --2- 172.21.15.16:0/3675881364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413e360 0x7f49e40cf290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f49e406ded0 tx=0x7f49d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd096c73f73b84a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49e94e6640 1 -- 172.21.15.16:0/3675881364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4031830 con 0x7f49e413e360 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49e94e6640 1 -- 172.21.15.16:0/3675881364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4031990 con 0x7f49e413e360 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49e94e6640 1 -- 172.21.15.16:0/3675881364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4031d20 con 0x7f49e413e360 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49ec74c640 1 -- 172.21.15.16:0/3675881364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413e360 msgr2=0x7f49e40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49ec74c640 1 --2- 172.21.15.16:0/3675881364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413e360 0x7f49e40cf290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f49e406ded0 tx=0x7f49d402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49ec74c640 1 -- 172.21.15.16:0/3675881364 shutdown_connections 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49ec74c640 1 --2- 172.21.15.16:0/3675881364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e415f9c0 0x7f49e40cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49ec74c640 1 --2- 172.21.15.16:0/3675881364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413e360 0x7f49e40cf290 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49ec74c640 1 -- 172.21.15.16:0/3675881364 >> 172.21.15.16:0/3675881364 conn(0x7f49e406a800 msgr2=0x7f49e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49ec74c640 1 -- 172.21.15.16:0/3675881364 shutdown_connections 2023-11-24T06:27:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.971+0000 7f49ec74c640 1 -- 172.21.15.16:0/3675881364 wait complete. 2023-11-24T06:27:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.973+0000 7f49ec74c640 1 Processor -- start 2023-11-24T06:27:07.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.973+0000 7f49ec74c640 1 -- start start 2023-11-24T06:27:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.974+0000 7f49ec74c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413df80 0x7f49e414dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.974+0000 7f49ec74c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e413e360 0x7f49e40b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.974+0000 7f49ec74c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e415f9c0 0x7f49e41f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.974+0000 7f49ec74c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e414c020 con 0x7f49e413df80 2023-11-24T06:27:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.974+0000 7f49ec74c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e41f0af0 con 0x7f49e415f9c0 2023-11-24T06:27:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.974+0000 7f49ec74c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4150c20 con 0x7f49e413e360 2023-11-24T06:27:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.974+0000 7f49ea4e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413df80 0x7f49e414dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49ea4e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413df80 0x7f49e414dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38526/0 (socket says 172.21.15.16:38526) 2023-11-24T06:27:07.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49ea4e8640 1 -- 172.21.15.16:0/3768521801 learned_addr learned my addr 172.21.15.16:0/3768521801 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49e9ce7640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e413e360 0x7f49e40b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49eace9640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e415f9c0 0x7f49e41f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49ea4e8640 1 -- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e413e360 msgr2=0x7f49e40b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49ea4e8640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e413e360 0x7f49e40b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49ea4e8640 1 -- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e415f9c0 msgr2=0x7f49e41f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49ea4e8640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e415f9c0 0x7f49e41f12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49ea4e8640 1 -- 172.21.15.16:0/3768521801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40094d0 con 0x7f49e413df80 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49eace9640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e415f9c0 0x7f49e41f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49ea4e8640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413df80 0x7f49e414dfa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f49e000b3f0 tx=0x7f49e000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.975+0000 7f49daffd640 1 -- 172.21.15.16:0/3768521801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e0011830 con 0x7f49e413df80 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.976+0000 7f49daffd640 1 -- 172.21.15.16:0/3768521801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e0011990 con 0x7f49e413df80 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.976+0000 7f49daffd640 1 -- 172.21.15.16:0/3768521801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e0011d40 con 0x7f49e413df80 2023-11-24T06:27:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.976+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e4150f00 con 0x7f49e413df80 2023-11-24T06:27:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.976+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e41f0dc0 con 0x7f49e413df80 2023-11-24T06:27:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.977+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e40675d0 con 0x7f49e413df80 2023-11-24T06:27:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.978+0000 7f49daffd640 1 -- 172.21.15.16:0/3768521801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49e001a400 con 0x7f49e413df80 2023-11-24T06:27:07.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.979+0000 7f49daffd640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49c806ae40 0x7f49c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:07.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.979+0000 7f49daffd640 1 -- 172.21.15.16:0/3768521801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49e0089440 con 0x7f49e413df80 2023-11-24T06:27:07.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.980+0000 7f49e9ce7640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49c806ae40 0x7f49c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.981+0000 7f49e9ce7640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49c806ae40 0x7f49c806d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49d4036000 tx=0x7f49d403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:07.984+0000 7f49daffd640 1 -- 172.21.15.16:0/3768521801 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49e00544e0 con 0x7f49e413df80 2023-11-24T06:27:08.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.222+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f49e406df00 con 0x7f49e413df80 2023-11-24T06:27:08.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.223+0000 7f49daffd640 1 -- 172.21.15.16:0/3768521801 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f49e0057b00 con 0x7f49e413df80 2023-11-24T06:27:08.226 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:08.226 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T06:27:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49c806ae40 msgr2=0x7f49c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49c806ae40 0x7f49c806d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49d4036000 tx=0x7f49d403c040 comp rx=0 tx=0).stop 2023-11-24T06:27:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413df80 msgr2=0x7f49e414dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413df80 0x7f49e414dfa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f49e000b3f0 tx=0x7f49e000b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 shutdown_connections 2023-11-24T06:27:08.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49c806ae40 0x7f49c806d300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49e415f9c0 0x7f49e41f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49e413e360 0x7f49e40b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 --2- 172.21.15.16:0/3768521801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49e413df80 0x7f49e414dfa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 >> 172.21.15.16:0/3768521801 conn(0x7f49e406a800 msgr2=0x7f49e415de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:08.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 shutdown_connections 2023-11-24T06:27:08.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.226+0000 7f49ec74c640 1 -- 172.21.15.16:0/3768521801 wait complete. 2023-11-24T06:27:08.247 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T06:27:08.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.466+0000 7ff31694e640 1 Processor -- start 2023-11-24T06:27:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.466+0000 7ff31694e640 1 -- start start 2023-11-24T06:27:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.468+0000 7ff31694e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff310160920 0x7ff3100b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.468+0000 7ff31694e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3100d0a10 con 0x7ff3100cfea0 2023-11-24T06:27:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.468+0000 7ff31694e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff310161e30 con 0x7ff3100cf9f0 2023-11-24T06:27:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.468+0000 7ff31694e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3100d1f50 con 0x7ff310160920 2023-11-24T06:27:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.468+0000 7ff31614d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff3100cfea0 0x7ff310161720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52882/0 (socket says 172.21.15.16:52882) 2023-11-24T06:27:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.468+0000 7ff31614d640 1 -- 172.21.15.16:0/638862488 learned_addr learned my addr 172.21.15.16:0/638862488 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.468+0000 7ff31594c640 1 --2- 172.21.15.16:0/638862488 >> v2:172.21.15.173:3300/0 conn(0x7ff310160920 0x7ff3100b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702224721 0 0) 0x7ff3100d0a10 con 0x7ff3100cfea0 2023-11-24T06:27:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2fc003520 con 0x7ff3100cfea0 2023-11-24T06:27:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3825619269 0 0) 0x7ff2fc003520 con 0x7ff3100cfea0 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 >> v2:172.21.15.173:3300/0 conn(0x7ff310160920 msgr2=0x7ff3100b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 --2- 172.21.15.16:0/638862488 >> v2:172.21.15.173:3300/0 conn(0x7ff310160920 0x7ff3100b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 >> v1:172.21.15.173:6789/0 conn(0x7ff3100cf9f0 legacy=0x7ff310161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100cf080 con 0x7ff3100cfea0 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7ff3080019f0 con 0x7ff3100cfea0 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 >> v1:172.21.15.16:6789/0 conn(0x7ff3100cfea0 legacy=0x7ff310161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2fc003970 0x7ff2fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0047c0 0x7ff2fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2fc005180 0x7ff2fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100d1f50 con 0x7ff2fc0047c0 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc00c020 con 0x7ff2fc005180 2023-11-24T06:27:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.469+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc00c1e0 con 0x7ff2fc003970 2023-11-24T06:27:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.470+0000 7ff31514b640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0047c0 0x7ff2fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.470+0000 7ff31594c640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2fc003970 0x7ff2fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.470+0000 7ff31514b640 1 -- 172.21.15.16:0/638862488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2fc003970 msgr2=0x7ff2fc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.470+0000 7ff31514b640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2fc003970 0x7ff2fc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.470+0000 7ff31514b640 1 -- 172.21.15.16:0/638862488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2fc005180 msgr2=0x7ff2fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.470+0000 7ff31514b640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2fc005180 0x7ff2fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.470+0000 7ff31514b640 1 -- 172.21.15.16:0/638862488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc003520 con 0x7ff2fc0047c0 2023-11-24T06:27:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.470+0000 7ff31594c640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2fc003970 0x7ff2fc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.470+0000 7ff31614d640 1 -- 172.21.15.16:0/638862488 reap_dead start 2023-11-24T06:27:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.471+0000 7ff31514b640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0047c0 0x7ff2fc004c40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff300001750 tx=0x7ff2fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5708475fe4ec219f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:08.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.471+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3000052d0 con 0x7ff2fc0047c0 2023-11-24T06:27:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.471+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30002e050 con 0x7ff2fc0047c0 2023-11-24T06:27:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.471+0000 7ff31494a640 1 -- 172.21.15.16:0/638862488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300003830 con 0x7ff2fc0047c0 2023-11-24T06:27:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.472+0000 7ff31694e640 1 -- 172.21.15.16:0/638862488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0047c0 msgr2=0x7ff2fc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.472+0000 7ff31694e640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0047c0 0x7ff2fc004c40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff300001750 tx=0x7ff2fc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.472+0000 7ff31694e640 1 -- 172.21.15.16:0/638862488 shutdown_connections 2023-11-24T06:27:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.472+0000 7ff31694e640 1 --2- 172.21.15.16:0/638862488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2fc0047c0 0x7ff2fc004c40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.472+0000 7ff31694e640 1 -- 172.21.15.16:0/638862488 >> 172.21.15.16:0/638862488 conn(0x7ff3100d3090 msgr2=0x7ff3100d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.472+0000 7ff31694e640 1 -- 172.21.15.16:0/638862488 shutdown_connections 2023-11-24T06:27:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.472+0000 7ff31694e640 1 -- 172.21.15.16:0/638862488 wait complete. 2023-11-24T06:27:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.474+0000 7ff31694e640 1 Processor -- start 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.474+0000 7ff31694e640 1 -- start start 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.475+0000 7ff31694e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2fc0047c0 0x7ff3100b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.475+0000 7ff31694e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff310150e90 0x7ff3100ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.475+0000 7ff31694e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff310151390 0x7ff3100ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.475+0000 7ff31694e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100b8230 con 0x7ff310150e90 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.475+0000 7ff31694e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100cbc90 con 0x7ff2fc0047c0 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.475+0000 7ff31694e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100cbf10 con 0x7ff310151390 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.475+0000 7ff31514b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff310150e90 0x7ff3100ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.475+0000 7ff31514b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff310150e90 0x7ff3100ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38554/0 (socket says 172.21.15.16:38554) 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.475+0000 7ff31514b640 1 -- 172.21.15.16:0/600273342 learned_addr learned my addr 172.21.15.16:0/600273342 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31594c640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2fc0047c0 0x7ff3100b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31614d640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff310151390 0x7ff3100ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31514b640 1 -- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff310151390 msgr2=0x7ff3100ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31514b640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff310151390 0x7ff3100ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31514b640 1 -- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2fc0047c0 msgr2=0x7ff3100b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31514b640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2fc0047c0 0x7ff3100b7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31514b640 1 -- 172.21.15.16:0/600273342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100cc1f0 con 0x7ff310150e90 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31614d640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff310151390 0x7ff3100ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31514b640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff310150e90 0x7ff3100ca160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff300000c00 tx=0x7ff300001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff3067fc640 1 -- 172.21.15.16:0/600273342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300007640 con 0x7ff310150e90 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff3067fc640 1 -- 172.21.15.16:0/600273342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3000069e0 con 0x7ff310150e90 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff3067fc640 1 -- 172.21.15.16:0/600273342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300006d90 con 0x7ff310150e90 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3100cc4d0 con 0x7ff310150e90 2023-11-24T06:27:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.476+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3100b61f0 con 0x7ff310150e90 2023-11-24T06:27:08.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.478+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff310067450 con 0x7ff310150e90 2023-11-24T06:27:08.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.478+0000 7ff3067fc640 1 -- 172.21.15.16:0/600273342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3000035f0 con 0x7ff310150e90 2023-11-24T06:27:08.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.479+0000 7ff3067fc640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff2f406ae70 0x7ff2f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.479+0000 7ff3067fc640 1 -- 172.21.15.16:0/600273342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3000ab260 con 0x7ff310150e90 2023-11-24T06:27:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.480+0000 7ff31594c640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff2f406ae70 0x7ff2f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.481+0000 7ff31594c640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff2f406ae70 0x7ff2f406d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff310067b00 tx=0x7ff30c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:08.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.484+0000 7ff3067fc640 1 -- 172.21.15.16:0/600273342 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3000761e0 con 0x7ff310150e90 2023-11-24T06:27:08.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.677+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff31008c7e0 con 0x7ff310150e90 2023-11-24T06:27:08.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.678+0000 7ff3067fc640 1 -- 172.21.15.16:0/600273342 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7ff300079800 con 0x7ff310150e90 2023-11-24T06:27:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:08.682 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff2f406ae70 msgr2=0x7ff2f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff2f406ae70 0x7ff2f406d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff310067b00 tx=0x7ff30c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff310150e90 msgr2=0x7ff3100ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff310150e90 0x7ff3100ca160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff300000c00 tx=0x7ff300001120 comp rx=0 tx=0).stop 2023-11-24T06:27:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 shutdown_connections 2023-11-24T06:27:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff2f406ae70 0x7ff2f406d330 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff310151390 0x7ff3100ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff310150e90 0x7ff3100ca160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 --2- 172.21.15.16:0/600273342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2fc0047c0 0x7ff3100b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 >> 172.21.15.16:0/600273342 conn(0x7ff3100d3090 msgr2=0x7ff31006c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 shutdown_connections 2023-11-24T06:27:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.681+0000 7ff31694e640 1 -- 172.21.15.16:0/600273342 wait complete. 2023-11-24T06:27:08.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:08.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.929+0000 7f3b01485640 1 Processor -- start 2023-11-24T06:27:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.929+0000 7f3b01485640 1 -- start start 2023-11-24T06:27:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.930+0000 7f3b01485640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3afc15d4b0 0x7f3afc0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.930+0000 7f3b01485640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cf9a0 0x7f3afc0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.930+0000 7f3b01485640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3afc0cddf0 0x7f3afc0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.930+0000 7f3b01485640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3afc0d0320 con 0x7f3afc15d4b0 2023-11-24T06:27:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.930+0000 7f3b01485640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3afc0b2730 con 0x7f3afc0cddf0 2023-11-24T06:27:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.930+0000 7f3b01485640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc0d1ec0 con 0x7f3afc0cf9a0 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.930+0000 7f3afbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3afc15d4b0 0x7f3afc0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.930+0000 7f3afbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3afc15d4b0 0x7f3afc0d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38558/0 (socket says 172.21.15.16:38558) 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.930+0000 7f3afbfff640 1 -- 172.21.15.16:0/2260424203 learned_addr learned my addr 172.21.15.16:0/2260424203 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.931+0000 7f3afb7fe640 1 --2- 172.21.15.16:0/2260424203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cf9a0 0x7f3afc0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.931+0000 7f3b00c84640 1 --2- 172.21.15.16:0/2260424203 >> v2:172.21.15.173:3300/0 conn(0x7f3afc0cddf0 0x7f3afc0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.931+0000 7f3afbfff640 1 -- 172.21.15.16:0/2260424203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cf9a0 msgr2=0x7f3afc0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.931+0000 7f3afbfff640 1 --2- 172.21.15.16:0/2260424203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cf9a0 0x7f3afc0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.931+0000 7f3afbfff640 1 -- 172.21.15.16:0/2260424203 >> v2:172.21.15.173:3300/0 conn(0x7f3afc0cddf0 msgr2=0x7f3afc0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.931+0000 7f3afbfff640 1 --2- 172.21.15.16:0/2260424203 >> v2:172.21.15.173:3300/0 conn(0x7f3afc0cddf0 0x7f3afc0ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.931+0000 7f3afbfff640 1 -- 172.21.15.16:0/2260424203 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc0b29d0 con 0x7f3afc15d4b0 2023-11-24T06:27:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.931+0000 7f3b00c84640 1 --2- 172.21.15.16:0/2260424203 >> v2:172.21.15.173:3300/0 conn(0x7f3afc0cddf0 0x7f3afc0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:08.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.932+0000 7f3afbfff640 1 --2- 172.21.15.16:0/2260424203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 0x7f3afc0d1550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009870 tx=0x7f3aec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a305195d3b76a04a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:08.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.932+0000 7f3afaffd640 1 -- 172.21.15.16:0/2260424203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec031640 con 0x7f3afc15d4b0 2023-11-24T06:27:08.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.932+0000 7f3afaffd640 1 -- 172.21.15.16:0/2260424203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec0317a0 con 0x7f3afc15d4b0 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.932+0000 7f3afaffd640 1 -- 172.21.15.16:0/2260424203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec031b30 con 0x7f3afc15d4b0 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.933+0000 7f3b01485640 1 -- 172.21.15.16:0/2260424203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 msgr2=0x7f3afc0d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.933+0000 7f3b01485640 1 --2- 172.21.15.16:0/2260424203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 0x7f3afc0d1550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009870 tx=0x7f3aec02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.933+0000 7f3b01485640 1 -- 172.21.15.16:0/2260424203 shutdown_connections 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.933+0000 7f3b01485640 1 --2- 172.21.15.16:0/2260424203 >> v2:172.21.15.173:3300/0 conn(0x7f3afc0cddf0 0x7f3afc0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.933+0000 7f3b01485640 1 --2- 172.21.15.16:0/2260424203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cf9a0 0x7f3afc0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.933+0000 7f3b01485640 1 --2- 172.21.15.16:0/2260424203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 0x7f3afc0d1550 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.933+0000 7f3b01485640 1 -- 172.21.15.16:0/2260424203 >> 172.21.15.16:0/2260424203 conn(0x7f3afc0d3000 msgr2=0x7f3afc0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.933+0000 7f3b01485640 1 -- 172.21.15.16:0/2260424203 shutdown_connections 2023-11-24T06:27:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.933+0000 7f3b01485640 1 -- 172.21.15.16:0/2260424203 wait complete. 2023-11-24T06:27:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.935+0000 7f3b01485640 1 Processor -- start 2023-11-24T06:27:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.935+0000 7f3b01485640 1 -- start start 2023-11-24T06:27:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.936+0000 7f3b01485640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cddf0 0x7f3afc154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.936+0000 7f3b01485640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3afc0cf9a0 0x7f3afc0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.936+0000 7f3b01485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 0x7f3afc0b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.936+0000 7f3b01485640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc0cb440 con 0x7f3afc15d4b0 2023-11-24T06:27:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.936+0000 7f3b01485640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc09ffd0 con 0x7f3afc0cf9a0 2023-11-24T06:27:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.936+0000 7f3b01485640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc0a02b0 con 0x7f3afc0cddf0 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.936+0000 7f3b00c84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 0x7f3afc0b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.936+0000 7f3b00c84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 0x7f3afc0b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38566/0 (socket says 172.21.15.16:38566) 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.936+0000 7f3b00c84640 1 -- 172.21.15.16:0/3493508542 learned_addr learned my addr 172.21.15.16:0/3493508542 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3afb7fe640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3afc0cf9a0 0x7f3afc0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3afbfff640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cddf0 0x7f3afc154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3b00c84640 1 -- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cddf0 msgr2=0x7f3afc154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3b00c84640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cddf0 0x7f3afc154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3b00c84640 1 -- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3afc0cf9a0 msgr2=0x7f3afc0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3b00c84640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3afc0cf9a0 0x7f3afc0b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3b00c84640 1 -- 172.21.15.16:0/3493508542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0094d0 con 0x7f3afc15d4b0 2023-11-24T06:27:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3afb7fe640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3afc0cf9a0 0x7f3afc0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3afbfff640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cddf0 0x7f3afc154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3b00c84640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 0x7f3afc0b0760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3af400cef0 tx=0x7f3af400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3af8ff9640 1 -- 172.21.15.16:0/3493508542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af400d840 con 0x7f3afc15d4b0 2023-11-24T06:27:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3af8ff9640 1 -- 172.21.15.16:0/3493508542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af400d9a0 con 0x7f3afc15d4b0 2023-11-24T06:27:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3af8ff9640 1 -- 172.21.15.16:0/3493508542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af400dd50 con 0x7f3afc15d4b0 2023-11-24T06:27:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.937+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc15da10 con 0x7f3afc15d4b0 2023-11-24T06:27:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.938+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc0a0870 con 0x7f3afc15d4b0 2023-11-24T06:27:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.939+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc0cb880 con 0x7f3afc15d4b0 2023-11-24T06:27:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.940+0000 7f3af8ff9640 1 -- 172.21.15.16:0/3493508542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3af401e3b0 con 0x7f3afc15d4b0 2023-11-24T06:27:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.941+0000 7f3af8ff9640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3adc06ae40 0x7f3adc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.941+0000 7f3af8ff9640 1 -- 172.21.15.16:0/3493508542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3af4089fb0 con 0x7f3afc15d4b0 2023-11-24T06:27:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.941+0000 7f3afbfff640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3adc06ae40 0x7f3adc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.942+0000 7f3afbfff640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3adc06ae40 0x7f3adc06d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3aec036000 tx=0x7f3aec03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:08.945+0000 7f3af8ff9640 1 -- 172.21.15.16:0/3493508542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3af40bce90 con 0x7f3afc15d4b0 2023-11-24T06:27:09.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.137+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3afc0d0460 con 0x7f3afc15d4b0 2023-11-24T06:27:09.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.138+0000 7f3af8ff9640 1 -- 172.21.15.16:0/3493508542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3af401a0a0 con 0x7f3afc15d4b0 2023-11-24T06:27:09.141 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3adc06ae40 msgr2=0x7f3adc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3adc06ae40 0x7f3adc06d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3aec036000 tx=0x7f3aec03c040 comp rx=0 tx=0).stop 2023-11-24T06:27:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 msgr2=0x7f3afc0b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 0x7f3afc0b0760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3af400cef0 tx=0x7f3af400a710 comp rx=0 tx=0).stop 2023-11-24T06:27:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 shutdown_connections 2023-11-24T06:27:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3adc06ae40 0x7f3adc06d300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3afc15d4b0 0x7f3afc0b0760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3afc0cf9a0 0x7f3afc0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 --2- 172.21.15.16:0/3493508542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3afc0cddf0 0x7f3afc154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 >> 172.21.15.16:0/3493508542 conn(0x7f3afc0d3000 msgr2=0x7f3afc0ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 shutdown_connections 2023-11-24T06:27:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.141+0000 7f3b01485640 1 -- 172.21.15.16:0/3493508542 wait complete. 2023-11-24T06:27:09.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.390+0000 7f630513a640 1 Processor -- start 2023-11-24T06:27:09.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.390+0000 7f630513a640 1 -- start start 2023-11-24T06:27:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.391+0000 7f630513a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6300159040 0x7f63000cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.391+0000 7f630513a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63000b34e0 0x7f63000d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.391+0000 7f630513a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63000c6060 con 0x7f630015c300 2023-11-24T06:27:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.392+0000 7f630513a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63000c62d0 con 0x7f6300159040 2023-11-24T06:27:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.392+0000 7f630513a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63000c65b0 con 0x7f63000b34e0 2023-11-24T06:27:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.392+0000 7f62fe59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f630015c300 0x7f630015f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52884/0 (socket says 172.21.15.16:52884) 2023-11-24T06:27:09.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.392+0000 7f62fe59c640 1 -- 172.21.15.16:0/579483505 learned_addr learned my addr 172.21.15.16:0/579483505 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.392+0000 7f62fed9d640 1 --2- 172.21.15.16:0/579483505 >> v2:172.21.15.173:3300/0 conn(0x7f6300159040 0x7f63000cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.392+0000 7f62ff59e640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63000b34e0 0x7f63000d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.392+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014330872 0 0) 0x7f63000c6060 con 0x7f630015c300 2023-11-24T06:27:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62e4003520 con 0x7f630015c300 2023-11-24T06:27:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2599167922 0 0) 0x7f62e4003520 con 0x7f630015c300 2023-11-24T06:27:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63000b34e0 msgr2=0x7f63000d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63000b34e0 0x7f63000d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 >> v2:172.21.15.173:3300/0 conn(0x7f6300159040 msgr2=0x7f63000cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:09.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 --2- 172.21.15.16:0/579483505 >> v2:172.21.15.173:3300/0 conn(0x7f6300159040 0x7f63000cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630015eec0 con 0x7f630015c300 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f62ec001670 con 0x7f630015c300 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 >> v1:172.21.15.16:6789/0 conn(0x7f630015c300 legacy=0x7f630015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f62e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62e40047c0 0x7f62e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63000b34e0 0x7f62e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63000c62d0 con 0x7f62e4003970 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63000c65b0 con 0x7f62e40047c0 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e400b980 con 0x7f63000b34e0 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fed9d640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f62e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.393+0000 7f62fe59c640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62e40047c0 0x7f62e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.394+0000 7f62ff59e640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63000b34e0 0x7f62e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.394+0000 7f62fed9d640 1 -- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63000b34e0 msgr2=0x7f62e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.394+0000 7f62fed9d640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63000b34e0 0x7f62e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.394+0000 7f62fed9d640 1 -- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62e40047c0 msgr2=0x7f62e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.394+0000 7f62fed9d640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62e40047c0 0x7f62e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.394+0000 7f62fed9d640 1 -- 172.21.15.16:0/579483505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e4003520 con 0x7f62e4003970 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.395+0000 7f62fed9d640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f62e4003d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009ac0 tx=0x7f62e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b072b53e33e9e01a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.395+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f002c710 con 0x7f62e4003970 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.395+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f002c870 con 0x7f62e4003970 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.395+0000 7f62fdd9b640 1 -- 172.21.15.16:0/579483505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f002cc00 con 0x7f62e4003970 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.396+0000 7f630513a640 1 -- 172.21.15.16:0/579483505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 msgr2=0x7f62e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.396+0000 7f630513a640 1 --2- 172.21.15.16:0/579483505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f62e4003d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009ac0 tx=0x7f62e40057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.396+0000 7f62ff59e640 1 -- 172.21.15.16:0/579483505 reap_dead start 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.396+0000 7f630513a640 1 -- 172.21.15.16:0/579483505 shutdown_connections 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.396+0000 7f630513a640 1 -- 172.21.15.16:0/579483505 >> 172.21.15.16:0/579483505 conn(0x7f6300066690 msgr2=0x7f630006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.396+0000 7f630513a640 1 -- 172.21.15.16:0/579483505 shutdown_connections 2023-11-24T06:27:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.396+0000 7f630513a640 1 -- 172.21.15.16:0/579483505 wait complete. 2023-11-24T06:27:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.398+0000 7f630513a640 1 Processor -- start 2023-11-24T06:27:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.398+0000 7f630513a640 1 -- start start 2023-11-24T06:27:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f630513a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f630014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f630513a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63000b4850 0x7f63000b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f630513a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f630015d330 0x7f630015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f630513a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630014f3f0 con 0x7f62e4003970 2023-11-24T06:27:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f630513a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630015dd00 con 0x7f63000b4850 2023-11-24T06:27:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f630513a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63000b63f0 con 0x7f630015d330 2023-11-24T06:27:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f630014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f630014eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38576/0 (socket says 172.21.15.16:38576) 2023-11-24T06:27:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62fed9d640 1 -- 172.21.15.16:0/4179387422 learned_addr learned my addr 172.21.15.16:0/4179387422 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62fe59c640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63000b4850 0x7f63000b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62ff59e640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f630015d330 0x7f630015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62fed9d640 1 -- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f630015d330 msgr2=0x7f630015d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62fed9d640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f630015d330 0x7f630015d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62fed9d640 1 -- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63000b4850 msgr2=0x7f63000b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62fed9d640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63000b4850 0x7f63000b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.399+0000 7f62fed9d640 1 -- 172.21.15.16:0/4179387422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000b5350 con 0x7f62e4003970 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.400+0000 7f62fed9d640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f630014eeb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f62f0000c00 tx=0x7f62f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.400+0000 7f62f77fe640 1 -- 172.21.15.16:0/4179387422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0003810 con 0x7f62e4003970 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.400+0000 7f62f77fe640 1 -- 172.21.15.16:0/4179387422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f0003990 con 0x7f62e4003970 2023-11-24T06:27:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.400+0000 7f62f77fe640 1 -- 172.21.15.16:0/4179387422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f0003d40 con 0x7f62e4003970 2023-11-24T06:27:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.400+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63000b5630 con 0x7f62e4003970 2023-11-24T06:27:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.400+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63000b5bf0 con 0x7f62e4003970 2023-11-24T06:27:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.402+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6300074a50 con 0x7f62e4003970 2023-11-24T06:27:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.402+0000 7f62f77fe640 1 -- 172.21.15.16:0/4179387422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62f003b070 con 0x7f62e4003970 2023-11-24T06:27:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.404+0000 7f62f77fe640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f62e806ae40 0x7f62e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.404+0000 7f62f77fe640 1 -- 172.21.15.16:0/4179387422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62f00ab2c0 con 0x7f62e4003970 2023-11-24T06:27:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.404+0000 7f62fe59c640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f62e806ae40 0x7f62e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.406+0000 7f62fe59c640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f62e806ae40 0x7f62e806d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62ec001810 tx=0x7f62ec000fd0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:09.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.408+0000 7f62f77fe640 1 -- 172.21.15.16:0/4179387422 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62f0076410 con 0x7f62e4003970 2023-11-24T06:27:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.603+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f630015eca0 con 0x7f62e4003970 2023-11-24T06:27:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.604+0000 7f62f77fe640 1 -- 172.21.15.16:0/4179387422 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62f0079a30 con 0x7f62e4003970 2023-11-24T06:27:09.607 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:09.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f62e806ae40 msgr2=0x7f62e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f62e806ae40 0x7f62e806d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62ec001810 tx=0x7f62ec000fd0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 msgr2=0x7f630014eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f630014eeb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f62f0000c00 tx=0x7f62f0004730 comp rx=0 tx=0).stop 2023-11-24T06:27:09.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 shutdown_connections 2023-11-24T06:27:09.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f62e806ae40 0x7f62e806d300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f630015d330 0x7f630015d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63000b4850 0x7f63000b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 --2- 172.21.15.16:0/4179387422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62e4003970 0x7f630014eeb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.606+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 >> 172.21.15.16:0/4179387422 conn(0x7f6300066690 msgr2=0x7f630006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:09.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.607+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 shutdown_connections 2023-11-24T06:27:09.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.607+0000 7f630513a640 1 -- 172.21.15.16:0/4179387422 wait complete. 2023-11-24T06:27:09.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.857+0000 7fb533367640 1 Processor -- start 2023-11-24T06:27:09.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.857+0000 7fb533367640 1 -- start start 2023-11-24T06:27:09.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.858+0000 7fb533367640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb52c0cf9a0 0x7fb52c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.858+0000 7fb533367640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 0x7fb52c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.858+0000 7fb533367640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c0d0320 con 0x7fb52c15d4b0 2023-11-24T06:27:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.858+0000 7fb533367640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c0d1c60 con 0x7fb52c0cf9a0 2023-11-24T06:27:09.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.858+0000 7fb533367640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0b2730 con 0x7fb52c0b31d0 2023-11-24T06:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb532365640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb52c15d4b0 0x7fb52c0d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52890/0 (socket says 172.21.15.16:52890) 2023-11-24T06:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb531b64640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb52c0cf9a0 0x7fb52c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb532365640 1 -- 172.21.15.16:0/1823028840 learned_addr learned my addr 172.21.15.16:0/1823028840 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb532b66640 1 --2- 172.21.15.16:0/1823028840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 0x7fb52c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb531363640 1 -- 172.21.15.16:0/1823028840 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918042160 0 0) 0x7fb52c0d0320 con 0x7fb52c15d4b0 2023-11-24T06:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb531363640 1 -- 172.21.15.16:0/1823028840 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb518003520 con 0x7fb52c15d4b0 2023-11-24T06:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb531b64640 1 -- 172.21.15.16:0/1823028840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 msgr2=0x7fb52c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb531b64640 1 --2- 172.21.15.16:0/1823028840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 0x7fb52c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb531b64640 1 -- 172.21.15.16:0/1823028840 >> v1:172.21.15.16:6789/0 conn(0x7fb52c15d4b0 legacy=0x7fb52c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb531b64640 1 -- 172.21.15.16:0/1823028840 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0d1ec0 con 0x7fb52c0cf9a0 2023-11-24T06:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.859+0000 7fb532b66640 1 --2- 172.21.15.16:0/1823028840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 0x7fb52c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.860+0000 7fb531b64640 1 --2- 172.21.15.16:0/1823028840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c0cf9a0 0x7fb52c0cfde0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0097a0 tx=0x7fb51c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630f10dcd7fd6317 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:09.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.861+0000 7fb531363640 1 -- 172.21.15.16:0/1823028840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c02d040 con 0x7fb52c0cf9a0 2023-11-24T06:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.861+0000 7fb531363640 1 -- 172.21.15.16:0/1823028840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c031810 con 0x7fb52c0cf9a0 2023-11-24T06:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.861+0000 7fb531363640 1 -- 172.21.15.16:0/1823028840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c031ba0 con 0x7fb52c0cf9a0 2023-11-24T06:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.862+0000 7fb533367640 1 -- 172.21.15.16:0/1823028840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c0cf9a0 msgr2=0x7fb52c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.862+0000 7fb533367640 1 --2- 172.21.15.16:0/1823028840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c0cf9a0 0x7fb52c0cfde0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0097a0 tx=0x7fb51c02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.862+0000 7fb533367640 1 -- 172.21.15.16:0/1823028840 shutdown_connections 2023-11-24T06:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.862+0000 7fb533367640 1 --2- 172.21.15.16:0/1823028840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 0x7fb52c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.862+0000 7fb533367640 1 --2- 172.21.15.16:0/1823028840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c0cf9a0 0x7fb52c0cfde0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.862+0000 7fb533367640 1 -- 172.21.15.16:0/1823028840 >> 172.21.15.16:0/1823028840 conn(0x7fb52c0d3000 msgr2=0x7fb52c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.862+0000 7fb533367640 1 -- 172.21.15.16:0/1823028840 shutdown_connections 2023-11-24T06:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.862+0000 7fb533367640 1 -- 172.21.15.16:0/1823028840 wait complete. 2023-11-24T06:27:09.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.864+0000 7fb533367640 1 Processor -- start 2023-11-24T06:27:09.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.864+0000 7fb533367640 1 -- start start 2023-11-24T06:27:09.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.865+0000 7fb533367640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 0x7fb52c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.865+0000 7fb533367640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb52c0cf9a0 0x7fb52c1f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.865+0000 7fb533367640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c15d4b0 0x7fb52c14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.865+0000 7fb533367640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c154270 con 0x7fb52c15d4b0 2023-11-24T06:27:09.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.865+0000 7fb533367640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1f30c0 con 0x7fb52c0cf9a0 2023-11-24T06:27:09.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.865+0000 7fb533367640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1f33a0 con 0x7fb52c0b31d0 2023-11-24T06:27:09.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.865+0000 7fb532b66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c15d4b0 0x7fb52c14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.865+0000 7fb532b66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c15d4b0 0x7fb52c14d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38584/0 (socket says 172.21.15.16:38584) 2023-11-24T06:27:09.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.865+0000 7fb532b66640 1 -- 172.21.15.16:0/902094732 learned_addr learned my addr 172.21.15.16:0/902094732 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb532365640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 0x7fb52c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb531b64640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb52c0cf9a0 0x7fb52c1f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb532b66640 1 -- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 msgr2=0x7fb52c15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb532b66640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 0x7fb52c15bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb532b66640 1 -- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb52c0cf9a0 msgr2=0x7fb52c1f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb532b66640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb52c0cf9a0 0x7fb52c1f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb532b66640 1 -- 172.21.15.16:0/902094732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c009400 con 0x7fb52c15d4b0 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb531b64640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb52c0cf9a0 0x7fb52c1f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb532b66640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c15d4b0 0x7fb52c14d660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb524007df0 tx=0x7fb52400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb522ffd640 1 -- 172.21.15.16:0/902094732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524003990 con 0x7fb52c15d4b0 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb522ffd640 1 -- 172.21.15.16:0/902094732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52400f040 con 0x7fb52c15d4b0 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c1f3680 con 0x7fb52c15d4b0 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.866+0000 7fb522ffd640 1 -- 172.21.15.16:0/902094732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524003d30 con 0x7fb52c15d4b0 2023-11-24T06:27:09.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.867+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c1f3960 con 0x7fb52c15d4b0 2023-11-24T06:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.868+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c150320 con 0x7fb52c15d4b0 2023-11-24T06:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.869+0000 7fb522ffd640 1 -- 172.21.15.16:0/902094732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb524004030 con 0x7fb52c15d4b0 2023-11-24T06:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.869+0000 7fb522ffd640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb51006ae40 0x7fb51006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:09.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.870+0000 7fb522ffd640 1 -- 172.21.15.16:0/902094732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5240899b0 con 0x7fb52c15d4b0 2023-11-24T06:27:09.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.870+0000 7fb532365640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb51006ae40 0x7fb51006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:09.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.871+0000 7fb532365640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb51006ae40 0x7fb51006d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb5280011e0 tx=0x7fb528009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:09.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:09.873+0000 7fb522ffd640 1 -- 172.21.15.16:0/902094732 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb52c150320 con 0x7fb52c15d4b0 2023-11-24T06:27:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.073+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb52c150320 con 0x7fb52c15d4b0 2023-11-24T06:27:10.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.074+0000 7fb522ffd640 1 -- 172.21.15.16:0/902094732 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb52c150320 con 0x7fb52c15d4b0 2023-11-24T06:27:10.078 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb51006ae40 msgr2=0x7fb51006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb51006ae40 0x7fb51006d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb5280011e0 tx=0x7fb528009040 comp rx=0 tx=0).stop 2023-11-24T06:27:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c15d4b0 msgr2=0x7fb52c14d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c15d4b0 0x7fb52c14d660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb524007df0 tx=0x7fb52400d4a0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 shutdown_connections 2023-11-24T06:27:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb51006ae40 0x7fb51006d300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb52c15d4b0 0x7fb52c14d660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb52c0cf9a0 0x7fb52c1f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 --2- 172.21.15.16:0/902094732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb52c0b31d0 0x7fb52c15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 >> 172.21.15.16:0/902094732 conn(0x7fb52c0d3000 msgr2=0x7fb52c0d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 shutdown_connections 2023-11-24T06:27:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.077+0000 7fb533367640 1 -- 172.21.15.16:0/902094732 wait complete. 2023-11-24T06:27:10.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.323+0000 7ff6a7091640 1 Processor -- start 2023-11-24T06:27:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.323+0000 7ff6a7091640 1 -- start start 2023-11-24T06:27:10.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.325+0000 7ff6a7091640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff6a00cc500 0x7ff6a00cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.325+0000 7ff6a7091640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a0070e50 con 0x7ff6a0143c40 2023-11-24T06:27:10.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.325+0000 7ff6a7091640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a0144020 con 0x7ff6a006d160 2023-11-24T06:27:10.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.325+0000 7ff6a7091640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a00723c0 con 0x7ff6a00cc500 2023-11-24T06:27:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.325+0000 7ff6a608f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff6a0143c40 0x7ff6a0071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52902/0 (socket says 172.21.15.16:52902) 2023-11-24T06:27:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.325+0000 7ff6a608f640 1 -- 172.21.15.16:0/2123851488 learned_addr learned my addr 172.21.15.16:0/2123851488 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.325+0000 7ff6a588e640 1 --2- 172.21.15.16:0/2123851488 >> v2:172.21.15.173:3300/0 conn(0x7ff6a00cc500 0x7ff6a00cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.326+0000 7ff6a508d640 1 -- 172.21.15.16:0/2123851488 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004399957 0 0) 0x7ff6a0070e50 con 0x7ff6a0143c40 2023-11-24T06:27:10.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.326+0000 7ff6a508d640 1 -- 172.21.15.16:0/2123851488 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff68c003520 con 0x7ff6a0143c40 2023-11-24T06:27:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.326+0000 7ff6a508d640 1 -- 172.21.15.16:0/2123851488 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3639020298 0 0) 0x7ff68c003520 con 0x7ff6a0143c40 2023-11-24T06:27:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.326+0000 7ff6a588e640 1 -- 172.21.15.16:0/2123851488 >> v1:172.21.15.173:6789/0 conn(0x7ff6a006d160 legacy=0x7ff6a006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.326+0000 7ff6a588e640 1 -- 172.21.15.16:0/2123851488 >> v1:172.21.15.16:6789/0 conn(0x7ff6a0143c40 legacy=0x7ff6a0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.326+0000 7ff6a588e640 1 -- 172.21.15.16:0/2123851488 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0071e00 con 0x7ff6a00cc500 2023-11-24T06:27:10.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.328+0000 7ff6a588e640 1 --2- 172.21.15.16:0/2123851488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6a00cc500 0x7ff6a00cc990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6900097a0 tx=0x7ff69002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=904b02d398236b30 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:10.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.328+0000 7ff6a508d640 1 -- 172.21.15.16:0/2123851488 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c001670 con 0x7ff6a00cc500 2023-11-24T06:27:10.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.328+0000 7ff6a508d640 1 -- 172.21.15.16:0/2123851488 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69002e050 con 0x7ff6a00cc500 2023-11-24T06:27:10.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.328+0000 7ff6a508d640 1 -- 172.21.15.16:0/2123851488 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690030730 con 0x7ff6a00cc500 2023-11-24T06:27:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.329+0000 7ff6a7091640 1 -- 172.21.15.16:0/2123851488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6a00cc500 msgr2=0x7ff6a00cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.329+0000 7ff6a7091640 1 --2- 172.21.15.16:0/2123851488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6a00cc500 0x7ff6a00cc990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6900097a0 tx=0x7ff69002b790 comp rx=0 tx=0).stop 2023-11-24T06:27:10.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.329+0000 7ff6a7091640 1 -- 172.21.15.16:0/2123851488 shutdown_connections 2023-11-24T06:27:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.329+0000 7ff6a7091640 1 --2- 172.21.15.16:0/2123851488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6a00cc500 0x7ff6a00cc990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.329+0000 7ff6a7091640 1 -- 172.21.15.16:0/2123851488 >> 172.21.15.16:0/2123851488 conn(0x7ff6a006a800 msgr2=0x7ff6a006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.329+0000 7ff6a7091640 1 -- 172.21.15.16:0/2123851488 shutdown_connections 2023-11-24T06:27:10.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.329+0000 7ff6a7091640 1 -- 172.21.15.16:0/2123851488 wait complete. 2023-11-24T06:27:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.331+0000 7ff6a7091640 1 Processor -- start 2023-11-24T06:27:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.331+0000 7ff6a7091640 1 -- start start 2023-11-24T06:27:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.332+0000 7ff6a7091640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6a006d160 0x7ff6a00ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.332+0000 7ff6a7091640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a00cc500 0x7ff6a00aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.332+0000 7ff6a7091640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6a0143c40 0x7ff6a01586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.332+0000 7ff6a7091640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a013ab50 con 0x7ff6a00cc500 2023-11-24T06:27:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.332+0000 7ff6a7091640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00abd10 con 0x7ff6a006d160 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.332+0000 7ff6a7091640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a00abfb0 con 0x7ff6a0143c40 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a588e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a00cc500 0x7ff6a00aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a588e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a00cc500 0x7ff6a00aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38596/0 (socket says 172.21.15.16:38596) 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a588e640 1 -- 172.21.15.16:0/4015687902 learned_addr learned my addr 172.21.15.16:0/4015687902 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a608f640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6a006d160 0x7ff6a00ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a6890640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6a0143c40 0x7ff6a01586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a588e640 1 -- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6a0143c40 msgr2=0x7ff6a01586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a588e640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6a0143c40 0x7ff6a01586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a588e640 1 -- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6a006d160 msgr2=0x7ff6a00ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a588e640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6a006d160 0x7ff6a00ab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.333+0000 7ff6a588e640 1 -- 172.21.15.16:0/4015687902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff690009400 con 0x7ff6a00cc500 2023-11-24T06:27:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.334+0000 7ff6a588e640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a00cc500 0x7ff6a00aff90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff6900098c0 tx=0x7ff6900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.334+0000 7ff696ffd640 1 -- 172.21.15.16:0/4015687902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69003c880 con 0x7ff6a00cc500 2023-11-24T06:27:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.334+0000 7ff696ffd640 1 -- 172.21.15.16:0/4015687902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69003c9e0 con 0x7ff6a00cc500 2023-11-24T06:27:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.334+0000 7ff696ffd640 1 -- 172.21.15.16:0/4015687902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69003cd90 con 0x7ff6a00cc500 2023-11-24T06:27:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.334+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a00ac290 con 0x7ff6a00cc500 2023-11-24T06:27:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.334+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a013a7e0 con 0x7ff6a00cc500 2023-11-24T06:27:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.335+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a013ad00 con 0x7ff6a00cc500 2023-11-24T06:27:10.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.336+0000 7ff696ffd640 1 -- 172.21.15.16:0/4015687902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff690039040 con 0x7ff6a00cc500 2023-11-24T06:27:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.338+0000 7ff696ffd640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff68406ae40 0x7ff68406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.338+0000 7ff696ffd640 1 -- 172.21.15.16:0/4015687902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6900ab250 con 0x7ff6a00cc500 2023-11-24T06:27:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.338+0000 7ff6a608f640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff68406ae40 0x7ff68406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.339+0000 7ff6a608f640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff68406ae40 0x7ff68406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff69c000f60 tx=0x7ff69c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.342+0000 7ff696ffd640 1 -- 172.21.15.16:0/4015687902 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6a013ad00 con 0x7ff6a00cc500 2023-11-24T06:27:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.578+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6a0067450 con 0x7ff6a00cc500 2023-11-24T06:27:10.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.579+0000 7ff696ffd640 1 -- 172.21.15.16:0/4015687902 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7ff690076260 con 0x7ff6a00cc500 2023-11-24T06:27:10.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:10.583 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3019663874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3019663874},{"type":"v1","addr":"172.21.15.173:6837","nonce":3019663874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2239024365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2239024365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2239024365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:26:56.460479+0000","modified":"2023-11-24T06:26:59.516602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/915142029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":915142029},{"type":"v1","addr":"172.21.15.173:6835","nonce":915142029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:27:10.583 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-11-24T06:27:10.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff68406ae40 msgr2=0x7ff68406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff68406ae40 0x7ff68406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff69c000f60 tx=0x7ff69c009040 comp rx=0 tx=0).stop 2023-11-24T06:27:10.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a00cc500 msgr2=0x7ff6a00aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a00cc500 0x7ff6a00aff90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff6900098c0 tx=0x7ff6900046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 shutdown_connections 2023-11-24T06:27:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff68406ae40 0x7ff68406d300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6a0143c40 0x7ff6a01586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a00cc500 0x7ff6a00aff90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 --2- 172.21.15.16:0/4015687902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6a006d160 0x7ff6a00ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 >> 172.21.15.16:0/4015687902 conn(0x7ff6a006a800 msgr2=0x7ff6a0144230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 shutdown_connections 2023-11-24T06:27:10.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.582+0000 7ff6a7091640 1 -- 172.21.15.16:0/4015687902 wait complete. 2023-11-24T06:27:10.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.824+0000 7f189dd9e640 1 Processor -- start 2023-11-24T06:27:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.824+0000 7f189dd9e640 1 -- start start 2023-11-24T06:27:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.826+0000 7f189dd9e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1898146770 0x7f189806c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.826+0000 7f189dd9e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18980b2e10 con 0x7f1898071410 2023-11-24T06:27:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.826+0000 7f189dd9e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1898146bc0 con 0x7f1898071f00 2023-11-24T06:27:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.826+0000 7f189dd9e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189806cec0 con 0x7f1898146770 2023-11-24T06:27:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.826+0000 7f189cd9c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1898071410 0x7f18980717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52908/0 (socket says 172.21.15.16:52908) 2023-11-24T06:27:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.826+0000 7f189cd9c640 1 -- 172.21.15.16:0/2045069203 learned_addr learned my addr 172.21.15.16:0/2045069203 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.826+0000 7f189d59d640 1 --2- 172.21.15.16:0/2045069203 >> v2:172.21.15.16:3300/0 conn(0x7f1898146770 0x7f189806c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.827+0000 7f188f7fe640 1 -- 172.21.15.16:0/2045069203 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24623560 0 0) 0x7f18980b2e10 con 0x7f1898071410 2023-11-24T06:27:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.827+0000 7f188f7fe640 1 -- 172.21.15.16:0/2045069203 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1880003520 con 0x7f1898071410 2023-11-24T06:27:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.827+0000 7f189d59d640 1 -- 172.21.15.16:0/2045069203 >> v1:172.21.15.173:6789/0 conn(0x7f1898071f00 legacy=0x7f1898164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.827+0000 7f189d59d640 1 -- 172.21.15.16:0/2045069203 >> v1:172.21.15.16:6789/0 conn(0x7f1898071410 legacy=0x7f18980717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.827+0000 7f189d59d640 1 -- 172.21.15.16:0/2045069203 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189806d1a0 con 0x7f1898146770 2023-11-24T06:27:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.828+0000 7f189d59d640 1 --2- 172.21.15.16:0/2045069203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898146770 0x7f189806c7c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1890009d90 tx=0x7f189002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e849b266df651d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.828+0000 7f188f7fe640 1 -- 172.21.15.16:0/2045069203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189002cc60 con 0x7f1898146770 2023-11-24T06:27:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.828+0000 7f188f7fe640 1 -- 172.21.15.16:0/2045069203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189002cdc0 con 0x7f1898146770 2023-11-24T06:27:10.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.828+0000 7f188f7fe640 1 -- 172.21.15.16:0/2045069203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890034660 con 0x7f1898146770 2023-11-24T06:27:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.829+0000 7f189dd9e640 1 -- 172.21.15.16:0/2045069203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898146770 msgr2=0x7f189806c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.829+0000 7f189dd9e640 1 --2- 172.21.15.16:0/2045069203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898146770 0x7f189806c7c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1890009d90 tx=0x7f189002beb0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.829+0000 7f189dd9e640 1 -- 172.21.15.16:0/2045069203 shutdown_connections 2023-11-24T06:27:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.829+0000 7f189dd9e640 1 --2- 172.21.15.16:0/2045069203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898146770 0x7f189806c7c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.829+0000 7f189dd9e640 1 -- 172.21.15.16:0/2045069203 >> 172.21.15.16:0/2045069203 conn(0x7f189806a800 msgr2=0x7f189806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.829+0000 7f189dd9e640 1 -- 172.21.15.16:0/2045069203 shutdown_connections 2023-11-24T06:27:10.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.829+0000 7f189dd9e640 1 -- 172.21.15.16:0/2045069203 wait complete. 2023-11-24T06:27:10.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.831+0000 7f189dd9e640 1 Processor -- start 2023-11-24T06:27:10.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.831+0000 7f189dd9e640 1 -- start start 2023-11-24T06:27:10.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.832+0000 7f189dd9e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1898071410 0x7f189815c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.832+0000 7f189dd9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898071f00 0x7f189815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.832+0000 7f189dd9e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1898146770 0x7f18981f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.832+0000 7f189dd9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981f3280 con 0x7f1898071f00 2023-11-24T06:27:10.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.832+0000 7f189dd9e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189815e7e0 con 0x7f1898146770 2023-11-24T06:27:10.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.832+0000 7f189dd9e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189815eac0 con 0x7f1898071410 2023-11-24T06:27:10.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.832+0000 7f188ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898071f00 0x7f189815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.832+0000 7f188ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898071f00 0x7f189815e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38600/0 (socket says 172.21.15.16:38600) 2023-11-24T06:27:10.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.832+0000 7f188ffff640 1 -- 172.21.15.16:0/3195244350 learned_addr learned my addr 172.21.15.16:0/3195244350 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f189d59d640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1898146770 0x7f18981f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f189cd9c640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1898071410 0x7f189815c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f188ffff640 1 -- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1898071410 msgr2=0x7f189815c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f188ffff640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1898071410 0x7f189815c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f188ffff640 1 -- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1898146770 msgr2=0x7f18981f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f188ffff640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1898146770 0x7f18981f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:10.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f188ffff640 1 -- 172.21.15.16:0/3195244350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890009a70 con 0x7f1898071f00 2023-11-24T06:27:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f188ffff640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898071f00 0x7f189815e150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f188800c450 tx=0x7f188800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f188d7fa640 1 -- 172.21.15.16:0/3195244350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18880052d0 con 0x7f1898071f00 2023-11-24T06:27:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.833+0000 7f188d7fa640 1 -- 172.21.15.16:0/3195244350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1888005430 con 0x7f1898071f00 2023-11-24T06:27:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.834+0000 7f188d7fa640 1 -- 172.21.15.16:0/3195244350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888012870 con 0x7f1898071f00 2023-11-24T06:27:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.834+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189815eda0 con 0x7f1898071f00 2023-11-24T06:27:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.834+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189815f360 con 0x7f1898071f00 2023-11-24T06:27:10.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.835+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1898067450 con 0x7f1898071f00 2023-11-24T06:27:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.836+0000 7f188d7fa640 1 -- 172.21.15.16:0/3195244350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18880129d0 con 0x7f1898071f00 2023-11-24T06:27:10.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.837+0000 7f188d7fa640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f188406ae40 0x7f188406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.837+0000 7f188d7fa640 1 -- 172.21.15.16:0/3195244350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18880891f0 con 0x7f1898071f00 2023-11-24T06:27:10.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.837+0000 7f189cd9c640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f188406ae40 0x7f188406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:10.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.838+0000 7f189cd9c640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f188406ae40 0x7f188406d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18940011e0 tx=0x7f1894009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:10.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:10.841+0000 7f188d7fa640 1 -- 172.21.15.16:0/3195244350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1888054200 con 0x7f1898071f00 2023-11-24T06:27:11.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.035+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f189815f520 con 0x7f1898071f00 2023-11-24T06:27:11.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.038+0000 7f188d7fa640 1 -- 172.21.15.16:0/3195244350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f1888057820 con 0x7f1898071f00 2023-11-24T06:27:11.042 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:11.043 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f188406ae40 msgr2=0x7f188406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f188406ae40 0x7f188406d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18940011e0 tx=0x7f1894009040 comp rx=0 tx=0).stop 2023-11-24T06:27:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898071f00 msgr2=0x7f189815e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898071f00 0x7f189815e150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f188800c450 tx=0x7f188800c910 comp rx=0 tx=0).stop 2023-11-24T06:27:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 shutdown_connections 2023-11-24T06:27:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f188406ae40 0x7f188406d300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1898146770 0x7f18981f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1898071f00 0x7f189815e150 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 --2- 172.21.15.16:0/3195244350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1898071410 0x7f189815c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 >> 172.21.15.16:0/3195244350 conn(0x7f189806a800 msgr2=0x7f18980b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:11.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.041+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 shutdown_connections 2023-11-24T06:27:11.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.042+0000 7f189dd9e640 1 -- 172.21.15.16:0/3195244350 wait complete. 2023-11-24T06:27:11.064 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:11.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.269+0000 7fc6e583d640 1 Processor -- start 2023-11-24T06:27:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.269+0000 7fc6e583d640 1 -- start start 2023-11-24T06:27:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6e583d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc6e013df80 0x7fc6e015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6e583d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6e013e360 0x7fc6e00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6e583d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e015f9c0 0x7fc6e00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6e583d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e006c470 con 0x7fc6e013e360 2023-11-24T06:27:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6e583d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e006c750 con 0x7fc6e013df80 2023-11-24T06:27:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6e583d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e006ca30 con 0x7fc6e015f9c0 2023-11-24T06:27:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6de7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6e013e360 0x7fc6e00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6de7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6e013e360 0x7fc6e00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38606/0 (socket says 172.21.15.16:38606) 2023-11-24T06:27:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6de7fc640 1 -- 172.21.15.16:0/333396309 learned_addr learned my addr 172.21.15.16:0/333396309 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6deffd640 1 --2- 172.21.15.16:0/333396309 >> v2:172.21.15.173:3300/0 conn(0x7fc6e013df80 0x7fc6e015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6de7fc640 1 -- 172.21.15.16:0/333396309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e015f9c0 msgr2=0x7fc6e00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6df7fe640 1 --2- 172.21.15.16:0/333396309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e015f9c0 0x7fc6e00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6de7fc640 1 --2- 172.21.15.16:0/333396309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e015f9c0 0x7fc6e00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6de7fc640 1 -- 172.21.15.16:0/333396309 >> v2:172.21.15.173:3300/0 conn(0x7fc6e013df80 msgr2=0x7fc6e015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6de7fc640 1 --2- 172.21.15.16:0/333396309 >> v2:172.21.15.173:3300/0 conn(0x7fc6e013df80 0x7fc6e015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6de7fc640 1 -- 172.21.15.16:0/333396309 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e006cd10 con 0x7fc6e013e360 2023-11-24T06:27:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6deffd640 1 --2- 172.21.15.16:0/333396309 >> v2:172.21.15.173:3300/0 conn(0x7fc6e013df80 0x7fc6e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.271+0000 7fc6df7fe640 1 --2- 172.21.15.16:0/333396309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e015f9c0 0x7fc6e00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:11.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.273+0000 7fc6de7fc640 1 --2- 172.21.15.16:0/333396309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 0x7fc6e00cf290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0072770 tx=0x7fc6d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2609fd2e1b99e18 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.273+0000 7fc6ddffb640 1 -- 172.21.15.16:0/333396309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4031870 con 0x7fc6e013e360 2023-11-24T06:27:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.273+0000 7fc6ddffb640 1 -- 172.21.15.16:0/333396309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d40319d0 con 0x7fc6e013e360 2023-11-24T06:27:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.273+0000 7fc6ddffb640 1 -- 172.21.15.16:0/333396309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4031d60 con 0x7fc6e013e360 2023-11-24T06:27:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.274+0000 7fc6e583d640 1 -- 172.21.15.16:0/333396309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 msgr2=0x7fc6e00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.274+0000 7fc6e583d640 1 --2- 172.21.15.16:0/333396309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 0x7fc6e00cf290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0072770 tx=0x7fc6d402b910 comp rx=0 tx=0).stop 2023-11-24T06:27:11.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.274+0000 7fc6e583d640 1 -- 172.21.15.16:0/333396309 shutdown_connections 2023-11-24T06:27:11.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.274+0000 7fc6e583d640 1 --2- 172.21.15.16:0/333396309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e015f9c0 0x7fc6e00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.274+0000 7fc6e583d640 1 --2- 172.21.15.16:0/333396309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 0x7fc6e00cf290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.274+0000 7fc6e583d640 1 --2- 172.21.15.16:0/333396309 >> v2:172.21.15.173:3300/0 conn(0x7fc6e013df80 0x7fc6e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.274+0000 7fc6e583d640 1 -- 172.21.15.16:0/333396309 >> 172.21.15.16:0/333396309 conn(0x7fc6e006a800 msgr2=0x7fc6e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:11.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.274+0000 7fc6e583d640 1 -- 172.21.15.16:0/333396309 shutdown_connections 2023-11-24T06:27:11.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.274+0000 7fc6e583d640 1 -- 172.21.15.16:0/333396309 wait complete. 2023-11-24T06:27:11.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.276+0000 7fc6e583d640 1 Processor -- start 2023-11-24T06:27:11.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.276+0000 7fc6e583d640 1 -- start start 2023-11-24T06:27:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6e583d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e013df80 0x7fc6e01545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6e583d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 0x7fc6e00a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6e583d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e015f9c0 0x7fc6e00aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6e583d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e00acf90 con 0x7fc6e013e360 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6e583d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e00ad270 con 0x7fc6e015f9c0 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6e583d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e00ad550 con 0x7fc6e013df80 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 0x7fc6e00a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 0x7fc6e00a9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38612/0 (socket says 172.21.15.16:38612) 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6de7fc640 1 -- 172.21.15.16:0/108668271 learned_addr learned my addr 172.21.15.16:0/108668271 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6de7fc640 1 -- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e013df80 msgr2=0x7fc6e01545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6de7fc640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e013df80 0x7fc6e01545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6de7fc640 1 -- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e015f9c0 msgr2=0x7fc6e00aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6de7fc640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e015f9c0 0x7fc6e00aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.277+0000 7fc6de7fc640 1 -- 172.21.15.16:0/108668271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d40094d0 con 0x7fc6e013e360 2023-11-24T06:27:11.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.278+0000 7fc6de7fc640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 0x7fc6e00a9e70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc6d402b950 tx=0x7fc6d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.278+0000 7fc6c7fff640 1 -- 172.21.15.16:0/108668271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4031860 con 0x7fc6e013e360 2023-11-24T06:27:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.278+0000 7fc6c7fff640 1 -- 172.21.15.16:0/108668271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d40443f0 con 0x7fc6e013e360 2023-11-24T06:27:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.278+0000 7fc6c7fff640 1 -- 172.21.15.16:0/108668271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d40447a0 con 0x7fc6e013e360 2023-11-24T06:27:11.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.278+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e00ad830 con 0x7fc6e013e360 2023-11-24T06:27:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.278+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e00a9740 con 0x7fc6e013e360 2023-11-24T06:27:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.279+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e00add50 con 0x7fc6e013e360 2023-11-24T06:27:11.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.280+0000 7fc6c7fff640 1 -- 172.21.15.16:0/108668271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc6d404d450 con 0x7fc6e013e360 2023-11-24T06:27:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.281+0000 7fc6c7fff640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc6cc06ae40 0x7fc6cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.281+0000 7fc6c7fff640 1 -- 172.21.15.16:0/108668271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6d402d040 con 0x7fc6e013e360 2023-11-24T06:27:11.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.281+0000 7fc6deffd640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc6cc06ae40 0x7fc6cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.282+0000 7fc6deffd640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc6cc06ae40 0x7fc6cc06d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0006fd0 tx=0x7fc6d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:11.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.285+0000 7fc6c7fff640 1 -- 172.21.15.16:0/108668271 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6e00add50 con 0x7fc6e013e360 2023-11-24T06:27:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.476+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6e00add50 con 0x7fc6e013e360 2023-11-24T06:27:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.477+0000 7fc6c7fff640 1 -- 172.21.15.16:0/108668271 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6e00add50 con 0x7fc6e013e360 2023-11-24T06:27:11.481 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.479+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc6cc06ae40 msgr2=0x7fc6cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.479+0000 7fc6e583d640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc6cc06ae40 0x7fc6cc06d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0006fd0 tx=0x7fc6d0008040 comp rx=0 tx=0).stop 2023-11-24T06:27:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.479+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 msgr2=0x7fc6e00a9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.479+0000 7fc6e583d640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 0x7fc6e00a9e70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc6d402b950 tx=0x7fc6d40046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.480+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 shutdown_connections 2023-11-24T06:27:11.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.480+0000 7fc6e583d640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc6cc06ae40 0x7fc6cc06d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.480+0000 7fc6e583d640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6e015f9c0 0x7fc6e00aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.480+0000 7fc6e583d640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6e013e360 0x7fc6e00a9e70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.480+0000 7fc6e583d640 1 --2- 172.21.15.16:0/108668271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6e013df80 0x7fc6e01545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.480+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 >> 172.21.15.16:0/108668271 conn(0x7fc6e006a800 msgr2=0x7fc6e00ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.480+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 shutdown_connections 2023-11-24T06:27:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.480+0000 7fc6e583d640 1 -- 172.21.15.16:0/108668271 wait complete. 2023-11-24T06:27:11.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.733+0000 7ffa30fcb640 1 Processor -- start 2023-11-24T06:27:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.733+0000 7ffa30fcb640 1 -- start start 2023-11-24T06:27:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.735+0000 7ffa30fcb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ffa2c13fcb0 0x7ffa2c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.735+0000 7ffa30fcb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa2c0ce440 con 0x7ffa2c0cdc20 2023-11-24T06:27:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.735+0000 7ffa30fcb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa2c0ce720 con 0x7ffa2c13cf10 2023-11-24T06:27:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.735+0000 7ffa30fcb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c0cea00 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.735+0000 7ffa2bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffa2c0cdc20 0x7ffa2c0b2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52914/0 (socket says 172.21.15.16:52914) 2023-11-24T06:27:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.735+0000 7ffa2bfff640 1 -- 172.21.15.16:0/2533274325 learned_addr learned my addr 172.21.15.16:0/2533274325 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.735+0000 7ffa2affd640 1 --2- 172.21.15.16:0/2533274325 >> v2:172.21.15.173:3300/0 conn(0x7ffa2c13fcb0 0x7ffa2c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.736+0000 7ffa2a7fc640 1 -- 172.21.15.16:0/2533274325 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579582411 0 0) 0x7ffa2c0ce440 con 0x7ffa2c0cdc20 2023-11-24T06:27:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.736+0000 7ffa2a7fc640 1 -- 172.21.15.16:0/2533274325 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa14003520 con 0x7ffa2c0cdc20 2023-11-24T06:27:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.736+0000 7ffa2affd640 1 -- 172.21.15.16:0/2533274325 >> v1:172.21.15.173:6789/0 conn(0x7ffa2c13cf10 legacy=0x7ffa2c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.736+0000 7ffa2a7fc640 1 -- 172.21.15.16:0/2533274325 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4054263015 0 0) 0x7ffa14003520 con 0x7ffa2c0cdc20 2023-11-24T06:27:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.736+0000 7ffa2affd640 1 -- 172.21.15.16:0/2533274325 >> v1:172.21.15.16:6789/0 conn(0x7ffa2c0cdc20 legacy=0x7ffa2c0b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.736+0000 7ffa2affd640 1 -- 172.21.15.16:0/2533274325 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c0b1d90 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.738+0000 7ffa2affd640 1 --2- 172.21.15.16:0/2533274325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa2c13fcb0 0x7ffa2c0cd6e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa200097a0 tx=0x7ffa2002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c23c0ab7deda5744 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.738+0000 7ffa2a7fc640 1 -- 172.21.15.16:0/2533274325 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2002f050 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.738+0000 7ffa2a7fc640 1 -- 172.21.15.16:0/2533274325 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa20031500 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.738+0000 7ffa2a7fc640 1 -- 172.21.15.16:0/2533274325 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24001670 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.739+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2533274325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa2c13fcb0 msgr2=0x7ffa2c0cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.739+0000 7ffa30fcb640 1 --2- 172.21.15.16:0/2533274325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa2c13fcb0 0x7ffa2c0cd6e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa200097a0 tx=0x7ffa2002b820 comp rx=0 tx=0).stop 2023-11-24T06:27:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.739+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2533274325 shutdown_connections 2023-11-24T06:27:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.739+0000 7ffa30fcb640 1 --2- 172.21.15.16:0/2533274325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa2c13fcb0 0x7ffa2c0cd6e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.739+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2533274325 >> 172.21.15.16:0/2533274325 conn(0x7ffa2c06a7d0 msgr2=0x7ffa2c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.739+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2533274325 shutdown_connections 2023-11-24T06:27:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.739+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2533274325 wait complete. 2023-11-24T06:27:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.741+0000 7ffa30fcb640 1 Processor -- start 2023-11-24T06:27:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.741+0000 7ffa30fcb640 1 -- start start 2023-11-24T06:27:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.742+0000 7ffa30fcb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2c0cdc20 0x7ffa2c1f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.742+0000 7ffa30fcb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa2c13cf10 0x7ffa2c15a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.742+0000 7ffa30fcb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c13fcb0 0x7ffa2c153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.742+0000 7ffa30fcb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c137390 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.742+0000 7ffa30fcb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0ad4e0 con 0x7ffa2c13cf10 2023-11-24T06:27:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.742+0000 7ffa30fcb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0ad7c0 con 0x7ffa2c0cdc20 2023-11-24T06:27:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.742+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c13fcb0 0x7ffa2c153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.742+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c13fcb0 0x7ffa2c153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38618/0 (socket says 172.21.15.16:38618) 2023-11-24T06:27:11.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.742+0000 7ffa2bfff640 1 -- 172.21.15.16:0/2707532496 learned_addr learned my addr 172.21.15.16:0/2707532496 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:11.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa2affd640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa2c13cf10 0x7ffa2c15a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa2b7fe640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2c0cdc20 0x7ffa2c1f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa2bfff640 1 -- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2c0cdc20 msgr2=0x7ffa2c1f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa2bfff640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2c0cdc20 0x7ffa2c1f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa2bfff640 1 -- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa2c13cf10 msgr2=0x7ffa2c15a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa2bfff640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa2c13cf10 0x7ffa2c15a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa2bfff640 1 -- 172.21.15.16:0/2707532496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa20009400 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa2bfff640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c13fcb0 0x7ffa2c153530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c750 tx=0x7ffa2400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:11.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa13fff640 1 -- 172.21.15.16:0/2707532496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24005ad0 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa13fff640 1 -- 172.21.15.16:0/2707532496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa24005c30 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa13fff640 1 -- 172.21.15.16:0/2707532496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24007070 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.743+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c0adaa0 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.744+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c1f2c70 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.745+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2c137500 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.746+0000 7ffa13fff640 1 -- 172.21.15.16:0/2707532496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa240071d0 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.746+0000 7ffa13fff640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa1806ae40 0x7ffa1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.747+0000 7ffa13fff640 1 -- 172.21.15.16:0/2707532496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa24089350 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.747+0000 7ffa2b7fe640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa1806ae40 0x7ffa1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.748+0000 7ffa2b7fe640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa1806ae40 0x7ffa1806d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0012e0 tx=0x7ffa1c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:11.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.751+0000 7ffa13fff640 1 -- 172.21.15.16:0/2707532496 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa240543f0 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.944+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa2c067450 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.945+0000 7ffa13fff640 1 -- 172.21.15.16:0/2707532496 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa24057a10 con 0x7ffa2c13fcb0 2023-11-24T06:27:11.949 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:11.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa1806ae40 msgr2=0x7ffa1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa1806ae40 0x7ffa1806d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0012e0 tx=0x7ffa1c009040 comp rx=0 tx=0).stop 2023-11-24T06:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c13fcb0 msgr2=0x7ffa2c153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c13fcb0 0x7ffa2c153530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c750 tx=0x7ffa2400cc10 comp rx=0 tx=0).stop 2023-11-24T06:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 shutdown_connections 2023-11-24T06:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa1806ae40 0x7ffa1806d300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa2c13fcb0 0x7ffa2c153530 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa2c13cf10 0x7ffa2c15a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 --2- 172.21.15.16:0/2707532496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2c0cdc20 0x7ffa2c1f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 >> 172.21.15.16:0/2707532496 conn(0x7ffa2c06a7d0 msgr2=0x7ffa2c0b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 shutdown_connections 2023-11-24T06:27:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:11.948+0000 7ffa30fcb640 1 -- 172.21.15.16:0/2707532496 wait complete. 2023-11-24T06:27:11.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.202+0000 7f63d9963640 1 Processor -- start 2023-11-24T06:27:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.202+0000 7f63d9963640 1 -- start start 2023-11-24T06:27:12.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.204+0000 7f63d9963640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63d415d2e0 0x7f63d406c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.204+0000 7f63d9963640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f63d406b500 0x7f63d40ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.204+0000 7f63d9963640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d406f2e0 con 0x7f63d40ad2e0 2023-11-24T06:27:12.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.204+0000 7f63d9963640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d40adcb0 con 0x7f63d415d2e0 2023-11-24T06:27:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.204+0000 7f63d9963640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d406b9d0 con 0x7f63d406b500 2023-11-24T06:27:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.204+0000 7f63d8961640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63d415d2e0 0x7f63d406c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.204+0000 7f63d8961640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63d415d2e0 0x7f63d406c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38634/0 (socket says 172.21.15.16:38634) 2023-11-24T06:27:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.204+0000 7f63d8961640 1 -- 172.21.15.16:0/4243858077 learned_addr learned my addr 172.21.15.16:0/4243858077 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:12.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.205+0000 7f63d9162640 1 --2- 172.21.15.16:0/4243858077 >> v2:172.21.15.173:3300/0 conn(0x7f63d406b500 0x7f63d40ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.205+0000 7f63d8961640 1 -- 172.21.15.16:0/4243858077 >> v2:172.21.15.173:3300/0 conn(0x7f63d406b500 msgr2=0x7f63d40ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.205+0000 7f63d8961640 1 --2- 172.21.15.16:0/4243858077 >> v2:172.21.15.173:3300/0 conn(0x7f63d406b500 0x7f63d40ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.205+0000 7f63d8961640 1 -- 172.21.15.16:0/4243858077 >> v1:172.21.15.173:6789/0 conn(0x7f63d40ad2e0 legacy=0x7f63d406adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.205+0000 7f63d8961640 1 -- 172.21.15.16:0/4243858077 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d406bcb0 con 0x7f63d415d2e0 2023-11-24T06:27:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.205+0000 7f63d9162640 1 --2- 172.21.15.16:0/4243858077 >> v2:172.21.15.173:3300/0 conn(0x7f63d406b500 0x7f63d40ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:12.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.206+0000 7f63d8961640 1 --2- 172.21.15.16:0/4243858077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d415d2e0 0x7f63d406c8a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f63d4070970 tx=0x7f63c802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e30f8c50975bcaf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d37fe640 1 -- 172.21.15.16:0/4243858077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80317e0 con 0x7f63d415d2e0 2023-11-24T06:27:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d37fe640 1 -- 172.21.15.16:0/4243858077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c8031940 con 0x7f63d415d2e0 2023-11-24T06:27:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d37fe640 1 -- 172.21.15.16:0/4243858077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8031cd0 con 0x7f63d415d2e0 2023-11-24T06:27:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d9963640 1 -- 172.21.15.16:0/4243858077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d415d2e0 msgr2=0x7f63d406c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d9963640 1 --2- 172.21.15.16:0/4243858077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d415d2e0 0x7f63d406c8a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f63d4070970 tx=0x7f63c802b880 comp rx=0 tx=0).stop 2023-11-24T06:27:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d9963640 1 -- 172.21.15.16:0/4243858077 shutdown_connections 2023-11-24T06:27:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d9963640 1 --2- 172.21.15.16:0/4243858077 >> v2:172.21.15.173:3300/0 conn(0x7f63d406b500 0x7f63d40ad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d9963640 1 --2- 172.21.15.16:0/4243858077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d415d2e0 0x7f63d406c8a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d9963640 1 -- 172.21.15.16:0/4243858077 >> 172.21.15.16:0/4243858077 conn(0x7f63d40cf6c0 msgr2=0x7f63d40cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.207+0000 7f63d9963640 1 -- 172.21.15.16:0/4243858077 shutdown_connections 2023-11-24T06:27:12.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.208+0000 7f63d9963640 1 -- 172.21.15.16:0/4243858077 wait complete. 2023-11-24T06:27:12.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.209+0000 7f63d9963640 1 Processor -- start 2023-11-24T06:27:12.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.210+0000 7f63d9963640 1 -- start start 2023-11-24T06:27:12.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.210+0000 7f63d9963640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63d406b500 0x7f63d4142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d9963640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d40ad2e0 0x7f63d4140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d9963640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63d415d2e0 0x7f63d41418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d9963640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4140590 con 0x7f63d40ad2e0 2023-11-24T06:27:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d9963640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4140830 con 0x7f63d406b500 2023-11-24T06:27:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d9963640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4140b10 con 0x7f63d415d2e0 2023-11-24T06:27:12.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d40ad2e0 0x7f63d4140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d40ad2e0 0x7f63d4140050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38638/0 (socket says 172.21.15.16:38638) 2023-11-24T06:27:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d3fff640 1 -- 172.21.15.16:0/2861182395 learned_addr learned my addr 172.21.15.16:0/2861182395 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d8961640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63d406b500 0x7f63d4142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d9162640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63d415d2e0 0x7f63d41418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d3fff640 1 -- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63d415d2e0 msgr2=0x7f63d41418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d3fff640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63d415d2e0 0x7f63d41418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d3fff640 1 -- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63d406b500 msgr2=0x7f63d4142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d3fff640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63d406b500 0x7f63d4142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d3fff640 1 -- 172.21.15.16:0/2861182395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c80094d0 con 0x7f63d40ad2e0 2023-11-24T06:27:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d9162640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63d415d2e0 0x7f63d41418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.211+0000 7f63d8961640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63d406b500 0x7f63d4142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.212+0000 7f63d3fff640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d40ad2e0 0x7f63d4140050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63c000c430 tx=0x7f63c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.212+0000 7f63d17fa640 1 -- 172.21.15.16:0/2861182395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0011790 con 0x7f63d40ad2e0 2023-11-24T06:27:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.212+0000 7f63d17fa640 1 -- 172.21.15.16:0/2861182395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c00118f0 con 0x7f63d40ad2e0 2023-11-24T06:27:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.212+0000 7f63d17fa640 1 -- 172.21.15.16:0/2861182395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0011ce0 con 0x7f63d40ad2e0 2023-11-24T06:27:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.212+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d4140df0 con 0x7f63d40ad2e0 2023-11-24T06:27:12.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.212+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d41413b0 con 0x7f63d40ad2e0 2023-11-24T06:27:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.213+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d4067b60 con 0x7f63d40ad2e0 2023-11-24T06:27:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.215+0000 7f63d17fa640 1 -- 172.21.15.16:0/2861182395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63c001c460 con 0x7f63d40ad2e0 2023-11-24T06:27:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.216+0000 7f63d17fa640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f63b806ae40 0x7f63b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.216+0000 7f63d17fa640 1 -- 172.21.15.16:0/2861182395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63c0089700 con 0x7f63d40ad2e0 2023-11-24T06:27:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.217+0000 7f63d8961640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f63b806ae40 0x7f63b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.218+0000 7f63d8961640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f63b806ae40 0x7f63b806d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f63c8036000 tx=0x7f63c803d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:12.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.221+0000 7f63d17fa640 1 -- 172.21.15.16:0/2861182395 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63c0054680 con 0x7f63d40ad2e0 2023-11-24T06:27:12.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.414+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63d40cefa0 con 0x7f63d40ad2e0 2023-11-24T06:27:12.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.415+0000 7f63d17fa640 1 -- 172.21.15.16:0/2861182395 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63c0057ca0 con 0x7f63d40ad2e0 2023-11-24T06:27:12.418 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:12.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f63b806ae40 msgr2=0x7f63b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f63b806ae40 0x7f63b806d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f63c8036000 tx=0x7f63c803d040 comp rx=0 tx=0).stop 2023-11-24T06:27:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d40ad2e0 msgr2=0x7f63d4140050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d40ad2e0 0x7f63d4140050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63c000c430 tx=0x7f63c000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 shutdown_connections 2023-11-24T06:27:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f63b806ae40 0x7f63b806d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63d415d2e0 0x7f63d41418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d40ad2e0 0x7f63d4140050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 --2- 172.21.15.16:0/2861182395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63d406b500 0x7f63d4142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 >> 172.21.15.16:0/2861182395 conn(0x7f63d40cf6c0 msgr2=0x7f63d406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.417+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 shutdown_connections 2023-11-24T06:27:12.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.418+0000 7f63d9963640 1 -- 172.21.15.16:0/2861182395 wait complete. 2023-11-24T06:27:12.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.651+0000 7f701f085640 1 Processor -- start 2023-11-24T06:27:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.651+0000 7f701f085640 1 -- start start 2023-11-24T06:27:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.653+0000 7f701f085640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f70180cc500 0x7f70180cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.653+0000 7f701f085640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.653+0000 7f701f085640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7018070e50 con 0x7f7018143c40 2023-11-24T06:27:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.653+0000 7f701f085640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70180723c0 con 0x7f70180cc500 2023-11-24T06:27:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.653+0000 7f701f085640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70180cced0 con 0x7f701806d160 2023-11-24T06:27:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.654+0000 7f701e884640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.654+0000 7f701d882640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f70180cc500 0x7f70180cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.654+0000 7f701e884640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701806d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:50654/0 (socket says 172.21.15.16:50654) 2023-11-24T06:27:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.654+0000 7f701e884640 1 -- 172.21.15.16:0/3726250593 learned_addr learned my addr 172.21.15.16:0/3726250593 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.654+0000 7f701e884640 1 -- 172.21.15.16:0/3726250593 >> v2:172.21.15.173:3300/0 conn(0x7f70180cc500 msgr2=0x7f70180cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.654+0000 7f701e884640 1 --2- 172.21.15.16:0/3726250593 >> v2:172.21.15.173:3300/0 conn(0x7f70180cc500 0x7f70180cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.654+0000 7f701e884640 1 -- 172.21.15.16:0/3726250593 >> v1:172.21.15.173:6789/0 conn(0x7f7018143c40 legacy=0x7f7018071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.654+0000 7f701e884640 1 -- 172.21.15.16:0/3726250593 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7018144020 con 0x7f701806d160 2023-11-24T06:27:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.656+0000 7f701e884640 1 --2- 172.21.15.16:0/3726250593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701806d560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f70180b44a0 tx=0x7f701002be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc5d9e8c7b351c3b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.656+0000 7f701d081640 1 -- 172.21.15.16:0/3726250593 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701002c9a0 con 0x7f701806d160 2023-11-24T06:27:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.656+0000 7f701d081640 1 -- 172.21.15.16:0/3726250593 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701002cb00 con 0x7f701806d160 2023-11-24T06:27:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.656+0000 7f701d081640 1 -- 172.21.15.16:0/3726250593 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701002ce90 con 0x7f701806d160 2023-11-24T06:27:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.657+0000 7f701f085640 1 -- 172.21.15.16:0/3726250593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 msgr2=0x7f701806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.657+0000 7f701f085640 1 --2- 172.21.15.16:0/3726250593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701806d560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f70180b44a0 tx=0x7f701002be00 comp rx=0 tx=0).stop 2023-11-24T06:27:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.657+0000 7f701f085640 1 -- 172.21.15.16:0/3726250593 shutdown_connections 2023-11-24T06:27:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.657+0000 7f701f085640 1 --2- 172.21.15.16:0/3726250593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701806d560 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.657+0000 7f701f085640 1 --2- 172.21.15.16:0/3726250593 >> v2:172.21.15.173:3300/0 conn(0x7f70180cc500 0x7f70180cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.657+0000 7f701f085640 1 -- 172.21.15.16:0/3726250593 >> 172.21.15.16:0/3726250593 conn(0x7f701806a800 msgr2=0x7f701806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.657+0000 7f701f085640 1 -- 172.21.15.16:0/3726250593 shutdown_connections 2023-11-24T06:27:12.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.657+0000 7f701f085640 1 -- 172.21.15.16:0/3726250593 wait complete. 2023-11-24T06:27:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.659+0000 7f701f085640 1 Processor -- start 2023-11-24T06:27:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.659+0000 7f701f085640 1 -- start start 2023-11-24T06:27:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.660+0000 7f701f085640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.660+0000 7f701f085640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70180cc500 0x7f7018142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.660+0000 7f701f085640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7018143c40 0x7f7018137e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.660+0000 7f701f085640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70180b22a0 con 0x7f7018143c40 2023-11-24T06:27:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.660+0000 7f701f085640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70180b2580 con 0x7f70180cc500 2023-11-24T06:27:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.660+0000 7f701f085640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70180b2860 con 0x7f701806d160 2023-11-24T06:27:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.660+0000 7f701e884640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7018143c40 0x7f7018137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.660+0000 7f701e884640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7018143c40 0x7f7018137e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56060/0 (socket says 172.21.15.16:56060) 2023-11-24T06:27:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.660+0000 7f701e884640 1 -- 172.21.15.16:0/2732743536 learned_addr learned my addr 172.21.15.16:0/2732743536 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701d882640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70180cc500 0x7f7018142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701e083640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701e884640 1 -- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 msgr2=0x7f701815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701e884640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701e884640 1 -- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70180cc500 msgr2=0x7f7018142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701e884640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70180cc500 0x7f7018142d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701e884640 1 -- 172.21.15.16:0/2732743536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7010009b30 con 0x7f7018143c40 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701e083640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701d882640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70180cc500 0x7f7018142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701e884640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7018143c40 0x7f7018137e70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7010036040 tx=0x7f7010004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f700effd640 1 -- 172.21.15.16:0/2732743536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010009510 con 0x7f7018143c40 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f700effd640 1 -- 172.21.15.16:0/2732743536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701003e450 con 0x7f7018143c40 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f700effd640 1 -- 172.21.15.16:0/2732743536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701003e7e0 con 0x7f7018143c40 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.661+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70180b2b40 con 0x7f7018143c40 2023-11-24T06:27:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.662+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701815ae70 con 0x7f7018143c40 2023-11-24T06:27:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.663+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7018067450 con 0x7f7018143c40 2023-11-24T06:27:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.664+0000 7f700effd640 1 -- 172.21.15.16:0/2732743536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f701003e940 con 0x7f7018143c40 2023-11-24T06:27:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.665+0000 7f700effd640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ffc06ae40 0x7f6ffc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.665+0000 7f700effd640 1 -- 172.21.15.16:0/2732743536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70100ac2b0 con 0x7f7018143c40 2023-11-24T06:27:12.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.665+0000 7f701e083640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ffc06ae40 0x7f6ffc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:12.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.666+0000 7f701e083640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ffc06ae40 0x7f6ffc06d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f701400a830 tx=0x7f7014008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:12.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.669+0000 7f700effd640 1 -- 172.21.15.16:0/2732743536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70100773f0 con 0x7f7018143c40 2023-11-24T06:27:12.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.901+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70180ceb30 con 0x7f7018143c40 2023-11-24T06:27:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.902+0000 7f700effd640 1 -- 172.21.15.16:0/2732743536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f701007aa10 con 0x7f7018143c40 2023-11-24T06:27:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:12.905 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3019663874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3019663874},{"type":"v1","addr":"172.21.15.173:6837","nonce":3019663874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2239024365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2239024365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2239024365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:26:56.460479+0000","modified":"2023-11-24T06:26:59.516602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/915142029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":915142029},{"type":"v1","addr":"172.21.15.173:6835","nonce":915142029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:27:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-11-24T06:27:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.904+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ffc06ae40 msgr2=0x7f6ffc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.904+0000 7f701f085640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ffc06ae40 0x7f6ffc06d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f701400a830 tx=0x7f7014008040 comp rx=0 tx=0).stop 2023-11-24T06:27:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7018143c40 msgr2=0x7f7018137e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7018143c40 0x7f7018137e70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7010036040 tx=0x7f7010004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 shutdown_connections 2023-11-24T06:27:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ffc06ae40 0x7f6ffc06d300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7018143c40 0x7f7018137e70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70180cc500 0x7f7018142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 --2- 172.21.15.16:0/2732743536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f701806d160 0x7f701815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 >> 172.21.15.16:0/2732743536 conn(0x7f701806a800 msgr2=0x7f7018071290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 shutdown_connections 2023-11-24T06:27:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:12.905+0000 7f701f085640 1 -- 172.21.15.16:0/2732743536 wait complete. 2023-11-24T06:27:12.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.141+0000 7f5b3c0da640 1 Processor -- start 2023-11-24T06:27:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.141+0000 7f5b3c0da640 1 -- start start 2023-11-24T06:27:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.143+0000 7f5b3c0da640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b340cf9a0 0x7f5b34166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.143+0000 7f5b3c0da640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3415c5f0 con 0x7f5b3413df80 2023-11-24T06:27:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.143+0000 7f5b3c0da640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3406c470 con 0x7f5b3413e360 2023-11-24T06:27:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.143+0000 7f5b3c0da640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3406c750 con 0x7f5b340cf9a0 2023-11-24T06:27:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.143+0000 7f5b39e76640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5b3413df80 0x7f5b3415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47630/0 (socket says 172.21.15.16:47630) 2023-11-24T06:27:13.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.143+0000 7f5b39e76640 1 -- 172.21.15.16:0/2746989305 learned_addr learned my addr 172.21.15.16:0/2746989305 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.143+0000 7f5b3a677640 1 --2- 172.21.15.16:0/2746989305 >> v2:172.21.15.16:3300/0 conn(0x7f5b340cf9a0 0x7f5b34166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.144+0000 7f5b38e74640 1 -- 172.21.15.16:0/2746989305 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356457703 0 0) 0x7f5b3415c5f0 con 0x7f5b3413df80 2023-11-24T06:27:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.144+0000 7f5b38e74640 1 -- 172.21.15.16:0/2746989305 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b20003520 con 0x7f5b3413df80 2023-11-24T06:27:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.144+0000 7f5b3a677640 1 -- 172.21.15.16:0/2746989305 >> v1:172.21.15.173:6789/0 conn(0x7f5b3413e360 legacy=0x7f5b340cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.144+0000 7f5b3a677640 1 -- 172.21.15.16:0/2746989305 >> v1:172.21.15.16:6789/0 conn(0x7f5b3413df80 legacy=0x7f5b3415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.144+0000 7f5b3a677640 1 -- 172.21.15.16:0/2746989305 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3406ca30 con 0x7f5b340cf9a0 2023-11-24T06:27:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.145+0000 7f5b3a677640 1 --2- 172.21.15.16:0/2746989305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b340cf9a0 0x7f5b34166030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009d90 tx=0x7f5b2c02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7191e1ac7e5353 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.146+0000 7f5b38e74640 1 -- 172.21.15.16:0/2746989305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c02ccf0 con 0x7f5b340cf9a0 2023-11-24T06:27:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.146+0000 7f5b38e74640 1 -- 172.21.15.16:0/2746989305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c02ce50 con 0x7f5b340cf9a0 2023-11-24T06:27:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.146+0000 7f5b38e74640 1 -- 172.21.15.16:0/2746989305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c034660 con 0x7f5b340cf9a0 2023-11-24T06:27:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.147+0000 7f5b3c0da640 1 -- 172.21.15.16:0/2746989305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b340cf9a0 msgr2=0x7f5b34166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.147+0000 7f5b3c0da640 1 --2- 172.21.15.16:0/2746989305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b340cf9a0 0x7f5b34166030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009d90 tx=0x7f5b2c02bf40 comp rx=0 tx=0).stop 2023-11-24T06:27:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.147+0000 7f5b3c0da640 1 -- 172.21.15.16:0/2746989305 shutdown_connections 2023-11-24T06:27:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.147+0000 7f5b3c0da640 1 --2- 172.21.15.16:0/2746989305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b340cf9a0 0x7f5b34166030 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.147+0000 7f5b3c0da640 1 -- 172.21.15.16:0/2746989305 >> 172.21.15.16:0/2746989305 conn(0x7f5b3406a800 msgr2=0x7f5b3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.147+0000 7f5b3c0da640 1 -- 172.21.15.16:0/2746989305 shutdown_connections 2023-11-24T06:27:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.147+0000 7f5b3c0da640 1 -- 172.21.15.16:0/2746989305 wait complete. 2023-11-24T06:27:13.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.149+0000 7f5b3c0da640 1 Processor -- start 2023-11-24T06:27:13.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.149+0000 7f5b3c0da640 1 -- start start 2023-11-24T06:27:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.150+0000 7f5b3c0da640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b340cf9a0 0x7f5b34137700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.150+0000 7f5b3c0da640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b3413df80 0x7f5b340b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.150+0000 7f5b3c0da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3413e360 0x7f5b340b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.150+0000 7f5b3c0da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34139650 con 0x7f5b3413e360 2023-11-24T06:27:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.150+0000 7f5b3c0da640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b341398f0 con 0x7f5b3413df80 2023-11-24T06:27:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.150+0000 7f5b3c0da640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34139bd0 con 0x7f5b340cf9a0 2023-11-24T06:27:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.150+0000 7f5b3a677640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3413e360 0x7f5b340b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.150+0000 7f5b3a677640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3413e360 0x7f5b340b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56090/0 (socket says 172.21.15.16:56090) 2023-11-24T06:27:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.150+0000 7f5b3a677640 1 -- 172.21.15.16:0/1084336958 learned_addr learned my addr 172.21.15.16:0/1084336958 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b39e76640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b340cf9a0 0x7f5b34137700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b3a677640 1 -- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b340cf9a0 msgr2=0x7f5b34137700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b3a677640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b340cf9a0 0x7f5b34137700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b3a677640 1 -- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b3413df80 msgr2=0x7f5b340b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b39675640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b3413df80 0x7f5b340b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b3a677640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b3413df80 0x7f5b340b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b3a677640 1 -- 172.21.15.16:0/1084336958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c009a70 con 0x7f5b3413e360 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b39e76640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b340cf9a0 0x7f5b34137700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b39675640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b3413df80 0x7f5b340b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b3a677640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3413e360 0x7f5b340b3230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c037040 tx=0x7f5b2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b2a7fc640 1 -- 172.21.15.16:0/1084336958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c03d600 con 0x7f5b3413e360 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b2a7fc640 1 -- 172.21.15.16:0/1084336958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c03d760 con 0x7f5b3413e360 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b2a7fc640 1 -- 172.21.15.16:0/1084336958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c03db10 con 0x7f5b3413e360 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.151+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b34139eb0 con 0x7f5b3413e360 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.152+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b34137c40 con 0x7f5b3413e360 2023-11-24T06:27:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.153+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b340675d0 con 0x7f5b3413e360 2023-11-24T06:27:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.153+0000 7f5b2a7fc640 1 -- 172.21.15.16:0/1084336958 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5b2c03b040 con 0x7f5b3413e360 2023-11-24T06:27:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.154+0000 7f5b2a7fc640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1806ae40 0x7f5b1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.154+0000 7f5b2a7fc640 1 -- 172.21.15.16:0/1084336958 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b2c0abc40 con 0x7f5b3413e360 2023-11-24T06:27:13.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.155+0000 7f5b39e76640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1806ae40 0x7f5b1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.156+0000 7f5b39e76640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1806ae40 0x7f5b1806d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b300011e0 tx=0x7f5b30009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.159+0000 7f5b2a7fc640 1 -- 172.21.15.16:0/1084336958 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b2c076d70 con 0x7f5b3413e360 2023-11-24T06:27:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.346+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b3415ed20 con 0x7f5b3413e360 2023-11-24T06:27:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.348+0000 7f5b2a7fc640 1 -- 172.21.15.16:0/1084336958 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f5b2c039020 con 0x7f5b3413e360 2023-11-24T06:27:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:13.352 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1806ae40 msgr2=0x7f5b1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1806ae40 0x7f5b1806d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b300011e0 tx=0x7f5b30009040 comp rx=0 tx=0).stop 2023-11-24T06:27:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3413e360 msgr2=0x7f5b340b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3413e360 0x7f5b340b3230 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c037040 tx=0x7f5b2c004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 shutdown_connections 2023-11-24T06:27:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1806ae40 0x7f5b1806d300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3413e360 0x7f5b340b3230 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b3413df80 0x7f5b340b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 --2- 172.21.15.16:0/1084336958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b340cf9a0 0x7f5b34137700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 >> 172.21.15.16:0/1084336958 conn(0x7f5b3406a800 msgr2=0x7f5b340b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 shutdown_connections 2023-11-24T06:27:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.351+0000 7f5b3c0da640 1 -- 172.21.15.16:0/1084336958 wait complete. 2023-11-24T06:27:13.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:27:13.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.580+0000 7fafe2b81640 1 Processor -- start 2023-11-24T06:27:13.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.580+0000 7fafe2b81640 1 -- start start 2023-11-24T06:27:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.582+0000 7fafe2b81640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fafdc071410 0x7fafdc146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.582+0000 7fafe2b81640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc06c7c0 0x7fafdc0717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.582+0000 7fafe2b81640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc06cc20 con 0x7fafdc071d30 2023-11-24T06:27:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.582+0000 7fafe2b81640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc06cec0 con 0x7fafdc071410 2023-11-24T06:27:13.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.582+0000 7fafe2b81640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc06d1a0 con 0x7fafdc06c7c0 2023-11-24T06:27:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.582+0000 7fafe137e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc06c7c0 0x7fafdc0717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.582+0000 7fafe1b7f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fafdc071410 0x7fafdc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.583+0000 7fafe137e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc06c7c0 0x7fafdc0717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:50686/0 (socket says 172.21.15.16:50686) 2023-11-24T06:27:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.583+0000 7fafe137e640 1 -- 172.21.15.16:0/2495889138 learned_addr learned my addr 172.21.15.16:0/2495889138 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.583+0000 7fafe137e640 1 -- 172.21.15.16:0/2495889138 >> v2:172.21.15.173:3300/0 conn(0x7fafdc071410 msgr2=0x7fafdc146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:13.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.583+0000 7fafe137e640 1 --2- 172.21.15.16:0/2495889138 >> v2:172.21.15.173:3300/0 conn(0x7fafdc071410 0x7fafdc146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.583+0000 7fafe137e640 1 -- 172.21.15.16:0/2495889138 >> v1:172.21.15.173:6789/0 conn(0x7fafdc071d30 legacy=0x7fafdc0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.583+0000 7fafe137e640 1 -- 172.21.15.16:0/2495889138 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc06d480 con 0x7fafdc06c7c0 2023-11-24T06:27:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.585+0000 7fafe137e640 1 --2- 172.21.15.16:0/2495889138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc06c7c0 0x7fafdc0717f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009870 tx=0x7fafcc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a253fbede6340ed9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.585+0000 7fafe0b7d640 1 -- 172.21.15.16:0/2495889138 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc030840 con 0x7fafdc06c7c0 2023-11-24T06:27:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.585+0000 7fafe0b7d640 1 -- 172.21.15.16:0/2495889138 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc0309a0 con 0x7fafdc06c7c0 2023-11-24T06:27:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.585+0000 7fafe0b7d640 1 -- 172.21.15.16:0/2495889138 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc030d30 con 0x7fafdc06c7c0 2023-11-24T06:27:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.586+0000 7fafe2b81640 1 -- 172.21.15.16:0/2495889138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc06c7c0 msgr2=0x7fafdc0717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.586+0000 7fafe2b81640 1 --2- 172.21.15.16:0/2495889138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc06c7c0 0x7fafdc0717f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009870 tx=0x7fafcc02b980 comp rx=0 tx=0).stop 2023-11-24T06:27:13.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.586+0000 7fafe2b81640 1 -- 172.21.15.16:0/2495889138 shutdown_connections 2023-11-24T06:27:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.586+0000 7fafe2b81640 1 --2- 172.21.15.16:0/2495889138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc06c7c0 0x7fafdc0717f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.586+0000 7fafe2b81640 1 --2- 172.21.15.16:0/2495889138 >> v2:172.21.15.173:3300/0 conn(0x7fafdc071410 0x7fafdc146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.586+0000 7fafe2b81640 1 -- 172.21.15.16:0/2495889138 >> 172.21.15.16:0/2495889138 conn(0x7fafdc06a800 msgr2=0x7fafdc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.586+0000 7fafe2b81640 1 -- 172.21.15.16:0/2495889138 shutdown_connections 2023-11-24T06:27:13.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.586+0000 7fafe2b81640 1 -- 172.21.15.16:0/2495889138 wait complete. 2023-11-24T06:27:13.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.588+0000 7fafe2b81640 1 Processor -- start 2023-11-24T06:27:13.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.589+0000 7fafe2b81640 1 -- start start 2023-11-24T06:27:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.589+0000 7fafe2b81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc06c7c0 0x7fafdc12d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.589+0000 7fafe2b81640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafdc071410 0x7fafdc12fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.589+0000 7fafe2b81640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc071d30 0x7fafdc1f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.589+0000 7fafe2b81640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc1330e0 con 0x7fafdc06c7c0 2023-11-24T06:27:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.589+0000 7fafe2b81640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc132e00 con 0x7fafdc071410 2023-11-24T06:27:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.589+0000 7fafe2b81640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc140870 con 0x7fafdc071d30 2023-11-24T06:27:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.590+0000 7fafe1b7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc06c7c0 0x7fafdc12d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.590+0000 7fafe1b7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc06c7c0 0x7fafdc12d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56106/0 (socket says 172.21.15.16:56106) 2023-11-24T06:27:13.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.590+0000 7fafe1b7f640 1 -- 172.21.15.16:0/504138259 learned_addr learned my addr 172.21.15.16:0/504138259 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.590+0000 7fafe1b7f640 1 -- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc071d30 msgr2=0x7fafdc1f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.590+0000 7fafe1b7f640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc071d30 0x7fafdc1f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.590+0000 7fafe1b7f640 1 -- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafdc071410 msgr2=0x7fafdc12fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.590+0000 7fafe1b7f640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafdc071410 0x7fafdc12fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.590+0000 7fafe1b7f640 1 -- 172.21.15.16:0/504138259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc0094d0 con 0x7fafdc06c7c0 2023-11-24T06:27:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.590+0000 7fafe1b7f640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc06c7c0 0x7fafdc12d1a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafd800e9b0 tx=0x7fafd800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.591+0000 7fafd27fc640 1 -- 172.21.15.16:0/504138259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8009670 con 0x7fafdc06c7c0 2023-11-24T06:27:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.591+0000 7fafd27fc640 1 -- 172.21.15.16:0/504138259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd800c9a0 con 0x7fafdc06c7c0 2023-11-24T06:27:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.591+0000 7fafd27fc640 1 -- 172.21.15.16:0/504138259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd800cd50 con 0x7fafdc06c7c0 2023-11-24T06:27:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.591+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc140b50 con 0x7fafdc06c7c0 2023-11-24T06:27:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.591+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc130120 con 0x7fafdc06c7c0 2023-11-24T06:27:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.592+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc1406d0 con 0x7fafdc06c7c0 2023-11-24T06:27:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.593+0000 7fafd27fc640 1 -- 172.21.15.16:0/504138259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fafd8013470 con 0x7fafdc06c7c0 2023-11-24T06:27:13.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.594+0000 7fafd27fc640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fafc006ae40 0x7fafc006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:13.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.594+0000 7fafd27fc640 1 -- 172.21.15.16:0/504138259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafd8089900 con 0x7fafdc06c7c0 2023-11-24T06:27:13.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.595+0000 7fafe137e640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fafc006ae40 0x7fafc006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:13.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.596+0000 7fafe137e640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fafc006ae40 0x7fafc006d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fafcc036000 tx=0x7fafcc03b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:13.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.598+0000 7fafd27fc640 1 -- 172.21.15.16:0/504138259 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fafdc1406d0 con 0x7fafdc06c7c0 2023-11-24T06:27:13.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.829+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafdc1406d0 con 0x7fafdc06c7c0 2023-11-24T06:27:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.830+0000 7fafd27fc640 1 -- 172.21.15.16:0/504138259 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fafdc1406d0 con 0x7fafdc06c7c0 2023-11-24T06:27:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:13.834 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3019663874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3019663874},{"type":"v1","addr":"172.21.15.173:6837","nonce":3019663874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2239024365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2239024365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2239024365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:26:56.460479+0000","modified":"2023-11-24T06:26:59.516602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/915142029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":915142029},{"type":"v1","addr":"172.21.15.173:6835","nonce":915142029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-11-24T06:27:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.832+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fafc006ae40 msgr2=0x7fafc006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.832+0000 7fafe2b81640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fafc006ae40 0x7fafc006d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fafcc036000 tx=0x7fafcc03b040 comp rx=0 tx=0).stop 2023-11-24T06:27:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.832+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc06c7c0 msgr2=0x7fafdc12d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.832+0000 7fafe2b81640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc06c7c0 0x7fafdc12d1a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafd800e9b0 tx=0x7fafd800ee70 comp rx=0 tx=0).stop 2023-11-24T06:27:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.833+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 shutdown_connections 2023-11-24T06:27:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.833+0000 7fafe2b81640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fafc006ae40 0x7fafc006d300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.833+0000 7fafe2b81640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fafdc071d30 0x7fafdc1f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.833+0000 7fafe2b81640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fafdc071410 0x7fafdc12fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.833+0000 7fafe2b81640 1 --2- 172.21.15.16:0/504138259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafdc06c7c0 0x7fafdc12d1a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.833+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 >> 172.21.15.16:0/504138259 conn(0x7fafdc06a800 msgr2=0x7fafdc13ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.833+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 shutdown_connections 2023-11-24T06:27:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:13.833+0000 7fafe2b81640 1 -- 172.21.15.16:0/504138259 wait complete. 2023-11-24T06:27:14.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:26:56.460479+0000', 'modified': '2023-11-24T06:26:59.516602+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6835/915142029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 915142029}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 915142029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:27:14.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:27:14.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-24T06:27:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.493+0000 7febbffff640 1 Processor -- start 2023-11-24T06:27:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.493+0000 7febbffff640 1 -- start start 2023-11-24T06:27:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.495+0000 7febbffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc015afb0 0x7febc013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.495+0000 7febbffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7febc013fae0 0x7febc00b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.495+0000 7febbffff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc013ff00 con 0x7febc013cfa0 2023-11-24T06:27:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.495+0000 7febbffff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc00cdc40 con 0x7febc013fae0 2023-11-24T06:27:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.495+0000 7febbffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc00cdec0 con 0x7febc015afb0 2023-11-24T06:27:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.496+0000 7febb67fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc015afb0 0x7febc013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.496+0000 7febb67fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc015afb0 0x7febc013f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:50694/0 (socket says 172.21.15.16:50694) 2023-11-24T06:27:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.496+0000 7febb67fc640 1 -- 172.21.15.16:0/3089967273 learned_addr learned my addr 172.21.15.16:0/3089967273 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.496+0000 7febbf7fe640 1 --2- 172.21.15.16:0/3089967273 >> v2:172.21.15.173:3300/0 conn(0x7febc013fae0 0x7febc00b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.497+0000 7febb67fc640 1 -- 172.21.15.16:0/3089967273 >> v2:172.21.15.173:3300/0 conn(0x7febc013fae0 msgr2=0x7febc00b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.497+0000 7febb67fc640 1 --2- 172.21.15.16:0/3089967273 >> v2:172.21.15.173:3300/0 conn(0x7febc013fae0 0x7febc00b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.497+0000 7febb67fc640 1 -- 172.21.15.16:0/3089967273 >> v1:172.21.15.173:6789/0 conn(0x7febc013cfa0 legacy=0x7febc00cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.497+0000 7febb67fc640 1 -- 172.21.15.16:0/3089967273 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc00ce1a0 con 0x7febc015afb0 2023-11-24T06:27:14.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.497+0000 7febbf7fe640 1 --2- 172.21.15.16:0/3089967273 >> v2:172.21.15.173:3300/0 conn(0x7febc013fae0 0x7febc00b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.499+0000 7febb67fc640 1 --2- 172.21.15.16:0/3089967273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc015afb0 0x7febc013f5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feba8009e50 tx=0x7feba802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb16adcb4fb7e29 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.499+0000 7febbe7fc640 1 -- 172.21.15.16:0/3089967273 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba802ca80 con 0x7febc015afb0 2023-11-24T06:27:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.499+0000 7febbe7fc640 1 -- 172.21.15.16:0/3089967273 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba802cbe0 con 0x7febc015afb0 2023-11-24T06:27:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.499+0000 7febbe7fc640 1 -- 172.21.15.16:0/3089967273 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba80343f0 con 0x7febc015afb0 2023-11-24T06:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.500+0000 7febbffff640 1 -- 172.21.15.16:0/3089967273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc015afb0 msgr2=0x7febc013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.500+0000 7febbffff640 1 --2- 172.21.15.16:0/3089967273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc015afb0 0x7febc013f5a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feba8009e50 tx=0x7feba802bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.500+0000 7febbffff640 1 -- 172.21.15.16:0/3089967273 shutdown_connections 2023-11-24T06:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.500+0000 7febbffff640 1 --2- 172.21.15.16:0/3089967273 >> v2:172.21.15.173:3300/0 conn(0x7febc013fae0 0x7febc00b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.500+0000 7febbffff640 1 --2- 172.21.15.16:0/3089967273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc015afb0 0x7febc013f5a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.500+0000 7febbffff640 1 -- 172.21.15.16:0/3089967273 >> 172.21.15.16:0/3089967273 conn(0x7febc006a800 msgr2=0x7febc006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.500+0000 7febbffff640 1 -- 172.21.15.16:0/3089967273 shutdown_connections 2023-11-24T06:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.500+0000 7febbffff640 1 -- 172.21.15.16:0/3089967273 wait complete. 2023-11-24T06:27:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.502+0000 7febbffff640 1 Processor -- start 2023-11-24T06:27:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.502+0000 7febbffff640 1 -- start start 2023-11-24T06:27:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.503+0000 7febbffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc013cfa0 0x7febc013e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.503+0000 7febbffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc013fae0 0x7febc01f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.503+0000 7febbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc015afb0 0x7febc013b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.503+0000 7febbffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc013f540 con 0x7febc015afb0 2023-11-24T06:27:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.503+0000 7febbffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc013ef80 con 0x7febc013cfa0 2023-11-24T06:27:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.503+0000 7febbffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc013f260 con 0x7febc013fae0 2023-11-24T06:27:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.503+0000 7febbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc015afb0 0x7febc013b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.503+0000 7febbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc015afb0 0x7febc013b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56112/0 (socket says 172.21.15.16:56112) 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.503+0000 7febbf7fe640 1 -- 172.21.15.16:0/1236123051 learned_addr learned my addr 172.21.15.16:0/1236123051 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febb67fc640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc013fae0 0x7febc01f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febbeffd640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc013cfa0 0x7febc013e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febbf7fe640 1 -- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc013fae0 msgr2=0x7febc01f6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febbf7fe640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc013fae0 0x7febc01f6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febbf7fe640 1 -- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc013cfa0 msgr2=0x7febc013e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febbf7fe640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc013cfa0 0x7febc013e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febbf7fe640 1 -- 172.21.15.16:0/1236123051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb800ba70 con 0x7febc015afb0 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febb67fc640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc013fae0 0x7febc01f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febbeffd640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc013cfa0 0x7febc013e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febbf7fe640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc015afb0 0x7febc013b3e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7febb80089e0 tx=0x7febb8008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febb7fff640 1 -- 172.21.15.16:0/1236123051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8010040 con 0x7febc015afb0 2023-11-24T06:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febb7fff640 1 -- 172.21.15.16:0/1236123051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb8009ca0 con 0x7febc015afb0 2023-11-24T06:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.504+0000 7febb7fff640 1 -- 172.21.15.16:0/1236123051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8004250 con 0x7febc015afb0 2023-11-24T06:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.505+0000 7febbffff640 1 -- 172.21.15.16:0/1236123051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba8009b30 con 0x7febc015afb0 2023-11-24T06:27:14.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.505+0000 7febbffff640 1 -- 172.21.15.16:0/1236123051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc01f6940 con 0x7febc015afb0 2023-11-24T06:27:14.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.507+0000 7febb7fff640 1 -- 172.21.15.16:0/1236123051 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febb80043b0 con 0x7febc015afb0 2023-11-24T06:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.508+0000 7febb7fff640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feba006ae40 0x7feba006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.508+0000 7febb7fff640 1 -- 172.21.15.16:0/1236123051 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febb8089670 con 0x7febc015afb0 2023-11-24T06:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.508+0000 7febc8164740 1 Processor -- start 2023-11-24T06:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.508+0000 7febbeffd640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feba006ae40 0x7feba006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.509+0000 7febc8164740 1 -- start start 2023-11-24T06:27:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.509+0000 7febbeffd640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feba006ae40 0x7feba006d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7febc00cc6a0 tx=0x7febb0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.509+0000 7febc8164740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b544e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.509+0000 7febc8164740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b5448d70 0x55e6b544d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.509+0000 7febc8164740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5491730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.509+0000 7febc8164740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b53ac880 con 0x55e6b544d7b0 2023-11-24T06:27:14.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.509+0000 7febc8164740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b53bbae0 con 0x55e6b544e070 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febc8164740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6b53b91d0 con 0x55e6b5448d70 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5491730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5491730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56118/0 (socket says 172.21.15.16:56118) 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbf7fe640 1 -- 172.21.15.16:0/702029944 learned_addr learned my addr 172.21.15.16:0/702029944 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbeffd640 1 --2- 172.21.15.16:0/702029944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b544e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febb67fc640 1 --2- 172.21.15.16:0/702029944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b5448d70 0x55e6b544d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbf7fe640 1 -- 172.21.15.16:0/702029944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b5448d70 msgr2=0x55e6b544d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbf7fe640 1 --2- 172.21.15.16:0/702029944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b5448d70 0x55e6b544d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbf7fe640 1 -- 172.21.15.16:0/702029944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 msgr2=0x55e6b544e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbf7fe640 1 --2- 172.21.15.16:0/702029944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b544e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbf7fe640 1 -- 172.21.15.16:0/702029944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6b537c860 con 0x55e6b544d7b0 2023-11-24T06:27:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.510+0000 7febbf7fe640 1 --2- 172.21.15.16:0/702029944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5491730 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febb8087a40 tx=0x7febb8058400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ed81e456265ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febb4ff9640 1 -- 172.21.15.16:0/702029944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8009e10 con 0x55e6b544d7b0 2023-11-24T06:27:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febb4ff9640 1 -- 172.21.15.16:0/702029944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb8056480 con 0x55e6b544d7b0 2023-11-24T06:27:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febb4ff9640 1 -- 172.21.15.16:0/702029944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb805c120 con 0x55e6b544d7b0 2023-11-24T06:27:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febc8164740 1 -- 172.21.15.16:0/702029944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 msgr2=0x55e6b5491730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febc8164740 1 --2- 172.21.15.16:0/702029944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5491730 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febb8087a40 tx=0x7febb8058400 comp rx=0 tx=0).stop 2023-11-24T06:27:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febc8164740 1 -- 172.21.15.16:0/702029944 shutdown_connections 2023-11-24T06:27:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febc8164740 1 --2- 172.21.15.16:0/702029944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5491730 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febc8164740 1 --2- 172.21.15.16:0/702029944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b5448d70 0x55e6b544d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febc8164740 1 --2- 172.21.15.16:0/702029944 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b544e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.511+0000 7febc8164740 1 -- 172.21.15.16:0/702029944 >> 172.21.15.16:0/702029944 conn(0x55e6b540fc70 msgr2=0x55e6b5493b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.512+0000 7febc8164740 1 -- 172.21.15.16:0/702029944 shutdown_connections 2023-11-24T06:27:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.512+0000 7febc8164740 1 -- 172.21.15.16:0/702029944 wait complete. 2023-11-24T06:27:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.512+0000 7febc8164740 1 -- start start 2023-11-24T06:27:14.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.513+0000 7febc8164740 1 Processor -- start 2023-11-24T06:27:14.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febc8164740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b5448d70 0x55e6b54741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febc8164740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5477870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febc8164740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544e070 0x55e6b547c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febc8164740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b53b4ac0 con 0x55e6b544d7b0 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febc8164740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b53512f0 con 0x55e6b5448d70 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febc8164740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6b536ec70 con 0x55e6b544e070 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febc8164740 10 client.0 _ll_register_callbacks cb 0x55e6b53733b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febb67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5477870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febb67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5477870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56134/0 (socket says 172.21.15.16:56134) 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febb67fc640 1 -- 172.21.15.16:0/962516399 learned_addr learned my addr 172.21.15.16:0/962516399 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febbf7fe640 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544e070 0x55e6b547c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febbeffd640 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b5448d70 0x55e6b54741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febb67fc640 1 -- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544e070 msgr2=0x55e6b547c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febb67fc640 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544e070 0x55e6b547c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febb67fc640 1 -- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b5448d70 msgr2=0x55e6b54741b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.514+0000 7febb67fc640 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b5448d70 0x55e6b54741b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.515+0000 7febb67fc640 1 -- 172.21.15.16:0/962516399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc01f6940 con 0x55e6b544d7b0 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.515+0000 7febbeffd640 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b5448d70 0x55e6b54741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.515+0000 7febb67fc640 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5477870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feba8009e50 tx=0x7feba8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbf70d166128242 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.515+0000 7feb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.515+0000 7feb9dffb640 1 -- 172.21.15.16:0/962516399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8034b80 con 0x55e6b544d7b0 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.515+0000 7febc8164740 1 -- 172.21.15.16:0/962516399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6b53adaa0 con 0x55e6b544d7b0 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.515+0000 7feb9dffb640 1 -- 172.21.15.16:0/962516399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feba8044070 con 0x55e6b544d7b0 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7feb9dffb640 1 -- 172.21.15.16:0/962516399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7feba8034ce0 con 0x55e6b544d7b0 2023-11-24T06:27:14.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7febc8164740 10 client.4632 fetch_fsmap learned FSMap version 8 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7febc8164740 1 -- 172.21.15.16:0/962516399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e6b53b3140 con 0x55e6b544d7b0 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7feb9dffb640 1 -- 172.21.15.16:0/962516399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e6b53adaa0 con 0x55e6b544d7b0 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7feb9dffb640 1 -- 172.21.15.16:0/962516399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feba8048430 con 0x55e6b544d7b0 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7febc8164740 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7febc8164740 10 client.4632 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7febc8164740 20 client.4632 populate_metadata read hostname 'smithi016' 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7febc8164740 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] conn(0x55e6b547c850 0x55e6b5499740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7febc8164740 4 client.4632 mds_command: new command op to 4239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.516+0000 7febc8164740 1 -- 172.21.15.16:0/962516399 --> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6b53ad7c0 con 0x55e6b547c850 2023-11-24T06:27:14.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.517+0000 7febbeffd640 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] conn(0x55e6b547c850 0x55e6b5499740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.518+0000 7febbeffd640 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] conn(0x55e6b547c850 0x55e6b5499740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c35b29701ea396b server_cookie=57866d5396ceb503 in_seq=0 out_seq=0 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.518+0000 7feb9dffb640 10 client.4632 ms_handle_connect on v2:172.21.15.173:6834/915142029 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7feb9dffb640 1 -- 172.21.15.16:0/962516399 <== mds.0 v2:172.21.15.173:6834/915142029 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e6b53ad7c0 con 0x55e6b547c850 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7feb9dffb640 10 client.4632 handle_command_reply: tid=0 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 client.4632 shutdown 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 -- 172.21.15.16:0/962516399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 msgr2=0x55e6b5477870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5477870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feba8009e50 tx=0x7feba8004ab0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 -- 172.21.15.16:0/962516399 shutdown_connections 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] conn(0x55e6b547c850 0x55e6b5499740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544e070 0x55e6b547c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b544d7b0 0x55e6b5477870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7feb9dffb640 0 client.4632 ms_handle_reset on v2:172.21.15.173:6834/915142029 2023-11-24T06:27:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 --2- 172.21.15.16:0/962516399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b5448d70 0x55e6b54741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 -- 172.21.15.16:0/962516399 >> 172.21.15.16:0/962516399 conn(0x55e6b540fc70 msgr2=0x55e6b5447dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 -- 172.21.15.16:0/962516399 shutdown_connections 2023-11-24T06:27:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.520+0000 7febc8164740 1 -- 172.21.15.16:0/962516399 wait complete. 2023-11-24T06:27:14.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.521+0000 7febc8164740 20 client.4632 trim_cache size 0 max 16384 2023-11-24T06:27:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.544+0000 7febc8164740 1 Processor -- start 2023-11-24T06:27:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.544+0000 7febc8164740 1 -- start start 2023-11-24T06:27:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.545+0000 7febc8164740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b549ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.545+0000 7febc8164740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 0x55e6b549b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.545+0000 7febc8164740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b54a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.545+0000 7febc8164740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b5117f20 con 0x55e6b5448d70 2023-11-24T06:27:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.545+0000 7febc8164740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b547c360 con 0x55e6b544e070 2023-11-24T06:27:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.545+0000 7febc8164740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6b5444600 con 0x55e6b544d7b0 2023-11-24T06:27:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.545+0000 7febbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b549ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.545+0000 7febbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b549ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56136/0 (socket says 172.21.15.16:56136) 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.545+0000 7febbeffd640 1 -- 172.21.15.16:0/3570268395 learned_addr learned my addr 172.21.15.16:0/3570268395 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febbf7fe640 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b54a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febbeffd640 1 -- 172.21.15.16:0/3570268395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 msgr2=0x55e6b549b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febb67fc640 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 0x55e6b549b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febbeffd640 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 0x55e6b549b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febbeffd640 1 -- 172.21.15.16:0/3570268395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 msgr2=0x55e6b54a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febbeffd640 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b54a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febbeffd640 1 -- 172.21.15.16:0/3570268395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6b5350b80 con 0x55e6b5448d70 2023-11-24T06:27:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febbf7fe640 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b54a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febbeffd640 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b549ade0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7febb0007fd0 tx=0x7febb0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404fa55e58a62fbc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febb57fa640 1 -- 172.21.15.16:0/3570268395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb000dc70 con 0x55e6b5448d70 2023-11-24T06:27:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febb57fa640 1 -- 172.21.15.16:0/3570268395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb001d030 con 0x55e6b5448d70 2023-11-24T06:27:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.546+0000 7febb57fa640 1 -- 172.21.15.16:0/3570268395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0003610 con 0x55e6b5448d70 2023-11-24T06:27:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.547+0000 7febc8164740 1 -- 172.21.15.16:0/3570268395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 msgr2=0x55e6b549ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.547+0000 7febc8164740 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b549ade0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7febb0007fd0 tx=0x7febb0013000 comp rx=0 tx=0).stop 2023-11-24T06:27:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.547+0000 7febc8164740 1 -- 172.21.15.16:0/3570268395 shutdown_connections 2023-11-24T06:27:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.547+0000 7febc8164740 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b54a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.547+0000 7febc8164740 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 0x55e6b549b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.547+0000 7febc8164740 1 --2- 172.21.15.16:0/3570268395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b549ade0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.547+0000 7febc8164740 1 -- 172.21.15.16:0/3570268395 >> 172.21.15.16:0/3570268395 conn(0x55e6b540fc70 msgr2=0x55e6b544e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.547+0000 7febc8164740 1 -- 172.21.15.16:0/3570268395 shutdown_connections 2023-11-24T06:27:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.547+0000 7febc8164740 1 -- 172.21.15.16:0/3570268395 wait complete. 2023-11-24T06:27:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.548+0000 7febc8164740 1 -- start start 2023-11-24T06:27:14.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.548+0000 7febc8164740 1 Processor -- start 2023-11-24T06:27:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febc8164740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b5505a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febc8164740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 0x55e6b54a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febc8164740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b54a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febc8164740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b54a68c0 con 0x55e6b5448d70 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febc8164740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b5506210 con 0x55e6b544e070 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febc8164740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e6b5479070 con 0x55e6b544d7b0 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febc8164740 10 client.0 _ll_register_callbacks cb 0x7febb800b5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b5505a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b5505a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56138/0 (socket says 172.21.15.16:56138) 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.549+0000 7febbeffd640 1 -- 172.21.15.16:0/3468316270 learned_addr learned my addr 172.21.15.16:0/3468316270 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7febbf7fe640 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b54a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7febb67fc640 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 0x55e6b54a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7febbeffd640 1 -- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 msgr2=0x55e6b54a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7febbeffd640 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 0x55e6b54a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7febbeffd640 1 -- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 msgr2=0x55e6b54a5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7febbeffd640 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b54a5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7febbeffd640 1 -- 172.21.15.16:0/3468316270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc013ff00 con 0x55e6b5448d70 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7febb67fc640 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 0x55e6b54a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7febbeffd640 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b5505a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55e6b53adbc0 tx=0x7febb0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf61d0cf60235bfb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.550+0000 7feb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7feb9dffb640 1 -- 172.21.15.16:0/3468316270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0034070 con 0x55e6b5448d70 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7feb9dffb640 1 -- 172.21.15.16:0/3468316270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febb000de80 con 0x55e6b5448d70 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7febc8164740 1 -- 172.21.15.16:0/3468316270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febb8056480 con 0x55e6b5448d70 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7feb9dffb640 1 -- 172.21.15.16:0/3468316270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7febb0003610 con 0x55e6b5448d70 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7febc8164740 10 client.4638 fetch_fsmap learned FSMap version 8 2023-11-24T06:27:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7febc8164740 1 -- 172.21.15.16:0/3468316270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e6b5376170 con 0x55e6b5448d70 2023-11-24T06:27:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7feb9dffb640 1 -- 172.21.15.16:0/3468316270 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb8056480 con 0x55e6b5448d70 2023-11-24T06:27:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7feb9dffb640 1 -- 172.21.15.16:0/3468316270 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7febb002c030 con 0x55e6b5448d70 2023-11-24T06:27:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7febc8164740 10 client.4638 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:27:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7febc8164740 10 client.4638 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T06:27:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7febc8164740 20 client.4638 populate_metadata read hostname 'smithi016' 2023-11-24T06:27:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7febc8164740 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] conn(0x55e6b5509910 0x55e6b5509cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7febc8164740 4 client.4638 mds_command: new command op to 4239 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:27:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.551+0000 7febc8164740 1 -- 172.21.15.16:0/3468316270 --> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e6b50b56c0 con 0x55e6b5509910 2023-11-24T06:27:14.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.552+0000 7febb67fc640 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] conn(0x55e6b5509910 0x55e6b5509cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.553+0000 7febb67fc640 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] conn(0x55e6b5509910 0x55e6b5509cf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acff59c02cbff024 server_cookie=bca909d608e82891 in_seq=0 out_seq=0 2023-11-24T06:27:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.553+0000 7feb9dffb640 10 client.4638 ms_handle_connect on v2:172.21.15.173:6834/915142029 2023-11-24T06:27:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.553+0000 7feb9dffb640 1 -- 172.21.15.16:0/3468316270 <== mds.0 v2:172.21.15.173:6834/915142029 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55e6b50b56c0 con 0x55e6b5509910 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7feb9dffb640 10 client.4638 handle_command_reply: tid=0 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 client.4638 shutdown 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 -- 172.21.15.16:0/3468316270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 msgr2=0x55e6b5505a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b5505a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55e6b53adbc0 tx=0x7febb0000ed0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 -- 172.21.15.16:0/3468316270 shutdown_connections 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:6834/915142029,v1:172.21.15.173:6835/915142029] conn(0x55e6b5509910 0x55e6b5509cf0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e6b544e070 0x55e6b54a5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e6b544d7b0 0x55e6b54a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 --2- 172.21.15.16:0/3468316270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e6b5448d70 0x55e6b5505a00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7feb9dffb640 0 client.4638 ms_handle_reset on v2:172.21.15.173:6834/915142029 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 -- 172.21.15.16:0/3468316270 >> 172.21.15.16:0/3468316270 conn(0x55e6b540fc70 msgr2=0x55e6b549ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 -- 172.21.15.16:0/3468316270 shutdown_connections 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 1 -- 172.21.15.16:0/3468316270 wait complete. 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.554+0000 7febc8164740 20 client.4638 trim_cache size 0 max 16384 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6dc1668b-58b0-47dd-b8dc-4164e5a228ab", 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4239, 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-24T06:27:14.561 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 17.575063621000002, 2023-11-24T06:27:14.562 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 8, 2023-11-24T06:27:14.562 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 20, 2023-11-24T06:27:14.562 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 20, 2023-11-24T06:27:14.562 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 25.189648200000001 2023-11-24T06:27:14.562 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 -- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feba006ae40 msgr2=0x7feba006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feba006ae40 0x7feba006d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7febc00cc6a0 tx=0x7febb0009040 comp rx=0 tx=0).stop 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 -- 172.21.15.16:0/1236123051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc015afb0 msgr2=0x7febc013b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc015afb0 0x7febc013b3e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7febb80089e0 tx=0x7febb8008ad0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 -- 172.21.15.16:0/1236123051 shutdown_connections 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feba006ae40 0x7feba006d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc015afb0 0x7febc013b3e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc013fae0 0x7febc01f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 --2- 172.21.15.16:0/1236123051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc013cfa0 0x7febc013e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 -- 172.21.15.16:0/1236123051 >> 172.21.15.16:0/1236123051 conn(0x7febc006a800 msgr2=0x7febc013c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 -- 172.21.15.16:0/1236123051 shutdown_connections 2023-11-24T06:27:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:14.555+0000 7febbffff640 1 -- 172.21.15.16:0/1236123051 wait complete. 2023-11-24T06:27:14.579 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-24T06:27:14.602 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-24T06:27:14.602 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T06:27:14.602 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-24T06:27:14.602 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:27:14.602 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:14.602 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-24T06:27:14.602 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:27:14.603 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:27:14.603 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-24T06:27:14.603 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:27:14.603 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:14.603 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-24T06:27:14.603 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:27:14.603 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:27:14.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:14.603 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2023-11-24T06:27:14.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:14.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link delete ceph-brx 2023-11-24T06:27:14.766 INFO:teuthology.orchestra.run.smithi173.stderr:Cannot find device "ceph-brx" 2023-11-24T06:27:14.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:27:14.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:14.771 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-24T06:27:14.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:14.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2023-11-24T06:27:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2023-11-24T06:27:14.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:27:14.949 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-24T06:27:14.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.171+0000 7f78f1106640 1 Processor -- start 2023-11-24T06:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.172+0000 7f78f1106640 1 -- start start 2023-11-24T06:27:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.173+0000 7f78f1106640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec160920 0x7f78ec0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.173+0000 7f78f1106640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f78ec0cf9f0 0x7f78ec0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.173+0000 7f78f1106640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78ec161010 0x7f78ec1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.173+0000 7f78f1106640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ec0d03b0 con 0x7f78ec161010 2023-11-24T06:27:15.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.173+0000 7f78f1106640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ec0d0a10 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.173+0000 7f78f1106640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0cf080 con 0x7f78ec160920 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78f0905640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78ec161010 0x7f78ec1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78f0905640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78ec161010 0x7f78ec1614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56154/0 (socket says 172.21.15.16:56154) 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78f0905640 1 -- 172.21.15.16:0/3064543088 learned_addr learned my addr 172.21.15.16:0/3064543088 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78eb7fe640 1 --2- 172.21.15.16:0/3064543088 >> v2:172.21.15.173:3300/0 conn(0x7f78ec0cf9f0 0x7f78ec0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78ebfff640 1 --2- 172.21.15.16:0/3064543088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec160920 0x7f78ec0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78f0905640 1 -- 172.21.15.16:0/3064543088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec160920 msgr2=0x7f78ec0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78f0905640 1 --2- 172.21.15.16:0/3064543088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec160920 0x7f78ec0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78f0905640 1 -- 172.21.15.16:0/3064543088 >> v2:172.21.15.173:3300/0 conn(0x7f78ec0cf9f0 msgr2=0x7f78ec0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78f0905640 1 --2- 172.21.15.16:0/3064543088 >> v2:172.21.15.173:3300/0 conn(0x7f78ec0cf9f0 0x7f78ec0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78f0905640 1 -- 172.21.15.16:0/3064543088 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec0c6be0 con 0x7f78ec161010 2023-11-24T06:27:15.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.174+0000 7f78ebfff640 1 --2- 172.21.15.16:0/3064543088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec160920 0x7f78ec0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:15.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.175+0000 7f78f0905640 1 --2- 172.21.15.16:0/3064543088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec161010 0x7f78ec1614a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f78ec08d300 tx=0x7f78e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c8d3225c1ac724 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:15.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.176+0000 7f78eaffd640 1 -- 172.21.15.16:0/3064543088 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4031870 con 0x7f78ec161010 2023-11-24T06:27:15.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.176+0000 7f78eaffd640 1 -- 172.21.15.16:0/3064543088 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e40319d0 con 0x7f78ec161010 2023-11-24T06:27:15.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.176+0000 7f78eaffd640 1 -- 172.21.15.16:0/3064543088 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4031d60 con 0x7f78ec161010 2023-11-24T06:27:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.177+0000 7f78f1106640 1 -- 172.21.15.16:0/3064543088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec161010 msgr2=0x7f78ec1614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.177+0000 7f78f1106640 1 --2- 172.21.15.16:0/3064543088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec161010 0x7f78ec1614a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f78ec08d300 tx=0x7f78e402b910 comp rx=0 tx=0).stop 2023-11-24T06:27:15.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.177+0000 7f78f1106640 1 -- 172.21.15.16:0/3064543088 shutdown_connections 2023-11-24T06:27:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.177+0000 7f78f1106640 1 --2- 172.21.15.16:0/3064543088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec161010 0x7f78ec1614a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.177+0000 7f78f1106640 1 --2- 172.21.15.16:0/3064543088 >> v2:172.21.15.173:3300/0 conn(0x7f78ec0cf9f0 0x7f78ec0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.177+0000 7f78f1106640 1 --2- 172.21.15.16:0/3064543088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec160920 0x7f78ec0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.177+0000 7f78f1106640 1 -- 172.21.15.16:0/3064543088 >> 172.21.15.16:0/3064543088 conn(0x7f78ec0d3090 msgr2=0x7f78ec0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.177+0000 7f78f1106640 1 -- 172.21.15.16:0/3064543088 shutdown_connections 2023-11-24T06:27:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.177+0000 7f78f1106640 1 -- 172.21.15.16:0/3064543088 wait complete. 2023-11-24T06:27:15.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.179+0000 7f78f1106640 1 Processor -- start 2023-11-24T06:27:15.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.179+0000 7f78f1106640 1 -- start start 2023-11-24T06:27:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.180+0000 7f78f1106640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec0cf9f0 0x7f78ec15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78f1106640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78ec160920 0x7f78ec0b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78f1106640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec161010 0x7f78ec0b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78f1106640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec15b270 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78f1106640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0cc620 con 0x7f78ec160920 2023-11-24T06:27:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78f1106640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0cbcf0 con 0x7f78ec161010 2023-11-24T06:27:15.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec0cf9f0 0x7f78ec15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec0cf9f0 0x7f78ec15fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56166/0 (socket says 172.21.15.16:56166) 2023-11-24T06:27:15.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78ebfff640 1 -- 172.21.15.16:0/1884613230 learned_addr learned my addr 172.21.15.16:0/1884613230 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78f0905640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec161010 0x7f78ec0b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78eb7fe640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78ec160920 0x7f78ec0b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78ebfff640 1 -- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec161010 msgr2=0x7f78ec0b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78ebfff640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec161010 0x7f78ec0b7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78ebfff640 1 -- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78ec160920 msgr2=0x7f78ec0b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78ebfff640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78ec160920 0x7f78ec0b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78ebfff640 1 -- 172.21.15.16:0/1884613230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e40094d0 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.181+0000 7f78eb7fe640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78ec160920 0x7f78ec0b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.182+0000 7f78ebfff640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec0cf9f0 0x7f78ec15fb10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00cf60 tx=0x7f78dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.182+0000 7f78e8ff9640 1 -- 172.21.15.16:0/1884613230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00d840 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.182+0000 7f78e8ff9640 1 -- 172.21.15.16:0/1884613230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78dc00d9a0 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.182+0000 7f78e8ff9640 1 -- 172.21.15.16:0/1884613230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00dd30 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.182+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ec0cbfd0 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.182+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ec0ccab0 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.183+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ec0cea80 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.184+0000 7f78e8ff9640 1 -- 172.21.15.16:0/1884613230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78dc015440 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.185+0000 7f78e8ff9640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f78cc06ae40 0x7f78cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:15.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.186+0000 7f78e8ff9640 1 -- 172.21.15.16:0/1884613230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78dc089fd0 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.186+0000 7f78eb7fe640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f78cc06ae40 0x7f78cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:15.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.187+0000 7f78eb7fe640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f78cc06ae40 0x7f78cc06d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f78ec071ae0 tx=0x7f78d4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:15.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.189+0000 7f78e8ff9640 1 -- 172.21.15.16:0/1884613230 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78dc055130 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.426+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f78ec067450 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.427+0000 7f78e8ff9640 1 -- 172.21.15.16:0/1884613230 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f78dc01a060 con 0x7f78ec0cf9f0 2023-11-24T06:27:15.430 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:27:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.429+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f78cc06ae40 msgr2=0x7f78cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.429+0000 7f78f1106640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f78cc06ae40 0x7f78cc06d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f78ec071ae0 tx=0x7f78d4008040 comp rx=0 tx=0).stop 2023-11-24T06:27:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.429+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec0cf9f0 msgr2=0x7f78ec15fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.429+0000 7f78f1106640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec0cf9f0 0x7f78ec15fb10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00cf60 tx=0x7f78dc00cf90 comp rx=0 tx=0).stop 2023-11-24T06:27:15.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.430+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 shutdown_connections 2023-11-24T06:27:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.430+0000 7f78f1106640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f78cc06ae40 0x7f78cc06d300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.430+0000 7f78f1106640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78ec161010 0x7f78ec0b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.430+0000 7f78f1106640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78ec160920 0x7f78ec0b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.430+0000 7f78f1106640 1 --2- 172.21.15.16:0/1884613230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ec0cf9f0 0x7f78ec15fb10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.430+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 >> 172.21.15.16:0/1884613230 conn(0x7f78ec0d3090 msgr2=0x7f78ec06c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.430+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 shutdown_connections 2023-11-24T06:27:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:15.430+0000 7f78f1106640 1 -- 172.21.15.16:0/1884613230 wait complete. 2023-11-24T06:27:15.453 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:27:15.453 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:27:15.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-24T06:27:15.454 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:27:15.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:15.454 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:27:15.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:27:15.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:27:15.454 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:27:15.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:15.455 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-24T06:27:15.512 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:15.513 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42308sec preferred_lft 42308sec 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:15.514 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-24T06:27:15.516 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:27:15.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:15.516 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:27:15.516 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:27:15.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:27:15.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2023-11-24T06:27:15.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2023-11-24T06:27:15.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:27:15.516 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:27:15.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:15.999 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:27:16.086 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-24T06:27:16.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:16.090 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-24T06:27:16.143 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-24T06:27:16.143 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:27:16.143 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:27:16.143 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:27:16.143 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:27:16.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:16.144 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:27:16.144 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:27:16.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:27:16.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:27:16.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:27:16.144 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:27:16.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:16.874 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-24T06:27:16.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:16.933 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-11-24T06:27:16.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:16.986 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:27:16.987 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:27:16.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:16.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T06:27:16.987 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:27:17.135 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T06:27:17.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:17.135 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:27:17.135 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:27:17.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T06:27:17.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:27:17.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T06:27:17.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T06:27:17.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T06:27:17.136 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:27:17.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:17.453 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:27:17.453 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:27:17.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-24T06:27:17.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:27:17.453 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:27:17.631 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:27:17.631 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:17.631 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:17.688 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:27:17.689 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:17.689 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:17.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:27:17.835 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:17.886 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:27:17.886 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:27:17.886 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:27:17.886 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:27:17.886 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:27:17.886 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:27:17.886 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:27:17.886 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:27:17.887 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:17.888 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:17.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:17.889 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:27:17.940 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:27:17.940 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:27:17.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:27:18.025 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:18.069 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:27:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:27:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:27:18.072 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:27:18.072 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:27:18.072 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:27:18.072 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:27:18.072 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:27:18.072 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:18.072 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:18.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:18.073 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:27:18.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-24T06:27:18.160+0000 7fa5b5ffb480 -1 init, newargv = 0x55917471d8c0 newargc=15 2023-11-24T06:27:18.164 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[95601]: starting ceph client 2023-11-24T06:27:18.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[95601]: starting fuse 2023-11-24T06:27:19.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:27:19.377 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:19.466 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:27:19.467 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:19.468 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:27:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:27:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:27:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:27:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:27:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:27:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:27:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:19.469 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:19.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:19.470 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:27:19.528 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-24T06:27:19.528 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:27:19.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:27:19.528 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-24T06:27:19.528 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-24T06:27:19.529 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-24T06:27:19.530 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-24T06:27:19.723 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.95601.asok 2023-11-24T06:27:19.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:27:19.741 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95601.asok 2023-11-24T06:27:19.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:19.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95601.asok status 2023-11-24T06:27:19.999 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "95601", 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-24T06:27:20.000 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4653, 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4653 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1370831219 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:27:20.001 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1370831219 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4653 192.168.0.1:0/1370831219", 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1370831219", 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 8, 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 20, 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-24T06:27:20.002 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-24T06:27:20.003 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:27:20.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:27:20.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.240+0000 7f7036d89640 1 Processor -- start 2023-11-24T06:27:20.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.240+0000 7f7036d89640 1 -- start start 2023-11-24T06:27:20.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.241+0000 7f7036d89640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7030157c30 0x7f70300cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:20.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.241+0000 7f7036d89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70301618f0 0x7f7030070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:20.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.241+0000 7f7036d89640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7030071450 con 0x7f7030071030 2023-11-24T06:27:20.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.241+0000 7f7036d89640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703015ea30 con 0x7f70301618f0 2023-11-24T06:27:20.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.241+0000 7f7036d89640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703006d860 con 0x7f7030157c30 2023-11-24T06:27:20.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.242+0000 7f7027fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70301618f0 0x7f7030070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:20.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.242+0000 7f7027fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70301618f0 0x7f7030070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56168/0 (socket says 172.21.15.16:56168) 2023-11-24T06:27:20.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.242+0000 7f7027fff640 1 -- 172.21.15.16:0/3911376926 learned_addr learned my addr 172.21.15.16:0/3911376926 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:20.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.242+0000 7f7034b25640 1 --2- 172.21.15.16:0/3911376926 >> v2:172.21.15.173:3300/0 conn(0x7f7030157c30 0x7f70300cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:20.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.242+0000 7f7027fff640 1 -- 172.21.15.16:0/3911376926 >> v2:172.21.15.173:3300/0 conn(0x7f7030157c30 msgr2=0x7f70300cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:20.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.242+0000 7f7027fff640 1 --2- 172.21.15.16:0/3911376926 >> v2:172.21.15.173:3300/0 conn(0x7f7030157c30 0x7f70300cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.242+0000 7f7027fff640 1 -- 172.21.15.16:0/3911376926 >> v1:172.21.15.173:6789/0 conn(0x7f7030071030 legacy=0x7f703006dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:20.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.242+0000 7f7027fff640 1 -- 172.21.15.16:0/3911376926 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703015deb0 con 0x7f70301618f0 2023-11-24T06:27:20.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.244+0000 7f7027fff640 1 --2- 172.21.15.16:0/3911376926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 0x7f7030070af0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f70300723f0 tx=0x7f702002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa1198a62a50791 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:20.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.244+0000 7f70277fe640 1 -- 172.21.15.16:0/3911376926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020031870 con 0x7f70301618f0 2023-11-24T06:27:20.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.244+0000 7f70277fe640 1 -- 172.21.15.16:0/3911376926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70200319d0 con 0x7f70301618f0 2023-11-24T06:27:20.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.244+0000 7f70277fe640 1 -- 172.21.15.16:0/3911376926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020031d60 con 0x7f70301618f0 2023-11-24T06:27:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.245+0000 7f7036d89640 1 -- 172.21.15.16:0/3911376926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 msgr2=0x7f7030070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.245+0000 7f7036d89640 1 --2- 172.21.15.16:0/3911376926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 0x7f7030070af0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f70300723f0 tx=0x7f702002b910 comp rx=0 tx=0).stop 2023-11-24T06:27:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.245+0000 7f7036d89640 1 -- 172.21.15.16:0/3911376926 shutdown_connections 2023-11-24T06:27:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.245+0000 7f7036d89640 1 --2- 172.21.15.16:0/3911376926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 0x7f7030070af0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.245+0000 7f7036d89640 1 --2- 172.21.15.16:0/3911376926 >> v2:172.21.15.173:3300/0 conn(0x7f7030157c30 0x7f70300cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.245+0000 7f7036d89640 1 -- 172.21.15.16:0/3911376926 >> 172.21.15.16:0/3911376926 conn(0x7f70300cf700 msgr2=0x7f70300cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.245+0000 7f7036d89640 1 -- 172.21.15.16:0/3911376926 shutdown_connections 2023-11-24T06:27:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.245+0000 7f7036d89640 1 -- 172.21.15.16:0/3911376926 wait complete. 2023-11-24T06:27:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.247+0000 7f7036d89640 1 Processor -- start 2023-11-24T06:27:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.247+0000 7f7036d89640 1 -- start start 2023-11-24T06:27:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.248+0000 7f7036d89640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7030071030 0x7f7030156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.248+0000 7f7036d89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7030157c30 0x7f70300cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.248+0000 7f7036d89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 0x7f70300cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.248+0000 7f7036d89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7030158940 con 0x7f70301618f0 2023-11-24T06:27:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.248+0000 7f7036d89640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7030156b80 con 0x7f7030071030 2023-11-24T06:27:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.248+0000 7f7036d89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7030158b90 con 0x7f7030157c30 2023-11-24T06:27:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.248+0000 7f7035326640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 0x7f70300cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.248+0000 7f7035326640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 0x7f70300cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56174/0 (socket says 172.21.15.16:56174) 2023-11-24T06:27:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.248+0000 7f7035326640 1 -- 172.21.15.16:0/3746115430 learned_addr learned my addr 172.21.15.16:0/3746115430 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.249+0000 7f7034b25640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7030071030 0x7f7030156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.249+0000 7f7027fff640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7030157c30 0x7f70300cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.249+0000 7f7035326640 1 -- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7030157c30 msgr2=0x7f70300cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.249+0000 7f7035326640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7030157c30 0x7f70300cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.249+0000 7f7035326640 1 -- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7030071030 msgr2=0x7f7030156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.249+0000 7f7035326640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7030071030 0x7f7030156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.249+0000 7f7035326640 1 -- 172.21.15.16:0/3746115430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70200094d0 con 0x7f70301618f0 2023-11-24T06:27:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.249+0000 7f7035326640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 0x7f70300cdb40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f702800b9c0 tx=0x7f702800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.250+0000 7f70257fa640 1 -- 172.21.15.16:0/3746115430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702800cba0 con 0x7f70301618f0 2023-11-24T06:27:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.250+0000 7f70257fa640 1 -- 172.21.15.16:0/3746115430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702800cd00 con 0x7f70301618f0 2023-11-24T06:27:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.250+0000 7f70257fa640 1 -- 172.21.15.16:0/3746115430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028014680 con 0x7f70301618f0 2023-11-24T06:27:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.250+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7030158e70 con 0x7f70301618f0 2023-11-24T06:27:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.250+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7030159430 con 0x7f70301618f0 2023-11-24T06:27:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.251+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70300675d0 con 0x7f70301618f0 2023-11-24T06:27:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.252+0000 7f70257fa640 1 -- 172.21.15.16:0/3746115430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f702801d3b0 con 0x7f70301618f0 2023-11-24T06:27:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.253+0000 7f70257fa640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f701c06ae40 0x7f701c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.253+0000 7f70257fa640 1 -- 172.21.15.16:0/3746115430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70280895b0 con 0x7f70301618f0 2023-11-24T06:27:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.253+0000 7f7034b25640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f701c06ae40 0x7f701c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.255+0000 7f7034b25640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f701c06ae40 0x7f701c06d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f702c006fd0 tx=0x7f702c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.258+0000 7f70257fa640 1 -- 172.21.15.16:0/3746115430 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7028054650 con 0x7f70301618f0 2023-11-24T06:27:20.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.492+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7030159620 con 0x7f70301618f0 2023-11-24T06:27:20.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.493+0000 7f70257fa640 1 -- 172.21.15.16:0/3746115430 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7028019060 con 0x7f70301618f0 2023-11-24T06:27:20.496 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:27:20.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.495+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f701c06ae40 msgr2=0x7f701c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.495+0000 7f7036d89640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f701c06ae40 0x7f701c06d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f702c006fd0 tx=0x7f702c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.495+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 msgr2=0x7f70300cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.495+0000 7f7036d89640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 0x7f70300cdb40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f702800b9c0 tx=0x7f702800be80 comp rx=0 tx=0).stop 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.495+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 shutdown_connections 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.496+0000 7f7036d89640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f701c06ae40 0x7f701c06d300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.496+0000 7f7036d89640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70301618f0 0x7f70300cdb40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.496+0000 7f7036d89640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7030157c30 0x7f70300cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.496+0000 7f7036d89640 1 --2- 172.21.15.16:0/3746115430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7030071030 0x7f7030156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.496+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 >> 172.21.15.16:0/3746115430 conn(0x7f70300cf700 msgr2=0x7f70300b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.496+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 shutdown_connections 2023-11-24T06:27:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:20.496+0000 7f7036d89640 1 -- 172.21.15.16:0/3746115430 wait complete. 2023-11-24T06:27:20.518 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:27:20.518 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:27:20.518 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-24T06:27:20.518 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T06:27:20.518 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:27:20.518 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:27:20.518 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:27:20.519 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:27:20.519 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:27:20.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:20.519 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-24T06:27:20.569 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:20.570 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42303sec preferred_lft 42303sec 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-24T06:27:20.571 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:27:20.572 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:20.572 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:27:20.572 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:20.572 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8cfd:d2ff:fee1:5b18/64 scope link 2023-11-24T06:27:20.572 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:20.572 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:27:20.572 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:20.572 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8c89:e0ff:fefc:a3fb/64 scope link 2023-11-24T06:27:20.572 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:20.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:20.573 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:27:20.661 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-24T06:27:20.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:20.665 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-24T06:27:20.723 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-24T06:27:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:27:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:27:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:27:20.724 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:27:20.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:20.724 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:27:20.724 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:27:20.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:27:20.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:27:20.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:27:20.725 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:27:20.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:20.922 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-24T06:27:20.977 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:27:20.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:20.978 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-11-24T06:27:21.032 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T06:27:21.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:21.033 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:27:21.033 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:27:21.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:27:21.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T06:27:21.034 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:27:21.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:21.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:27:21.281 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:21.281 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:21.281 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:21.282 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:21.282 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:21.282 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:21.282 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:27:21.282 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5a:cb:a4:6b:e7:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:27:21.282 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:27:21.282 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:21.282 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::58cb:a4ff:fe6b:e7e7/64 scope link 2023-11-24T06:27:21.282 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:21.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:21.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:27:21.395 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:21.395 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:21.395 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:21.396 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:21.396 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:21.396 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:21.396 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:27:21.396 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5a:cb:a4:6b:e7:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:27:21.396 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:27:21.396 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:21.396 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::58cb:a4ff:fe6b:e7e7/64 scope link 2023-11-24T06:27:21.396 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:21.400 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T06:27:21.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:21.400 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:27:21.401 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:27:21.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T06:27:21.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:27:21.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T06:27:21.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T06:27:21.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T06:27:21.401 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:27:21.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:21.685 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:27:21.685 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:27:21.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2023-11-24T06:27:21.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:27:21.685 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:27:21.865 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-24T06:27:21.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:21.865 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T06:27:21.916 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:27:21.916 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:21.917 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:27:21.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:27:22.073 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:22.137 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:27:22.137 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:27:22.137 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:27:22.137 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:27:22.137 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:27:22.137 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:27:22.138 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:27:22.139 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:27:22.140 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:22.140 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:22.140 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:22.140 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:27:22.140 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:27:22.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:22.141 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:27:22.199 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-24T06:27:22.199 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-11-24T06:27:22.199 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-24T06:27:22.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:27:22.293 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:22.338 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:27:22.338 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:27:22.339 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:27:22.340 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:27:22.341 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:27:22.341 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:27:22.341 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:27:22.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:27:22.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:22.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:27:22.341 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:22.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:27:22.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:27:22.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:22.342 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:27:22.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:2023-11-24T06:27:22.405+0000 7f7133e2a480 -1 init, newargv = 0x5607e7e84b60 newargc=15 2023-11-24T06:27:22.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[96164]: starting ceph client 2023-11-24T06:27:22.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[96164]: starting fuse 2023-11-24T06:27:22.432 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-24T06:27:22.433 INFO:teuthology.orchestra.run.smithi016.stdout:42 2023-11-24T06:27:22.433 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 42] 2023-11-24T06:27:22.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:27:22.433 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-24T06:27:22.433 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-24T06:27:22.433 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-24T06:27:22.433 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-24T06:27:22.433 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:27:22.433 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-24T06:27:22.434 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:27:22.435 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:27:22.435 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:27:22.435 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-24T06:27:22.435 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-24T06:27:22.435 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-24T06:27:22.435 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:27:22.435 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:27:22.435 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.2")) 2023-11-24T06:27:22.435 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-24T06:27:22.640 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.2.96164.asok 2023-11-24T06:27:22.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:27:22.658 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.96164.asok 2023-11-24T06:27:22.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:22.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.96164.asok status 2023-11-24T06:27:22.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:22.752 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:22.894 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:27:22.895 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-24T06:27:22.895 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:27:22.895 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:27:22.895 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "2", 2023-11-24T06:27:22.895 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "96164", 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4665, 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-24T06:27:22.896 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4665 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1148797986 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:27:22.897 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1148797986 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4665 192.168.0.2:0/1148797986", 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.2:0/1148797986", 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 8, 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 20, 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-24T06:27:22.898 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:27:22.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:27:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.119+0000 7f256b448640 1 Processor -- start 2023-11-24T06:27:23.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.119+0000 7f256b448640 1 -- start start 2023-11-24T06:27:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.120+0000 7f256b448640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413df80 0x7f256415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.120+0000 7f256b448640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25640cf9a0 0x7f25640cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.120+0000 7f256b448640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256406c470 con 0x7f256413e360 2023-11-24T06:27:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f256b448640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256406c750 con 0x7f25640cf9a0 2023-11-24T06:27:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f256b448640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256406ca30 con 0x7f256413df80 2023-11-24T06:27:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f25699e5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25640cf9a0 0x7f25640cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f25699e5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25640cf9a0 0x7f25640cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52758/0 (socket says 172.21.15.16:52758) 2023-11-24T06:27:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f25699e5640 1 -- 172.21.15.16:0/2488106485 learned_addr learned my addr 172.21.15.16:0/2488106485 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f25699e5640 1 -- 172.21.15.16:0/2488106485 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413df80 msgr2=0x7f256415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f25691e4640 1 --2- 172.21.15.16:0/2488106485 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413df80 0x7f256415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f25699e5640 1 --2- 172.21.15.16:0/2488106485 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413df80 0x7f256415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f25699e5640 1 -- 172.21.15.16:0/2488106485 >> v1:172.21.15.16:6789/0 conn(0x7f256413e360 legacy=0x7f25640cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f25699e5640 1 -- 172.21.15.16:0/2488106485 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256406cd10 con 0x7f25640cf9a0 2023-11-24T06:27:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.121+0000 7f25691e4640 1 --2- 172.21.15.16:0/2488106485 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413df80 0x7f256415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.123+0000 7f25699e5640 1 --2- 172.21.15.16:0/2488106485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25640cf9a0 0x7f25640cd840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f255c009870 tx=0x7f255c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6270d26cb04c0615 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.123+0000 7f2563fff640 1 -- 172.21.15.16:0/2488106485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c030840 con 0x7f25640cf9a0 2023-11-24T06:27:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.123+0000 7f2563fff640 1 -- 172.21.15.16:0/2488106485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c0309a0 con 0x7f25640cf9a0 2023-11-24T06:27:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.123+0000 7f2563fff640 1 -- 172.21.15.16:0/2488106485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c030d30 con 0x7f25640cf9a0 2023-11-24T06:27:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.123+0000 7f256b448640 1 -- 172.21.15.16:0/2488106485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25640cf9a0 msgr2=0x7f25640cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.123+0000 7f256b448640 1 --2- 172.21.15.16:0/2488106485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25640cf9a0 0x7f25640cd840 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f255c009870 tx=0x7f255c02b980 comp rx=0 tx=0).stop 2023-11-24T06:27:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.124+0000 7f256b448640 1 -- 172.21.15.16:0/2488106485 shutdown_connections 2023-11-24T06:27:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.124+0000 7f256b448640 1 --2- 172.21.15.16:0/2488106485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25640cf9a0 0x7f25640cd840 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.124+0000 7f256b448640 1 --2- 172.21.15.16:0/2488106485 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413df80 0x7f256415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.124+0000 7f256b448640 1 -- 172.21.15.16:0/2488106485 >> 172.21.15.16:0/2488106485 conn(0x7f256406a800 msgr2=0x7f256406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.124+0000 7f256b448640 1 -- 172.21.15.16:0/2488106485 shutdown_connections 2023-11-24T06:27:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.124+0000 7f256b448640 1 -- 172.21.15.16:0/2488106485 wait complete. 2023-11-24T06:27:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.126+0000 7f256b448640 1 Processor -- start 2023-11-24T06:27:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.126+0000 7f256b448640 1 -- start start 2023-11-24T06:27:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.127+0000 7f256b448640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25640cf9a0 0x7f25641491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.127+0000 7f256b448640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256413df80 0x7f25640b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.127+0000 7f256b448640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413e360 0x7f2564071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.127+0000 7f256b448640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25640718a0 con 0x7f256413df80 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.127+0000 7f256b448640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25640b2d20 con 0x7f25640cf9a0 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.127+0000 7f256b448640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25640b3000 con 0x7f256413e360 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.127+0000 7f25689e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256413df80 0x7f25640b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.127+0000 7f25689e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256413df80 0x7f25640b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52774/0 (socket says 172.21.15.16:52774) 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.127+0000 7f25689e3640 1 -- 172.21.15.16:0/2119911343 learned_addr learned my addr 172.21.15.16:0/2119911343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f25689e3640 1 -- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413e360 msgr2=0x7f2564071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f25691e4640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25640cf9a0 0x7f25641491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f25699e5640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413e360 0x7f2564071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f25689e3640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413e360 0x7f2564071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f25689e3640 1 -- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25640cf9a0 msgr2=0x7f25641491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f25689e3640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25640cf9a0 0x7f25641491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f25689e3640 1 -- 172.21.15.16:0/2119911343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0094d0 con 0x7f256413df80 2023-11-24T06:27:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f25691e4640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25640cf9a0 0x7f25641491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f25689e3640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256413df80 0x7f25640b26b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f255800c450 tx=0x7f255800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f2561ffb640 1 -- 172.21.15.16:0/2119911343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580052d0 con 0x7f256413df80 2023-11-24T06:27:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f2561ffb640 1 -- 172.21.15.16:0/2119911343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2558005430 con 0x7f256413df80 2023-11-24T06:27:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f2561ffb640 1 -- 172.21.15.16:0/2119911343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558012870 con 0x7f256413df80 2023-11-24T06:27:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.128+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25640b32e0 con 0x7f256413df80 2023-11-24T06:27:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.129+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25640b38a0 con 0x7f256413df80 2023-11-24T06:27:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.130+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25640675d0 con 0x7f256413df80 2023-11-24T06:27:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.131+0000 7f2561ffb640 1 -- 172.21.15.16:0/2119911343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f25580129d0 con 0x7f256413df80 2023-11-24T06:27:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.132+0000 7f2561ffb640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f255006ae40 0x7f255006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.132+0000 7f2561ffb640 1 -- 172.21.15.16:0/2119911343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25580891d0 con 0x7f256413df80 2023-11-24T06:27:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.132+0000 7f25691e4640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f255006ae40 0x7f255006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:23.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.133+0000 7f25691e4640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f255006ae40 0x7f255006d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f255400d430 tx=0x7f2554007530 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:23.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.140+0000 7f2561ffb640 1 -- 172.21.15.16:0/2119911343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25580541f0 con 0x7f256413df80 2023-11-24T06:27:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.379+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f256406df00 con 0x7f256413df80 2023-11-24T06:27:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.380+0000 7f2561ffb640 1 -- 172.21.15.16:0/2119911343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2558057810 con 0x7f256413df80 2023-11-24T06:27:23.383 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f255006ae40 msgr2=0x7f255006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f255006ae40 0x7f255006d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f255400d430 tx=0x7f2554007530 comp rx=0 tx=0).stop 2023-11-24T06:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256413df80 msgr2=0x7f25640b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:23.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256413df80 0x7f25640b26b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f255800c450 tx=0x7f255800c910 comp rx=0 tx=0).stop 2023-11-24T06:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 shutdown_connections 2023-11-24T06:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f255006ae40 0x7f255006d300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f256413e360 0x7f2564071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f256413df80 0x7f25640b26b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 --2- 172.21.15.16:0/2119911343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25640cf9a0 0x7f25641491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 >> 172.21.15.16:0/2119911343 conn(0x7f256406a800 msgr2=0x7f25640ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 shutdown_connections 2023-11-24T06:27:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:23.383+0000 7f256b448640 1 -- 172.21.15.16:0/2119911343 wait complete. 2023-11-24T06:27:23.406 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:27:23.406 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:27:23.406 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2023-11-24T06:27:23.406 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:27:23.406 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:23.406 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:27:23.406 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:27:23.406 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:27:23.407 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:27:23.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:23.407 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2023-11-24T06:27:23.467 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:23.467 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:23.468 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:27:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42360sec preferred_lft 42360sec 2023-11-24T06:27:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2023-11-24T06:27:23.469 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:23.469 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:27:23.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:23.469 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:27:23.469 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:27:23.469 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:27:23.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip addr flush dev ceph-brx 2023-11-24T06:27:23.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set ceph-brx up 2023-11-24T06:27:23.470 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:27:23.470 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:27:23.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:23.924 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:27:24.008 INFO:teuthology.orchestra.run.smithi173.stdout:1 2023-11-24T06:27:24.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:24.012 DEBUG:teuthology.orchestra.run.smithi173:> route 2023-11-24T06:27:24.070 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2023-11-24T06:27:24.070 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:27:24.070 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:27:24.071 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:27:24.071 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:27:24.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:24.071 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:27:24.071 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:27:24.071 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:27:24.072 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:27:24.072 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:27:24.072 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:27:24.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:24.827 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2023-11-24T06:27:24.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:24.886 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list-id 2023-11-24T06:27:24.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:24.943 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:27:24.943 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:27:24.943 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:24.943 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T06:27:24.943 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:27:25.083 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T06:27:25.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:25.083 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:27:25.083 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:27:25.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T06:27:25.083 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:27:25.084 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T06:27:25.084 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T06:27:25.084 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T06:27:25.084 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:27:25.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:25.362 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:27:25.362 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:27:25.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2023-11-24T06:27:25.363 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:27:25.363 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:27:25.529 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:27:25.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:25.529 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:25.576 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:27:25.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:25.576 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:25.619 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:27:25.759 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:25.810 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:27:25.810 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:27:25.810 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:27:25.810 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:27:25.811 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:27:25.812 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:27:25.813 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:27:25.813 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:25.813 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:25.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:25.814 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:27:25.864 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:27:25.865 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:27:25.870 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:27:25.944 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:25.987 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:27:25.987 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:27:25.987 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:27:25.987 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:27:25.987 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:27:25.987 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:25.988 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:27:25.989 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:25.990 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:25.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:25.991 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:27:26.094 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:2023-11-24T06:27:26.092+0000 7f61ffa3f480 -1 init, newargv = 0x559747cfe9a0 newargc=15 2023-11-24T06:27:26.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[94298]: starting ceph client 2023-11-24T06:27:26.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[94298]: starting fuse 2023-11-24T06:27:27.180 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:27:27.285 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:27.346 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:27:27.346 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:27:27.347 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:27:27.347 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:27:27.347 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:27:27.348 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:27:27.348 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:27:27.348 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:27:27.349 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:27:27.349 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:27:27.349 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:27:27.350 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:27.350 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:27:27.350 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:27:27.351 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:27:27.351 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:27:27.352 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:27.352 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:27.352 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:27:27.353 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:27.354 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:27.354 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:27.354 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:27:27.355 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:27:27.355 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:27:27.355 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:27:27.356 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:27:27.356 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:27:27.357 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:27:27.357 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:27.358 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:27.358 INFO:teuthology.orchestra.run.smithi173.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:27.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:27.360 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:27:27.415 INFO:teuthology.orchestra.run.smithi173.stdout:39 2023-11-24T06:27:27.416 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-11-24T06:27:27.416 DEBUG:teuthology.orchestra.run.smithi173:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:27:27.416 DEBUG:teuthology.orchestra.run.smithi173:> import glob 2023-11-24T06:27:27.416 DEBUG:teuthology.orchestra.run.smithi173:> import re 2023-11-24T06:27:27.417 DEBUG:teuthology.orchestra.run.smithi173:> import os 2023-11-24T06:27:27.417 DEBUG:teuthology.orchestra.run.smithi173:> import subprocess 2023-11-24T06:27:27.417 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:27:27.417 DEBUG:teuthology.orchestra.run.smithi173:> def _find_admin_socket(client_name): 2023-11-24T06:27:27.417 DEBUG:teuthology.orchestra.run.smithi173:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:27:27.417 DEBUG:teuthology.orchestra.run.smithi173:> files = glob.glob(asok_path) 2023-11-24T06:27:27.418 DEBUG:teuthology.orchestra.run.smithi173:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:27:27.418 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:27:27.418 DEBUG:teuthology.orchestra.run.smithi173:> # Given a non-glob path, it better be there 2023-11-24T06:27:27.418 DEBUG:teuthology.orchestra.run.smithi173:> if "*" not in asok_path: 2023-11-24T06:27:27.418 DEBUG:teuthology.orchestra.run.smithi173:> assert(len(files) == 1) 2023-11-24T06:27:27.418 DEBUG:teuthology.orchestra.run.smithi173:> return files[0] 2023-11-24T06:27:27.419 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:27:27.419 DEBUG:teuthology.orchestra.run.smithi173:> for f in files: 2023-11-24T06:27:27.419 DEBUG:teuthology.orchestra.run.smithi173:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:27:27.419 DEBUG:teuthology.orchestra.run.smithi173:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:27:27.419 DEBUG:teuthology.orchestra.run.smithi173:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:27:27.419 DEBUG:teuthology.orchestra.run.smithi173:> contents = proc_f.read() 2023-11-24T06:27:27.419 DEBUG:teuthology.orchestra.run.smithi173:> if mountpoint in contents: 2023-11-24T06:27:27.420 DEBUG:teuthology.orchestra.run.smithi173:> return f 2023-11-24T06:27:27.420 DEBUG:teuthology.orchestra.run.smithi173:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:27:27.420 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:27:27.420 DEBUG:teuthology.orchestra.run.smithi173:> print(_find_admin_socket("client.1")) 2023-11-24T06:27:27.420 DEBUG:teuthology.orchestra.run.smithi173:> ' 2023-11-24T06:27:27.597 INFO:teuthology.orchestra.run.smithi173.stdout:/var/run/ceph/ceph-client.1.94298.asok 2023-11-24T06:27:27.601 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:27:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:27:27.615 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94298.asok 2023-11-24T06:27:27.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:27.615 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94298.asok status 2023-11-24T06:27:27.856 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-11-24T06:27:27.857 INFO:teuthology.orchestra.run.smithi173.stdout: "metadata": { 2023-11-24T06:27:27.857 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:27:27.857 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "entity_id": "1", 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "hostname": "smithi173", 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "pid": "94298", 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "root": "/" 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_count": 0, 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_pinned_count": 0, 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "id": 4460, 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "inst": { 2023-11-24T06:27:27.858 INFO:teuthology.orchestra.run.smithi173.stdout: "name": { 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "client", 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: "num": 4460 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 2107872673 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:27:27.859 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 2107872673 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: "inst_str": "client.4460 192.168.0.1:0/2107872673", 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: "addr_str": "192.168.0.1:0/2107872673", 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: "inode_count": 1, 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_epoch": 8, 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch": 20, 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: "blocklisted": false, 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout: "fs_name": "cephfs" 2023-11-24T06:27:27.860 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-11-24T06:27:27.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:27:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.103+0000 7fa73ec85640 1 Processor -- start 2023-11-24T06:27:28.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.103+0000 7fa73ec85640 1 -- start start 2023-11-24T06:27:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.104+0000 7fa73ec85640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa73813ff20 0x7fa73815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.105+0000 7fa73ec85640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7380b5440 con 0x7fa73806aaf0 2023-11-24T06:27:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.105+0000 7fa73ec85640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa738161c80 con 0x7fa738140300 2023-11-24T06:27:28.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.105+0000 7fa73ec85640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73815e6f0 con 0x7fa73813ff20 2023-11-24T06:27:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.105+0000 7fa73e484640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa73806aaf0 0x7fa738141c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35978/0 (socket says 172.21.15.16:35978) 2023-11-24T06:27:28.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.105+0000 7fa73e484640 1 -- 172.21.15.16:0/2805919173 learned_addr learned my addr 172.21.15.16:0/2805919173 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.105+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787105288 0 0) 0x7fa7380b5440 con 0x7fa73806aaf0 2023-11-24T06:27:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.105+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa724003520 con 0x7fa73806aaf0 2023-11-24T06:27:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73dc83640 1 --2- 172.21.15.16:0/2805919173 >> v2:172.21.15.173:3300/0 conn(0x7fa73813ff20 0x7fa73815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 191512212 0 0) 0x7fa724003520 con 0x7fa73806aaf0 2023-11-24T06:27:28.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 >> v2:172.21.15.173:3300/0 conn(0x7fa73813ff20 msgr2=0x7fa73815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 --2- 172.21.15.16:0/2805919173 >> v2:172.21.15.173:3300/0 conn(0x7fa73813ff20 0x7fa73815e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 >> v1:172.21.15.173:6789/0 conn(0x7fa738140300 legacy=0x7fa73806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7380b4830 con 0x7fa73806aaf0 2023-11-24T06:27:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fa7300019f0 con 0x7fa73806aaf0 2023-11-24T06:27:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 >> v1:172.21.15.16:6789/0 conn(0x7fa73806aaf0 legacy=0x7fa738141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa724003970 0x7fa724003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7240047c0 0x7fa724004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7240052a0 0x7fa72400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73815e6f0 con 0x7fa7240047c0 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72400c170 con 0x7fa7240052a0 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.106+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72400c330 con 0x7fa724003970 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.107+0000 7fa73d482640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7240047c0 0x7fa724004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.107+0000 7fa73dc83640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa724003970 0x7fa724003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.107+0000 7fa73e484640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7240052a0 0x7fa72400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.107+0000 7fa73d482640 1 -- 172.21.15.16:0/2805919173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa724003970 msgr2=0x7fa724003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.107+0000 7fa73d482640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa724003970 0x7fa724003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.107+0000 7fa73d482640 1 -- 172.21.15.16:0/2805919173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7240052a0 msgr2=0x7fa72400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.107+0000 7fa73d482640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7240052a0 0x7fa72400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.107+0000 7fa73d482640 1 -- 172.21.15.16:0/2805919173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724003520 con 0x7fa7240047c0 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.107+0000 7fa73e484640 1 -- 172.21.15.16:0/2805919173 reap_dead start 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.108+0000 7fa73d482640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7240047c0 0x7fa724004c40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa738140790 tx=0x7fa7240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b9b995a9a986bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7280052d0 con 0x7fa7240047c0 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72802e050 con 0x7fa7240047c0 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73cc81640 1 -- 172.21.15.16:0/2805919173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7280035f0 con 0x7fa7240047c0 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73ec85640 1 -- 172.21.15.16:0/2805919173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7240047c0 msgr2=0x7fa724004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73ec85640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7240047c0 0x7fa724004c40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa738140790 tx=0x7fa7240057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73ec85640 1 -- 172.21.15.16:0/2805919173 shutdown_connections 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73ec85640 1 --2- 172.21.15.16:0/2805919173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7240047c0 0x7fa724004c40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73ec85640 1 -- 172.21.15.16:0/2805919173 >> 172.21.15.16:0/2805919173 conn(0x7fa7380b1cd0 msgr2=0x7fa7380b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73ec85640 1 -- 172.21.15.16:0/2805919173 shutdown_connections 2023-11-24T06:27:28.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.109+0000 7fa73ec85640 1 -- 172.21.15.16:0/2805919173 wait complete. 2023-11-24T06:27:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.111+0000 7fa73ec85640 1 Processor -- start 2023-11-24T06:27:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.111+0000 7fa73ec85640 1 -- start start 2023-11-24T06:27:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.112+0000 7fa73ec85640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7240047c0 0x7fa73815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.112+0000 7fa73ec85640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa738157220 0x7fa7381f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.112+0000 7fa73ec85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738157fb0 0x7fa738158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.112+0000 7fa73ec85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738158970 con 0x7fa738157fb0 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.112+0000 7fa73ec85640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738160270 con 0x7fa738157220 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.112+0000 7fa73ec85640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73815b850 con 0x7fa7240047c0 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73e484640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738157fb0 0x7fa738158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73e484640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738157fb0 0x7fa738158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52804/0 (socket says 172.21.15.16:52804) 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73e484640 1 -- 172.21.15.16:0/1504574963 learned_addr learned my addr 172.21.15.16:0/1504574963 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73dc83640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7240047c0 0x7fa73815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73d482640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa738157220 0x7fa7381f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73e484640 1 -- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7240047c0 msgr2=0x7fa73815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73e484640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7240047c0 0x7fa73815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73e484640 1 -- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa738157220 msgr2=0x7fa7381f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73e484640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa738157220 0x7fa7381f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73e484640 1 -- 172.21.15.16:0/1504574963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73815bb30 con 0x7fa738157fb0 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73dc83640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7240047c0 0x7fa73815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.113+0000 7fa73e484640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738157fb0 0x7fa738158430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa7380722f0 tx=0x7fa730007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.114+0000 7fa72e7fc640 1 -- 172.21.15.16:0/1504574963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73000eb00 con 0x7fa738157fb0 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.114+0000 7fa72e7fc640 1 -- 172.21.15.16:0/1504574963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73000ec60 con 0x7fa738157fb0 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.114+0000 7fa72e7fc640 1 -- 172.21.15.16:0/1504574963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa730013420 con 0x7fa738157fb0 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.114+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73815be10 con 0x7fa738157fb0 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.114+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73815c3d0 con 0x7fa738157fb0 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.115+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa738067450 con 0x7fa738157fb0 2023-11-24T06:27:28.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.116+0000 7fa72e7fc640 1 -- 172.21.15.16:0/1504574963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa730013580 con 0x7fa738157fb0 2023-11-24T06:27:28.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.117+0000 7fa72e7fc640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa71c06ae70 0x7fa71c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:28.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.117+0000 7fa72e7fc640 1 -- 172.21.15.16:0/1504574963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa730089a00 con 0x7fa738157fb0 2023-11-24T06:27:28.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.118+0000 7fa73dc83640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa71c06ae70 0x7fa71c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:28.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.119+0000 7fa73dc83640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa71c06ae70 0x7fa71c06d330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa738070e70 tx=0x7fa734008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.122+0000 7fa72e7fc640 1 -- 172.21.15.16:0/1504574963 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa730054aa0 con 0x7fa738157fb0 2023-11-24T06:27:28.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.361+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa73813df60 con 0x7fa738157fb0 2023-11-24T06:27:28.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.362+0000 7fa72e7fc640 1 -- 172.21.15.16:0/1504574963 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa7300580c0 con 0x7fa738157fb0 2023-11-24T06:27:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:27:28.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa71c06ae70 msgr2=0x7fa71c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa71c06ae70 0x7fa71c06d330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa738070e70 tx=0x7fa734008040 comp rx=0 tx=0).stop 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738157fb0 msgr2=0x7fa738158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738157fb0 0x7fa738158430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa7380722f0 tx=0x7fa730007bc0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 shutdown_connections 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa71c06ae70 0x7fa71c06d330 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738157fb0 0x7fa738158430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa738157220 0x7fa7381f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 --2- 172.21.15.16:0/1504574963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7240047c0 0x7fa73815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 >> 172.21.15.16:0/1504574963 conn(0x7fa7380b1cd0 msgr2=0x7fa7380cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 shutdown_connections 2023-11-24T06:27:28.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:28.365+0000 7fa73ec85640 1 -- 172.21.15.16:0/1504574963 wait complete. 2023-11-24T06:27:28.388 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:27:28.389 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:27:28.389 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2023-11-24T06:27:28.389 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T06:27:28.389 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:27:28.389 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:27:28.389 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:27:28.389 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:27:28.389 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:27:28.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:28.390 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2023-11-24T06:27:28.447 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:28.447 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:28.447 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:28.448 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42355sec preferred_lft 42355sec 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ea:3b:9f:a2:9b:a6 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::c4e1:59ff:feae:2f56/64 scope link 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:28.449 INFO:teuthology.orchestra.run.smithi173.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:27:28.450 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ea:3b:9f:a2:9b:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:28.450 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::e83b:9fff:fea2:9ba6/64 scope link 2023-11-24T06:27:28.450 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:28.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:28.451 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:27:28.544 INFO:teuthology.orchestra.run.smithi173.stdout:1 2023-11-24T06:27:28.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:28.548 DEBUG:teuthology.orchestra.run.smithi173:> route 2023-11-24T06:27:28.602 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2023-11-24T06:27:28.602 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:27:28.603 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:27:28.603 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:27:28.603 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:27:28.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:28.603 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:27:28.603 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:27:28.603 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:27:28.603 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:27:28.604 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:27:28.604 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:27:28.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:28.783 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2023-11-24T06:27:28.832 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:27:28.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:28.832 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list-id 2023-11-24T06:27:28.882 INFO:teuthology.orchestra.run.smithi173.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T06:27:28.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:28.884 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:27:28.884 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:27:28.885 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:27:28.885 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T06:27:28.885 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:27:29.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:29.026 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:27:29.117 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:29.117 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:29.117 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:29.117 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:29.117 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:29.118 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:29.118 INFO:teuthology.orchestra.run.smithi173.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:27:29.118 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7a:73:06:9f:a1:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:27:29.118 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:27:29.118 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:29.118 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7873:6ff:fe9f:a170/64 scope link 2023-11-24T06:27:29.118 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:29.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:29.121 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:27:29.217 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:27:29.217 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:27:29.217 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:27:29.217 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:29.217 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2023-11-24T06:27:29.217 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:29.217 INFO:teuthology.orchestra.run.smithi173.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:27:29.217 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7a:73:06:9f:a1:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:27:29.217 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:27:29.218 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:29.218 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7873:6ff:fe9f:a170/64 scope link 2023-11-24T06:27:29.218 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:27:29.221 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T06:27:29.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:29.221 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:27:29.221 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:27:29.221 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T06:27:29.222 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:27:29.222 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T06:27:29.222 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T06:27:29.222 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T06:27:29.222 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:27:29.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:29.479 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:27:29.479 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:27:29.479 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2023-11-24T06:27:29.479 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:27:29.479 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:27:29.642 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-24T06:27:29.642 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:29.642 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T06:27:29.689 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:27:29.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:27:29.690 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:27:29.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:27:29.872 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:27:29.924 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:29.925 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:29.926 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:29.927 INFO:teuthology.orchestra.run.smithi173.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:29.927 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:27:29.927 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:27:29.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:29.928 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:27:29.978 INFO:teuthology.orchestra.run.smithi173.stdout:39 2023-11-24T06:27:29.979 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2023-11-24T06:27:29.979 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-24T06:27:29.984 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:27:30.062 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:27:30.106 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:27:30.107 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:27:30.108 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:27:30.109 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:27:30.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:30.110 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:27:30.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi173.stderr:2023-11-24T06:27:30.172+0000 7f383b84b480 -1 init, newargv = 0x558a576b1a50 newargc=15 2023-11-24T06:27:30.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi173.stderr:ceph-fuse[94811]: starting ceph client 2023-11-24T06:27:30.184 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi173.stderr:ceph-fuse[94811]: starting fuse 2023-11-24T06:27:30.197 INFO:teuthology.orchestra.run.smithi173.stdout:39 2023-11-24T06:27:30.197 INFO:teuthology.orchestra.run.smithi173.stdout:42 2023-11-24T06:27:30.197 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 42] 2023-11-24T06:27:30.197 DEBUG:teuthology.orchestra.run.smithi173:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:27:30.197 DEBUG:teuthology.orchestra.run.smithi173:> import glob 2023-11-24T06:27:30.197 DEBUG:teuthology.orchestra.run.smithi173:> import re 2023-11-24T06:27:30.197 DEBUG:teuthology.orchestra.run.smithi173:> import os 2023-11-24T06:27:30.197 DEBUG:teuthology.orchestra.run.smithi173:> import subprocess 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> def _find_admin_socket(client_name): 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> files = glob.glob(asok_path) 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> # Given a non-glob path, it better be there 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> if "*" not in asok_path: 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> assert(len(files) == 1) 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> return files[0] 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> for f in files: 2023-11-24T06:27:30.198 DEBUG:teuthology.orchestra.run.smithi173:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:27:30.199 DEBUG:teuthology.orchestra.run.smithi173:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:27:30.199 DEBUG:teuthology.orchestra.run.smithi173:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:27:30.199 DEBUG:teuthology.orchestra.run.smithi173:> contents = proc_f.read() 2023-11-24T06:27:30.199 DEBUG:teuthology.orchestra.run.smithi173:> if mountpoint in contents: 2023-11-24T06:27:30.199 DEBUG:teuthology.orchestra.run.smithi173:> return f 2023-11-24T06:27:30.199 DEBUG:teuthology.orchestra.run.smithi173:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:27:30.199 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:27:30.199 DEBUG:teuthology.orchestra.run.smithi173:> print(_find_admin_socket("client.3")) 2023-11-24T06:27:30.199 DEBUG:teuthology.orchestra.run.smithi173:> ' 2023-11-24T06:27:30.399 INFO:teuthology.orchestra.run.smithi173.stdout:/var/run/ceph/ceph-client.3.94811.asok 2023-11-24T06:27:30.403 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:27:30.416 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.94811.asok 2023-11-24T06:27:30.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:30.417 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.94811.asok status 2023-11-24T06:27:30.653 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-11-24T06:27:30.653 INFO:teuthology.orchestra.run.smithi173.stdout: "metadata": { 2023-11-24T06:27:30.653 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "entity_id": "3", 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "hostname": "smithi173", 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "pid": "94811", 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "root": "/" 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_count": 0, 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_pinned_count": 0, 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "id": 4466, 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "inst": { 2023-11-24T06:27:30.654 INFO:teuthology.orchestra.run.smithi173.stdout: "name": { 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "client", 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: "num": 4466 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 3144993190 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:27:30.655 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 3144993190 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: "inst_str": "client.4466 192.168.0.2:0/3144993190", 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: "addr_str": "192.168.0.2:0/3144993190", 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: "inode_count": 1, 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_epoch": 8, 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch": 20, 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: "blocklisted": false, 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout: "fs_name": "cephfs" 2023-11-24T06:27:30.656 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-11-24T06:27:30.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:30.669 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-24T06:27:30.669 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:30.732 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-24T06:27:30.732 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:30.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:30.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:30.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:30.832 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-24T06:27:30.832 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-24T06:27:30.894 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-24T06:27:30.894 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-24T06:27:30.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:30.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:27:31.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:31.005 DEBUG:teuthology.orchestra.run.smithi173:> stat --file-system '--printf=%T 2023-11-24T06:27:31.005 DEBUG:teuthology.orchestra.run.smithi173:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:31.066 INFO:teuthology.orchestra.run.smithi173.stdout:fuseblk 2023-11-24T06:27:31.066 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:31.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:31.067 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:31.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:31.164 DEBUG:teuthology.orchestra.run.smithi173:> stat --file-system '--printf=%T 2023-11-24T06:27:31.165 DEBUG:teuthology.orchestra.run.smithi173:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-24T06:27:31.220 INFO:teuthology.orchestra.run.smithi173.stdout:fuseblk 2023-11-24T06:27:31.220 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-24T06:27:31.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:31.220 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:27:31.311 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T06:27:31.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.573+0000 7f5d5a373640 1 Processor -- start 2023-11-24T06:27:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.573+0000 7f5d5a373640 1 -- start start 2023-11-24T06:27:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.574+0000 7f5d5a373640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540cfcf0 0x7f5d54160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.575+0000 7f5d5a373640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5d540d3910 0x7f5d54160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.575+0000 7f5d5a373640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d540d4160 con 0x7f5d540d00d0 2023-11-24T06:27:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.575+0000 7f5d5a373640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540ceff0 con 0x7f5d540d3910 2023-11-24T06:27:31.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.575+0000 7f5d5a373640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540cf410 con 0x7f5d540cfcf0 2023-11-24T06:27:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.575+0000 7f5d4bfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5d540d3910 0x7f5d54160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.576+0000 7f5d4bfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5d540d3910 0x7f5d54160c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:33302/0 (socket says 172.21.15.16:33302) 2023-11-24T06:27:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.576+0000 7f5d4bfff640 1 -- 172.21.15.16:0/1672390749 learned_addr learned my addr 172.21.15.16:0/1672390749 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.576+0000 7f5d53fff640 1 --2- 172.21.15.16:0/1672390749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540cfcf0 0x7f5d54160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.576+0000 7f5d4bfff640 1 -- 172.21.15.16:0/1672390749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540cfcf0 msgr2=0x7f5d54160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.576+0000 7f5d4bfff640 1 --2- 172.21.15.16:0/1672390749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540cfcf0 0x7f5d54160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.576+0000 7f5d4bfff640 1 -- 172.21.15.16:0/1672390749 >> v1:172.21.15.173:6789/0 conn(0x7f5d540d00d0 legacy=0x7f5d540b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.576+0000 7f5d4bfff640 1 -- 172.21.15.16:0/1672390749 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540b2420 con 0x7f5d540d3910 2023-11-24T06:27:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.578+0000 7f5d4bfff640 1 --2- 172.21.15.16:0/1672390749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d540d3910 0x7f5d54160c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0098a0 tx=0x7f5d3c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ab5218a6887fbc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.578+0000 7f5d537fe640 1 -- 172.21.15.16:0/1672390749 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c031640 con 0x7f5d540d3910 2023-11-24T06:27:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.579+0000 7f5d537fe640 1 -- 172.21.15.16:0/1672390749 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c0317a0 con 0x7f5d540d3910 2023-11-24T06:27:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.579+0000 7f5d537fe640 1 -- 172.21.15.16:0/1672390749 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c031b30 con 0x7f5d540d3910 2023-11-24T06:27:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.579+0000 7f5d5a373640 1 -- 172.21.15.16:0/1672390749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d540d3910 msgr2=0x7f5d54160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.579+0000 7f5d5a373640 1 --2- 172.21.15.16:0/1672390749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d540d3910 0x7f5d54160c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0098a0 tx=0x7f5d3c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.579+0000 7f5d5a373640 1 -- 172.21.15.16:0/1672390749 shutdown_connections 2023-11-24T06:27:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.579+0000 7f5d5a373640 1 --2- 172.21.15.16:0/1672390749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d540d3910 0x7f5d54160c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.579+0000 7f5d5a373640 1 --2- 172.21.15.16:0/1672390749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540cfcf0 0x7f5d54160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.579+0000 7f5d5a373640 1 -- 172.21.15.16:0/1672390749 >> 172.21.15.16:0/1672390749 conn(0x7f5d5409c530 msgr2=0x7f5d540d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.580+0000 7f5d5a373640 1 -- 172.21.15.16:0/1672390749 shutdown_connections 2023-11-24T06:27:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.580+0000 7f5d5a373640 1 -- 172.21.15.16:0/1672390749 wait complete. 2023-11-24T06:27:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.582+0000 7f5d5a373640 1 Processor -- start 2023-11-24T06:27:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.582+0000 7f5d5a373640 1 -- start start 2023-11-24T06:27:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d5a373640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d540cfcf0 0x7f5d5415da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d5a373640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d540d00d0 0x7f5d540b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d5a373640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540d3910 0x7f5d540b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d5a373640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541548d0 con 0x7f5d540d00d0 2023-11-24T06:27:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d5a373640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540b6c30 con 0x7f5d540cfcf0 2023-11-24T06:27:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d5a373640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540b6f10 con 0x7f5d540d3910 2023-11-24T06:27:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d540d00d0 0x7f5d540b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d540d00d0 0x7f5d540b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52820/0 (socket says 172.21.15.16:52820) 2023-11-24T06:27:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d4bfff640 1 -- 172.21.15.16:0/859118251 learned_addr learned my addr 172.21.15.16:0/859118251 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.584+0000 7f5d53fff640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d540cfcf0 0x7f5d5415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d58910640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540d3910 0x7f5d540b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d4bfff640 1 -- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540d3910 msgr2=0x7f5d540b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d4bfff640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540d3910 0x7f5d540b6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d4bfff640 1 -- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d540cfcf0 msgr2=0x7f5d5415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d4bfff640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d540cfcf0 0x7f5d5415da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d4bfff640 1 -- 172.21.15.16:0/859118251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0094d0 con 0x7f5d540d00d0 2023-11-24T06:27:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d4bfff640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d540d00d0 0x7f5d540b6050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0098a0 tx=0x7f5d3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d517fa640 1 -- 172.21.15.16:0/859118251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c03a930 con 0x7f5d540d00d0 2023-11-24T06:27:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d517fa640 1 -- 172.21.15.16:0/859118251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c03aa90 con 0x7f5d540d00d0 2023-11-24T06:27:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d517fa640 1 -- 172.21.15.16:0/859118251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c03ae40 con 0x7f5d540d00d0 2023-11-24T06:27:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.585+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d540b71f0 con 0x7f5d540d00d0 2023-11-24T06:27:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.586+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540b77b0 con 0x7f5d540d00d0 2023-11-24T06:27:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.587+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d540ca140 con 0x7f5d540d00d0 2023-11-24T06:27:31.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.588+0000 7f5d517fa640 1 -- 172.21.15.16:0/859118251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d3c02f050 con 0x7f5d540d00d0 2023-11-24T06:27:31.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.588+0000 7f5d517fa640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3406ae40 0x7f5d3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:31.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.588+0000 7f5d517fa640 1 -- 172.21.15.16:0/859118251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d3c0ab660 con 0x7f5d540d00d0 2023-11-24T06:27:31.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.589+0000 7f5d53fff640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3406ae40 0x7f5d3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:31.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.590+0000 7f5d53fff640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3406ae40 0x7f5d3406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009a30 tx=0x7f5d44008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:31.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.593+0000 7f5d517fa640 1 -- 172.21.15.16:0/859118251 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d3c076790 con 0x7f5d540d00d0 2023-11-24T06:27:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.788+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d540675d0 con 0x7f5d540d00d0 2023-11-24T06:27:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.791+0000 7f5d517fa640 1 -- 172.21.15.16:0/859118251 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f5d3c079db0 con 0x7f5d540d00d0 2023-11-24T06:27:31.794 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:31.794 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3406ae40 msgr2=0x7f5d3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3406ae40 0x7f5d3406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009a30 tx=0x7f5d44008040 comp rx=0 tx=0).stop 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d540d00d0 msgr2=0x7f5d540b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d540d00d0 0x7f5d540b6050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0098a0 tx=0x7f5d3c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 shutdown_connections 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5d3406ae40 0x7f5d3406d300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d540d3910 0x7f5d540b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d540d00d0 0x7f5d540b6050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 --2- 172.21.15.16:0/859118251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d540cfcf0 0x7f5d5415da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 >> 172.21.15.16:0/859118251 conn(0x7f5d5409c530 msgr2=0x7f5d5406c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 shutdown_connections 2023-11-24T06:27:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:31.794+0000 7f5d5a373640 1 -- 172.21.15.16:0/859118251 wait complete. 2023-11-24T06:27:31.818 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:31.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.038+0000 7fa53ccf1640 1 Processor -- start 2023-11-24T06:27:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.039+0000 7fa53ccf1640 1 -- start start 2023-11-24T06:27:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.040+0000 7fa53ccf1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa53815d4b0 0x7fa5380d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.040+0000 7fa53ccf1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5380cf9a0 0x7fa5380cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.040+0000 7fa53ccf1640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5380d0320 con 0x7fa5380cddf0 2023-11-24T06:27:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.040+0000 7fa53ccf1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5380b2730 con 0x7fa5380cf9a0 2023-11-24T06:27:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.040+0000 7fa53ccf1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380d1ec0 con 0x7fa53815d4b0 2023-11-24T06:27:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.040+0000 7fa52ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5380cf9a0 0x7fa5380cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.040+0000 7fa52ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa5380cf9a0 0x7fa5380cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52828/0 (socket says 172.21.15.16:52828) 2023-11-24T06:27:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.040+0000 7fa52ffff640 1 -- 172.21.15.16:0/1927462668 learned_addr learned my addr 172.21.15.16:0/1927462668 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.041+0000 7fa5377fe640 1 --2- 172.21.15.16:0/1927462668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa53815d4b0 0x7fa5380d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.041+0000 7fa52ffff640 1 -- 172.21.15.16:0/1927462668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa53815d4b0 msgr2=0x7fa5380d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.041+0000 7fa52ffff640 1 --2- 172.21.15.16:0/1927462668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa53815d4b0 0x7fa5380d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.041+0000 7fa52ffff640 1 -- 172.21.15.16:0/1927462668 >> v1:172.21.15.173:6789/0 conn(0x7fa5380cddf0 legacy=0x7fa5380ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:32.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.041+0000 7fa52ffff640 1 -- 172.21.15.16:0/1927462668 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5380b29d0 con 0x7fa5380cf9a0 2023-11-24T06:27:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.042+0000 7fa52ffff640 1 --2- 172.21.15.16:0/1927462668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 0x7fa5380cfde0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5200098a0 tx=0x7fa52002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980163e72688bee2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.042+0000 7fa536ffd640 1 -- 172.21.15.16:0/1927462668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520031640 con 0x7fa5380cf9a0 2023-11-24T06:27:32.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.042+0000 7fa536ffd640 1 -- 172.21.15.16:0/1927462668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5200317a0 con 0x7fa5380cf9a0 2023-11-24T06:27:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.042+0000 7fa536ffd640 1 -- 172.21.15.16:0/1927462668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520031b30 con 0x7fa5380cf9a0 2023-11-24T06:27:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.043+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1927462668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 msgr2=0x7fa5380cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.043+0000 7fa53ccf1640 1 --2- 172.21.15.16:0/1927462668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 0x7fa5380cfde0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5200098a0 tx=0x7fa52002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.043+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1927462668 shutdown_connections 2023-11-24T06:27:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.043+0000 7fa53ccf1640 1 --2- 172.21.15.16:0/1927462668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 0x7fa5380cfde0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.043+0000 7fa53ccf1640 1 --2- 172.21.15.16:0/1927462668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa53815d4b0 0x7fa5380d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.043+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1927462668 >> 172.21.15.16:0/1927462668 conn(0x7fa5380d3000 msgr2=0x7fa5380d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.043+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1927462668 shutdown_connections 2023-11-24T06:27:32.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.043+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1927462668 wait complete. 2023-11-24T06:27:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.045+0000 7fa53ccf1640 1 Processor -- start 2023-11-24T06:27:32.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.045+0000 7fa53ccf1640 1 -- start start 2023-11-24T06:27:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.046+0000 7fa53ccf1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5380cddf0 0x7fa53815f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.046+0000 7fa53ccf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 0x7fa5380b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.046+0000 7fa53ccf1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa53815d4b0 0x7fa5380b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.046+0000 7fa53ccf1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa538155020 con 0x7fa5380cf9a0 2023-11-24T06:27:32.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.046+0000 7fa53ccf1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa538160640 con 0x7fa53815d4b0 2023-11-24T06:27:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.046+0000 7fa53ccf1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53815fa30 con 0x7fa5380cddf0 2023-11-24T06:27:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.046+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 0x7fa5380b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.046+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 0x7fa5380b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52832/0 (socket says 172.21.15.16:52832) 2023-11-24T06:27:32.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.046+0000 7fa52ffff640 1 -- 172.21.15.16:0/1587863221 learned_addr learned my addr 172.21.15.16:0/1587863221 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa5377fe640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5380cddf0 0x7fa53815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa537fff640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa53815d4b0 0x7fa5380b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa52ffff640 1 -- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5380cddf0 msgr2=0x7fa53815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa52ffff640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5380cddf0 0x7fa53815f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa52ffff640 1 -- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa53815d4b0 msgr2=0x7fa5380b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa52ffff640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa53815d4b0 0x7fa5380b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa52ffff640 1 -- 172.21.15.16:0/1587863221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200094d0 con 0x7fa5380cf9a0 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa5377fe640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5380cddf0 0x7fa53815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa537fff640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa53815d4b0 0x7fa5380b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa52ffff640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 0x7fa5380b0be0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa520000c00 tx=0x7fa5200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa534ff9640 1 -- 172.21.15.16:0/1587863221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52003ac60 con 0x7fa5380cf9a0 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa534ff9640 1 -- 172.21.15.16:0/1587863221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52003ade0 con 0x7fa5380cf9a0 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53815da10 con 0x7fa5380cf9a0 2023-11-24T06:27:32.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.047+0000 7fa534ff9640 1 -- 172.21.15.16:0/1587863221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520044630 con 0x7fa5380cf9a0 2023-11-24T06:27:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.048+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53815fff0 con 0x7fa5380cf9a0 2023-11-24T06:27:32.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.049+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa538067450 con 0x7fa5380cf9a0 2023-11-24T06:27:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.050+0000 7fa534ff9640 1 -- 172.21.15.16:0/1587863221 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa52002f050 con 0x7fa5380cf9a0 2023-11-24T06:27:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.051+0000 7fa534ff9640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa51806ae90 0x7fa51806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.051+0000 7fa534ff9640 1 -- 172.21.15.16:0/1587863221 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5200ab3d0 con 0x7fa5380cf9a0 2023-11-24T06:27:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.051+0000 7fa5377fe640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa51806ae90 0x7fa51806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.053+0000 7fa5377fe640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa51806ae90 0x7fa51806d350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.056+0000 7fa534ff9640 1 -- 172.21.15.16:0/1587863221 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa5200764f0 con 0x7fa5380cf9a0 2023-11-24T06:27:32.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.260+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5380b04e0 con 0x7fa5380cf9a0 2023-11-24T06:27:32.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.261+0000 7fa534ff9640 1 -- 172.21.15.16:0/1587863221 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa520031e60 con 0x7fa5380cf9a0 2023-11-24T06:27:32.264 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa51806ae90 msgr2=0x7fa51806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa51806ae90 0x7fa51806d350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528008040 comp rx=0 tx=0).stop 2023-11-24T06:27:32.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 msgr2=0x7fa5380b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 0x7fa5380b0be0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa520000c00 tx=0x7fa5200046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 shutdown_connections 2023-11-24T06:27:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa51806ae90 0x7fa51806d350 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa53815d4b0 0x7fa5380b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5380cf9a0 0x7fa5380b0be0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 --2- 172.21.15.16:0/1587863221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5380cddf0 0x7fa53815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 >> 172.21.15.16:0/1587863221 conn(0x7fa5380d3000 msgr2=0x7fa5380b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:32.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 shutdown_connections 2023-11-24T06:27:32.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.264+0000 7fa53ccf1640 1 -- 172.21.15.16:0/1587863221 wait complete. 2023-11-24T06:27:32.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.517+0000 7fa9a926d640 1 Processor -- start 2023-11-24T06:27:32.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.517+0000 7fa9a926d640 1 -- start start 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a926d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a40c3ee0 0x7fa9a41630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a926d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9a40c42c0 0x7fa9a406b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a926d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa9a406ba30 0x7fa9a40b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a926d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a40b2920 con 0x7fa9a40c42c0 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a926d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a4162ca0 con 0x7fa9a406ba30 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a926d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a4162120 con 0x7fa9a40c3ee0 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9a40c42c0 0x7fa9a406b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9a40c42c0 0x7fa9a406b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52838/0 (socket says 172.21.15.16:52838) 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a37fe640 1 -- 172.21.15.16:0/3907898996 learned_addr learned my addr 172.21.15.16:0/3907898996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a37fe640 1 -- 172.21.15.16:0/3907898996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a40c3ee0 msgr2=0x7fa9a41630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a37fe640 1 --2- 172.21.15.16:0/3907898996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a40c3ee0 0x7fa9a41630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a37fe640 1 -- 172.21.15.16:0/3907898996 >> v2:172.21.15.173:3300/0 conn(0x7fa9a406ba30 msgr2=0x7fa9a40b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.519+0000 7fa9a8a6c640 1 --2- 172.21.15.16:0/3907898996 >> v2:172.21.15.173:3300/0 conn(0x7fa9a406ba30 0x7fa9a40b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.520+0000 7fa9a37fe640 1 --2- 172.21.15.16:0/3907898996 >> v2:172.21.15.173:3300/0 conn(0x7fa9a406ba30 0x7fa9a40b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.520+0000 7fa9a37fe640 1 -- 172.21.15.16:0/3907898996 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a4162400 con 0x7fa9a40c42c0 2023-11-24T06:27:32.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.520+0000 7fa9a8a6c640 1 --2- 172.21.15.16:0/3907898996 >> v2:172.21.15.173:3300/0 conn(0x7fa9a406ba30 0x7fa9a40b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.521+0000 7fa9a37fe640 1 --2- 172.21.15.16:0/3907898996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 0x7fa9a406b4f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4071880 tx=0x7fa98c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5a53c5d6fbec83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.521+0000 7fa9a2ffd640 1 -- 172.21.15.16:0/3907898996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c02c9f0 con 0x7fa9a40c42c0 2023-11-24T06:27:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.521+0000 7fa9a2ffd640 1 -- 172.21.15.16:0/3907898996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c02cb50 con 0x7fa9a40c42c0 2023-11-24T06:27:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.521+0000 7fa9a2ffd640 1 -- 172.21.15.16:0/3907898996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0343f0 con 0x7fa9a40c42c0 2023-11-24T06:27:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.522+0000 7fa9a926d640 1 -- 172.21.15.16:0/3907898996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 msgr2=0x7fa9a406b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.522+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3907898996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 0x7fa9a406b4f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4071880 tx=0x7fa98c02be50 comp rx=0 tx=0).stop 2023-11-24T06:27:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.522+0000 7fa9a926d640 1 -- 172.21.15.16:0/3907898996 shutdown_connections 2023-11-24T06:27:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.522+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3907898996 >> v2:172.21.15.173:3300/0 conn(0x7fa9a406ba30 0x7fa9a40b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.522+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3907898996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 0x7fa9a406b4f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.522+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3907898996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a40c3ee0 0x7fa9a41630a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.522+0000 7fa9a926d640 1 -- 172.21.15.16:0/3907898996 >> 172.21.15.16:0/3907898996 conn(0x7fa9a40b2a60 msgr2=0x7fa9a40b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.522+0000 7fa9a926d640 1 -- 172.21.15.16:0/3907898996 shutdown_connections 2023-11-24T06:27:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.522+0000 7fa9a926d640 1 -- 172.21.15.16:0/3907898996 wait complete. 2023-11-24T06:27:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.524+0000 7fa9a926d640 1 Processor -- start 2023-11-24T06:27:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.524+0000 7fa9a926d640 1 -- start start 2023-11-24T06:27:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.525+0000 7fa9a926d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a406ba30 0x7fa9a40c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.525+0000 7fa9a926d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a40c3ee0 0x7fa9a40b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.525+0000 7fa9a926d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 0x7fa9a415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.525+0000 7fa9a926d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40c9100 con 0x7fa9a40c42c0 2023-11-24T06:27:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.525+0000 7fa9a926d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40b5550 con 0x7fa9a40c3ee0 2023-11-24T06:27:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.525+0000 7fa9a926d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40c4f10 con 0x7fa9a406ba30 2023-11-24T06:27:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.525+0000 7fa9a8a6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 0x7fa9a415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.525+0000 7fa9a8a6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 0x7fa9a415ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52842/0 (socket says 172.21.15.16:52842) 2023-11-24T06:27:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.525+0000 7fa9a8a6c640 1 -- 172.21.15.16:0/3578928227 learned_addr learned my addr 172.21.15.16:0/3578928227 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.526+0000 7fa9a37fe640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a40c3ee0 0x7fa9a40b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.526+0000 7fa9a3fff640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a406ba30 0x7fa9a40c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.526+0000 7fa9a8a6c640 1 -- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a406ba30 msgr2=0x7fa9a40c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.526+0000 7fa9a8a6c640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a406ba30 0x7fa9a40c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.526+0000 7fa9a8a6c640 1 -- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a40c3ee0 msgr2=0x7fa9a40b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.526+0000 7fa9a8a6c640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a40c3ee0 0x7fa9a40b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.526+0000 7fa9a8a6c640 1 -- 172.21.15.16:0/3578928227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009b30 con 0x7fa9a40c42c0 2023-11-24T06:27:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.526+0000 7fa9a8a6c640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 0x7fa9a415ef20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00c7f0 tx=0x7fa99c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.526+0000 7fa9a0ff9640 1 -- 172.21.15.16:0/3578928227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c007820 con 0x7fa9a40c42c0 2023-11-24T06:27:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.527+0000 7fa9a0ff9640 1 -- 172.21.15.16:0/3578928227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c007980 con 0x7fa9a40c42c0 2023-11-24T06:27:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.527+0000 7fa9a0ff9640 1 -- 172.21.15.16:0/3578928227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c007d70 con 0x7fa9a40c42c0 2023-11-24T06:27:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.527+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a40c51f0 con 0x7fa9a40c42c0 2023-11-24T06:27:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.527+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a40c57b0 con 0x7fa9a40c42c0 2023-11-24T06:27:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.528+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a40c92d0 con 0x7fa9a40c42c0 2023-11-24T06:27:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.529+0000 7fa9a0ff9640 1 -- 172.21.15.16:0/3578928227 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa99c0113f0 con 0x7fa9a40c42c0 2023-11-24T06:27:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.530+0000 7fa9a0ff9640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa98406ae40 0x7fa98406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.530+0000 7fa9a0ff9640 1 -- 172.21.15.16:0/3578928227 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa99c089a00 con 0x7fa9a40c42c0 2023-11-24T06:27:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.531+0000 7fa9a3fff640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa98406ae40 0x7fa98406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.532+0000 7fa9a3fff640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa98406ae40 0x7fa98406d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa9a41fabd0 tx=0x7fa994008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.534+0000 7fa9a0ff9640 1 -- 172.21.15.16:0/3578928227 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9a40c92d0 con 0x7fa9a40c42c0 2023-11-24T06:27:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.730+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9a40c92d0 con 0x7fa9a40c42c0 2023-11-24T06:27:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.731+0000 7fa9a0ff9640 1 -- 172.21.15.16:0/3578928227 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa99c01f020 con 0x7fa9a40c42c0 2023-11-24T06:27:32.734 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa98406ae40 msgr2=0x7fa98406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa98406ae40 0x7fa98406d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa9a41fabd0 tx=0x7fa994008040 comp rx=0 tx=0).stop 2023-11-24T06:27:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 msgr2=0x7fa9a415ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 0x7fa9a415ef20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00c7f0 tx=0x7fa99c00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 shutdown_connections 2023-11-24T06:27:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa98406ae40 0x7fa98406d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9a40c42c0 0x7fa9a415ef20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a40c3ee0 0x7fa9a40b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 --2- 172.21.15.16:0/3578928227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a406ba30 0x7fa9a40c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 >> 172.21.15.16:0/3578928227 conn(0x7fa9a40b2a60 msgr2=0x7fa9a40afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 shutdown_connections 2023-11-24T06:27:32.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.734+0000 7fa9a926d640 1 -- 172.21.15.16:0/3578928227 wait complete. 2023-11-24T06:27:32.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.982+0000 7f836359e640 1 Processor -- start 2023-11-24T06:27:32.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.982+0000 7f836359e640 1 -- start start 2023-11-24T06:27:32.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.983+0000 7f836359e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f83640d1d40 0x7f83640b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.983+0000 7f836359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83640b1f70 0x7f83640b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f836359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83640d0090 con 0x7f836415d1d0 2023-11-24T06:27:32.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f836359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83640d02e0 con 0x7f83640b1f70 2023-11-24T06:27:32.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f836359e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83640d05c0 con 0x7f83640d1d40 2023-11-24T06:27:32.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f836259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f836415d1d0 0x7f83640d1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52560/0 (socket says 172.21.15.16:52560) 2023-11-24T06:27:32.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f836259c640 1 -- 172.21.15.16:0/3420078953 learned_addr learned my addr 172.21.15.16:0/3420078953 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:32.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f8362d9d640 1 --2- 172.21.15.16:0/3420078953 >> v2:172.21.15.16:3300/0 conn(0x7f83640b1f70 0x7f83640b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f835bfff640 1 --2- 172.21.15.16:0/3420078953 >> v2:172.21.15.173:3300/0 conn(0x7f83640d1d40 0x7f83640b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f8361d9b640 1 -- 172.21.15.16:0/3420078953 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655550937 0 0) 0x7f83640d0090 con 0x7f836415d1d0 2023-11-24T06:27:32.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f8361d9b640 1 -- 172.21.15.16:0/3420078953 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8350003520 con 0x7f836415d1d0 2023-11-24T06:27:32.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f8362d9d640 1 -- 172.21.15.16:0/3420078953 >> v2:172.21.15.173:3300/0 conn(0x7f83640d1d40 msgr2=0x7f83640b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:32.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f8362d9d640 1 --2- 172.21.15.16:0/3420078953 >> v2:172.21.15.173:3300/0 conn(0x7f83640d1d40 0x7f83640b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f8362d9d640 1 -- 172.21.15.16:0/3420078953 >> v1:172.21.15.16:6789/0 conn(0x7f836415d1d0 legacy=0x7f83640d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f8362d9d640 1 -- 172.21.15.16:0/3420078953 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640d08a0 con 0x7f83640b1f70 2023-11-24T06:27:32.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.984+0000 7f835bfff640 1 --2- 172.21.15.16:0/3420078953 >> v2:172.21.15.173:3300/0 conn(0x7f83640d1d40 0x7f83640b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.986+0000 7f8362d9d640 1 --2- 172.21.15.16:0/3420078953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640b1f70 0x7f83640b0700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f835c009d90 tx=0x7f835c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c925cc2e7fae63 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.986+0000 7f8361d9b640 1 -- 172.21.15.16:0/3420078953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c02cc60 con 0x7f83640b1f70 2023-11-24T06:27:32.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.986+0000 7f8361d9b640 1 -- 172.21.15.16:0/3420078953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c02cdc0 con 0x7f83640b1f70 2023-11-24T06:27:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.986+0000 7f8361d9b640 1 -- 172.21.15.16:0/3420078953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c034660 con 0x7f83640b1f70 2023-11-24T06:27:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.986+0000 7f836359e640 1 -- 172.21.15.16:0/3420078953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640b1f70 msgr2=0x7f83640b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.986+0000 7f836359e640 1 --2- 172.21.15.16:0/3420078953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640b1f70 0x7f83640b0700 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f835c009d90 tx=0x7f835c02beb0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.987+0000 7f836359e640 1 -- 172.21.15.16:0/3420078953 shutdown_connections 2023-11-24T06:27:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.987+0000 7f836359e640 1 --2- 172.21.15.16:0/3420078953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640b1f70 0x7f83640b0700 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.987+0000 7f836359e640 1 --2- 172.21.15.16:0/3420078953 >> v2:172.21.15.173:3300/0 conn(0x7f83640d1d40 0x7f83640b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.987+0000 7f836359e640 1 -- 172.21.15.16:0/3420078953 >> 172.21.15.16:0/3420078953 conn(0x7f83640d3050 msgr2=0x7f83640d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.987+0000 7f836359e640 1 -- 172.21.15.16:0/3420078953 shutdown_connections 2023-11-24T06:27:32.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.987+0000 7f836359e640 1 -- 172.21.15.16:0/3420078953 wait complete. 2023-11-24T06:27:32.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.988+0000 7f836359e640 1 Processor -- start 2023-11-24T06:27:32.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.989+0000 7f836359e640 1 -- start start 2023-11-24T06:27:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.989+0000 7f836359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83640b1f70 0x7f836415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.989+0000 7f836359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640d1d40 0x7f83640b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f836359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f836415d1d0 0x7f83640b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f836359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836415b430 con 0x7f83640d1d40 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f836359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836409ce20 con 0x7f83640b1f70 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f836359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836415e4c0 con 0x7f836415d1d0 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f835bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640d1d40 0x7f83640b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f835bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640d1d40 0x7f83640b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48192/0 (socket says 172.21.15.16:48192) 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f835bfff640 1 -- 172.21.15.16:0/4037134119 learned_addr learned my addr 172.21.15.16:0/4037134119 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f836259c640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83640b1f70 0x7f836415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f8362d9d640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f836415d1d0 0x7f83640b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f835bfff640 1 -- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f836415d1d0 msgr2=0x7f83640b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f835bfff640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f836415d1d0 0x7f83640b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f835bfff640 1 -- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83640b1f70 msgr2=0x7f836415aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f835bfff640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83640b1f70 0x7f836415aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.990+0000 7f835bfff640 1 -- 172.21.15.16:0/4037134119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c009a70 con 0x7f83640d1d40 2023-11-24T06:27:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.991+0000 7f836259c640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83640b1f70 0x7f836415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.991+0000 7f835bfff640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640d1d40 0x7f83640b70c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c7f0 tx=0x7f834c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.991+0000 7f835affd640 1 -- 172.21.15.16:0/4037134119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c00cea0 con 0x7f83640d1d40 2023-11-24T06:27:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.991+0000 7f835affd640 1 -- 172.21.15.16:0/4037134119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c004230 con 0x7f83640d1d40 2023-11-24T06:27:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.991+0000 7f835affd640 1 -- 172.21.15.16:0/4037134119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c007830 con 0x7f83640d1d40 2023-11-24T06:27:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.991+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836415dbd0 con 0x7f83640d1d40 2023-11-24T06:27:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.991+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836415e950 con 0x7f83640d1d40 2023-11-24T06:27:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.993+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8364160580 con 0x7f83640d1d40 2023-11-24T06:27:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.994+0000 7f835affd640 1 -- 172.21.15.16:0/4037134119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f834c007990 con 0x7f83640d1d40 2023-11-24T06:27:32.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.995+0000 7f835affd640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f834406ae40 0x7f834406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:32.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.995+0000 7f835affd640 1 -- 172.21.15.16:0/4037134119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f834c0896d0 con 0x7f83640d1d40 2023-11-24T06:27:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.995+0000 7f836259c640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f834406ae40 0x7f834406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.997+0000 7f836259c640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f834406ae40 0x7f834406d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f83540011e0 tx=0x7f8354009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:32.999+0000 7f835affd640 1 -- 172.21.15.16:0/4037134119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f834c054650 con 0x7f83640d1d40 2023-11-24T06:27:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.192+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8364067450 con 0x7f83640d1d40 2023-11-24T06:27:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.192+0000 7f835affd640 1 -- 172.21.15.16:0/4037134119 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f834c057c70 con 0x7f83640d1d40 2023-11-24T06:27:33.196 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:33.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f834406ae40 msgr2=0x7f834406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f834406ae40 0x7f834406d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f83540011e0 tx=0x7f8354009040 comp rx=0 tx=0).stop 2023-11-24T06:27:33.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640d1d40 msgr2=0x7f83640b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640d1d40 0x7f83640b70c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c7f0 tx=0x7f834c00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 shutdown_connections 2023-11-24T06:27:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f834406ae40 0x7f834406d300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f836415d1d0 0x7f83640b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83640d1d40 0x7f83640b70c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 --2- 172.21.15.16:0/4037134119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83640b1f70 0x7f836415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.195+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 >> 172.21.15.16:0/4037134119 conn(0x7f83640d3050 msgr2=0x7f83640b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:33.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.196+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 shutdown_connections 2023-11-24T06:27:33.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.196+0000 7f836359e640 1 -- 172.21.15.16:0/4037134119 wait complete. 2023-11-24T06:27:33.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:33.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.435+0000 7fd48e306640 1 Processor -- start 2023-11-24T06:27:33.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.435+0000 7fd48e306640 1 -- start start 2023-11-24T06:27:33.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.437+0000 7fd48e306640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 0x7fd488161900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.437+0000 7fd48e306640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd488161470 con 0x7fd4880b2050 2023-11-24T06:27:33.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.437+0000 7fd48e306640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48815e410 con 0x7fd4880b2430 2023-11-24T06:27:33.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.437+0000 7fd48e306640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880b3ff0 con 0x7fd4880cf740 2023-11-24T06:27:33.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.438+0000 7fd48d304640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd4880b2050 0x7fd4880cf030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52572/0 (socket says 172.21.15.16:52572) 2023-11-24T06:27:33.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.438+0000 7fd48d304640 1 -- 172.21.15.16:0/4208660400 learned_addr learned my addr 172.21.15.16:0/4208660400 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:33.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.438+0000 7fd48cb03640 1 --2- 172.21.15.16:0/4208660400 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 0x7fd488161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.438+0000 7fd47ffff640 1 -- 172.21.15.16:0/4208660400 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520149252 0 0) 0x7fd488161470 con 0x7fd4880b2050 2023-11-24T06:27:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.438+0000 7fd47ffff640 1 -- 172.21.15.16:0/4208660400 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd474003520 con 0x7fd4880b2050 2023-11-24T06:27:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.438+0000 7fd48cb03640 1 -- 172.21.15.16:0/4208660400 >> v1:172.21.15.173:6789/0 conn(0x7fd4880b2430 legacy=0x7fd48815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.438+0000 7fd48cb03640 1 -- 172.21.15.16:0/4208660400 >> v1:172.21.15.16:6789/0 conn(0x7fd4880b2050 legacy=0x7fd4880cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.438+0000 7fd48cb03640 1 -- 172.21.15.16:0/4208660400 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48815e6f0 con 0x7fd4880cf740 2023-11-24T06:27:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.439+0000 7fd48cb03640 1 --2- 172.21.15.16:0/4208660400 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 0x7fd488161900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4780097d0 tx=0x7fd47802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b77a9c3cc76cce5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.440+0000 7fd47ffff640 1 -- 172.21.15.16:0/4208660400 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478031770 con 0x7fd4880cf740 2023-11-24T06:27:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.440+0000 7fd47ffff640 1 -- 172.21.15.16:0/4208660400 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4780318d0 con 0x7fd4880cf740 2023-11-24T06:27:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.440+0000 7fd47ffff640 1 -- 172.21.15.16:0/4208660400 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478031c60 con 0x7fd4880cf740 2023-11-24T06:27:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.441+0000 7fd48e306640 1 -- 172.21.15.16:0/4208660400 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 msgr2=0x7fd488161900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.441+0000 7fd48e306640 1 --2- 172.21.15.16:0/4208660400 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 0x7fd488161900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd4780097d0 tx=0x7fd47802b810 comp rx=0 tx=0).stop 2023-11-24T06:27:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.441+0000 7fd48e306640 1 -- 172.21.15.16:0/4208660400 shutdown_connections 2023-11-24T06:27:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.441+0000 7fd48e306640 1 --2- 172.21.15.16:0/4208660400 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 0x7fd488161900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.441+0000 7fd48e306640 1 -- 172.21.15.16:0/4208660400 >> 172.21.15.16:0/4208660400 conn(0x7fd48806a800 msgr2=0x7fd48806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:33.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.441+0000 7fd48e306640 1 -- 172.21.15.16:0/4208660400 shutdown_connections 2023-11-24T06:27:33.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.441+0000 7fd48e306640 1 -- 172.21.15.16:0/4208660400 wait complete. 2023-11-24T06:27:33.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.443+0000 7fd48e306640 1 Processor -- start 2023-11-24T06:27:33.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.443+0000 7fd48e306640 1 -- start start 2023-11-24T06:27:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48e306640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4880b2050 0x7fd48813e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48e306640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4880b2430 0x7fd488099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48e306640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 0x7fd4881e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48e306640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4881435d0 con 0x7fd4880b2430 2023-11-24T06:27:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48e306640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488142ca0 con 0x7fd4880b2050 2023-11-24T06:27:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48e306640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd488142f80 con 0x7fd4880cf740 2023-11-24T06:27:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48cb03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4880b2430 0x7fd488099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48cb03640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4880b2430 0x7fd488099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48206/0 (socket says 172.21.15.16:48206) 2023-11-24T06:27:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48cb03640 1 -- 172.21.15.16:0/991691340 learned_addr learned my addr 172.21.15.16:0/991691340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.444+0000 7fd48db05640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 0x7fd4881e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.445+0000 7fd48d304640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4880b2050 0x7fd48813e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.445+0000 7fd48cb03640 1 -- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 msgr2=0x7fd4881e7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.445+0000 7fd48cb03640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 0x7fd4881e7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.445+0000 7fd48cb03640 1 -- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4880b2050 msgr2=0x7fd48813e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.445+0000 7fd48cb03640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4880b2050 0x7fd48813e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.445+0000 7fd48cb03640 1 -- 172.21.15.16:0/991691340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd478009400 con 0x7fd4880b2430 2023-11-24T06:27:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.445+0000 7fd48d304640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4880b2050 0x7fd48813e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.445+0000 7fd48cb03640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4880b2430 0x7fd488099a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd478034040 tx=0x7fd4780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.446+0000 7fd47dffb640 1 -- 172.21.15.16:0/991691340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478032810 con 0x7fd4880b2430 2023-11-24T06:27:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.446+0000 7fd47dffb640 1 -- 172.21.15.16:0/991691340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd478032990 con 0x7fd4880b2430 2023-11-24T06:27:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.446+0000 7fd47dffb640 1 -- 172.21.15.16:0/991691340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478032d40 con 0x7fd4880b2430 2023-11-24T06:27:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.446+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488143260 con 0x7fd4880b2430 2023-11-24T06:27:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.446+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd488142990 con 0x7fd4880b2430 2023-11-24T06:27:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.447+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488143740 con 0x7fd4880b2430 2023-11-24T06:27:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.448+0000 7fd47dffb640 1 -- 172.21.15.16:0/991691340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd47804d3f0 con 0x7fd4880b2430 2023-11-24T06:27:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.449+0000 7fd47dffb640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd46c06ae70 0x7fd46c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.449+0000 7fd47dffb640 1 -- 172.21.15.16:0/991691340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4780ab0d0 con 0x7fd4880b2430 2023-11-24T06:27:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.449+0000 7fd48d304640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd46c06ae70 0x7fd46c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.451+0000 7fd48d304640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd46c06ae70 0x7fd46c06d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd4840011e0 tx=0x7fd484009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.453+0000 7fd47dffb640 1 -- 172.21.15.16:0/991691340 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd478076060 con 0x7fd4880b2430 2023-11-24T06:27:33.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.647+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd488067450 con 0x7fd4880b2430 2023-11-24T06:27:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.649+0000 7fd47dffb640 1 -- 172.21.15.16:0/991691340 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fd478079680 con 0x7fd4880b2430 2023-11-24T06:27:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:33.653 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.652+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd46c06ae70 msgr2=0x7fd46c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.652+0000 7fd48e306640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd46c06ae70 0x7fd46c06d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd4840011e0 tx=0x7fd484009040 comp rx=0 tx=0).stop 2023-11-24T06:27:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.652+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4880b2430 msgr2=0x7fd488099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.652+0000 7fd48e306640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4880b2430 0x7fd488099a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd478034040 tx=0x7fd4780046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.653+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 shutdown_connections 2023-11-24T06:27:33.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.653+0000 7fd48e306640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd46c06ae70 0x7fd46c06d330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.653+0000 7fd48e306640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4880cf740 0x7fd4881e7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.653+0000 7fd48e306640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4880b2430 0x7fd488099a60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.653+0000 7fd48e306640 1 --2- 172.21.15.16:0/991691340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4880b2050 0x7fd48813e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.653+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 >> 172.21.15.16:0/991691340 conn(0x7fd48806a800 msgr2=0x7fd48806be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.653+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 shutdown_connections 2023-11-24T06:27:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.653+0000 7fd48e306640 1 -- 172.21.15.16:0/991691340 wait complete. 2023-11-24T06:27:33.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:33.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:33.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.905+0000 7f151599e640 1 Processor -- start 2023-11-24T06:27:33.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.905+0000 7f151599e640 1 -- start start 2023-11-24T06:27:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.909+0000 7f151599e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1510143810 0x7f1510071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.909+0000 7f151599e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15100ae170 0x7f151015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.909+0000 7f151599e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151006c730 con 0x7f151015d770 2023-11-24T06:27:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.909+0000 7f151599e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1510143bf0 con 0x7f15100ae170 2023-11-24T06:27:33.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.909+0000 7f151599e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15100ae5f0 con 0x7f1510143810 2023-11-24T06:27:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.909+0000 7f151519d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f151015d770 0x7f151015dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52588/0 (socket says 172.21.15.16:52588) 2023-11-24T06:27:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.909+0000 7f151519d640 1 -- 172.21.15.16:0/4246319914 learned_addr learned my addr 172.21.15.16:0/4246319914 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.909+0000 7f150ffff640 1 --2- 172.21.15.16:0/4246319914 >> v2:172.21.15.16:3300/0 conn(0x7f15100ae170 0x7f151015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.909+0000 7f151499c640 1 --2- 172.21.15.16:0/4246319914 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1510143810 0x7f1510071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.910+0000 7f150f7fe640 1 -- 172.21.15.16:0/4246319914 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389163214 0 0) 0x7f151006c730 con 0x7f151015d770 2023-11-24T06:27:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.910+0000 7f150f7fe640 1 -- 172.21.15.16:0/4246319914 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1500003520 con 0x7f151015d770 2023-11-24T06:27:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.910+0000 7f150ffff640 1 -- 172.21.15.16:0/4246319914 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1510143810 msgr2=0x7f1510071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.910+0000 7f150ffff640 1 --2- 172.21.15.16:0/4246319914 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1510143810 0x7f1510071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.910+0000 7f150ffff640 1 -- 172.21.15.16:0/4246319914 >> v1:172.21.15.16:6789/0 conn(0x7f151015d770 legacy=0x7f151015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.910+0000 7f150ffff640 1 -- 172.21.15.16:0/4246319914 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15100724d0 con 0x7f15100ae170 2023-11-24T06:27:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.911+0000 7f150ffff640 1 --2- 172.21.15.16:0/4246319914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15100ae170 0x7f151015e8a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0097a0 tx=0x7f14fc02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94b7ffab2533ad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.912+0000 7f150f7fe640 1 -- 172.21.15.16:0/4246319914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc031780 con 0x7f15100ae170 2023-11-24T06:27:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.912+0000 7f150f7fe640 1 -- 172.21.15.16:0/4246319914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc0318e0 con 0x7f15100ae170 2023-11-24T06:27:33.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.912+0000 7f150f7fe640 1 -- 172.21.15.16:0/4246319914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc031c70 con 0x7f15100ae170 2023-11-24T06:27:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.912+0000 7f151599e640 1 -- 172.21.15.16:0/4246319914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15100ae170 msgr2=0x7f151015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.912+0000 7f151599e640 1 --2- 172.21.15.16:0/4246319914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15100ae170 0x7f151015e8a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0097a0 tx=0x7f14fc02b820 comp rx=0 tx=0).stop 2023-11-24T06:27:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.912+0000 7f151599e640 1 -- 172.21.15.16:0/4246319914 shutdown_connections 2023-11-24T06:27:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.912+0000 7f151599e640 1 --2- 172.21.15.16:0/4246319914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15100ae170 0x7f151015e8a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.912+0000 7f151599e640 1 --2- 172.21.15.16:0/4246319914 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1510143810 0x7f1510071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.912+0000 7f151599e640 1 -- 172.21.15.16:0/4246319914 >> 172.21.15.16:0/4246319914 conn(0x7f151006a7d0 msgr2=0x7f151006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.913+0000 7f151599e640 1 -- 172.21.15.16:0/4246319914 shutdown_connections 2023-11-24T06:27:33.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.913+0000 7f151599e640 1 -- 172.21.15.16:0/4246319914 wait complete. 2023-11-24T06:27:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.914+0000 7f151599e640 1 Processor -- start 2023-11-24T06:27:33.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.915+0000 7f151599e640 1 -- start start 2023-11-24T06:27:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.915+0000 7f151599e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15100ae170 0x7f15100b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.915+0000 7f151599e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1510143810 0x7f15100cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f151599e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151015d770 0x7f151013acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f151599e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15100cc7d0 con 0x7f1510143810 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f151599e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15100cc530 con 0x7f15100ae170 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f151599e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15100aeba0 con 0x7f151015d770 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f150ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1510143810 0x7f15100cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f150ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1510143810 0x7f15100cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48226/0 (socket says 172.21.15.16:48226) 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f150ffff640 1 -- 172.21.15.16:0/1013437331 learned_addr learned my addr 172.21.15.16:0/1013437331 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f151499c640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15100ae170 0x7f15100b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f150ffff640 1 -- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151015d770 msgr2=0x7f151013acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f150ffff640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151015d770 0x7f151013acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f150ffff640 1 -- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15100ae170 msgr2=0x7f15100b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f150ffff640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15100ae170 0x7f15100b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f150ffff640 1 -- 172.21.15.16:0/1013437331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc009400 con 0x7f1510143810 2023-11-24T06:27:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.916+0000 7f151499c640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15100ae170 0x7f15100b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:33.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.917+0000 7f150ffff640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1510143810 0x7f15100cbff0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f14fc000c00 tx=0x7f14fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.917+0000 7f150d7fa640 1 -- 172.21.15.16:0/1013437331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc032820 con 0x7f1510143810 2023-11-24T06:27:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.917+0000 7f150d7fa640 1 -- 172.21.15.16:0/1013437331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc0329c0 con 0x7f1510143810 2023-11-24T06:27:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.917+0000 7f150d7fa640 1 -- 172.21.15.16:0/1013437331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc032d70 con 0x7f1510143810 2023-11-24T06:27:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.917+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15100aee80 con 0x7f1510143810 2023-11-24T06:27:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.917+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15100af440 con 0x7f1510143810 2023-11-24T06:27:33.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.919+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15100af660 con 0x7f1510143810 2023-11-24T06:27:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.919+0000 7f150d7fa640 1 -- 172.21.15.16:0/1013437331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14fc04e450 con 0x7f1510143810 2023-11-24T06:27:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.920+0000 7f150d7fa640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14f406ae40 0x7f14f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:33.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.920+0000 7f150d7fa640 1 -- 172.21.15.16:0/1013437331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14fc0ab520 con 0x7f1510143810 2023-11-24T06:27:33.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.922+0000 7f151499c640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14f406ae40 0x7f14f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:33.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.924+0000 7f150d7fa640 1 -- 172.21.15.16:0/1013437331 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14fc076660 con 0x7f1510143810 2023-11-24T06:27:33.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:33.925+0000 7f151499c640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14f406ae40 0x7f14f406d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1504009ac0 tx=0x7f1504008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.121+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1510067450 con 0x7f1510143810 2023-11-24T06:27:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.122+0000 7f150d7fa640 1 -- 172.21.15.16:0/1013437331 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14fc079c80 con 0x7f1510143810 2023-11-24T06:27:34.125 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.124+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14f406ae40 msgr2=0x7f14f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.124+0000 7f151599e640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14f406ae40 0x7f14f406d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1504009ac0 tx=0x7f1504008040 comp rx=0 tx=0).stop 2023-11-24T06:27:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.124+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1510143810 msgr2=0x7f15100cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.124+0000 7f151599e640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1510143810 0x7f15100cbff0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f14fc000c00 tx=0x7f14fc0046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.125+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 shutdown_connections 2023-11-24T06:27:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.125+0000 7f151599e640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14f406ae40 0x7f14f406d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.125+0000 7f151599e640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f151015d770 0x7f151013acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.125+0000 7f151599e640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1510143810 0x7f15100cbff0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.125+0000 7f151599e640 1 --2- 172.21.15.16:0/1013437331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15100ae170 0x7f15100b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.125+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 >> 172.21.15.16:0/1013437331 conn(0x7f151006a7d0 msgr2=0x7f15100b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.125+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 shutdown_connections 2023-11-24T06:27:34.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.125+0000 7f151599e640 1 -- 172.21.15.16:0/1013437331 wait complete. 2023-11-24T06:27:34.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.374+0000 7f6c08885640 1 Processor -- start 2023-11-24T06:27:34.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.375+0000 7f6c08885640 1 -- start start 2023-11-24T06:27:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.376+0000 7f6c08885640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c040cf9a0 0x7f6c040cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.376+0000 7f6c08885640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c040b31d0 0x7f6c040b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.376+0000 7f6c08885640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c040d0320 con 0x7f6c0415d4b0 2023-11-24T06:27:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.376+0000 7f6c08885640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c040d1c60 con 0x7f6c040cf9a0 2023-11-24T06:27:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.376+0000 7f6c08885640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c040b2730 con 0x7f6c040b31d0 2023-11-24T06:27:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.377+0000 7f6bfbfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c040cf9a0 0x7f6c040cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.377+0000 7f6c03fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c040b31d0 0x7f6c040b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.377+0000 7f6bfbfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c040cf9a0 0x7f6c040cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:48296/0 (socket says 172.21.15.16:48296) 2023-11-24T06:27:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.377+0000 7f6bfbfff640 1 -- 172.21.15.16:0/1023578980 learned_addr learned my addr 172.21.15.16:0/1023578980 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.378+0000 7f6c03fff640 1 -- 172.21.15.16:0/1023578980 >> v2:172.21.15.173:3300/0 conn(0x7f6c040cf9a0 msgr2=0x7f6c040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.378+0000 7f6c03fff640 1 --2- 172.21.15.16:0/1023578980 >> v2:172.21.15.173:3300/0 conn(0x7f6c040cf9a0 0x7f6c040cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.378+0000 7f6c03fff640 1 -- 172.21.15.16:0/1023578980 >> v1:172.21.15.173:6789/0 conn(0x7f6c0415d4b0 legacy=0x7f6c040d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.378+0000 7f6c03fff640 1 -- 172.21.15.16:0/1023578980 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c040d1ec0 con 0x7f6c040b31d0 2023-11-24T06:27:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.378+0000 7f6bfbfff640 1 --2- 172.21.15.16:0/1023578980 >> v2:172.21.15.173:3300/0 conn(0x7f6c040cf9a0 0x7f6c040cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:27:34.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.379+0000 7f6c03fff640 1 --2- 172.21.15.16:0/1023578980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c040b31d0 0x7f6c040b3660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009d90 tx=0x7f6bfc02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf61a7154abd1e37 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.380+0000 7f6c02ffd640 1 -- 172.21.15.16:0/1023578980 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc02cc50 con 0x7f6c040b31d0 2023-11-24T06:27:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.380+0000 7f6c02ffd640 1 -- 172.21.15.16:0/1023578980 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc02cdb0 con 0x7f6c040b31d0 2023-11-24T06:27:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.380+0000 7f6c02ffd640 1 -- 172.21.15.16:0/1023578980 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc034660 con 0x7f6c040b31d0 2023-11-24T06:27:34.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.381+0000 7f6c08885640 1 -- 172.21.15.16:0/1023578980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c040b31d0 msgr2=0x7f6c040b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.381+0000 7f6c08885640 1 --2- 172.21.15.16:0/1023578980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c040b31d0 0x7f6c040b3660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009d90 tx=0x7f6bfc02bea0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.381+0000 7f6c08885640 1 -- 172.21.15.16:0/1023578980 shutdown_connections 2023-11-24T06:27:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.381+0000 7f6c08885640 1 --2- 172.21.15.16:0/1023578980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c040b31d0 0x7f6c040b3660 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.381+0000 7f6c08885640 1 --2- 172.21.15.16:0/1023578980 >> v2:172.21.15.173:3300/0 conn(0x7f6c040cf9a0 0x7f6c040cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.381+0000 7f6c08885640 1 -- 172.21.15.16:0/1023578980 >> 172.21.15.16:0/1023578980 conn(0x7f6c040d3000 msgr2=0x7f6c040d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.381+0000 7f6c08885640 1 -- 172.21.15.16:0/1023578980 shutdown_connections 2023-11-24T06:27:34.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.381+0000 7f6c08885640 1 -- 172.21.15.16:0/1023578980 wait complete. 2023-11-24T06:27:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.383+0000 7f6c08885640 1 Processor -- start 2023-11-24T06:27:34.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.383+0000 7f6c08885640 1 -- start start 2023-11-24T06:27:34.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.384+0000 7f6c08885640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c040b31d0 0x7f6c0409e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.384+0000 7f6c08885640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040cf9a0 0x7f6c040ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.384+0000 7f6c08885640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c0415d4b0 0x7f6c0409cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.384+0000 7f6c08885640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0409d4c0 con 0x7f6c040cf9a0 2023-11-24T06:27:34.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.384+0000 7f6c08885640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0409fbf0 con 0x7f6c040b31d0 2023-11-24T06:27:34.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.384+0000 7f6c08885640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0409feb0 con 0x7f6c0415d4b0 2023-11-24T06:27:34.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6bfbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040cf9a0 0x7f6c040ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6bfbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040cf9a0 0x7f6c040ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48242/0 (socket says 172.21.15.16:48242) 2023-11-24T06:27:34.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6bfbfff640 1 -- 172.21.15.16:0/1472585837 learned_addr learned my addr 172.21.15.16:0/1472585837 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:34.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6c037fe640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c040b31d0 0x7f6c0409e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6c03fff640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c0415d4b0 0x7f6c0409cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6bfbfff640 1 -- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c0415d4b0 msgr2=0x7f6c0409cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6bfbfff640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c0415d4b0 0x7f6c0409cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6bfbfff640 1 -- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c040b31d0 msgr2=0x7f6c0409e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6bfbfff640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c040b31d0 0x7f6c0409e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.385+0000 7f6bfbfff640 1 -- 172.21.15.16:0/1472585837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0094d0 con 0x7f6c040cf9a0 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.386+0000 7f6bfbfff640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040cf9a0 0x7f6c040ca0d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00e9e0 tx=0x7f6bec00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.386+0000 7f6c00ff9640 1 -- 172.21.15.16:0/1472585837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec00c9d0 con 0x7f6c040cf9a0 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.386+0000 7f6c00ff9640 1 -- 172.21.15.16:0/1472585837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec00cb30 con 0x7f6c040cf9a0 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.386+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc009a70 con 0x7f6c040cf9a0 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.386+0000 7f6c00ff9640 1 -- 172.21.15.16:0/1472585837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec00cea0 con 0x7f6c040cf9a0 2023-11-24T06:27:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.386+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c040ca7b0 con 0x7f6c040cf9a0 2023-11-24T06:27:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.388+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c04067450 con 0x7f6c040cf9a0 2023-11-24T06:27:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.388+0000 7f6c00ff9640 1 -- 172.21.15.16:0/1472585837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bec013470 con 0x7f6c040cf9a0 2023-11-24T06:27:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.389+0000 7f6c00ff9640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6be406ae40 0x7f6be406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.389+0000 7f6c00ff9640 1 -- 172.21.15.16:0/1472585837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bec0898c0 con 0x7f6c040cf9a0 2023-11-24T06:27:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.390+0000 7f6c037fe640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6be406ae40 0x7f6be406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.391+0000 7f6c037fe640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6be406ae40 0x7f6be406d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c0406c2f0 tx=0x7f6bf4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.394+0000 7f6c00ff9640 1 -- 172.21.15.16:0/1472585837 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bec054960 con 0x7f6c040cf9a0 2023-11-24T06:27:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.586+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c0409fff0 con 0x7f6c040cf9a0 2023-11-24T06:27:34.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.587+0000 7f6c00ff9640 1 -- 172.21.15.16:0/1472585837 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6bec017020 con 0x7f6c040cf9a0 2023-11-24T06:27:34.591 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6be406ae40 msgr2=0x7f6be406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6be406ae40 0x7f6be406d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6c0406c2f0 tx=0x7f6bf4009040 comp rx=0 tx=0).stop 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040cf9a0 msgr2=0x7f6c040ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040cf9a0 0x7f6c040ca0d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00e9e0 tx=0x7f6bec00eea0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 shutdown_connections 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6be406ae40 0x7f6be406d300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c0415d4b0 0x7f6c0409cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c040cf9a0 0x7f6c040ca0d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 --2- 172.21.15.16:0/1472585837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c040b31d0 0x7f6c0409e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.589+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 >> 172.21.15.16:0/1472585837 conn(0x7f6c040d3000 msgr2=0x7f6c0406c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.590+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 shutdown_connections 2023-11-24T06:27:34.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.590+0000 7f6c08885640 1 -- 172.21.15.16:0/1472585837 wait complete. 2023-11-24T06:27:34.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.839+0000 7efe04e4b640 1 Processor -- start 2023-11-24T06:27:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.839+0000 7efe04e4b640 1 -- start start 2023-11-24T06:27:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efe04e4b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7efe0015d4b0 0x7efe000d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efe04e4b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe000cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efe04e4b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe000d0320 con 0x7efe000cddf0 2023-11-24T06:27:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efe04e4b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000b2730 con 0x7efe0015d4b0 2023-11-24T06:27:34.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efe04e4b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000d1ec0 con 0x7efe000cf9a0 2023-11-24T06:27:34.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efdff7fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7efe0015d4b0 0x7efe000d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efdfeffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe000cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efdff7fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7efe0015d4b0 0x7efe000d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:48304/0 (socket says 172.21.15.16:48304) 2023-11-24T06:27:34.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efdff7fe640 1 -- 172.21.15.16:0/723689622 learned_addr learned my addr 172.21.15.16:0/723689622 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:34.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.841+0000 7efdfeffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe000cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:36774/0 (socket says 172.21.15.16:36774) 2023-11-24T06:27:34.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.842+0000 7efdfeffd640 1 -- 172.21.15.16:0/723689622 >> v2:172.21.15.173:3300/0 conn(0x7efe0015d4b0 msgr2=0x7efe000d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:34.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.842+0000 7efdfeffd640 1 --2- 172.21.15.16:0/723689622 >> v2:172.21.15.173:3300/0 conn(0x7efe0015d4b0 0x7efe000d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.842+0000 7efdfeffd640 1 -- 172.21.15.16:0/723689622 >> v1:172.21.15.173:6789/0 conn(0x7efe000cddf0 legacy=0x7efe000ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.842+0000 7efdfeffd640 1 -- 172.21.15.16:0/723689622 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe000b29d0 con 0x7efe000cf9a0 2023-11-24T06:27:34.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.844+0000 7efdfeffd640 1 --2- 172.21.15.16:0/723689622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe000cfde0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efde80097d0 tx=0x7efde802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ee6c44fc52a72f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.844+0000 7efdfe7fc640 1 -- 172.21.15.16:0/723689622 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8031800 con 0x7efe000cf9a0 2023-11-24T06:27:34.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.844+0000 7efdfe7fc640 1 -- 172.21.15.16:0/723689622 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde8031960 con 0x7efe000cf9a0 2023-11-24T06:27:34.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.844+0000 7efdfe7fc640 1 -- 172.21.15.16:0/723689622 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8031cf0 con 0x7efe000cf9a0 2023-11-24T06:27:34.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.845+0000 7efe04e4b640 1 -- 172.21.15.16:0/723689622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 msgr2=0x7efe000cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.845+0000 7efe04e4b640 1 --2- 172.21.15.16:0/723689622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe000cfde0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efde80097d0 tx=0x7efde802b8a0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.845+0000 7efe04e4b640 1 -- 172.21.15.16:0/723689622 shutdown_connections 2023-11-24T06:27:34.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.845+0000 7efe04e4b640 1 --2- 172.21.15.16:0/723689622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe000cfde0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.845+0000 7efe04e4b640 1 --2- 172.21.15.16:0/723689622 >> v2:172.21.15.173:3300/0 conn(0x7efe0015d4b0 0x7efe000d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.845+0000 7efe04e4b640 1 -- 172.21.15.16:0/723689622 >> 172.21.15.16:0/723689622 conn(0x7efe000d3000 msgr2=0x7efe000d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:34.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.845+0000 7efe04e4b640 1 -- 172.21.15.16:0/723689622 shutdown_connections 2023-11-24T06:27:34.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.845+0000 7efe04e4b640 1 -- 172.21.15.16:0/723689622 wait complete. 2023-11-24T06:27:34.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.847+0000 7efe04e4b640 1 Processor -- start 2023-11-24T06:27:34.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.847+0000 7efe04e4b640 1 -- start start 2023-11-24T06:27:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.848+0000 7efe04e4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe000cddf0 0x7efe000caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.848+0000 7efe04e4b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe0015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.848+0000 7efe04e4b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0015d4b0 0x7efe001f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.848+0000 7efe04e4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015dba0 con 0x7efe000cddf0 2023-11-24T06:27:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.848+0000 7efe04e4b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0009cdf0 con 0x7efe0015d4b0 2023-11-24T06:27:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.848+0000 7efe04e4b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000cb330 con 0x7efe000cf9a0 2023-11-24T06:27:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.848+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe000cddf0 0x7efe000caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.848+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe000cddf0 0x7efe000caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48254/0 (socket says 172.21.15.16:48254) 2023-11-24T06:27:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.848+0000 7efdff7fe640 1 -- 172.21.15.16:0/1428696059 learned_addr learned my addr 172.21.15.16:0/1428696059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdfeffd640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe0015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdfffff640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0015d4b0 0x7efe001f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdff7fe640 1 -- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 msgr2=0x7efe0015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdff7fe640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe0015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdff7fe640 1 -- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0015d4b0 msgr2=0x7efe001f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdff7fe640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0015d4b0 0x7efe001f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdff7fe640 1 -- 172.21.15.16:0/1428696059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde8009400 con 0x7efe000cddf0 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdfffff640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0015d4b0 0x7efe001f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdff7fe640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe000cddf0 0x7efe000caab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdf000efc0 tx=0x7efdf000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdf7fff640 1 -- 172.21.15.16:0/1428696059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0009b60 con 0x7efe000cddf0 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdf7fff640 1 -- 172.21.15.16:0/1428696059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000f040 con 0x7efe000cddf0 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0015da10 con 0x7efe000cddf0 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.849+0000 7efdf7fff640 1 -- 172.21.15.16:0/1428696059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00092e0 con 0x7efe000cddf0 2023-11-24T06:27:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.850+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe000cb8f0 con 0x7efe000cddf0 2023-11-24T06:27:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.851+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe00067450 con 0x7efe000cddf0 2023-11-24T06:27:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.852+0000 7efdf7fff640 1 -- 172.21.15.16:0/1428696059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efdf0014480 con 0x7efe000cddf0 2023-11-24T06:27:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.853+0000 7efdf7fff640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efde006ae40 0x7efde006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.853+0000 7efdf7fff640 1 -- 172.21.15.16:0/1428696059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdf0089dd0 con 0x7efe000cddf0 2023-11-24T06:27:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.853+0000 7efdfeffd640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efde006ae40 0x7efde006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:34.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.854+0000 7efdfeffd640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efde006ae40 0x7efde006d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efde8036000 tx=0x7efde803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:34.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:34.857+0000 7efdf7fff640 1 -- 172.21.15.16:0/1428696059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efdf0054df0 con 0x7efe000cddf0 2023-11-24T06:27:35.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.057+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe000b2040 con 0x7efe000cddf0 2023-11-24T06:27:35.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.059+0000 7efdf7fff640 1 -- 172.21.15.16:0/1428696059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7efdf008b020 con 0x7efe000cddf0 2023-11-24T06:27:35.062 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.061+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efde006ae40 msgr2=0x7efde006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.061+0000 7efe04e4b640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efde006ae40 0x7efde006d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efde8036000 tx=0x7efde803c040 comp rx=0 tx=0).stop 2023-11-24T06:27:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.061+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe000cddf0 msgr2=0x7efe000caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.061+0000 7efe04e4b640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe000cddf0 0x7efe000caab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efdf000efc0 tx=0x7efdf000c490 comp rx=0 tx=0).stop 2023-11-24T06:27:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.062+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 shutdown_connections 2023-11-24T06:27:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.062+0000 7efe04e4b640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7efde006ae40 0x7efde006d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.062+0000 7efe04e4b640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe0015d4b0 0x7efe001f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.062+0000 7efe04e4b640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe000cf9a0 0x7efe0015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.062+0000 7efe04e4b640 1 --2- 172.21.15.16:0/1428696059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe000cddf0 0x7efe000caab0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.062+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 >> 172.21.15.16:0/1428696059 conn(0x7efe000d3000 msgr2=0x7efe0006bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.062+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 shutdown_connections 2023-11-24T06:27:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.062+0000 7efe04e4b640 1 -- 172.21.15.16:0/1428696059 wait complete. 2023-11-24T06:27:35.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.306+0000 7f6bdd133640 1 Processor -- start 2023-11-24T06:27:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.307+0000 7f6bdd133640 1 -- start start 2023-11-24T06:27:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.308+0000 7f6bdd133640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd813df80 0x7f6bd815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.308+0000 7f6bdd133640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6bd813e360 0x7f6bd80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.308+0000 7f6bdd133640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd806c470 con 0x7f6bd815f9c0 2023-11-24T06:27:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.308+0000 7f6bdd133640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd806c750 con 0x7f6bd813e360 2023-11-24T06:27:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.308+0000 7f6bdd133640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd806ca30 con 0x7f6bd813df80 2023-11-24T06:27:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.309+0000 7f6bd759e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6bd815f9c0 0x7f6bd81408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52600/0 (socket says 172.21.15.16:52600) 2023-11-24T06:27:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.309+0000 7f6bd759e640 1 -- 172.21.15.16:0/3519892056 learned_addr learned my addr 172.21.15.16:0/3519892056 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.309+0000 7f6bcffff640 1 --2- 172.21.15.16:0/3519892056 >> v2:172.21.15.173:3300/0 conn(0x7f6bd813e360 0x7f6bd80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.309+0000 7f6bd6d9d640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd813df80 0x7f6bd815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.309+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543757020 0 0) 0x7f6bd806c470 con 0x7f6bd815f9c0 2023-11-24T06:27:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc4003520 con 0x7f6bd815f9c0 2023-11-24T06:27:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2471951696 0 0) 0x7f6bc4003520 con 0x7f6bd815f9c0 2023-11-24T06:27:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd813df80 msgr2=0x7f6bd815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd813df80 0x7f6bd815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 >> v2:172.21.15.173:3300/0 conn(0x7f6bd813e360 msgr2=0x7f6bd80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 --2- 172.21.15.16:0/3519892056 >> v2:172.21.15.173:3300/0 conn(0x7f6bd813e360 0x7f6bd80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd806cd10 con 0x7f6bd815f9c0 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f6bd00019f0 con 0x7f6bd815f9c0 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 >> v1:172.21.15.16:6789/0 conn(0x7f6bd815f9c0 legacy=0x7f6bd81408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd813df80 0x7f6bc4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4003ef0 0x7f6bc40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc4004d00 0x7f6bc4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd806c750 con 0x7f6bc4004d00 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd806ca30 con 0x7f6bc4003ef0 2023-11-24T06:27:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc400b980 con 0x7f6bd813df80 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.310+0000 7f6bd759e640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc4004d00 0x7f6bc4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.311+0000 7f6bd6d9d640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd813df80 0x7f6bc4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.311+0000 7f6bcffff640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4003ef0 0x7f6bc40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.311+0000 7f6bd759e640 1 -- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd813df80 msgr2=0x7f6bc4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.311+0000 7f6bd759e640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd813df80 0x7f6bc4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.311+0000 7f6bd759e640 1 -- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4003ef0 msgr2=0x7f6bc40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.311+0000 7f6bd759e640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4003ef0 0x7f6bc40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.311+0000 7f6bd759e640 1 -- 172.21.15.16:0/3519892056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc4003520 con 0x7f6bc4004d00 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.312+0000 7f6bd759e640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc4004d00 0x7f6bc4005180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000a8d0 tx=0x7f6bc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef213d1b21e400a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.312+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0034630 con 0x7f6bc4004d00 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.312+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd002f070 con 0x7f6bc4004d00 2023-11-24T06:27:35.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.312+0000 7f6bd659c640 1 -- 172.21.15.16:0/3519892056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd00349d0 con 0x7f6bc4004d00 2023-11-24T06:27:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.313+0000 7f6bdd133640 1 -- 172.21.15.16:0/3519892056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc4004d00 msgr2=0x7f6bc4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.313+0000 7f6bdd133640 1 --2- 172.21.15.16:0/3519892056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc4004d00 0x7f6bc4005180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000a8d0 tx=0x7f6bc40057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.313+0000 7f6bd759e640 1 -- 172.21.15.16:0/3519892056 reap_dead start 2023-11-24T06:27:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.313+0000 7f6bdd133640 1 -- 172.21.15.16:0/3519892056 shutdown_connections 2023-11-24T06:27:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.313+0000 7f6bdd133640 1 -- 172.21.15.16:0/3519892056 >> 172.21.15.16:0/3519892056 conn(0x7f6bd806a800 msgr2=0x7f6bd806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.313+0000 7f6bdd133640 1 -- 172.21.15.16:0/3519892056 shutdown_connections 2023-11-24T06:27:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.313+0000 7f6bdd133640 1 -- 172.21.15.16:0/3519892056 wait complete. 2023-11-24T06:27:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.315+0000 7f6bdd133640 1 Processor -- start 2023-11-24T06:27:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.315+0000 7f6bdd133640 1 -- start start 2023-11-24T06:27:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.316+0000 7f6bdd133640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4004d00 0x7f6bd812f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.316+0000 7f6bdd133640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd814d690 0x7f6bd814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.316+0000 7f6bdd133640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd815d850 0x7f6bd815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.316+0000 7f6bdd133640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd814daf0 con 0x7f6bd814d690 2023-11-24T06:27:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.316+0000 7f6bdd133640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd815cab0 con 0x7f6bc4004d00 2023-11-24T06:27:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.316+0000 7f6bdd133640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd8150270 con 0x7f6bd815d850 2023-11-24T06:27:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.317+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd814d690 0x7f6bd814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.317+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd814d690 0x7f6bd814fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48272/0 (socket says 172.21.15.16:48272) 2023-11-24T06:27:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.317+0000 7f6bcffff640 1 -- 172.21.15.16:0/4154929407 learned_addr learned my addr 172.21.15.16:0/4154929407 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.317+0000 7f6bcffff640 1 -- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd815d850 msgr2=0x7f6bd815dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.317+0000 7f6bcffff640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd815d850 0x7f6bd815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.317+0000 7f6bcffff640 1 -- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4004d00 msgr2=0x7f6bd812f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.317+0000 7f6bcffff640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4004d00 0x7f6bd812f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.317+0000 7f6bcffff640 1 -- 172.21.15.16:0/4154929407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd815cd70 con 0x7f6bd814d690 2023-11-24T06:27:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.318+0000 7f6bcffff640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd814d690 0x7f6bd814fa20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0011600 tx=0x7f6bc0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.318+0000 7f6bcf7fe640 1 -- 172.21.15.16:0/4154929407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0015870 con 0x7f6bd814d690 2023-11-24T06:27:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.318+0000 7f6bcf7fe640 1 -- 172.21.15.16:0/4154929407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc00159d0 con 0x7f6bd814d690 2023-11-24T06:27:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.318+0000 7f6bcf7fe640 1 -- 172.21.15.16:0/4154929407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0015d40 con 0x7f6bd814d690 2023-11-24T06:27:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.318+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd815d050 con 0x7f6bd814d690 2023-11-24T06:27:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.318+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd814ff60 con 0x7f6bd814d690 2023-11-24T06:27:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.319+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd80675d0 con 0x7f6bd814d690 2023-11-24T06:27:35.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.320+0000 7f6bcf7fe640 1 -- 172.21.15.16:0/4154929407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bc000f490 con 0x7f6bd814d690 2023-11-24T06:27:35.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.321+0000 7f6bcf7fe640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bb806ae70 0x7f6bb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.322+0000 7f6bcf7fe640 1 -- 172.21.15.16:0/4154929407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bc0089a60 con 0x7f6bd814d690 2023-11-24T06:27:35.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.322+0000 7f6bd6d9d640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bb806ae70 0x7f6bb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.323+0000 7f6bd6d9d640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bb806ae70 0x7f6bb806d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009910 tx=0x7f6bc8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.326+0000 7f6bcf7fe640 1 -- 172.21.15.16:0/4154929407 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bc0054b00 con 0x7f6bd814d690 2023-11-24T06:27:35.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.520+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bd8150570 con 0x7f6bd814d690 2023-11-24T06:27:35.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.523+0000 7f6bcf7fe640 1 -- 172.21.15.16:0/4154929407 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f6bc0022090 con 0x7f6bd814d690 2023-11-24T06:27:35.526 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:35.527 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.525+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bb806ae70 msgr2=0x7f6bb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.525+0000 7f6bdd133640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bb806ae70 0x7f6bb806d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009910 tx=0x7f6bc8008040 comp rx=0 tx=0).stop 2023-11-24T06:27:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd814d690 msgr2=0x7f6bd814fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd814d690 0x7f6bd814fa20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0011600 tx=0x7f6bc0011ac0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 shutdown_connections 2023-11-24T06:27:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bb806ae70 0x7f6bb806d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bd815d850 0x7f6bd815dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bd814d690 0x7f6bd814fa20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 --2- 172.21.15.16:0/4154929407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4004d00 0x7f6bd812f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 >> 172.21.15.16:0/4154929407 conn(0x7f6bd806a800 msgr2=0x7f6bd80b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 shutdown_connections 2023-11-24T06:27:35.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.526+0000 7f6bdd133640 1 -- 172.21.15.16:0/4154929407 wait complete. 2023-11-24T06:27:35.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:35.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.769+0000 7fbd4517c640 1 Processor -- start 2023-11-24T06:27:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.770+0000 7fbd4517c640 1 -- start start 2023-11-24T06:27:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.771+0000 7fbd4517c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd400adf20 0x7fbd40070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.771+0000 7fbd4517c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400ccb50 0x7fbd40143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.771+0000 7fbd4517c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4015f4e0 con 0x7fbd4015bf10 2023-11-24T06:27:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.771+0000 7fbd4517c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd4015f970 con 0x7fbd400adf20 2023-11-24T06:27:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.771+0000 7fbd4517c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd40071130 con 0x7fbd400ccb50 2023-11-24T06:27:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.772+0000 7fbd3ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd400adf20 0x7fbd40070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.772+0000 7fbd3ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd400adf20 0x7fbd40070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48288/0 (socket says 172.21.15.16:48288) 2023-11-24T06:27:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.772+0000 7fbd3ffff640 1 -- 172.21.15.16:0/755829249 learned_addr learned my addr 172.21.15.16:0/755829249 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.772+0000 7fbd4497b640 1 --2- 172.21.15.16:0/755829249 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400ccb50 0x7fbd40143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.772+0000 7fbd3ffff640 1 -- 172.21.15.16:0/755829249 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400ccb50 msgr2=0x7fbd40143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.772+0000 7fbd3ffff640 1 --2- 172.21.15.16:0/755829249 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400ccb50 0x7fbd40143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.772+0000 7fbd3ffff640 1 -- 172.21.15.16:0/755829249 >> v1:172.21.15.173:6789/0 conn(0x7fbd4015bf10 legacy=0x7fbd400cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.772+0000 7fbd3ffff640 1 -- 172.21.15.16:0/755829249 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4015c390 con 0x7fbd400adf20 2023-11-24T06:27:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.773+0000 7fbd3ffff640 1 --2- 172.21.15.16:0/755829249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400adf20 0x7fbd40070a30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009870 tx=0x7fbd3002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316741129e25fcd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.774+0000 7fbd3f7fe640 1 -- 172.21.15.16:0/755829249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd300317c0 con 0x7fbd400adf20 2023-11-24T06:27:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.774+0000 7fbd3f7fe640 1 -- 172.21.15.16:0/755829249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd30031920 con 0x7fbd400adf20 2023-11-24T06:27:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.774+0000 7fbd3f7fe640 1 -- 172.21.15.16:0/755829249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd30031cb0 con 0x7fbd400adf20 2023-11-24T06:27:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.775+0000 7fbd4517c640 1 -- 172.21.15.16:0/755829249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400adf20 msgr2=0x7fbd40070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.775+0000 7fbd4517c640 1 --2- 172.21.15.16:0/755829249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400adf20 0x7fbd40070a30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009870 tx=0x7fbd3002b860 comp rx=0 tx=0).stop 2023-11-24T06:27:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.775+0000 7fbd4517c640 1 -- 172.21.15.16:0/755829249 shutdown_connections 2023-11-24T06:27:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.775+0000 7fbd4517c640 1 --2- 172.21.15.16:0/755829249 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400ccb50 0x7fbd40143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.775+0000 7fbd4517c640 1 --2- 172.21.15.16:0/755829249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400adf20 0x7fbd40070a30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.775+0000 7fbd4517c640 1 -- 172.21.15.16:0/755829249 >> 172.21.15.16:0/755829249 conn(0x7fbd4006a7d0 msgr2=0x7fbd4006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.775+0000 7fbd4517c640 1 -- 172.21.15.16:0/755829249 shutdown_connections 2023-11-24T06:27:35.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.775+0000 7fbd4517c640 1 -- 172.21.15.16:0/755829249 wait complete. 2023-11-24T06:27:35.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.777+0000 7fbd4517c640 1 Processor -- start 2023-11-24T06:27:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.778+0000 7fbd4517c640 1 -- start start 2023-11-24T06:27:35.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.779+0000 7fbd4517c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400adf20 0x7fbd40159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.779+0000 7fbd4517c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd400ccb50 0x7fbd4013edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.779+0000 7fbd4517c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd4015bf10 0x7fbd401f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.779+0000 7fbd4517c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4015a4a0 con 0x7fbd4015bf10 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.779+0000 7fbd4517c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4013f5f0 con 0x7fbd400ccb50 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.779+0000 7fbd4517c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd401591a0 con 0x7fbd400adf20 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.779+0000 7fbd4497b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd4015bf10 0x7fbd401f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.779+0000 7fbd4497b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd4015bf10 0x7fbd401f6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48296/0 (socket says 172.21.15.16:48296) 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.779+0000 7fbd4497b640 1 -- 172.21.15.16:0/3086037920 learned_addr learned my addr 172.21.15.16:0/3086037920 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.780+0000 7fbd37fff640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd400ccb50 0x7fbd4013edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.780+0000 7fbd3ffff640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400adf20 0x7fbd40159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.780+0000 7fbd4497b640 1 -- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400adf20 msgr2=0x7fbd40159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.780+0000 7fbd4497b640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400adf20 0x7fbd40159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.780+0000 7fbd4497b640 1 -- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd400ccb50 msgr2=0x7fbd4013edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.780+0000 7fbd4497b640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd400ccb50 0x7fbd4013edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.780+0000 7fbd4497b640 1 -- 172.21.15.16:0/3086037920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd300094d0 con 0x7fbd4015bf10 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.780+0000 7fbd4497b640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd4015bf10 0x7fbd401f6c30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd38007e30 tx=0x7fbd3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.781+0000 7fbd3d7fa640 1 -- 172.21.15.16:0/3086037920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38014620 con 0x7fbd4015bf10 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.781+0000 7fbd3d7fa640 1 -- 172.21.15.16:0/3086037920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3800ee90 con 0x7fbd4015bf10 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.781+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd40159480 con 0x7fbd4015bf10 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.781+0000 7fbd3d7fa640 1 -- 172.21.15.16:0/3086037920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd380149c0 con 0x7fbd4015bf10 2023-11-24T06:27:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.781+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd40159a40 con 0x7fbd4015bf10 2023-11-24T06:27:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.782+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd40159c60 con 0x7fbd4015bf10 2023-11-24T06:27:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.783+0000 7fbd3d7fa640 1 -- 172.21.15.16:0/3086037920 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd38014b20 con 0x7fbd4015bf10 2023-11-24T06:27:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.785+0000 7fbd3d7fa640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd2006ae90 0x7fbd2006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.785+0000 7fbd3d7fa640 1 -- 172.21.15.16:0/3086037920 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd3808a050 con 0x7fbd4015bf10 2023-11-24T06:27:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.785+0000 7fbd3ffff640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd2006ae90 0x7fbd2006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.786+0000 7fbd3ffff640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd2006ae90 0x7fbd2006d350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd30036000 tx=0x7fbd3003d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.788+0000 7fbd3d7fa640 1 -- 172.21.15.16:0/3086037920 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd380bce90 con 0x7fbd4015bf10 2023-11-24T06:27:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.982+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd40067450 con 0x7fbd4015bf10 2023-11-24T06:27:35.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.983+0000 7fbd3d7fa640 1 -- 172.21.15.16:0/3086037920 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd38055050 con 0x7fbd4015bf10 2023-11-24T06:27:35.986 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.985+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd2006ae90 msgr2=0x7fbd2006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.985+0000 7fbd4517c640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd2006ae90 0x7fbd2006d350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd30036000 tx=0x7fbd3003d040 comp rx=0 tx=0).stop 2023-11-24T06:27:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.985+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd4015bf10 msgr2=0x7fbd401f6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.985+0000 7fbd4517c640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd4015bf10 0x7fbd401f6c30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd38007e30 tx=0x7fbd3800e4b0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.986+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 shutdown_connections 2023-11-24T06:27:35.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.986+0000 7fbd4517c640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd2006ae90 0x7fbd2006d350 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.986+0000 7fbd4517c640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd4015bf10 0x7fbd401f6c30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.986+0000 7fbd4517c640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd400ccb50 0x7fbd4013edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.986+0000 7fbd4517c640 1 --2- 172.21.15.16:0/3086037920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd400adf20 0x7fbd40159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.986+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 >> 172.21.15.16:0/3086037920 conn(0x7fbd4006a7d0 msgr2=0x7fbd40071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.986+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 shutdown_connections 2023-11-24T06:27:35.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:35.986+0000 7fbd4517c640 1 -- 172.21.15.16:0/3086037920 wait complete. 2023-11-24T06:27:36.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.230+0000 7f759f0bb640 1 Processor -- start 2023-11-24T06:27:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.231+0000 7f759f0bb640 1 -- start start 2023-11-24T06:27:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.232+0000 7f759f0bb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f75980cf9a0 0x7f75980cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.232+0000 7f759f0bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75980d0320 con 0x7f75980b31d0 2023-11-24T06:27:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.232+0000 7f759f0bb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75980d1c60 con 0x7f759815d4b0 2023-11-24T06:27:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.232+0000 7f759f0bb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75980b2730 con 0x7f75980cf9a0 2023-11-24T06:27:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.233+0000 7f759e8ba640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f75980b31d0 0x7f75980b3660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52606/0 (socket says 172.21.15.16:52606) 2023-11-24T06:27:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.233+0000 7f759e8ba640 1 -- 172.21.15.16:0/1917329098 learned_addr learned my addr 172.21.15.16:0/1917329098 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.233+0000 7f759d8b8640 1 --2- 172.21.15.16:0/1917329098 >> v2:172.21.15.173:3300/0 conn(0x7f75980cf9a0 0x7f75980cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.233+0000 7f759d0b7640 1 -- 172.21.15.16:0/1917329098 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5324181 0 0) 0x7f75980d0320 con 0x7f75980b31d0 2023-11-24T06:27:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.233+0000 7f759d0b7640 1 -- 172.21.15.16:0/1917329098 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7584003520 con 0x7f75980b31d0 2023-11-24T06:27:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.234+0000 7f759d8b8640 1 -- 172.21.15.16:0/1917329098 >> v1:172.21.15.173:6789/0 conn(0x7f759815d4b0 legacy=0x7f75980d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.234+0000 7f759d8b8640 1 -- 172.21.15.16:0/1917329098 >> v1:172.21.15.16:6789/0 conn(0x7f75980b31d0 legacy=0x7f75980b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.234+0000 7f759d0b7640 1 -- 172.21.15.16:0/1917329098 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 96149245 0 0) 0x7f7584003520 con 0x7f75980b31d0 2023-11-24T06:27:36.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.234+0000 7f759d8b8640 1 -- 172.21.15.16:0/1917329098 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980d1ec0 con 0x7f75980cf9a0 2023-11-24T06:27:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.235+0000 7f759d8b8640 1 --2- 172.21.15.16:0/1917329098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75980cf9a0 0x7f75980cfde0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f75880097a0 tx=0x7f758802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=821131706546eb55 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.236+0000 7f759d0b7640 1 -- 172.21.15.16:0/1917329098 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758802f050 con 0x7f75980cf9a0 2023-11-24T06:27:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.236+0000 7f759d0b7640 1 -- 172.21.15.16:0/1917329098 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588031500 con 0x7f75980cf9a0 2023-11-24T06:27:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.236+0000 7f759d0b7640 1 -- 172.21.15.16:0/1917329098 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590001670 con 0x7f75980cf9a0 2023-11-24T06:27:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.236+0000 7f759f0bb640 1 -- 172.21.15.16:0/1917329098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75980cf9a0 msgr2=0x7f75980cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.236+0000 7f759f0bb640 1 --2- 172.21.15.16:0/1917329098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75980cf9a0 0x7f75980cfde0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f75880097a0 tx=0x7f758802b820 comp rx=0 tx=0).stop 2023-11-24T06:27:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.237+0000 7f759f0bb640 1 -- 172.21.15.16:0/1917329098 shutdown_connections 2023-11-24T06:27:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.237+0000 7f759f0bb640 1 --2- 172.21.15.16:0/1917329098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75980cf9a0 0x7f75980cfde0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.237+0000 7f759f0bb640 1 -- 172.21.15.16:0/1917329098 >> 172.21.15.16:0/1917329098 conn(0x7f75980d3000 msgr2=0x7f75980d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.237+0000 7f759f0bb640 1 -- 172.21.15.16:0/1917329098 shutdown_connections 2023-11-24T06:27:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.237+0000 7f759f0bb640 1 -- 172.21.15.16:0/1917329098 wait complete. 2023-11-24T06:27:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.238+0000 7f759f0bb640 1 Processor -- start 2023-11-24T06:27:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.239+0000 7f759f0bb640 1 -- start start 2023-11-24T06:27:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759f0bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75980b31d0 0x7f75980a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759f0bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75980cf9a0 0x7f75980cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759f0bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f759815d4b0 0x7f75980b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759f0bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759809f600 con 0x7f75980cf9a0 2023-11-24T06:27:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759f0bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759809f8a0 con 0x7f759815d4b0 2023-11-24T06:27:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759f0bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759809fb40 con 0x7f75980b31d0 2023-11-24T06:27:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759d8b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75980cf9a0 0x7f75980cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759d8b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75980cf9a0 0x7f75980cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48312/0 (socket says 172.21.15.16:48312) 2023-11-24T06:27:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759d8b8640 1 -- 172.21.15.16:0/2298185101 learned_addr learned my addr 172.21.15.16:0/2298185101 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759e8ba640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f759815d4b0 0x7f75980b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759d8b8640 1 -- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75980b31d0 msgr2=0x7f75980a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759d8b8640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75980b31d0 0x7f75980a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759d8b8640 1 -- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f759815d4b0 msgr2=0x7f75980b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759d8b8640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f759815d4b0 0x7f75980b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759d8b8640 1 -- 172.21.15.16:0/2298185101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7588009400 con 0x7f75980cf9a0 2023-11-24T06:27:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.240+0000 7f759e8ba640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f759815d4b0 0x7f75980b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.241+0000 7f759d8b8640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75980cf9a0 0x7f75980cb480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7588036040 tx=0x7f75880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.241+0000 7f758effd640 1 -- 172.21.15.16:0/2298185101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758803d940 con 0x7f75980cf9a0 2023-11-24T06:27:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.241+0000 7f758effd640 1 -- 172.21.15.16:0/2298185101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758803daa0 con 0x7f75980cf9a0 2023-11-24T06:27:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.241+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759809fe20 con 0x7f75980cf9a0 2023-11-24T06:27:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.241+0000 7f758effd640 1 -- 172.21.15.16:0/2298185101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758803de50 con 0x7f75980cf9a0 2023-11-24T06:27:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.241+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75980a03e0 con 0x7f75980cf9a0 2023-11-24T06:27:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.243+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75980a05d0 con 0x7f75980cf9a0 2023-11-24T06:27:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.244+0000 7f758effd640 1 -- 172.21.15.16:0/2298185101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f758803a040 con 0x7f75980cf9a0 2023-11-24T06:27:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.245+0000 7f758effd640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f757c06ae70 0x7f757c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.245+0000 7f758effd640 1 -- 172.21.15.16:0/2298185101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75880ab290 con 0x7f75980cf9a0 2023-11-24T06:27:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.245+0000 7f759e0b9640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f757c06ae70 0x7f757c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.247+0000 7f759e0b9640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f757c06ae70 0x7f757c06d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75940012e0 tx=0x7f7594007d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.249+0000 7f758effd640 1 -- 172.21.15.16:0/2298185101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7588076330 con 0x7f75980cf9a0 2023-11-24T06:27:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.442+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7598067450 con 0x7f75980cf9a0 2023-11-24T06:27:36.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.443+0000 7f758effd640 1 -- 172.21.15.16:0/2298185101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7588079950 con 0x7f75980cf9a0 2023-11-24T06:27:36.447 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f757c06ae70 msgr2=0x7f757c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f757c06ae70 0x7f757c06d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75940012e0 tx=0x7f7594007d30 comp rx=0 tx=0).stop 2023-11-24T06:27:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75980cf9a0 msgr2=0x7f75980cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75980cf9a0 0x7f75980cb480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7588036040 tx=0x7f75880046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 shutdown_connections 2023-11-24T06:27:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f757c06ae70 0x7f757c06d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f759815d4b0 0x7f75980b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75980cf9a0 0x7f75980cb480 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 --2- 172.21.15.16:0/2298185101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f75980b31d0 0x7f75980a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 >> 172.21.15.16:0/2298185101 conn(0x7f75980d3000 msgr2=0x7f75980b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 shutdown_connections 2023-11-24T06:27:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.446+0000 7f759f0bb640 1 -- 172.21.15.16:0/2298185101 wait complete. 2023-11-24T06:27:36.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.696+0000 7f6d54015640 1 Processor -- start 2023-11-24T06:27:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.696+0000 7f6d54015640 1 -- start start 2023-11-24T06:27:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.697+0000 7f6d54015640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d4c15d940 0x7f6d4c06b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.697+0000 7f6d54015640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4c0c7290 con 0x7f6d4c15ef70 2023-11-24T06:27:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.697+0000 7f6d54015640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4c0c7ef0 con 0x7f6d4c0c7a90 2023-11-24T06:27:36.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.697+0000 7f6d54015640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4c06d1d0 con 0x7f6d4c15d940 2023-11-24T06:27:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.698+0000 7f6d51db1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d4c15d940 0x7f6d4c06b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.698+0000 7f6d51db1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d4c15d940 0x7f6d4c06b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48326/0 (socket says 172.21.15.16:48326) 2023-11-24T06:27:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.698+0000 7f6d51db1640 1 -- 172.21.15.16:0/3709142226 learned_addr learned my addr 172.21.15.16:0/3709142226 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.698+0000 7f6d51db1640 1 -- 172.21.15.16:0/3709142226 >> v1:172.21.15.173:6789/0 conn(0x7f6d4c0c7a90 legacy=0x7f6d4c15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:36.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.698+0000 7f6d51db1640 1 -- 172.21.15.16:0/3709142226 >> v1:172.21.15.16:6789/0 conn(0x7f6d4c15ef70 legacy=0x7f6d4c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.698+0000 7f6d50daf640 1 -- 172.21.15.16:0/3709142226 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908577601 0 0) 0x7f6d4c0c7290 con 0x7f6d4c15ef70 2023-11-24T06:27:36.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.698+0000 7f6d51db1640 1 -- 172.21.15.16:0/3709142226 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c06bb80 con 0x7f6d4c15d940 2023-11-24T06:27:36.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.700+0000 7f6d51db1640 1 --2- 172.21.15.16:0/3709142226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 0x7f6d4c06b4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009e50 tx=0x7f6d4802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6349a445e25abaad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.700+0000 7f6d50daf640 1 -- 172.21.15.16:0/3709142226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4802ca80 con 0x7f6d4c15d940 2023-11-24T06:27:36.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.700+0000 7f6d50daf640 1 -- 172.21.15.16:0/3709142226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4802cbe0 con 0x7f6d4c15d940 2023-11-24T06:27:36.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.700+0000 7f6d50daf640 1 -- 172.21.15.16:0/3709142226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d480343f0 con 0x7f6d4c15d940 2023-11-24T06:27:36.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.701+0000 7f6d54015640 1 -- 172.21.15.16:0/3709142226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 msgr2=0x7f6d4c06b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.701+0000 7f6d54015640 1 --2- 172.21.15.16:0/3709142226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 0x7f6d4c06b4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009e50 tx=0x7f6d4802bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.701+0000 7f6d54015640 1 -- 172.21.15.16:0/3709142226 shutdown_connections 2023-11-24T06:27:36.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.701+0000 7f6d54015640 1 --2- 172.21.15.16:0/3709142226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 0x7f6d4c06b4f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.701+0000 7f6d54015640 1 -- 172.21.15.16:0/3709142226 >> 172.21.15.16:0/3709142226 conn(0x7f6d4c0b2ce0 msgr2=0x7f6d4c0b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.701+0000 7f6d54015640 1 -- 172.21.15.16:0/3709142226 shutdown_connections 2023-11-24T06:27:36.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.701+0000 7f6d54015640 1 -- 172.21.15.16:0/3709142226 wait complete. 2023-11-24T06:27:36.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.703+0000 7f6d54015640 1 Processor -- start 2023-11-24T06:27:36.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.703+0000 7f6d54015640 1 -- start start 2023-11-24T06:27:36.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.704+0000 7f6d54015640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d4c0c7a90 0x7f6d4c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.704+0000 7f6d54015640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 0x7f6d4c0cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.704+0000 7f6d54015640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4c15ef70 0x7f6d4c0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.704+0000 7f6d54015640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0cd200 con 0x7f6d4c15d940 2023-11-24T06:27:36.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.704+0000 7f6d54015640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0ccf20 con 0x7f6d4c15ef70 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.704+0000 7f6d54015640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0b3f30 con 0x7f6d4c0c7a90 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.704+0000 7f6d515b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 0x7f6d4c0cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.704+0000 7f6d515b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 0x7f6d4c0cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48334/0 (socket says 172.21.15.16:48334) 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.704+0000 7f6d515b0640 1 -- 172.21.15.16:0/2647250566 learned_addr learned my addr 172.21.15.16:0/2647250566 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d51db1640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d4c0c7a90 0x7f6d4c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d525b2640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4c15ef70 0x7f6d4c0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d515b0640 1 -- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d4c0c7a90 msgr2=0x7f6d4c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d515b0640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d4c0c7a90 0x7f6d4c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d515b0640 1 -- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4c15ef70 msgr2=0x7f6d4c0cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d515b0640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4c15ef70 0x7f6d4c0cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d515b0640 1 -- 172.21.15.16:0/2647250566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d48009b30 con 0x7f6d4c15d940 2023-11-24T06:27:36.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d51db1640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d4c0c7a90 0x7f6d4c0cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:36.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d525b2640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4c15ef70 0x7f6d4c0cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:36.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d515b0640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 0x7f6d4c0cd380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00b3f0 tx=0x7f6d3c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d427fc640 1 -- 172.21.15.16:0/2647250566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c011830 con 0x7f6d4c15d940 2023-11-24T06:27:36.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d427fc640 1 -- 172.21.15.16:0/2647250566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c011990 con 0x7f6d4c15d940 2023-11-24T06:27:36.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d427fc640 1 -- 172.21.15.16:0/2647250566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c011d40 con 0x7f6d4c15d940 2023-11-24T06:27:36.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.705+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c0b4210 con 0x7f6d4c15d940 2023-11-24T06:27:36.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.706+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c0ccab0 con 0x7f6d4c15d940 2023-11-24T06:27:36.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.707+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c0b3d90 con 0x7f6d4c15d940 2023-11-24T06:27:36.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.708+0000 7f6d427fc640 1 -- 172.21.15.16:0/2647250566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d3c01a400 con 0x7f6d4c15d940 2023-11-24T06:27:36.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.709+0000 7f6d427fc640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6d3006ae40 0x7f6d3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:36.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.709+0000 7f6d427fc640 1 -- 172.21.15.16:0/2647250566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d3c089560 con 0x7f6d4c15d940 2023-11-24T06:27:36.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.709+0000 7f6d51db1640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6d3006ae40 0x7f6d3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:36.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.710+0000 7f6d51db1640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6d3006ae40 0x7f6d3006d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6d48035000 tx=0x7f6d48030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:36.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.713+0000 7f6d427fc640 1 -- 172.21.15.16:0/2647250566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d4c0b3d90 con 0x7f6d4c15d940 2023-11-24T06:27:36.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.910+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d4c0b4730 con 0x7f6d4c15d940 2023-11-24T06:27:36.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.911+0000 7f6d427fc640 1 -- 172.21.15.16:0/2647250566 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d4c0b4730 con 0x7f6d4c15d940 2023-11-24T06:27:36.915 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:36.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6d3006ae40 msgr2=0x7f6d3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6d3006ae40 0x7f6d3006d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6d48035000 tx=0x7f6d48030000 comp rx=0 tx=0).stop 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 msgr2=0x7f6d4c0cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 0x7f6d4c0cd380 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00b3f0 tx=0x7f6d3c00b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 shutdown_connections 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6d3006ae40 0x7f6d3006d300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d4c15ef70 0x7f6d4c0cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d4c15d940 0x7f6d4c0cd380 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 --2- 172.21.15.16:0/2647250566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d4c0c7a90 0x7f6d4c0cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 >> 172.21.15.16:0/2647250566 conn(0x7f6d4c0b2ce0 msgr2=0x7f6d4c0b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 shutdown_connections 2023-11-24T06:27:36.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:36.914+0000 7f6d54015640 1 -- 172.21.15.16:0/2647250566 wait complete. 2023-11-24T06:27:36.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.153+0000 7f9fd946d640 1 Processor -- start 2023-11-24T06:27:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.153+0000 7f9fd946d640 1 -- start start 2023-11-24T06:27:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.155+0000 7f9fd946d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fd4146770 0x7f9fd406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.155+0000 7f9fd946d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd40b2e10 con 0x7f9fd4071f00 2023-11-24T06:27:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.155+0000 7f9fd946d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd4146bc0 con 0x7f9fd4071410 2023-11-24T06:27:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.155+0000 7f9fd946d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd406cec0 con 0x7f9fd4146770 2023-11-24T06:27:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.155+0000 7f9fd8c6c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fd4146770 0x7f9fd406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.155+0000 7f9fd8c6c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fd4146770 0x7f9fd406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48350/0 (socket says 172.21.15.16:48350) 2023-11-24T06:27:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.155+0000 7f9fd8c6c640 1 -- 172.21.15.16:0/3286995760 learned_addr learned my addr 172.21.15.16:0/3286995760 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.156+0000 7f9fd8c6c640 1 -- 172.21.15.16:0/3286995760 >> v1:172.21.15.173:6789/0 conn(0x7f9fd4071410 legacy=0x7f9fd40717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.156+0000 7f9fd8c6c640 1 -- 172.21.15.16:0/3286995760 >> v1:172.21.15.16:6789/0 conn(0x7f9fd4071f00 legacy=0x7f9fd4164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.156+0000 7f9fd8c6c640 1 -- 172.21.15.16:0/3286995760 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd406d1a0 con 0x7f9fd4146770 2023-11-24T06:27:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.157+0000 7f9fd8c6c640 1 --2- 172.21.15.16:0/3286995760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 0x7f9fd406c7c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009e50 tx=0x7f9fcc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c393e407b67319c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.157+0000 7f9fd2ffd640 1 -- 172.21.15.16:0/3286995760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc02ca80 con 0x7f9fd4146770 2023-11-24T06:27:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.158+0000 7f9fd2ffd640 1 -- 172.21.15.16:0/3286995760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc02cbe0 con 0x7f9fd4146770 2023-11-24T06:27:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.158+0000 7f9fd2ffd640 1 -- 172.21.15.16:0/3286995760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0343f0 con 0x7f9fd4146770 2023-11-24T06:27:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.158+0000 7f9fd946d640 1 -- 172.21.15.16:0/3286995760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 msgr2=0x7f9fd406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.158+0000 7f9fd946d640 1 --2- 172.21.15.16:0/3286995760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 0x7f9fd406c7c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009e50 tx=0x7f9fcc02bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.158+0000 7f9fd946d640 1 -- 172.21.15.16:0/3286995760 shutdown_connections 2023-11-24T06:27:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.158+0000 7f9fd946d640 1 --2- 172.21.15.16:0/3286995760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 0x7f9fd406c7c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.158+0000 7f9fd946d640 1 -- 172.21.15.16:0/3286995760 >> 172.21.15.16:0/3286995760 conn(0x7f9fd406a800 msgr2=0x7f9fd406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.158+0000 7f9fd946d640 1 -- 172.21.15.16:0/3286995760 shutdown_connections 2023-11-24T06:27:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.159+0000 7f9fd946d640 1 -- 172.21.15.16:0/3286995760 wait complete. 2023-11-24T06:27:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.160+0000 7f9fd946d640 1 Processor -- start 2023-11-24T06:27:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.160+0000 7f9fd946d640 1 -- start start 2023-11-24T06:27:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.161+0000 7f9fd946d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fd4071410 0x7f9fd4142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.161+0000 7f9fd946d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fd4071f00 0x7f9fd40c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.161+0000 7f9fd946d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 0x7f9fd40c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd946d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41600c0 con 0x7f9fd4146770 2023-11-24T06:27:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd946d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd415f210 con 0x7f9fd4071f00 2023-11-24T06:27:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd946d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd415f4b0 con 0x7f9fd4071410 2023-11-24T06:27:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd8c6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 0x7f9fd40c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd8c6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 0x7f9fd40c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48354/0 (socket says 172.21.15.16:48354) 2023-11-24T06:27:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd8c6c640 1 -- 172.21.15.16:0/1500083544 learned_addr learned my addr 172.21.15.16:0/1500083544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd37fe640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fd4071f00 0x7f9fd40c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd8c6c640 1 -- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fd4071410 msgr2=0x7f9fd4142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd8c6c640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fd4071410 0x7f9fd4142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd8c6c640 1 -- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fd4071f00 msgr2=0x7f9fd40c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd8c6c640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fd4071f00 0x7f9fd40c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd8c6c640 1 -- 172.21.15.16:0/1500083544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc009b30 con 0x7f9fd4146770 2023-11-24T06:27:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.162+0000 7f9fd37fe640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fd4071f00 0x7f9fd40c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.163+0000 7f9fd8c6c640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 0x7f9fd40c4ef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc036040 tx=0x7f9fcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.163+0000 7f9fd0ff9640 1 -- 172.21.15.16:0/1500083544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc03e450 con 0x7f9fd4146770 2023-11-24T06:27:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.163+0000 7f9fd0ff9640 1 -- 172.21.15.16:0/1500083544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc03e5d0 con 0x7f9fd4146770 2023-11-24T06:27:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.163+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd415f790 con 0x7f9fd4146770 2023-11-24T06:27:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.163+0000 7f9fd0ff9640 1 -- 172.21.15.16:0/1500083544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc03e980 con 0x7f9fd4146770 2023-11-24T06:27:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.163+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd415fd50 con 0x7f9fd4146770 2023-11-24T06:27:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.164+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd4067450 con 0x7f9fd4146770 2023-11-24T06:27:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.165+0000 7f9fd0ff9640 1 -- 172.21.15.16:0/1500083544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9fcc02cd50 con 0x7f9fd4146770 2023-11-24T06:27:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.166+0000 7f9fd0ff9640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9fc006ae40 0x7f9fc006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.166+0000 7f9fd0ff9640 1 -- 172.21.15.16:0/1500083544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fcc0ac2b0 con 0x7f9fd4146770 2023-11-24T06:27:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.167+0000 7f9fd3fff640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9fc006ae40 0x7f9fc006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.168+0000 7f9fd3fff640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9fc006ae40 0x7f9fc006d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4003cc0 tx=0x7f9fc40079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.171+0000 7f9fd0ff9640 1 -- 172.21.15.16:0/1500083544 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fcc077490 con 0x7f9fd4146770 2023-11-24T06:27:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.366+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fd40ce360 con 0x7f9fd4146770 2023-11-24T06:27:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.368+0000 7f9fd0ff9640 1 -- 172.21.15.16:0/1500083544 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f9fcc07aab0 con 0x7f9fd4146770 2023-11-24T06:27:37.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:37.371 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.370+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9fc006ae40 msgr2=0x7f9fc006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.370+0000 7f9fd946d640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9fc006ae40 0x7f9fc006d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4003cc0 tx=0x7f9fc40079b0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.370+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 msgr2=0x7f9fd40c4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.370+0000 7f9fd946d640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 0x7f9fd40c4ef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc036040 tx=0x7f9fcc004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.371+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 shutdown_connections 2023-11-24T06:27:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.371+0000 7f9fd946d640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9fc006ae40 0x7f9fc006d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.371+0000 7f9fd946d640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd4146770 0x7f9fd40c4ef0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.371+0000 7f9fd946d640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fd4071f00 0x7f9fd40c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.371+0000 7f9fd946d640 1 --2- 172.21.15.16:0/1500083544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fd4071410 0x7f9fd4142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.371+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 >> 172.21.15.16:0/1500083544 conn(0x7f9fd406a800 msgr2=0x7f9fd41440a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.371+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 shutdown_connections 2023-11-24T06:27:37.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.371+0000 7f9fd946d640 1 -- 172.21.15.16:0/1500083544 wait complete. 2023-11-24T06:27:37.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:37.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:37.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.623+0000 7f3bb7fff640 1 Processor -- start 2023-11-24T06:27:37.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.623+0000 7f3bb7fff640 1 -- start start 2023-11-24T06:27:37.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.625+0000 7f3bb7fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3bb80adf20 0x7f3bb8070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.625+0000 7f3bb7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3bb80ccb50 0x7f3bb8143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.625+0000 7f3bb7fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb815f4e0 con 0x7f3bb815bf10 2023-11-24T06:27:37.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.625+0000 7f3bb7fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb815f970 con 0x7f3bb80ccb50 2023-11-24T06:27:37.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.625+0000 7f3bb7fff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb8071130 con 0x7f3bb80adf20 2023-11-24T06:27:37.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.625+0000 7f3bb67fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3bb815bf10 0x7f3bb80cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52614/0 (socket says 172.21.15.16:52614) 2023-11-24T06:27:37.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.625+0000 7f3bb67fc640 1 -- 172.21.15.16:0/2077153469 learned_addr learned my addr 172.21.15.16:0/2077153469 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:37.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.626+0000 7f3bb77fe640 1 --2- 172.21.15.16:0/2077153469 >> v2:172.21.15.16:3300/0 conn(0x7f3bb80ccb50 0x7f3bb8143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.626+0000 7f3bb6ffd640 1 --2- 172.21.15.16:0/2077153469 >> v2:172.21.15.173:3300/0 conn(0x7f3bb80adf20 0x7f3bb8070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.626+0000 7f3bb5ffb640 1 -- 172.21.15.16:0/2077153469 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330769474 0 0) 0x7f3bb815f4e0 con 0x7f3bb815bf10 2023-11-24T06:27:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.626+0000 7f3bb5ffb640 1 -- 172.21.15.16:0/2077153469 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ba0003520 con 0x7f3bb815bf10 2023-11-24T06:27:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.626+0000 7f3bb77fe640 1 -- 172.21.15.16:0/2077153469 >> v2:172.21.15.173:3300/0 conn(0x7f3bb80adf20 msgr2=0x7f3bb8070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.626+0000 7f3bb77fe640 1 --2- 172.21.15.16:0/2077153469 >> v2:172.21.15.173:3300/0 conn(0x7f3bb80adf20 0x7f3bb8070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.626+0000 7f3bb77fe640 1 -- 172.21.15.16:0/2077153469 >> v1:172.21.15.16:6789/0 conn(0x7f3bb815bf10 legacy=0x7f3bb80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.626+0000 7f3bb77fe640 1 -- 172.21.15.16:0/2077153469 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb815c390 con 0x7f3bb80ccb50 2023-11-24T06:27:37.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.627+0000 7f3bb77fe640 1 --2- 172.21.15.16:0/2077153469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb80ccb50 0x7f3bb8143d30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80b4150 tx=0x7f3bb002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6f7163353e781f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.628+0000 7f3bb5ffb640 1 -- 172.21.15.16:0/2077153469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb002cbf0 con 0x7f3bb80ccb50 2023-11-24T06:27:37.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.628+0000 7f3bb5ffb640 1 -- 172.21.15.16:0/2077153469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb002cd50 con 0x7f3bb80ccb50 2023-11-24T06:27:37.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.628+0000 7f3bb5ffb640 1 -- 172.21.15.16:0/2077153469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0034660 con 0x7f3bb80ccb50 2023-11-24T06:27:37.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.628+0000 7f3bb7fff640 1 -- 172.21.15.16:0/2077153469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb80ccb50 msgr2=0x7f3bb8143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.628+0000 7f3bb7fff640 1 --2- 172.21.15.16:0/2077153469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb80ccb50 0x7f3bb8143d30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80b4150 tx=0x7f3bb002be40 comp rx=0 tx=0).stop 2023-11-24T06:27:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.629+0000 7f3bb7fff640 1 -- 172.21.15.16:0/2077153469 shutdown_connections 2023-11-24T06:27:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.629+0000 7f3bb7fff640 1 --2- 172.21.15.16:0/2077153469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb80ccb50 0x7f3bb8143d30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.629+0000 7f3bb7fff640 1 --2- 172.21.15.16:0/2077153469 >> v2:172.21.15.173:3300/0 conn(0x7f3bb80adf20 0x7f3bb8070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.629+0000 7f3bb7fff640 1 -- 172.21.15.16:0/2077153469 >> 172.21.15.16:0/2077153469 conn(0x7f3bb806a7d0 msgr2=0x7f3bb806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.629+0000 7f3bb7fff640 1 -- 172.21.15.16:0/2077153469 shutdown_connections 2023-11-24T06:27:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.629+0000 7f3bb7fff640 1 -- 172.21.15.16:0/2077153469 wait complete. 2023-11-24T06:27:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.630+0000 7f3bb7fff640 1 Processor -- start 2023-11-24T06:27:37.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.631+0000 7f3bb7fff640 1 -- start start 2023-11-24T06:27:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb80adf20 0x7f3bb81f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb7fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb80ccb50 0x7f3bb813e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb815bf10 0x7f3bb81f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb7fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb81f6990 con 0x7f3bb815bf10 2023-11-24T06:27:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb7fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb81f6c70 con 0x7f3bb80adf20 2023-11-24T06:27:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb7fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb81f2c10 con 0x7f3bb80ccb50 2023-11-24T06:27:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb815bf10 0x7f3bb81f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb815bf10 0x7f3bb81f5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48372/0 (socket says 172.21.15.16:48372) 2023-11-24T06:27:37.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb77fe640 1 -- 172.21.15.16:0/3990445808 learned_addr learned my addr 172.21.15.16:0/3990445808 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb6ffd640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb80adf20 0x7f3bb81f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb67fc640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb80ccb50 0x7f3bb813e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb77fe640 1 -- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb80ccb50 msgr2=0x7f3bb813e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb77fe640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb80ccb50 0x7f3bb813e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb77fe640 1 -- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb80adf20 msgr2=0x7f3bb81f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb77fe640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb80adf20 0x7f3bb81f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb77fe640 1 -- 172.21.15.16:0/3990445808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb0009a70 con 0x7f3bb815bf10 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.632+0000 7f3bb6ffd640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb80adf20 0x7f3bb81f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.633+0000 7f3bb77fe640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb815bf10 0x7f3bb81f5f40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009e80 tx=0x7f3bb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.633+0000 7f3b9f7fe640 1 -- 172.21.15.16:0/3990445808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0040650 con 0x7f3bb815bf10 2023-11-24T06:27:37.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.633+0000 7f3b9f7fe640 1 -- 172.21.15.16:0/3990445808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb00407b0 con 0x7f3bb815bf10 2023-11-24T06:27:37.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.633+0000 7f3b9f7fe640 1 -- 172.21.15.16:0/3990445808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0040b60 con 0x7f3bb815bf10 2023-11-24T06:27:37.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.633+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb81f2ef0 con 0x7f3bb815bf10 2023-11-24T06:27:37.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.633+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb815e050 con 0x7f3bb815bf10 2023-11-24T06:27:37.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.634+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb81f3110 con 0x7f3bb815bf10 2023-11-24T06:27:37.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.636+0000 7f3b9f7fe640 1 -- 172.21.15.16:0/3990445808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3bb0040cc0 con 0x7f3bb815bf10 2023-11-24T06:27:37.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.636+0000 7f3b9f7fe640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ba406ae40 0x7f3ba406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:37.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.637+0000 7f3b9f7fe640 1 -- 172.21.15.16:0/3990445808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bb00ac330 con 0x7f3bb815bf10 2023-11-24T06:27:37.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.637+0000 7f3bb6ffd640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ba406ae40 0x7f3ba406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:37.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.638+0000 7f3bb6ffd640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ba406ae40 0x7f3ba406d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8006fd0 tx=0x7f3ba8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:37.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.641+0000 7f3b9f7fe640 1 -- 172.21.15.16:0/3990445808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3bb0077470 con 0x7f3bb815bf10 2023-11-24T06:27:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.834+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3bb815e1e0 con 0x7f3bb815bf10 2023-11-24T06:27:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.835+0000 7f3b9f7fe640 1 -- 172.21.15.16:0/3990445808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3bb0009450 con 0x7f3bb815bf10 2023-11-24T06:27:37.842 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ba406ae40 msgr2=0x7f3ba406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ba406ae40 0x7f3ba406d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8006fd0 tx=0x7f3ba8008040 comp rx=0 tx=0).stop 2023-11-24T06:27:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb815bf10 msgr2=0x7f3bb81f5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb815bf10 0x7f3bb81f5f40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009e80 tx=0x7f3bb0004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 shutdown_connections 2023-11-24T06:27:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ba406ae40 0x7f3ba406d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb815bf10 0x7f3bb81f5f40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb80ccb50 0x7f3bb813e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 --2- 172.21.15.16:0/3990445808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb80adf20 0x7f3bb81f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 >> 172.21.15.16:0/3990445808 conn(0x7f3bb806a7d0 msgr2=0x7f3bb81437a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.838+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 shutdown_connections 2023-11-24T06:27:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:37.839+0000 7f3bb7fff640 1 -- 172.21.15.16:0/3990445808 wait complete. 2023-11-24T06:27:37.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.090+0000 7fbda6a44640 1 Processor -- start 2023-11-24T06:27:38.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.090+0000 7fbda6a44640 1 -- start start 2023-11-24T06:27:38.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.091+0000 7fbda6a44640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbda00cfcf0 0x7fbda0160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.091+0000 7fbda6a44640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbda00d3910 0x7fbda0160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.091+0000 7fbda6a44640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 0x7fbda00b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.091+0000 7fbda6a44640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda00ceff0 con 0x7fbda00cfcf0 2023-11-24T06:27:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.092+0000 7fbda6a44640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda00cf410 con 0x7fbda00d3910 2023-11-24T06:27:38.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.092+0000 7fbda6a44640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00d4160 con 0x7fbda00d00d0 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.092+0000 7fbd9ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbda00cfcf0 0x7fbda0160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.092+0000 7fbd9ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbda00cfcf0 0x7fbda0160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48388/0 (socket says 172.21.15.16:48388) 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.092+0000 7fbd9ffff640 1 -- 172.21.15.16:0/2081025171 learned_addr learned my addr 172.21.15.16:0/2081025171 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.092+0000 7fbd97fff640 1 --2- 172.21.15.16:0/2081025171 >> v2:172.21.15.173:3300/0 conn(0x7fbda00d3910 0x7fbda0160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.092+0000 7fbda4fe1640 1 --2- 172.21.15.16:0/2081025171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 0x7fbda00b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.092+0000 7fbd9ffff640 1 -- 172.21.15.16:0/2081025171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 msgr2=0x7fbda00b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.093+0000 7fbd9ffff640 1 --2- 172.21.15.16:0/2081025171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 0x7fbda00b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.093+0000 7fbd9ffff640 1 -- 172.21.15.16:0/2081025171 >> v2:172.21.15.173:3300/0 conn(0x7fbda00d3910 msgr2=0x7fbda0160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.093+0000 7fbd9ffff640 1 --2- 172.21.15.16:0/2081025171 >> v2:172.21.15.173:3300/0 conn(0x7fbda00d3910 0x7fbda0160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.093+0000 7fbd9ffff640 1 -- 172.21.15.16:0/2081025171 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda00b2420 con 0x7fbda00cfcf0 2023-11-24T06:27:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.093+0000 7fbd97fff640 1 --2- 172.21.15.16:0/2081025171 >> v2:172.21.15.173:3300/0 conn(0x7fbda00d3910 0x7fbda0160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.093+0000 7fbda4fe1640 1 --2- 172.21.15.16:0/2081025171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 0x7fbda00b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.094+0000 7fbd9ffff640 1 --2- 172.21.15.16:0/2081025171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00cfcf0 0x7fbda0160700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbda008b640 tx=0x7fbd9002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422809fa1caf1ab9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.094+0000 7fbd9f7fe640 1 -- 172.21.15.16:0/2081025171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd900317e0 con 0x7fbda00cfcf0 2023-11-24T06:27:38.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.094+0000 7fbd9f7fe640 1 -- 172.21.15.16:0/2081025171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd90031940 con 0x7fbda00cfcf0 2023-11-24T06:27:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.094+0000 7fbd9f7fe640 1 -- 172.21.15.16:0/2081025171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90031cd0 con 0x7fbda00cfcf0 2023-11-24T06:27:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.095+0000 7fbda6a44640 1 -- 172.21.15.16:0/2081025171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00cfcf0 msgr2=0x7fbda0160700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.095+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2081025171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00cfcf0 0x7fbda0160700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbda008b640 tx=0x7fbd9002b880 comp rx=0 tx=0).stop 2023-11-24T06:27:38.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.095+0000 7fbda6a44640 1 -- 172.21.15.16:0/2081025171 shutdown_connections 2023-11-24T06:27:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.095+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2081025171 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 0x7fbda00b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.095+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2081025171 >> v2:172.21.15.173:3300/0 conn(0x7fbda00d3910 0x7fbda0160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.095+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2081025171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00cfcf0 0x7fbda0160700 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.095+0000 7fbda6a44640 1 -- 172.21.15.16:0/2081025171 >> 172.21.15.16:0/2081025171 conn(0x7fbda009c530 msgr2=0x7fbda00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.095+0000 7fbda6a44640 1 -- 172.21.15.16:0/2081025171 shutdown_connections 2023-11-24T06:27:38.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.095+0000 7fbda6a44640 1 -- 172.21.15.16:0/2081025171 wait complete. 2023-11-24T06:27:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.097+0000 7fbda6a44640 1 Processor -- start 2023-11-24T06:27:38.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.097+0000 7fbda6a44640 1 -- start start 2023-11-24T06:27:38.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbda6a44640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbda00cfcf0 0x7fbda01f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbda6a44640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 0x7fbda00b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbda6a44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00d3910 0x7fbda00cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbda6a44640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00cb410 con 0x7fbda00d3910 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbda6a44640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda01f3ce0 con 0x7fbda00cfcf0 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbda6a44640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00b8ea0 con 0x7fbda00d00d0 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbda4fe1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00d3910 0x7fbda00cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbda4fe1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00d3910 0x7fbda00cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48404/0 (socket says 172.21.15.16:48404) 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbda4fe1640 1 -- 172.21.15.16:0/2830852624 learned_addr learned my addr 172.21.15.16:0/2830852624 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.098+0000 7fbd9ffff640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbda00cfcf0 0x7fbda01f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.099+0000 7fbd97fff640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 0x7fbda00b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.099+0000 7fbda4fe1640 1 -- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 msgr2=0x7fbda00b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.099+0000 7fbda4fe1640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 0x7fbda00b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.099+0000 7fbda4fe1640 1 -- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbda00cfcf0 msgr2=0x7fbda01f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.099+0000 7fbda4fe1640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbda00cfcf0 0x7fbda01f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.099+0000 7fbda4fe1640 1 -- 172.21.15.16:0/2830852624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd900094d0 con 0x7fbda00d3910 2023-11-24T06:27:38.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.099+0000 7fbda4fe1640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00d3910 0x7fbda00cb580 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800ce20 tx=0x7fbd9800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.099+0000 7fbd9d7fa640 1 -- 172.21.15.16:0/2830852624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98003b90 con 0x7fbda00d3910 2023-11-24T06:27:38.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.100+0000 7fbd9d7fa640 1 -- 172.21.15.16:0/2830852624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd98003d10 con 0x7fbda00d3910 2023-11-24T06:27:38.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.100+0000 7fbd9d7fa640 1 -- 172.21.15.16:0/2830852624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98004260 con 0x7fbda00d3910 2023-11-24T06:27:38.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.100+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda00ca890 con 0x7fbda00d3910 2023-11-24T06:27:38.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.100+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda00cae50 con 0x7fbda00d3910 2023-11-24T06:27:38.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.101+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda00675d0 con 0x7fbda00d3910 2023-11-24T06:27:38.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.102+0000 7fbd9d7fa640 1 -- 172.21.15.16:0/2830852624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd980043c0 con 0x7fbda00d3910 2023-11-24T06:27:38.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.104+0000 7fbd9d7fa640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd8006ae40 0x7fbd8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.104+0000 7fbd9d7fa640 1 -- 172.21.15.16:0/2830852624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd98089f20 con 0x7fbda00d3910 2023-11-24T06:27:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.104+0000 7fbd9ffff640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd8006ae40 0x7fbd8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.105+0000 7fbd9ffff640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd8006ae40 0x7fbd8006d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbd90036000 tx=0x7fbd9003d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.108+0000 7fbd9d7fa640 1 -- 172.21.15.16:0/2830852624 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd98054f30 con 0x7fbda00d3910 2023-11-24T06:27:38.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.301+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbda01f3ee0 con 0x7fbda00d3910 2023-11-24T06:27:38.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.302+0000 7fbd9d7fa640 1 -- 172.21.15.16:0/2830852624 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd98003e80 con 0x7fbda00d3910 2023-11-24T06:27:38.305 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd8006ae40 msgr2=0x7fbd8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd8006ae40 0x7fbd8006d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbd90036000 tx=0x7fbd9003d040 comp rx=0 tx=0).stop 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00d3910 msgr2=0x7fbda00cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00d3910 0x7fbda00cb580 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800ce20 tx=0x7fbd9800a710 comp rx=0 tx=0).stop 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 shutdown_connections 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd8006ae40 0x7fbd8006d300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbda00d3910 0x7fbda00cb580 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbda00d00d0 0x7fbda00b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 --2- 172.21.15.16:0/2830852624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbda00cfcf0 0x7fbda01f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.304+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 >> 172.21.15.16:0/2830852624 conn(0x7fbda009c530 msgr2=0x7fbda00afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.305+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 shutdown_connections 2023-11-24T06:27:38.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.305+0000 7fbda6a44640 1 -- 172.21.15.16:0/2830852624 wait complete. 2023-11-24T06:27:38.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:38.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.553+0000 7f6c98973640 1 Processor -- start 2023-11-24T06:27:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.554+0000 7f6c98973640 1 -- start start 2023-11-24T06:27:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.555+0000 7f6c98973640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940cf9a0 0x7f6c940cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.555+0000 7f6c98973640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c940b31d0 0x7f6c940b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.555+0000 7f6c98973640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c940d0320 con 0x7f6c9415d4b0 2023-11-24T06:27:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.555+0000 7f6c98973640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c940d1c60 con 0x7f6c940b31d0 2023-11-24T06:27:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.555+0000 7f6c98973640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940b2730 con 0x7f6c940cf9a0 2023-11-24T06:27:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.556+0000 7f6c93fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c940b31d0 0x7f6c940b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.556+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940cf9a0 0x7f6c940cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.556+0000 7f6c937fe640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f6c9415d4b0 0x7f6c940d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:52046/0 (socket says 172.21.15.16:52046) 2023-11-24T06:27:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.556+0000 7f6c93fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c940b31d0 0x7f6c940b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:48422/0 (socket says 172.21.15.16:48422) 2023-11-24T06:27:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.556+0000 7f6c937fe640 1 -- 172.21.15.16:0/3550475274 learned_addr learned my addr 172.21.15.16:0/3550475274 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.557+0000 7f6c8bfff640 1 -- 172.21.15.16:0/3550475274 >> v2:172.21.15.173:3300/0 conn(0x7f6c940b31d0 msgr2=0x7f6c940b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.557+0000 7f6c8bfff640 1 --2- 172.21.15.16:0/3550475274 >> v2:172.21.15.173:3300/0 conn(0x7f6c940b31d0 0x7f6c940b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.557+0000 7f6c8bfff640 1 -- 172.21.15.16:0/3550475274 >> v1:172.21.15.173:6789/0 conn(0x7f6c9415d4b0 legacy=0x7f6c940d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.557+0000 7f6c8bfff640 1 -- 172.21.15.16:0/3550475274 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940d1ec0 con 0x7f6c940cf9a0 2023-11-24T06:27:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.557+0000 7f6c92ffd640 1 -- 172.21.15.16:0/3550475274 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057675578 0 0) 0x7f6c940d0320 con 0x7f6c9415d4b0 2023-11-24T06:27:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.558+0000 7f6c8bfff640 1 --2- 172.21.15.16:0/3550475274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940cf9a0 0x7f6c940cfde0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0097d0 tx=0x7f6c7c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f34b70dcee938294 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.559+0000 7f6c92ffd640 1 -- 172.21.15.16:0/3550475274 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c031800 con 0x7f6c940cf9a0 2023-11-24T06:27:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.559+0000 7f6c92ffd640 1 -- 172.21.15.16:0/3550475274 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c031960 con 0x7f6c940cf9a0 2023-11-24T06:27:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.559+0000 7f6c92ffd640 1 -- 172.21.15.16:0/3550475274 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c031cf0 con 0x7f6c940cf9a0 2023-11-24T06:27:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.560+0000 7f6c98973640 1 -- 172.21.15.16:0/3550475274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940cf9a0 msgr2=0x7f6c940cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.560+0000 7f6c98973640 1 --2- 172.21.15.16:0/3550475274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940cf9a0 0x7f6c940cfde0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0097d0 tx=0x7f6c7c02b8a0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.560+0000 7f6c98973640 1 -- 172.21.15.16:0/3550475274 shutdown_connections 2023-11-24T06:27:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.560+0000 7f6c98973640 1 --2- 172.21.15.16:0/3550475274 >> v2:172.21.15.173:3300/0 conn(0x7f6c940b31d0 0x7f6c940b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.560+0000 7f6c98973640 1 --2- 172.21.15.16:0/3550475274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940cf9a0 0x7f6c940cfde0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.560+0000 7f6c98973640 1 -- 172.21.15.16:0/3550475274 >> 172.21.15.16:0/3550475274 conn(0x7f6c940d3000 msgr2=0x7f6c940d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.560+0000 7f6c98973640 1 -- 172.21.15.16:0/3550475274 shutdown_connections 2023-11-24T06:27:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.560+0000 7f6c98973640 1 -- 172.21.15.16:0/3550475274 wait complete. 2023-11-24T06:27:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.562+0000 7f6c98973640 1 Processor -- start 2023-11-24T06:27:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.562+0000 7f6c98973640 1 -- start start 2023-11-24T06:27:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c98973640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940b31d0 0x7f6c94151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c98973640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c940cf9a0 0x7f6c940ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c98973640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c9415d4b0 0x7f6c940ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c98973640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940cb850 con 0x7f6c940cf9a0 2023-11-24T06:27:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c98973640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94160770 con 0x7f6c9415d4b0 2023-11-24T06:27:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c98973640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941f2f50 con 0x7f6c940b31d0 2023-11-24T06:27:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c940cf9a0 0x7f6c940ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c940cf9a0 0x7f6c940ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48412/0 (socket says 172.21.15.16:48412) 2023-11-24T06:27:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c8bfff640 1 -- 172.21.15.16:0/95799974 learned_addr learned my addr 172.21.15.16:0/95799974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c937fe640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940b31d0 0x7f6c94151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c93fff640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c9415d4b0 0x7f6c940ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c8bfff640 1 -- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940b31d0 msgr2=0x7f6c94151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c8bfff640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940b31d0 0x7f6c94151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c8bfff640 1 -- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c9415d4b0 msgr2=0x7f6c940ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c8bfff640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c9415d4b0 0x7f6c940ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c8bfff640 1 -- 172.21.15.16:0/95799974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c009400 con 0x7f6c940cf9a0 2023-11-24T06:27:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c937fe640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940b31d0 0x7f6c94151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.563+0000 7f6c93fff640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c9415d4b0 0x7f6c940ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.564+0000 7f6c8bfff640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c940cf9a0 0x7f6c940ca0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098f0 tx=0x7f6c7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.564+0000 7f6c90ff9640 1 -- 172.21.15.16:0/95799974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c03aca0 con 0x7f6c940cf9a0 2023-11-24T06:27:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.564+0000 7f6c90ff9640 1 -- 172.21.15.16:0/95799974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c03ae20 con 0x7f6c940cf9a0 2023-11-24T06:27:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.564+0000 7f6c90ff9640 1 -- 172.21.15.16:0/95799974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c044630 con 0x7f6c940cf9a0 2023-11-24T06:27:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.564+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c940cac40 con 0x7f6c940cf9a0 2023-11-24T06:27:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.564+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c940cb200 con 0x7f6c940cf9a0 2023-11-24T06:27:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.565+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c940d2180 con 0x7f6c940cf9a0 2023-11-24T06:27:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.566+0000 7f6c90ff9640 1 -- 172.21.15.16:0/95799974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c7c04d450 con 0x7f6c940cf9a0 2023-11-24T06:27:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.568+0000 7f6c90ff9640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6c7406ae40 0x7f6c7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.568+0000 7f6c90ff9640 1 -- 172.21.15.16:0/95799974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c7c02e070 con 0x7f6c940cf9a0 2023-11-24T06:27:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.568+0000 7f6c937fe640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6c7406ae40 0x7f6c7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:38.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.569+0000 7f6c937fe640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6c7406ae40 0x7f6c7406d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000f60 tx=0x7f6c84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.572+0000 7f6c90ff9640 1 -- 172.21.15.16:0/95799974 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c940d2180 con 0x7f6c940cf9a0 2023-11-24T06:27:38.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.764+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c940d2180 con 0x7f6c940cf9a0 2023-11-24T06:27:38.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.765+0000 7f6c90ff9640 1 -- 172.21.15.16:0/95799974 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c7c035020 con 0x7f6c940cf9a0 2023-11-24T06:27:38.768 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:38.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6c7406ae40 msgr2=0x7f6c7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6c7406ae40 0x7f6c7406d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000f60 tx=0x7f6c84009040 comp rx=0 tx=0).stop 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c940cf9a0 msgr2=0x7f6c940ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c940cf9a0 0x7f6c940ca0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098f0 tx=0x7f6c7c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 shutdown_connections 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6c7406ae40 0x7f6c7406d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c9415d4b0 0x7f6c940ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c940cf9a0 0x7f6c940ca0d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 --2- 172.21.15.16:0/95799974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c940b31d0 0x7f6c94151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 >> 172.21.15.16:0/95799974 conn(0x7f6c940d3000 msgr2=0x7f6c940d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.768+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 shutdown_connections 2023-11-24T06:27:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:38.769+0000 7f6c98973640 1 -- 172.21.15.16:0/95799974 wait complete. 2023-11-24T06:27:38.859 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:27:38.859 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:27:38.859 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:27:38.859 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:27:38.859 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:27:38.859 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:27:38.859 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:27:38.859 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:27:38.859 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:27:38.860 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:27:38.860 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:27:38.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:39.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.089+0000 7fee5ca4a640 1 Processor -- start 2023-11-24T06:27:39.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.090+0000 7fee5ca4a640 1 -- start start 2023-11-24T06:27:39.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.091+0000 7fee5ca4a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee58160920 0x7fee580b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.091+0000 7fee5ca4a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fee580cf9f0 0x7fee580cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.091+0000 7fee5ca4a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee58161010 0x7fee581614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.091+0000 7fee5ca4a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee580d03b0 con 0x7fee58161010 2023-11-24T06:27:39.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.091+0000 7fee5ca4a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee580d0a10 con 0x7fee580cf9f0 2023-11-24T06:27:39.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.091+0000 7fee5ca4a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee580cf080 con 0x7fee58160920 2023-11-24T06:27:39.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee5759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee58161010 0x7fee581614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee5759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee58161010 0x7fee581614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48424/0 (socket says 172.21.15.16:48424) 2023-11-24T06:27:39.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee5759e640 1 -- 172.21.15.16:0/758308520 learned_addr learned my addr 172.21.15.16:0/758308520 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:39.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee5659c640 1 --2- 172.21.15.16:0/758308520 >> v2:172.21.15.173:3300/0 conn(0x7fee580cf9f0 0x7fee580cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee56d9d640 1 --2- 172.21.15.16:0/758308520 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee58160920 0x7fee580b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee5759e640 1 -- 172.21.15.16:0/758308520 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee58160920 msgr2=0x7fee580b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:39.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee5759e640 1 --2- 172.21.15.16:0/758308520 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee58160920 0x7fee580b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee5759e640 1 -- 172.21.15.16:0/758308520 >> v2:172.21.15.173:3300/0 conn(0x7fee580cf9f0 msgr2=0x7fee580cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:39.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee5759e640 1 --2- 172.21.15.16:0/758308520 >> v2:172.21.15.173:3300/0 conn(0x7fee580cf9f0 0x7fee580cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.092+0000 7fee5759e640 1 -- 172.21.15.16:0/758308520 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee580c6be0 con 0x7fee58161010 2023-11-24T06:27:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.093+0000 7fee5759e640 1 --2- 172.21.15.16:0/758308520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 0x7fee581614a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee5808d300 tx=0x7fee5002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf49f3eac335280c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.094+0000 7fee55d9b640 1 -- 172.21.15.16:0/758308520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50031860 con 0x7fee58161010 2023-11-24T06:27:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.094+0000 7fee55d9b640 1 -- 172.21.15.16:0/758308520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee500319c0 con 0x7fee58161010 2023-11-24T06:27:39.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.094+0000 7fee55d9b640 1 -- 172.21.15.16:0/758308520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50031d50 con 0x7fee58161010 2023-11-24T06:27:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.095+0000 7fee5ca4a640 1 -- 172.21.15.16:0/758308520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 msgr2=0x7fee581614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.095+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/758308520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 0x7fee581614a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee5808d300 tx=0x7fee5002b900 comp rx=0 tx=0).stop 2023-11-24T06:27:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.095+0000 7fee5ca4a640 1 -- 172.21.15.16:0/758308520 shutdown_connections 2023-11-24T06:27:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.095+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/758308520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 0x7fee581614a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.095+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/758308520 >> v2:172.21.15.173:3300/0 conn(0x7fee580cf9f0 0x7fee580cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.095+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/758308520 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee58160920 0x7fee580b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.095+0000 7fee5ca4a640 1 -- 172.21.15.16:0/758308520 >> 172.21.15.16:0/758308520 conn(0x7fee580d3090 msgr2=0x7fee580d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.095+0000 7fee5ca4a640 1 -- 172.21.15.16:0/758308520 shutdown_connections 2023-11-24T06:27:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.095+0000 7fee5ca4a640 1 -- 172.21.15.16:0/758308520 wait complete. 2023-11-24T06:27:39.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.097+0000 7fee5ca4a640 1 Processor -- start 2023-11-24T06:27:39.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.097+0000 7fee5ca4a640 1 -- start start 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5ca4a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee580cf9f0 0x7fee58153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5ca4a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee58160920 0x7fee580b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5ca4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 0x7fee580caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5ca4a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58154510 con 0x7fee58161010 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5ca4a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee580b6610 con 0x7fee58160920 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5ca4a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee580b68f0 con 0x7fee580cf9f0 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 0x7fee580caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 0x7fee580caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48434/0 (socket says 172.21.15.16:48434) 2023-11-24T06:27:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5759e640 1 -- 172.21.15.16:0/28577179 learned_addr learned my addr 172.21.15.16:0/28577179 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee5659c640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee58160920 0x7fee580b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.098+0000 7fee56d9d640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee580cf9f0 0x7fee58153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee5759e640 1 -- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee580cf9f0 msgr2=0x7fee58153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee5759e640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee580cf9f0 0x7fee58153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee5759e640 1 -- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee58160920 msgr2=0x7fee580b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee5759e640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee58160920 0x7fee580b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee5759e640 1 -- 172.21.15.16:0/28577179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500094d0 con 0x7fee58161010 2023-11-24T06:27:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee5759e640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 0x7fee580caa50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee500090b0 tx=0x7fee500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee4f7fe640 1 -- 172.21.15.16:0/28577179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5003ac60 con 0x7fee58161010 2023-11-24T06:27:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee4f7fe640 1 -- 172.21.15.16:0/28577179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5003adc0 con 0x7fee58161010 2023-11-24T06:27:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee4f7fe640 1 -- 172.21.15.16:0/28577179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50044630 con 0x7fee58161010 2023-11-24T06:27:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.099+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee580b6bd0 con 0x7fee58161010 2023-11-24T06:27:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.100+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee580b7190 con 0x7fee58161010 2023-11-24T06:27:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.101+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee580b7380 con 0x7fee58161010 2023-11-24T06:27:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.102+0000 7fee4f7fe640 1 -- 172.21.15.16:0/28577179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee5004d450 con 0x7fee58161010 2023-11-24T06:27:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.103+0000 7fee4f7fe640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fee3806ae40 0x7fee3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.103+0000 7fee4f7fe640 1 -- 172.21.15.16:0/28577179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee500abc10 con 0x7fee58161010 2023-11-24T06:27:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.103+0000 7fee56d9d640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fee3806ae40 0x7fee3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.104+0000 7fee56d9d640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fee3806ae40 0x7fee3806d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee48009a60 tx=0x7fee48003990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.107+0000 7fee4f7fe640 1 -- 172.21.15.16:0/28577179 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee50076c20 con 0x7fee58161010 2023-11-24T06:27:39.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.305+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee58067450 con 0x7fee58161010 2023-11-24T06:27:39.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.307+0000 7fee4f7fe640 1 -- 172.21.15.16:0/28577179 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fee50035020 con 0x7fee58161010 2023-11-24T06:27:39.310 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:26:57.937586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.309+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fee3806ae40 msgr2=0x7fee3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.309+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fee3806ae40 0x7fee3806d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee48009a60 tx=0x7fee48003990 comp rx=0 tx=0).stop 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.309+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 msgr2=0x7fee580caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.309+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 0x7fee580caa50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee500090b0 tx=0x7fee500046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.310+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 shutdown_connections 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.310+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fee3806ae40 0x7fee3806d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.310+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58161010 0x7fee580caa50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.310+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee58160920 0x7fee580b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.310+0000 7fee5ca4a640 1 --2- 172.21.15.16:0/28577179 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fee580cf9f0 0x7fee58153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.310+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 >> 172.21.15.16:0/28577179 conn(0x7fee580d3090 msgr2=0x7fee580d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:39.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.310+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 shutdown_connections 2023-11-24T06:27:39.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.310+0000 7fee5ca4a640 1 -- 172.21.15.16:0/28577179 wait complete. 2023-11-24T06:27:39.333 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:39.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.559+0000 7f3389ada640 1 Processor -- start 2023-11-24T06:27:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.559+0000 7f3389ada640 1 -- start start 2023-11-24T06:27:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.561+0000 7f3389ada640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f338413cf10 0x7f338413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.561+0000 7f3389ada640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33840cddf0 0x7f33840b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.561+0000 7f3389ada640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33840ce440 con 0x7f338413fae0 2023-11-24T06:27:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.561+0000 7f3389ada640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338413ff90 con 0x7f338413cf10 2023-11-24T06:27:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.561+0000 7f3389ada640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33840ce260 con 0x7f33840cddf0 2023-11-24T06:27:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.561+0000 7f3388ad8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f338413cf10 0x7f338413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.561+0000 7f3388ad8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f338413cf10 0x7f338413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48448/0 (socket says 172.21.15.16:48448) 2023-11-24T06:27:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.562+0000 7f3388ad8640 1 -- 172.21.15.16:0/1416655973 learned_addr learned my addr 172.21.15.16:0/1416655973 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.562+0000 7f3388ad8640 1 -- 172.21.15.16:0/1416655973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33840cddf0 msgr2=0x7f33840b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.562+0000 7f3388ad8640 1 --2- 172.21.15.16:0/1416655973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33840cddf0 0x7f33840b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.562+0000 7f3388ad8640 1 -- 172.21.15.16:0/1416655973 >> v1:172.21.15.173:6789/0 conn(0x7f338413fae0 legacy=0x7f33840cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.562+0000 7f3388ad8640 1 -- 172.21.15.16:0/1416655973 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33840ce720 con 0x7f338413cf10 2023-11-24T06:27:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.563+0000 7f3388ad8640 1 --2- 172.21.15.16:0/1416655973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 0x7f338413f5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3380009e50 tx=0x7f338002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d78f2daa93d06a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.563+0000 7f337b7fe640 1 -- 172.21.15.16:0/1416655973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338002ca70 con 0x7f338413cf10 2023-11-24T06:27:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.563+0000 7f337b7fe640 1 -- 172.21.15.16:0/1416655973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338002cbd0 con 0x7f338413cf10 2023-11-24T06:27:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.563+0000 7f337b7fe640 1 -- 172.21.15.16:0/1416655973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800343f0 con 0x7f338413cf10 2023-11-24T06:27:39.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.564+0000 7f3389ada640 1 -- 172.21.15.16:0/1416655973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 msgr2=0x7f338413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.564+0000 7f3389ada640 1 --2- 172.21.15.16:0/1416655973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 0x7f338413f5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3380009e50 tx=0x7f338002bed0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.564+0000 7f3389ada640 1 -- 172.21.15.16:0/1416655973 shutdown_connections 2023-11-24T06:27:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.564+0000 7f3389ada640 1 --2- 172.21.15.16:0/1416655973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33840cddf0 0x7f33840b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.564+0000 7f3389ada640 1 --2- 172.21.15.16:0/1416655973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 0x7f338413f5a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.564+0000 7f3389ada640 1 -- 172.21.15.16:0/1416655973 >> 172.21.15.16:0/1416655973 conn(0x7f338406a7d0 msgr2=0x7f338406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.564+0000 7f3389ada640 1 -- 172.21.15.16:0/1416655973 shutdown_connections 2023-11-24T06:27:39.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.564+0000 7f3389ada640 1 -- 172.21.15.16:0/1416655973 wait complete. 2023-11-24T06:27:39.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.566+0000 7f3389ada640 1 Processor -- start 2023-11-24T06:27:39.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.566+0000 7f3389ada640 1 -- start start 2023-11-24T06:27:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.567+0000 7f3389ada640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33840cddf0 0x7f33841532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.567+0000 7f3389ada640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 0x7f3384139120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.567+0000 7f3389ada640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f338413fae0 0x7f338413dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.567+0000 7f3389ada640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33841537e0 con 0x7f338413cf10 2023-11-24T06:27:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.567+0000 7f3389ada640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338413e830 con 0x7f338413fae0 2023-11-24T06:27:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.567+0000 7f3389ada640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3384139660 con 0x7f33840cddf0 2023-11-24T06:27:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.567+0000 7f337bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 0x7f3384139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.568+0000 7f337bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 0x7f3384139120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48450/0 (socket says 172.21.15.16:48450) 2023-11-24T06:27:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.568+0000 7f337bfff640 1 -- 172.21.15.16:0/3546284331 learned_addr learned my addr 172.21.15.16:0/3546284331 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.568+0000 7f337bfff640 1 -- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33840cddf0 msgr2=0x7f33841532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.568+0000 7f337bfff640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33840cddf0 0x7f33841532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.568+0000 7f337bfff640 1 -- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f338413fae0 msgr2=0x7f338413dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.568+0000 7f337bfff640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f338413fae0 0x7f338413dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.568+0000 7f337bfff640 1 -- 172.21.15.16:0/3546284331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3380009b30 con 0x7f338413cf10 2023-11-24T06:27:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.568+0000 7f337bfff640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 0x7f3384139120 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f337400b450 tx=0x7f337400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.569+0000 7f33797fa640 1 -- 172.21.15.16:0/3546284331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374011890 con 0x7f338413cf10 2023-11-24T06:27:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.569+0000 7f33797fa640 1 -- 172.21.15.16:0/3546284331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33740119f0 con 0x7f338413cf10 2023-11-24T06:27:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.569+0000 7f33797fa640 1 -- 172.21.15.16:0/3546284331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374011da0 con 0x7f338413cf10 2023-11-24T06:27:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.569+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3384159920 con 0x7f338413cf10 2023-11-24T06:27:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.569+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3384159ee0 con 0x7f338413cf10 2023-11-24T06:27:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.570+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3384067450 con 0x7f338413cf10 2023-11-24T06:27:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.571+0000 7f33797fa640 1 -- 172.21.15.16:0/3546284331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f337401a400 con 0x7f338413cf10 2023-11-24T06:27:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.572+0000 7f33797fa640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f337006ae70 0x7f337006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.572+0000 7f33797fa640 1 -- 172.21.15.16:0/3546284331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3374089610 con 0x7f338413cf10 2023-11-24T06:27:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.573+0000 7f3388ad8640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f337006ae70 0x7f337006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.574+0000 7f3388ad8640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f337006ae70 0x7f337006d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3380035000 tx=0x7f338002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.577+0000 7f33797fa640 1 -- 172.21.15.16:0/3546284331 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3374054770 con 0x7f338413cf10 2023-11-24T06:27:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.774+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f338413e970 con 0x7f338413cf10 2023-11-24T06:27:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.775+0000 7f33797fa640 1 -- 172.21.15.16:0/3546284331 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3374057d90 con 0x7f338413cf10 2023-11-24T06:27:39.778 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.777+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f337006ae70 msgr2=0x7f337006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.777+0000 7f3389ada640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f337006ae70 0x7f337006d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3380035000 tx=0x7f338002e000 comp rx=0 tx=0).stop 2023-11-24T06:27:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.777+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 msgr2=0x7f3384139120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.777+0000 7f3389ada640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 0x7f3384139120 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f337400b450 tx=0x7f337400b910 comp rx=0 tx=0).stop 2023-11-24T06:27:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.778+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 shutdown_connections 2023-11-24T06:27:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.778+0000 7f3389ada640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f337006ae70 0x7f337006d330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.778+0000 7f3389ada640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f338413fae0 0x7f338413dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.778+0000 7f3389ada640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f338413cf10 0x7f3384139120 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.778+0000 7f3389ada640 1 --2- 172.21.15.16:0/3546284331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33840cddf0 0x7f33841532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.778+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 >> 172.21.15.16:0/3546284331 conn(0x7f338406a7d0 msgr2=0x7f3384157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.778+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 shutdown_connections 2023-11-24T06:27:39.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:39.778+0000 7f3389ada640 1 -- 172.21.15.16:0/3546284331 wait complete. 2023-11-24T06:27:39.799 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.029+0000 7ffa025ea640 1 Processor -- start 2023-11-24T06:27:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.029+0000 7ffa025ea640 1 -- start start 2023-11-24T06:27:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.030+0000 7ffa025ea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9fc15d610 0x7ff9fc0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.030+0000 7ffa025ea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9fc0ced60 0x7ff9fc0d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.030+0000 7ffa025ea640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff9fc0d1d20 0x7ff9fc0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.030+0000 7ffa025ea640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9fc0d1390 con 0x7ff9fc0ced60 2023-11-24T06:27:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.030+0000 7ffa025ea640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9fc0d21b0 con 0x7ff9fc0d1d20 2023-11-24T06:27:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.030+0000 7ffa025ea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0b3400 con 0x7ff9fc15d610 2023-11-24T06:27:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.031+0000 7ff9fb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9fc0ced60 0x7ff9fc0d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.031+0000 7ff9fb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9fc0ced60 0x7ff9fc0d17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48464/0 (socket says 172.21.15.16:48464) 2023-11-24T06:27:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.031+0000 7ff9fb7fe640 1 -- 172.21.15.16:0/1186609022 learned_addr learned my addr 172.21.15.16:0/1186609022 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.031+0000 7ffa00b87640 1 --2- 172.21.15.16:0/1186609022 >> v2:172.21.15.173:3300/0 conn(0x7ff9fc0d1d20 0x7ff9fc0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.031+0000 7ff9fb7fe640 1 -- 172.21.15.16:0/1186609022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9fc15d610 msgr2=0x7ff9fc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.031+0000 7ff9fb7fe640 1 --2- 172.21.15.16:0/1186609022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9fc15d610 0x7ff9fc0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.031+0000 7ff9fb7fe640 1 -- 172.21.15.16:0/1186609022 >> v2:172.21.15.173:3300/0 conn(0x7ff9fc0d1d20 msgr2=0x7ff9fc0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.031+0000 7ff9fb7fe640 1 --2- 172.21.15.16:0/1186609022 >> v2:172.21.15.173:3300/0 conn(0x7ff9fc0d1d20 0x7ff9fc0cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.031+0000 7ff9fb7fe640 1 -- 172.21.15.16:0/1186609022 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc0b27f0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.032+0000 7ff9fb7fe640 1 --2- 172.21.15.16:0/1186609022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 0x7ff9fc0d17e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0d42d0 tx=0x7ff9e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be9875c71c6cab4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ff9faffd640 1 -- 172.21.15.16:0/1186609022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4031870 con 0x7ff9fc0ced60 2023-11-24T06:27:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ff9faffd640 1 -- 172.21.15.16:0/1186609022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e40319d0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ff9faffd640 1 -- 172.21.15.16:0/1186609022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4031d60 con 0x7ff9fc0ced60 2023-11-24T06:27:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ffa025ea640 1 -- 172.21.15.16:0/1186609022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 msgr2=0x7ff9fc0d17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1186609022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 0x7ff9fc0d17e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0d42d0 tx=0x7ff9e402b910 comp rx=0 tx=0).stop 2023-11-24T06:27:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ffa025ea640 1 -- 172.21.15.16:0/1186609022 shutdown_connections 2023-11-24T06:27:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1186609022 >> v2:172.21.15.173:3300/0 conn(0x7ff9fc0d1d20 0x7ff9fc0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1186609022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 0x7ff9fc0d17e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1186609022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9fc15d610 0x7ff9fc0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.033+0000 7ffa025ea640 1 -- 172.21.15.16:0/1186609022 >> 172.21.15.16:0/1186609022 conn(0x7ff9fc09c540 msgr2=0x7ff9fc0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.034+0000 7ffa025ea640 1 -- 172.21.15.16:0/1186609022 shutdown_connections 2023-11-24T06:27:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.034+0000 7ffa025ea640 1 -- 172.21.15.16:0/1186609022 wait complete. 2023-11-24T06:27:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.035+0000 7ffa025ea640 1 Processor -- start 2023-11-24T06:27:40.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.035+0000 7ffa025ea640 1 -- start start 2023-11-24T06:27:40.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.036+0000 7ffa025ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 0x7ff9fc15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.036+0000 7ffa025ea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9fc0d1d20 0x7ff9fc09dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.036+0000 7ffa025ea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9fc15d610 0x7ff9fc0b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.036+0000 7ffa025ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc09e1b0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.036+0000 7ffa025ea640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0ca2d0 con 0x7ff9fc15d610 2023-11-24T06:27:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.036+0000 7ffa025ea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc0b8f80 con 0x7ff9fc0d1d20 2023-11-24T06:27:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ff9fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 0x7ff9fc15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ff9fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 0x7ff9fc15ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48476/0 (socket says 172.21.15.16:48476) 2023-11-24T06:27:40.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ff9fbfff640 1 -- 172.21.15.16:0/1786255539 learned_addr learned my addr 172.21.15.16:0/1786255539 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ffa00b87640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9fc15d610 0x7ff9fc0b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ff9fbfff640 1 -- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9fc0d1d20 msgr2=0x7ff9fc09dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ff9fbfff640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9fc0d1d20 0x7ff9fc09dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ff9fbfff640 1 -- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9fc15d610 msgr2=0x7ff9fc0b8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ff9fbfff640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9fc15d610 0x7ff9fc0b8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ff9fbfff640 1 -- 172.21.15.16:0/1786255539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40094d0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.037+0000 7ff9fbfff640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 0x7ff9fc15ba00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00ba50 tx=0x7ff9ec00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.038+0000 7ff9f8ff9640 1 -- 172.21.15.16:0/1786255539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00cbe0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.038+0000 7ff9f8ff9640 1 -- 172.21.15.16:0/1786255539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec00cd60 con 0x7ff9fc0ced60 2023-11-24T06:27:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.038+0000 7ff9f8ff9640 1 -- 172.21.15.16:0/1786255539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec014680 con 0x7ff9fc0ced60 2023-11-24T06:27:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.038+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc0ca570 con 0x7ff9fc0ced60 2023-11-24T06:27:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.038+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc0cab30 con 0x7ff9fc0ced60 2023-11-24T06:27:40.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.039+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc0675d0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.040+0000 7ff9f8ff9640 1 -- 172.21.15.16:0/1786255539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9ec01d3b0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.041+0000 7ff9f8ff9640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9dc06ae40 0x7ff9dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.041+0000 7ff9f8ff9640 1 -- 172.21.15.16:0/1786255539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9ec0896a0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.042+0000 7ff9fb7fe640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9dc06ae40 0x7ff9dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.043+0000 7ff9fb7fe640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9dc06ae40 0x7ff9dc06d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4036000 tx=0x7ff9e4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.045+0000 7ff9f8ff9640 1 -- 172.21.15.16:0/1786255539 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9ec0546b0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.241+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9fc0b4c20 con 0x7ff9fc0ced60 2023-11-24T06:27:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.241+0000 7ff9f8ff9640 1 -- 172.21.15.16:0/1786255539 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9ec057cd0 con 0x7ff9fc0ced60 2023-11-24T06:27:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9dc06ae40 msgr2=0x7ff9dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9dc06ae40 0x7ff9dc06d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4036000 tx=0x7ff9e4030000 comp rx=0 tx=0).stop 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 msgr2=0x7ff9fc15ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 0x7ff9fc15ba00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00ba50 tx=0x7ff9ec00bf10 comp rx=0 tx=0).stop 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 shutdown_connections 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff9dc06ae40 0x7ff9dc06d300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9fc15d610 0x7ff9fc0b8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9fc0d1d20 0x7ff9fc09dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 --2- 172.21.15.16:0/1786255539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9fc0ced60 0x7ff9fc15ba00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 >> 172.21.15.16:0/1786255539 conn(0x7ff9fc09c540 msgr2=0x7ff9fc0b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 shutdown_connections 2023-11-24T06:27:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.244+0000 7ffa025ea640 1 -- 172.21.15.16:0/1786255539 wait complete. 2023-11-24T06:27:40.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:40.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.489+0000 7f373b59e640 1 Processor -- start 2023-11-24T06:27:40.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.489+0000 7f373b59e640 1 -- start start 2023-11-24T06:27:40.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.491+0000 7f373b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c0cf9a0 0x7f373c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.491+0000 7f373b59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c0d0320 con 0x7f373c0b31d0 2023-11-24T06:27:40.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.491+0000 7f373b59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c0d1c60 con 0x7f373c15d4b0 2023-11-24T06:27:40.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.491+0000 7f373b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0b2730 con 0x7f373c0cf9a0 2023-11-24T06:27:40.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.491+0000 7f373ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f373c0b31d0 0x7f373c0b3660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52622/0 (socket says 172.21.15.16:52622) 2023-11-24T06:27:40.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.491+0000 7f373ad9d640 1 -- 172.21.15.16:0/1647382149 learned_addr learned my addr 172.21.15.16:0/1647382149 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.491+0000 7f3733fff640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c0cf9a0 0x7f373c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790486040 0 0) 0x7f373c0d0320 con 0x7f373c0b31d0 2023-11-24T06:27:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3728003520 con 0x7f373c0b31d0 2023-11-24T06:27:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1506790448 0 0) 0x7f3728003520 con 0x7f373c0b31d0 2023-11-24T06:27:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c0cf9a0 msgr2=0x7f373c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c0cf9a0 0x7f373c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 >> v1:172.21.15.173:6789/0 conn(0x7f373c15d4b0 legacy=0x7f373c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c0d1ec0 con 0x7f373c0b31d0 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f3734001670 con 0x7f373c0b31d0 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 >> v1:172.21.15.16:6789/0 conn(0x7f373c0b31d0 legacy=0x7f373c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f3728003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37280047c0 0x7f3728004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c0cf9a0 0x7f3728005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0b2730 con 0x7f37280039a0 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372800ba60 con 0x7f37280047c0 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372800bc20 con 0x7f373c0cf9a0 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.492+0000 7f373a59c640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f3728003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.493+0000 7f3733fff640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37280047c0 0x7f3728004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.493+0000 7f373a59c640 1 -- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c0cf9a0 msgr2=0x7f3728005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.493+0000 7f373a59c640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c0cf9a0 0x7f3728005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.493+0000 7f373a59c640 1 -- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37280047c0 msgr2=0x7f3728004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.493+0000 7f373a59c640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37280047c0 0x7f3728004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.493+0000 7f373a59c640 1 -- 172.21.15.16:0/1647382149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3728003520 con 0x7f37280039a0 2023-11-24T06:27:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.494+0000 7f373a59c640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f3728003dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f372c001e20 tx=0x7f37280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea0a6056f683287 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.494+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c003b70 con 0x7f37280039a0 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.494+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c003cd0 con 0x7f37280039a0 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.494+0000 7f3739d9b640 1 -- 172.21.15.16:0/1647382149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c005310 con 0x7f37280039a0 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.495+0000 7f373b59e640 1 -- 172.21.15.16:0/1647382149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 msgr2=0x7f3728003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.495+0000 7f373b59e640 1 --2- 172.21.15.16:0/1647382149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f3728003dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f372c001e20 tx=0x7f37280057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.495+0000 7f373ad9d640 1 -- 172.21.15.16:0/1647382149 reap_dead start 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.495+0000 7f373b59e640 1 -- 172.21.15.16:0/1647382149 shutdown_connections 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.495+0000 7f373b59e640 1 -- 172.21.15.16:0/1647382149 >> 172.21.15.16:0/1647382149 conn(0x7f373c0d3000 msgr2=0x7f373c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.495+0000 7f373b59e640 1 -- 172.21.15.16:0/1647382149 shutdown_connections 2023-11-24T06:27:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.495+0000 7f373b59e640 1 -- 172.21.15.16:0/1647382149 wait complete. 2023-11-24T06:27:40.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.497+0000 7f373b59e640 1 Processor -- start 2023-11-24T06:27:40.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.497+0000 7f373b59e640 1 -- start start 2023-11-24T06:27:40.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.498+0000 7f373b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f373c0b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.498+0000 7f373b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c151500 0x7f373c0b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.498+0000 7f373b59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f373c150970 0x7f373c150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.498+0000 7f373b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0b7060 con 0x7f37280039a0 2023-11-24T06:27:40.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.498+0000 7f373b59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c151330 con 0x7f373c150970 2023-11-24T06:27:40.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.498+0000 7f373b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c0b6100 con 0x7f373c151500 2023-11-24T06:27:40.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.498+0000 7f373a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f373c0b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.498+0000 7f373a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f373c0b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48498/0 (socket says 172.21.15.16:48498) 2023-11-24T06:27:40.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.498+0000 7f373a59c640 1 -- 172.21.15.16:0/2543166062 learned_addr learned my addr 172.21.15.16:0/2543166062 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f373a59c640 1 -- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c151500 msgr2=0x7f373c0b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f373a59c640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c151500 0x7f373c0b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f373a59c640 1 -- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f373c150970 msgr2=0x7f373c150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f373a59c640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f373c150970 0x7f373c150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f373a59c640 1 -- 172.21.15.16:0/2543166062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c0b63a0 con 0x7f37280039a0 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f373a59c640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f373c0b5bc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f372c02bf50 tx=0x7f372c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f3732ffd640 1 -- 172.21.15.16:0/2543166062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c03e070 con 0x7f37280039a0 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f3732ffd640 1 -- 172.21.15.16:0/2543166062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c006b50 con 0x7f37280039a0 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f3732ffd640 1 -- 172.21.15.16:0/2543166062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c007030 con 0x7f37280039a0 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c0b6680 con 0x7f37280039a0 2023-11-24T06:27:40.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.499+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c0b6c40 con 0x7f37280039a0 2023-11-24T06:27:40.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.501+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c067450 con 0x7f37280039a0 2023-11-24T06:27:40.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.501+0000 7f3732ffd640 1 -- 172.21.15.16:0/2543166062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f372c007190 con 0x7f37280039a0 2023-11-24T06:27:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.502+0000 7f3732ffd640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f371c06ae40 0x7f371c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.502+0000 7f3732ffd640 1 -- 172.21.15.16:0/2543166062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f372c0ab140 con 0x7f37280039a0 2023-11-24T06:27:40.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.503+0000 7f3733fff640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f371c06ae40 0x7f371c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.504+0000 7f3733fff640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f371c06ae40 0x7f371c06d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f373c0b1640 tx=0x7f3724008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.506+0000 7f3732ffd640 1 -- 172.21.15.16:0/2543166062 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f372c076150 con 0x7f37280039a0 2023-11-24T06:27:40.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.699+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f373c067db0 con 0x7f37280039a0 2023-11-24T06:27:40.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.700+0000 7f3732ffd640 1 -- 172.21.15.16:0/2543166062 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f372c079770 con 0x7f37280039a0 2023-11-24T06:27:40.703 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f371c06ae40 msgr2=0x7f371c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f371c06ae40 0x7f371c06d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f373c0b1640 tx=0x7f3724008040 comp rx=0 tx=0).stop 2023-11-24T06:27:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 msgr2=0x7f373c0b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f373c0b5bc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f372c02bf50 tx=0x7f372c0014a0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 shutdown_connections 2023-11-24T06:27:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f371c06ae40 0x7f371c06d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f373c150970 0x7f373c150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f373c151500 0x7f373c0b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 --2- 172.21.15.16:0/2543166062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37280039a0 0x7f373c0b5bc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.702+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 >> 172.21.15.16:0/2543166062 conn(0x7f373c0d3000 msgr2=0x7f373c0cf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.703+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 shutdown_connections 2023-11-24T06:27:40.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.703+0000 7f373b59e640 1 -- 172.21.15.16:0/2543166062 wait complete. 2023-11-24T06:27:40.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T06:27:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.937+0000 7f6efac6d640 1 Processor -- start 2023-11-24T06:27:40.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.937+0000 7f6efac6d640 1 -- start start 2023-11-24T06:27:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.939+0000 7f6efac6d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6ef4160920 0x7f6ef40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.939+0000 7f6efac6d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ef40cfea0 0x7f6ef4161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.939+0000 7f6efac6d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef40d0a10 con 0x7f6ef40cf9f0 2023-11-24T06:27:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.939+0000 7f6efac6d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef40d1f50 con 0x7f6ef40cfea0 2023-11-24T06:27:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.939+0000 7f6efac6d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef40cf080 con 0x7f6ef4160920 2023-11-24T06:27:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.939+0000 7f6efa46c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ef40cfea0 0x7f6ef4161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.939+0000 7f6efa46c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6ef40cfea0 0x7f6ef4161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48504/0 (socket says 172.21.15.16:48504) 2023-11-24T06:27:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.939+0000 7f6efa46c640 1 -- 172.21.15.16:0/3631102448 learned_addr learned my addr 172.21.15.16:0/3631102448 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:40.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.940+0000 7f6efa46c640 1 -- 172.21.15.16:0/3631102448 >> v2:172.21.15.173:3300/0 conn(0x7f6ef4160920 msgr2=0x7f6ef40b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.940+0000 7f6efa46c640 1 --2- 172.21.15.16:0/3631102448 >> v2:172.21.15.173:3300/0 conn(0x7f6ef4160920 0x7f6ef40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.940+0000 7f6efa46c640 1 -- 172.21.15.16:0/3631102448 >> v1:172.21.15.173:6789/0 conn(0x7f6ef40cf9f0 legacy=0x7f6ef4161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.940+0000 7f6efa46c640 1 -- 172.21.15.16:0/3631102448 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40d0310 con 0x7f6ef40cfea0 2023-11-24T06:27:40.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.941+0000 7f6efa46c640 1 --2- 172.21.15.16:0/3631102448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 0x7f6ef4161720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ef408c060 tx=0x7f6eec02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a810448ee90d4c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.941+0000 7f6ef8c69640 1 -- 172.21.15.16:0/3631102448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec02ca80 con 0x7f6ef40cfea0 2023-11-24T06:27:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.941+0000 7f6ef8c69640 1 -- 172.21.15.16:0/3631102448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eec02cbe0 con 0x7f6ef40cfea0 2023-11-24T06:27:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.941+0000 7f6ef8c69640 1 -- 172.21.15.16:0/3631102448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0343f0 con 0x7f6ef40cfea0 2023-11-24T06:27:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.942+0000 7f6efac6d640 1 -- 172.21.15.16:0/3631102448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 msgr2=0x7f6ef4161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.942+0000 7f6efac6d640 1 --2- 172.21.15.16:0/3631102448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 0x7f6ef4161720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ef408c060 tx=0x7f6eec02bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.942+0000 7f6efac6d640 1 -- 172.21.15.16:0/3631102448 shutdown_connections 2023-11-24T06:27:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.942+0000 7f6efac6d640 1 --2- 172.21.15.16:0/3631102448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 0x7f6ef4161720 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.942+0000 7f6efac6d640 1 --2- 172.21.15.16:0/3631102448 >> v2:172.21.15.173:3300/0 conn(0x7f6ef4160920 0x7f6ef40b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.942+0000 7f6efac6d640 1 -- 172.21.15.16:0/3631102448 >> 172.21.15.16:0/3631102448 conn(0x7f6ef40d3090 msgr2=0x7f6ef40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.942+0000 7f6efac6d640 1 -- 172.21.15.16:0/3631102448 shutdown_connections 2023-11-24T06:27:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.942+0000 7f6efac6d640 1 -- 172.21.15.16:0/3631102448 wait complete. 2023-11-24T06:27:40.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.943+0000 7f6efac6d640 1 Processor -- start 2023-11-24T06:27:40.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.944+0000 7f6efac6d640 1 -- start start 2023-11-24T06:27:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.944+0000 7f6efac6d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef40cf9f0 0x7f6ef40cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.944+0000 7f6efac6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 0x7f6ef4160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.944+0000 7f6efac6d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ef4160920 0x7f6ef41f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.944+0000 7f6efac6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef40cbf40 con 0x7f6ef40cfea0 2023-11-24T06:27:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6efac6d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41f7070 con 0x7f6ef40cf9f0 2023-11-24T06:27:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6efac6d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41f7350 con 0x7f6ef4160920 2023-11-24T06:27:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6ef946a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 0x7f6ef4160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6ef946a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 0x7f6ef4160f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48508/0 (socket says 172.21.15.16:48508) 2023-11-24T06:27:40.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6ef946a640 1 -- 172.21.15.16:0/1884197688 learned_addr learned my addr 172.21.15.16:0/1884197688 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6ef946a640 1 -- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ef4160920 msgr2=0x7f6ef41f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6ef946a640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ef4160920 0x7f6ef41f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6ef946a640 1 -- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef40cf9f0 msgr2=0x7f6ef40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:40.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6ef946a640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef40cf9f0 0x7f6ef40cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.945+0000 7f6ef946a640 1 -- 172.21.15.16:0/1884197688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec009b30 con 0x7f6ef40cfea0 2023-11-24T06:27:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.946+0000 7f6ef946a640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 0x7f6ef4160f70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400b3e0 tx=0x7f6ee400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.946+0000 7f6eea7fc640 1 -- 172.21.15.16:0/1884197688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4011830 con 0x7f6ef40cfea0 2023-11-24T06:27:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.946+0000 7f6eea7fc640 1 -- 172.21.15.16:0/1884197688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee40119b0 con 0x7f6ef40cfea0 2023-11-24T06:27:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.946+0000 7f6eea7fc640 1 -- 172.21.15.16:0/1884197688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4011d60 con 0x7f6ef40cfea0 2023-11-24T06:27:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.946+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef40cc1a0 con 0x7f6ef40cfea0 2023-11-24T06:27:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.946+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef40ccee0 con 0x7f6ef40cfea0 2023-11-24T06:27:40.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.947+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef40cc6c0 con 0x7f6ef40cfea0 2023-11-24T06:27:40.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.948+0000 7f6eea7fc640 1 -- 172.21.15.16:0/1884197688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ee401a400 con 0x7f6ef40cfea0 2023-11-24T06:27:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.949+0000 7f6eea7fc640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ed806ae40 0x7f6ed806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.949+0000 7f6eea7fc640 1 -- 172.21.15.16:0/1884197688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ee40897e0 con 0x7f6ef40cfea0 2023-11-24T06:27:40.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.950+0000 7f6ef9c6b640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ed806ae40 0x7f6ed806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:40.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.951+0000 7f6ef9c6b640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ed806ae40 0x7f6ed806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009710 tx=0x7f6ef0006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:40.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:40.953+0000 7f6eea7fc640 1 -- 172.21.15.16:0/1884197688 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ef40cc6c0 con 0x7f6ef40cfea0 2023-11-24T06:27:41.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:41.216+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f6ef40cc6c0 con 0x7f6ef40cfea0 2023-11-24T06:27:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.014+0000 7f6eea7fc640 1 -- 172.21.15.16:0/1884197688 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v59) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6ef40cc6c0 con 0x7f6ef40cfea0 2023-11-24T06:27:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ed806ae40 msgr2=0x7f6ed806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ed806ae40 0x7f6ed806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009710 tx=0x7f6ef0006c90 comp rx=0 tx=0).stop 2023-11-24T06:27:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 msgr2=0x7f6ef4160f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 0x7f6ef4160f70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400b3e0 tx=0x7f6ee400b8a0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 shutdown_connections 2023-11-24T06:27:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6ed806ae40 0x7f6ed806d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ef4160920 0x7f6ef41f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ef40cfea0 0x7f6ef4160f70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 --2- 172.21.15.16:0/1884197688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef40cf9f0 0x7f6ef40cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 >> 172.21.15.16:0/1884197688 conn(0x7f6ef40d3090 msgr2=0x7f6ef40c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 shutdown_connections 2023-11-24T06:27:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.019+0000 7f6efac6d640 1 -- 172.21.15.16:0/1884197688 wait complete. 2023-11-24T06:27:42.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:27:42.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.273+0000 7f20b4a8c640 1 Processor -- start 2023-11-24T06:27:42.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.273+0000 7f20b4a8c640 1 -- start start 2023-11-24T06:27:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.274+0000 7f20b4a8c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f20b00cef70 0x7f20b006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.275+0000 7f20b4a8c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20b0071290 0x7f20b01539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.275+0000 7f20b4a8c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b006aea0 con 0x7f20b006a920 2023-11-24T06:27:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.275+0000 7f20b4a8c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b00cd390 con 0x7f20b0071290 2023-11-24T06:27:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.275+0000 7f20b4a8c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b006b180 con 0x7f20b00cef70 2023-11-24T06:27:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.275+0000 7f20aeffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20b006a920 0x7f20b0070b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52630/0 (socket says 172.21.15.16:52630) 2023-11-24T06:27:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.275+0000 7f20aeffd640 1 -- 172.21.15.16:0/4133478006 learned_addr learned my addr 172.21.15.16:0/4133478006 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.275+0000 7f20affff640 1 --2- 172.21.15.16:0/4133478006 >> v2:172.21.15.16:3300/0 conn(0x7f20b0071290 0x7f20b01539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.275+0000 7f20af7fe640 1 --2- 172.21.15.16:0/4133478006 >> v2:172.21.15.173:3300/0 conn(0x7f20b00cef70 0x7f20b006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.276+0000 7f20affff640 1 -- 172.21.15.16:0/4133478006 >> v2:172.21.15.173:3300/0 conn(0x7f20b00cef70 msgr2=0x7f20b006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.276+0000 7f20affff640 1 --2- 172.21.15.16:0/4133478006 >> v2:172.21.15.173:3300/0 conn(0x7f20b00cef70 0x7f20b006a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.276+0000 7f20affff640 1 -- 172.21.15.16:0/4133478006 >> v1:172.21.15.16:6789/0 conn(0x7f20b006a920 legacy=0x7f20b0070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.276+0000 7f20affff640 1 -- 172.21.15.16:0/4133478006 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0160db0 con 0x7f20b0071290 2023-11-24T06:27:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.277+0000 7f20affff640 1 --2- 172.21.15.16:0/4133478006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 0x7f20b01539b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f20b00ccd40 tx=0x7f20a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4ebe7b503fc40e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.277+0000 7f20ae7fc640 1 -- 172.21.15.16:0/4133478006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8031830 con 0x7f20b0071290 2023-11-24T06:27:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.277+0000 7f20ae7fc640 1 -- 172.21.15.16:0/4133478006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a8031990 con 0x7f20b0071290 2023-11-24T06:27:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.277+0000 7f20ae7fc640 1 -- 172.21.15.16:0/4133478006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8031d20 con 0x7f20b0071290 2023-11-24T06:27:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.278+0000 7f20b4a8c640 1 -- 172.21.15.16:0/4133478006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 msgr2=0x7f20b01539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.278+0000 7f20b4a8c640 1 --2- 172.21.15.16:0/4133478006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 0x7f20b01539b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f20b00ccd40 tx=0x7f20a802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.278+0000 7f20b4a8c640 1 -- 172.21.15.16:0/4133478006 shutdown_connections 2023-11-24T06:27:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.278+0000 7f20b4a8c640 1 --2- 172.21.15.16:0/4133478006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 0x7f20b01539b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.278+0000 7f20b4a8c640 1 --2- 172.21.15.16:0/4133478006 >> v2:172.21.15.173:3300/0 conn(0x7f20b00cef70 0x7f20b006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.278+0000 7f20b4a8c640 1 -- 172.21.15.16:0/4133478006 >> 172.21.15.16:0/4133478006 conn(0x7f20b00cc1d0 msgr2=0x7f20b00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.278+0000 7f20b4a8c640 1 -- 172.21.15.16:0/4133478006 shutdown_connections 2023-11-24T06:27:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.278+0000 7f20b4a8c640 1 -- 172.21.15.16:0/4133478006 wait complete. 2023-11-24T06:27:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.280+0000 7f20b4a8c640 1 Processor -- start 2023-11-24T06:27:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.280+0000 7f20b4a8c640 1 -- start start 2023-11-24T06:27:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20b4a8c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b006a920 0x7f20b0156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20b4a8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 0x7f20b0153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20b4a8c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b00cef70 0x7f20b01e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20b4a8c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b015c140 con 0x7f20b0071290 2023-11-24T06:27:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20b4a8c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00b1230 con 0x7f20b006a920 2023-11-24T06:27:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20b4a8c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00b0f50 con 0x7f20b00cef70 2023-11-24T06:27:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20aeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 0x7f20b0153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20aeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 0x7f20b0153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48522/0 (socket says 172.21.15.16:48522) 2023-11-24T06:27:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20aeffd640 1 -- 172.21.15.16:0/2210862089 learned_addr learned my addr 172.21.15.16:0/2210862089 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20affff640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b00cef70 0x7f20b01e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20af7fe640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b006a920 0x7f20b0156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20aeffd640 1 -- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b00cef70 msgr2=0x7f20b01e6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20aeffd640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b00cef70 0x7f20b01e6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20aeffd640 1 -- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b006a920 msgr2=0x7f20b0156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20aeffd640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b006a920 0x7f20b0156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.281+0000 7f20aeffd640 1 -- 172.21.15.16:0/2210862089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a8009400 con 0x7f20b0071290 2023-11-24T06:27:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.282+0000 7f20aeffd640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 0x7f20b0153ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f209800c3c0 tx=0x7f209800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.282+0000 7f20a7fff640 1 -- 172.21.15.16:0/2210862089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980074e0 con 0x7f20b0071290 2023-11-24T06:27:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.282+0000 7f20a7fff640 1 -- 172.21.15.16:0/2210862089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2098007640 con 0x7f20b0071290 2023-11-24T06:27:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.282+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b015baf0 con 0x7f20b0071290 2023-11-24T06:27:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.282+0000 7f20a7fff640 1 -- 172.21.15.16:0/2210862089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980059a0 con 0x7f20b0071290 2023-11-24T06:27:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.282+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b00b0be0 con 0x7f20b0071290 2023-11-24T06:27:42.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.283+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b0067b60 con 0x7f20b0071290 2023-11-24T06:27:42.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.284+0000 7f20a7fff640 1 -- 172.21.15.16:0/2210862089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2098005b00 con 0x7f20b0071290 2023-11-24T06:27:42.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.285+0000 7f20a7fff640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209006ae40 0x7f209006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.285+0000 7f20a7fff640 1 -- 172.21.15.16:0/2210862089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2098088a90 con 0x7f20b0071290 2023-11-24T06:27:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.286+0000 7f20af7fe640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209006ae40 0x7f209006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:42.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.287+0000 7f20af7fe640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209006ae40 0x7f209006d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009f30 tx=0x7f20a000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:42.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.290+0000 7f20a7fff640 1 -- 172.21.15.16:0/2210862089 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2098053b50 con 0x7f20b0071290 2023-11-24T06:27:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.483+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f20b00668e0 con 0x7f20b0071290 2023-11-24T06:27:42.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.489+0000 7f20a7fff640 1 -- 172.21.15.16:0/2210862089 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2098017020 con 0x7f20b0071290 2023-11-24T06:27:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209006ae40 msgr2=0x7f209006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209006ae40 0x7f209006d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009f30 tx=0x7f20a000b040 comp rx=0 tx=0).stop 2023-11-24T06:27:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 msgr2=0x7f20b0153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 0x7f20b0153ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f209800c3c0 tx=0x7f209800c880 comp rx=0 tx=0).stop 2023-11-24T06:27:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 shutdown_connections 2023-11-24T06:27:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209006ae40 0x7f209006d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b00cef70 0x7f20b01e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b0071290 0x7f20b0153ca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 --2- 172.21.15.16:0/2210862089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b006a920 0x7f20b0156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.491+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 >> 172.21.15.16:0/2210862089 conn(0x7f20b00cc1d0 msgr2=0x7f20b00accb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.492+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 shutdown_connections 2023-11-24T06:27:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:42.492+0000 7f20b4a8c640 1 -- 172.21.15.16:0/2210862089 wait complete. 2023-11-24T06:27:42.514 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:27:42.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:27:42.643 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:27:42.768 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:27:42.768 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:27:42.768 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:42.832 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:27:42.832 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:42.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:43.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.117+0000 7f278c368640 1 Processor -- start 2023-11-24T06:27:43.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.117+0000 7f278c368640 1 -- start start 2023-11-24T06:27:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.118+0000 7f278c368640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2784143ae0 0x7f27840cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.118+0000 7f278c368640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27840cf7d0 0x7f278415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.118+0000 7f278c368640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f278415e1e0 0x7f278415a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.118+0000 7f278c368640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840cfc30 con 0x7f278415e1e0 2023-11-24T06:27:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.118+0000 7f278c368640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840b1ee0 con 0x7f2784143ae0 2023-11-24T06:27:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.118+0000 7f278c368640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27840b2780 con 0x7f27840cf7d0 2023-11-24T06:27:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a905640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f278415e1e0 0x7f278415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a905640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f278415e1e0 0x7f278415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33586/0 (socket says 172.21.15.16:33586) 2023-11-24T06:27:43.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a905640 1 -- 172.21.15.16:0/2981793268 learned_addr learned my addr 172.21.15.16:0/2981793268 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:43.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a104640 1 --2- 172.21.15.16:0/2981793268 >> v2:172.21.15.173:3300/0 conn(0x7f2784143ae0 0x7f27840cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a905640 1 -- 172.21.15.16:0/2981793268 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27840cf7d0 msgr2=0x7f278415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a905640 1 --2- 172.21.15.16:0/2981793268 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27840cf7d0 0x7f278415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a905640 1 -- 172.21.15.16:0/2981793268 >> v2:172.21.15.173:3300/0 conn(0x7f2784143ae0 msgr2=0x7f27840cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a905640 1 --2- 172.21.15.16:0/2981793268 >> v2:172.21.15.173:3300/0 conn(0x7f2784143ae0 0x7f27840cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a905640 1 -- 172.21.15.16:0/2981793268 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278415e990 con 0x7f278415e1e0 2023-11-24T06:27:43.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.119+0000 7f278a104640 1 --2- 172.21.15.16:0/2981793268 >> v2:172.21.15.173:3300/0 conn(0x7f2784143ae0 0x7f27840cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:43.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.121+0000 7f278a905640 1 --2- 172.21.15.16:0/2981793268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278415e1e0 0x7f278415a670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f27840cfea0 tx=0x7f277c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a07f74a1a03cab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.121+0000 7f2789102640 1 -- 172.21.15.16:0/2981793268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c02ca70 con 0x7f278415e1e0 2023-11-24T06:27:43.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.121+0000 7f2789102640 1 -- 172.21.15.16:0/2981793268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c02cbd0 con 0x7f278415e1e0 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.121+0000 7f2789102640 1 -- 172.21.15.16:0/2981793268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c0343f0 con 0x7f278415e1e0 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.121+0000 7f278c368640 1 -- 172.21.15.16:0/2981793268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278415e1e0 msgr2=0x7f278415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.121+0000 7f278c368640 1 --2- 172.21.15.16:0/2981793268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278415e1e0 0x7f278415a670 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f27840cfea0 tx=0x7f277c02bed0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.122+0000 7f278c368640 1 -- 172.21.15.16:0/2981793268 shutdown_connections 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.122+0000 7f278c368640 1 --2- 172.21.15.16:0/2981793268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f278415e1e0 0x7f278415a670 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.122+0000 7f278c368640 1 --2- 172.21.15.16:0/2981793268 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27840cf7d0 0x7f278415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.122+0000 7f278c368640 1 --2- 172.21.15.16:0/2981793268 >> v2:172.21.15.173:3300/0 conn(0x7f2784143ae0 0x7f27840cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.122+0000 7f278c368640 1 -- 172.21.15.16:0/2981793268 >> 172.21.15.16:0/2981793268 conn(0x7f278406a800 msgr2=0x7f278406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.122+0000 7f278c368640 1 -- 172.21.15.16:0/2981793268 shutdown_connections 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.122+0000 7f278c368640 1 -- 172.21.15.16:0/2981793268 wait complete. 2023-11-24T06:27:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.124+0000 7f278c368640 1 Processor -- start 2023-11-24T06:27:43.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.124+0000 7f278c368640 1 -- start start 2023-11-24T06:27:43.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278c368640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840cf7d0 0x7f278412e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278c368640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143ae0 0x7f278415c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278c368640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f278415e1e0 0x7f278415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278c368640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2784161b30 con 0x7f27840cf7d0 2023-11-24T06:27:43.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278c368640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2784160eb0 con 0x7f278415e1e0 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278c368640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2784161190 con 0x7f2784143ae0 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278a104640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840cf7d0 0x7f278412e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278a104640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840cf7d0 0x7f278412e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33592/0 (socket says 172.21.15.16:33592) 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278a104640 1 -- 172.21.15.16:0/4217013153 learned_addr learned my addr 172.21.15.16:0/4217013153 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278a104640 1 -- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143ae0 msgr2=0x7f278415c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278a104640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143ae0 0x7f278415c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278a104640 1 -- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f278415e1e0 msgr2=0x7f278415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278a104640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f278415e1e0 0x7f278415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.125+0000 7f278a104640 1 -- 172.21.15.16:0/4217013153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c009b30 con 0x7f27840cf7d0 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.126+0000 7f278a104640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840cf7d0 0x7f278412e470 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f278000c7f0 tx=0x7f278000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.126+0000 7f277affd640 1 -- 172.21.15.16:0/4217013153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780007a30 con 0x7f27840cf7d0 2023-11-24T06:27:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.126+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2784161470 con 0x7f27840cf7d0 2023-11-24T06:27:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.126+0000 7f277affd640 1 -- 172.21.15.16:0/4217013153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2780007b90 con 0x7f27840cf7d0 2023-11-24T06:27:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.126+0000 7f277affd640 1 -- 172.21.15.16:0/4217013153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278000ce40 con 0x7f27840cf7d0 2023-11-24T06:27:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.126+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278412e9b0 con 0x7f27840cf7d0 2023-11-24T06:27:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.127+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27840675d0 con 0x7f27840cf7d0 2023-11-24T06:27:43.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.128+0000 7f277affd640 1 -- 172.21.15.16:0/4217013153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27800113f0 con 0x7f27840cf7d0 2023-11-24T06:27:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.129+0000 7f277affd640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.129+0000 7f277affd640 1 -- 172.21.15.16:0/4217013153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f2780089990 con 0x7f27840cf7d0 2023-11-24T06:27:43.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.130+0000 7f2789903640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.131+0000 7f2789903640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f27840b3810 tx=0x7f2774006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.134+0000 7f277affd640 1 -- 172.21.15.16:0/4217013153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27800548a0 con 0x7f27840cf7d0 2023-11-24T06:27:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.334+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f278415d3f0 con 0x7f27840cf7d0 2023-11-24T06:27:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.336+0000 7f277affd640 1 -- 172.21.15.16:0/4217013153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16773 (secure 0 0 0) 0x7f2780057ec0 con 0x7f27840cf7d0 2023-11-24T06:27:43.340 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:43.340 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:43.016216+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 msgr2=0x7f276806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f27840b3810 tx=0x7f2774006c90 comp rx=0 tx=0).stop 2023-11-24T06:27:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840cf7d0 msgr2=0x7f278412e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840cf7d0 0x7f278412e470 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f278000c7f0 tx=0x7f278000cbf0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 shutdown_connections 2023-11-24T06:27:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f278415e1e0 0x7f278415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143ae0 0x7f278415c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 --2- 172.21.15.16:0/4217013153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840cf7d0 0x7f278412e470 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 >> 172.21.15.16:0/4217013153 conn(0x7f278406a800 msgr2=0x7f278406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 shutdown_connections 2023-11-24T06:27:43.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.340+0000 7f278c368640 1 -- 172.21.15.16:0/4217013153 wait complete. 2023-11-24T06:27:43.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:43.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.583+0000 7f2adb59e640 1 Processor -- start 2023-11-24T06:27:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.583+0000 7f2adb59e640 1 -- start start 2023-11-24T06:27:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.585+0000 7f2adb59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2adc13cf10 0x7f2adc13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.585+0000 7f2adb59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2adc0cddf0 0x7f2adc0b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.585+0000 7f2adb59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2adc0ce440 con 0x7f2adc13fae0 2023-11-24T06:27:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.585+0000 7f2adb59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2adc13ff90 con 0x7f2adc13cf10 2023-11-24T06:27:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.585+0000 7f2adb59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2adc0ce220 con 0x7f2adc0cddf0 2023-11-24T06:27:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2ada59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2adc13cf10 0x7f2adc13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2ad3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2adc13fae0 0x7f2adc0cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42434/0 (socket says 172.21.15.16:42434) 2023-11-24T06:27:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2ad3fff640 1 -- 172.21.15.16:0/4165687132 learned_addr learned my addr 172.21.15.16:0/4165687132 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2adad9d640 1 --2- 172.21.15.16:0/4165687132 >> v2:172.21.15.173:3300/0 conn(0x7f2adc0cddf0 0x7f2adc0b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2ad9d9b640 1 -- 172.21.15.16:0/4165687132 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671820281 0 0) 0x7f2adc0ce440 con 0x7f2adc13fae0 2023-11-24T06:27:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2ad9d9b640 1 -- 172.21.15.16:0/4165687132 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ac8003520 con 0x7f2adc13fae0 2023-11-24T06:27:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2ada59c640 1 -- 172.21.15.16:0/4165687132 >> v2:172.21.15.173:3300/0 conn(0x7f2adc0cddf0 msgr2=0x7f2adc0b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2ada59c640 1 --2- 172.21.15.16:0/4165687132 >> v2:172.21.15.173:3300/0 conn(0x7f2adc0cddf0 0x7f2adc0b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2ada59c640 1 -- 172.21.15.16:0/4165687132 >> v1:172.21.15.16:6789/0 conn(0x7f2adc13fae0 legacy=0x7f2adc0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2ada59c640 1 -- 172.21.15.16:0/4165687132 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc0ce720 con 0x7f2adc13cf10 2023-11-24T06:27:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.586+0000 7f2adad9d640 1 --2- 172.21.15.16:0/4165687132 >> v2:172.21.15.173:3300/0 conn(0x7f2adc0cddf0 0x7f2adc0b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.588+0000 7f2ada59c640 1 --2- 172.21.15.16:0/4165687132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13cf10 0x7f2adc13f5a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2adc157f90 tx=0x7f2acc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e43e2415c6948e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.588+0000 7f2ad9d9b640 1 -- 172.21.15.16:0/4165687132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc031830 con 0x7f2adc13cf10 2023-11-24T06:27:43.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.588+0000 7f2ad9d9b640 1 -- 172.21.15.16:0/4165687132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc031990 con 0x7f2adc13cf10 2023-11-24T06:27:43.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.588+0000 7f2ad9d9b640 1 -- 172.21.15.16:0/4165687132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc031d20 con 0x7f2adc13cf10 2023-11-24T06:27:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.589+0000 7f2adb59e640 1 -- 172.21.15.16:0/4165687132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13cf10 msgr2=0x7f2adc13f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.589+0000 7f2adb59e640 1 --2- 172.21.15.16:0/4165687132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13cf10 0x7f2adc13f5a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2adc157f90 tx=0x7f2acc02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.589+0000 7f2adb59e640 1 -- 172.21.15.16:0/4165687132 shutdown_connections 2023-11-24T06:27:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.589+0000 7f2adb59e640 1 --2- 172.21.15.16:0/4165687132 >> v2:172.21.15.173:3300/0 conn(0x7f2adc0cddf0 0x7f2adc0b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.589+0000 7f2adb59e640 1 --2- 172.21.15.16:0/4165687132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13cf10 0x7f2adc13f5a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.589+0000 7f2adb59e640 1 -- 172.21.15.16:0/4165687132 >> 172.21.15.16:0/4165687132 conn(0x7f2adc06a7d0 msgr2=0x7f2adc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.589+0000 7f2adb59e640 1 -- 172.21.15.16:0/4165687132 shutdown_connections 2023-11-24T06:27:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.589+0000 7f2adb59e640 1 -- 172.21.15.16:0/4165687132 wait complete. 2023-11-24T06:27:43.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.591+0000 7f2adb59e640 1 Processor -- start 2023-11-24T06:27:43.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.591+0000 7f2adb59e640 1 -- start start 2023-11-24T06:27:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.592+0000 7f2adb59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2adc0cddf0 0x7f2adc154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.592+0000 7f2adb59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2adc13cf10 0x7f2adc0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.592+0000 7f2adb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13fae0 0x7f2adc137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.592+0000 7f2adb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc13e390 con 0x7f2adc13fae0 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.592+0000 7f2adb59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc14a940 con 0x7f2adc13cf10 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.592+0000 7f2adb59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc13d780 con 0x7f2adc0cddf0 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2adad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13fae0 0x7f2adc137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2adad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13fae0 0x7f2adc137ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33616/0 (socket says 172.21.15.16:33616) 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2adad9d640 1 -- 172.21.15.16:0/1094554473 learned_addr learned my addr 172.21.15.16:0/1094554473 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2ad3fff640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2adc13cf10 0x7f2adc0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2ada59c640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2adc0cddf0 0x7f2adc154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2adad9d640 1 -- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2adc0cddf0 msgr2=0x7f2adc154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2adad9d640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2adc0cddf0 0x7f2adc154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2adad9d640 1 -- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2adc13cf10 msgr2=0x7f2adc0adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2adad9d640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2adc13cf10 0x7f2adc0adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2adad9d640 1 -- 172.21.15.16:0/1094554473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc009400 con 0x7f2adc13fae0 2023-11-24T06:27:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2ada59c640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2adc0cddf0 0x7f2adc154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.593+0000 7f2adad9d640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13fae0 0x7f2adc137ea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400cef0 tx=0x7f2ad400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.594+0000 7f2ad2ffd640 1 -- 172.21.15.16:0/1094554473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad400d840 con 0x7f2adc13fae0 2023-11-24T06:27:43.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.594+0000 7f2ad2ffd640 1 -- 172.21.15.16:0/1094554473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad400d9c0 con 0x7f2adc13fae0 2023-11-24T06:27:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.594+0000 7f2ad2ffd640 1 -- 172.21.15.16:0/1094554473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad400dd70 con 0x7f2adc13fae0 2023-11-24T06:27:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.594+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2adc13da60 con 0x7f2adc13fae0 2023-11-24T06:27:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.594+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2adc13e020 con 0x7f2adc13fae0 2023-11-24T06:27:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.595+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2adc13e540 con 0x7f2adc13fae0 2023-11-24T06:27:43.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.596+0000 7f2ad2ffd640 1 -- 172.21.15.16:0/1094554473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ad401e3b0 con 0x7f2adc13fae0 2023-11-24T06:27:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.597+0000 7f2ad2ffd640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2abc06ae40 0x7f2abc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:43.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.597+0000 7f2ad2ffd640 1 -- 172.21.15.16:0/1094554473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f2ad408a1d0 con 0x7f2adc13fae0 2023-11-24T06:27:43.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.598+0000 7f2ada59c640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2abc06ae40 0x7f2abc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:43.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.599+0000 7f2ada59c640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2abc06ae40 0x7f2abc06d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2acc036000 tx=0x7f2acc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:43.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.601+0000 7f2ad2ffd640 1 -- 172.21.15.16:0/1094554473 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2adc13e540 con 0x7f2adc13fae0 2023-11-24T06:27:43.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.792+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2adc067450 con 0x7f2adc13fae0 2023-11-24T06:27:43.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.793+0000 7f2ad2ffd640 1 -- 172.21.15.16:0/1094554473 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ad4055220 con 0x7f2adc13fae0 2023-11-24T06:27:43.796 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:43.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2abc06ae40 msgr2=0x7f2abc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2abc06ae40 0x7f2abc06d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2acc036000 tx=0x7f2acc008030 comp rx=0 tx=0).stop 2023-11-24T06:27:43.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13fae0 msgr2=0x7f2adc137ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:43.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13fae0 0x7f2adc137ea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400cef0 tx=0x7f2ad400a710 comp rx=0 tx=0).stop 2023-11-24T06:27:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 shutdown_connections 2023-11-24T06:27:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2abc06ae40 0x7f2abc06d300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2adc13fae0 0x7f2adc137ea0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2adc13cf10 0x7f2adc0adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 --2- 172.21.15.16:0/1094554473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2adc0cddf0 0x7f2adc154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:43.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 >> 172.21.15.16:0/1094554473 conn(0x7f2adc06a7d0 msgr2=0x7f2adc06cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:43.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 shutdown_connections 2023-11-24T06:27:43.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:43.796+0000 7f2adb59e640 1 -- 172.21.15.16:0/1094554473 wait complete. 2023-11-24T06:27:43.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.048+0000 7f4c6f090640 1 Processor -- start 2023-11-24T06:27:44.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.048+0000 7f4c6f090640 1 -- start start 2023-11-24T06:27:44.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.051+0000 7f4c6f090640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4c680af4c0 0x7f4c68167ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.051+0000 7f4c6f090640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680b3670 con 0x7f4c6806d7d0 2023-11-24T06:27:44.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.051+0000 7f4c6f090640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680b1d20 con 0x7f4c6815d240 2023-11-24T06:27:44.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.051+0000 7f4c6f090640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c680b38d0 con 0x7f4c680af4c0 2023-11-24T06:27:44.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.051+0000 7f4c67fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4c6806d7d0 0x7f4c680aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42440/0 (socket says 172.21.15.16:42440) 2023-11-24T06:27:44.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.051+0000 7f4c67fff640 1 -- 172.21.15.16:0/2965272654 learned_addr learned my addr 172.21.15.16:0/2965272654 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.051+0000 7f4c6d62d640 1 --2- 172.21.15.16:0/2965272654 >> v2:172.21.15.173:3300/0 conn(0x7f4c680af4c0 0x7f4c68167ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.052+0000 7f4c677fe640 1 -- 172.21.15.16:0/2965272654 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856408279 0 0) 0x7f4c680b3670 con 0x7f4c6806d7d0 2023-11-24T06:27:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.052+0000 7f4c677fe640 1 -- 172.21.15.16:0/2965272654 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c54003520 con 0x7f4c6806d7d0 2023-11-24T06:27:44.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.053+0000 7f4c6d62d640 1 -- 172.21.15.16:0/2965272654 >> v1:172.21.15.173:6789/0 conn(0x7f4c6815d240 legacy=0x7f4c680b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.053+0000 7f4c6d62d640 1 -- 172.21.15.16:0/2965272654 >> v1:172.21.15.16:6789/0 conn(0x7f4c6806d7d0 legacy=0x7f4c680aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.053+0000 7f4c6d62d640 1 -- 172.21.15.16:0/2965272654 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680b1fc0 con 0x7f4c680af4c0 2023-11-24T06:27:44.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.054+0000 7f4c6d62d640 1 --2- 172.21.15.16:0/2965272654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c680af4c0 0x7f4c68167ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009d90 tx=0x7f4c5002bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=778f60a06de5d2c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.055+0000 7f4c677fe640 1 -- 172.21.15.16:0/2965272654 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5002cc70 con 0x7f4c680af4c0 2023-11-24T06:27:44.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.055+0000 7f4c677fe640 1 -- 172.21.15.16:0/2965272654 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5002cdd0 con 0x7f4c680af4c0 2023-11-24T06:27:44.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.055+0000 7f4c677fe640 1 -- 172.21.15.16:0/2965272654 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50034640 con 0x7f4c680af4c0 2023-11-24T06:27:44.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.056+0000 7f4c6f090640 1 -- 172.21.15.16:0/2965272654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c680af4c0 msgr2=0x7f4c68167ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.056+0000 7f4c6f090640 1 --2- 172.21.15.16:0/2965272654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c680af4c0 0x7f4c68167ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009d90 tx=0x7f4c5002bec0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.056+0000 7f4c6f090640 1 -- 172.21.15.16:0/2965272654 shutdown_connections 2023-11-24T06:27:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.056+0000 7f4c6f090640 1 --2- 172.21.15.16:0/2965272654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c680af4c0 0x7f4c68167ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.056+0000 7f4c6f090640 1 -- 172.21.15.16:0/2965272654 >> 172.21.15.16:0/2965272654 conn(0x7f4c6806b4f0 msgr2=0x7f4c6806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:44.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.056+0000 7f4c6f090640 1 -- 172.21.15.16:0/2965272654 shutdown_connections 2023-11-24T06:27:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.056+0000 7f4c6f090640 1 -- 172.21.15.16:0/2965272654 wait complete. 2023-11-24T06:27:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.058+0000 7f4c6f090640 1 Processor -- start 2023-11-24T06:27:44.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.058+0000 7f4c6f090640 1 -- start start 2023-11-24T06:27:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.059+0000 7f4c6f090640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c6806d7d0 0x7f4c6814ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.059+0000 7f4c6f090640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c680af4c0 0x7f4c680b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.059+0000 7f4c6f090640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c6815d240 0x7f4c6806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.059+0000 7f4c6f090640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68159220 con 0x7f4c680af4c0 2023-11-24T06:27:44.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.059+0000 7f4c6f090640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681596b0 con 0x7f4c6806d7d0 2023-11-24T06:27:44.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.059+0000 7f4c6f090640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68158610 con 0x7f4c6815d240 2023-11-24T06:27:44.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.059+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c680af4c0 0x7f4c680b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.059+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c680af4c0 0x7f4c680b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33626/0 (socket says 172.21.15.16:33626) 2023-11-24T06:27:44.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.059+0000 7f4c67fff640 1 -- 172.21.15.16:0/1822843598 learned_addr learned my addr 172.21.15.16:0/1822843598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:44.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c6ce2c640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c6806d7d0 0x7f4c6814ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c6d62d640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c6815d240 0x7f4c6806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c67fff640 1 -- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c6815d240 msgr2=0x7f4c6806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c67fff640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c6815d240 0x7f4c6806eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c67fff640 1 -- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c6806d7d0 msgr2=0x7f4c6814ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c67fff640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c6806d7d0 0x7f4c6814ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c67fff640 1 -- 172.21.15.16:0/1822843598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c50009a70 con 0x7f4c680af4c0 2023-11-24T06:27:44.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c67fff640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c680af4c0 0x7f4c680b11c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c58001eb0 tx=0x7f4c5800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c657fa640 1 -- 172.21.15.16:0/1822843598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58005ce0 con 0x7f4c680af4c0 2023-11-24T06:27:44.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c657fa640 1 -- 172.21.15.16:0/1822843598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c58005e40 con 0x7f4c680af4c0 2023-11-24T06:27:44.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c681588f0 con 0x7f4c680af4c0 2023-11-24T06:27:44.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.060+0000 7f4c657fa640 1 -- 172.21.15.16:0/1822843598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58007270 con 0x7f4c680af4c0 2023-11-24T06:27:44.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.061+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c68158eb0 con 0x7f4c680af4c0 2023-11-24T06:27:44.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.062+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c680675d0 con 0x7f4c680af4c0 2023-11-24T06:27:44.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.063+0000 7f4c657fa640 1 -- 172.21.15.16:0/1822843598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c58007490 con 0x7f4c680af4c0 2023-11-24T06:27:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.064+0000 7f4c657fa640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4c4806ae40 0x7f4c4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.064+0000 7f4c657fa640 1 -- 172.21.15.16:0/1822843598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f4c58088cb0 con 0x7f4c680af4c0 2023-11-24T06:27:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.064+0000 7f4c6ce2c640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4c4806ae40 0x7f4c4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.067+0000 7f4c6ce2c640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4c4806ae40 0x7f4c4806d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4c60000f60 tx=0x7f4c60009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.069+0000 7f4c657fa640 1 -- 172.21.15.16:0/1822843598 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c58053cf0 con 0x7f4c680af4c0 2023-11-24T06:27:44.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.266+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c68159360 con 0x7f4c680af4c0 2023-11-24T06:27:44.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.267+0000 7f4c657fa640 1 -- 172.21.15.16:0/1822843598 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c58057310 con 0x7f4c680af4c0 2023-11-24T06:27:44.271 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4c4806ae40 msgr2=0x7f4c4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4c4806ae40 0x7f4c4806d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4c60000f60 tx=0x7f4c60009040 comp rx=0 tx=0).stop 2023-11-24T06:27:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c680af4c0 msgr2=0x7f4c680b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c680af4c0 0x7f4c680b11c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c58001eb0 tx=0x7f4c5800c960 comp rx=0 tx=0).stop 2023-11-24T06:27:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 shutdown_connections 2023-11-24T06:27:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4c4806ae40 0x7f4c4806d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c6815d240 0x7f4c6806eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c680af4c0 0x7f4c680b11c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 --2- 172.21.15.16:0/1822843598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c6806d7d0 0x7f4c6814ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 >> 172.21.15.16:0/1822843598 conn(0x7f4c6806b4f0 msgr2=0x7f4c6815ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 shutdown_connections 2023-11-24T06:27:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.270+0000 7f4c6f090640 1 -- 172.21.15.16:0/1822843598 wait complete. 2023-11-24T06:27:44.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.516+0000 7f662d1ef640 1 Processor -- start 2023-11-24T06:27:44.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.516+0000 7f662d1ef640 1 -- start start 2023-11-24T06:27:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.517+0000 7f662d1ef640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f662815d4b0 0x7f66280d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.517+0000 7f662d1ef640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f66280b41e0 0x7f66280cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.517+0000 7f662d1ef640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66280d0320 con 0x7f66280cf9a0 2023-11-24T06:27:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.517+0000 7f662d1ef640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66280b2730 con 0x7f662815d4b0 2023-11-24T06:27:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.517+0000 7f662d1ef640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66280d1ec0 con 0x7f66280b41e0 2023-11-24T06:27:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.518+0000 7f6627fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f662815d4b0 0x7f66280d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.518+0000 7f661f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f66280cf9a0 0x7f66280cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42450/0 (socket says 172.21.15.16:42450) 2023-11-24T06:27:44.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.518+0000 7f661f7fe640 1 -- 172.21.15.16:0/759637486 learned_addr learned my addr 172.21.15.16:0/759637486 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.518+0000 7f662c9ee640 1 --2- 172.21.15.16:0/759637486 >> v2:172.21.15.173:3300/0 conn(0x7f66280b41e0 0x7f66280cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.518+0000 7f6627fff640 1 -- 172.21.15.16:0/759637486 >> v2:172.21.15.173:3300/0 conn(0x7f66280b41e0 msgr2=0x7f66280cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.518+0000 7f6627fff640 1 --2- 172.21.15.16:0/759637486 >> v2:172.21.15.173:3300/0 conn(0x7f66280b41e0 0x7f66280cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.518+0000 7f6627fff640 1 -- 172.21.15.16:0/759637486 >> v1:172.21.15.16:6789/0 conn(0x7f66280cf9a0 legacy=0x7f66280cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.518+0000 7f6627fff640 1 -- 172.21.15.16:0/759637486 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66280b29d0 con 0x7f662815d4b0 2023-11-24T06:27:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.520+0000 7f6627fff640 1 --2- 172.21.15.16:0/759637486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662815d4b0 0x7f66280d1550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66280d0680 tx=0x7f661802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0853885327127b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.520+0000 7f66277fe640 1 -- 172.21.15.16:0/759637486 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618031830 con 0x7f662815d4b0 2023-11-24T06:27:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.520+0000 7f66277fe640 1 -- 172.21.15.16:0/759637486 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6618031990 con 0x7f662815d4b0 2023-11-24T06:27:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.520+0000 7f66277fe640 1 -- 172.21.15.16:0/759637486 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618031d20 con 0x7f662815d4b0 2023-11-24T06:27:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.520+0000 7f662d1ef640 1 -- 172.21.15.16:0/759637486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662815d4b0 msgr2=0x7f66280d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.520+0000 7f662d1ef640 1 --2- 172.21.15.16:0/759637486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662815d4b0 0x7f66280d1550 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66280d0680 tx=0x7f661802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.521+0000 7f662d1ef640 1 -- 172.21.15.16:0/759637486 shutdown_connections 2023-11-24T06:27:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.521+0000 7f662d1ef640 1 --2- 172.21.15.16:0/759637486 >> v2:172.21.15.173:3300/0 conn(0x7f66280b41e0 0x7f66280cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.521+0000 7f662d1ef640 1 --2- 172.21.15.16:0/759637486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662815d4b0 0x7f66280d1550 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.521+0000 7f662d1ef640 1 -- 172.21.15.16:0/759637486 >> 172.21.15.16:0/759637486 conn(0x7f66280d3000 msgr2=0x7f66280d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.521+0000 7f662d1ef640 1 -- 172.21.15.16:0/759637486 shutdown_connections 2023-11-24T06:27:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.521+0000 7f662d1ef640 1 -- 172.21.15.16:0/759637486 wait complete. 2023-11-24T06:27:44.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.522+0000 7f662d1ef640 1 Processor -- start 2023-11-24T06:27:44.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f662d1ef640 1 -- start start 2023-11-24T06:27:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f662d1ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66280b41e0 0x7f662815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f662d1ef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66280cf9a0 0x7f66280cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f662d1ef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f662815d4b0 0x7f662815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f662d1ef640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280caff0 con 0x7f66280b41e0 2023-11-24T06:27:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f662d1ef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280ccea0 con 0x7f662815d4b0 2023-11-24T06:27:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f662d1ef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66280cd180 con 0x7f66280cf9a0 2023-11-24T06:27:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f6627fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66280b41e0 0x7f662815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f6627fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66280b41e0 0x7f662815c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33642/0 (socket says 172.21.15.16:33642) 2023-11-24T06:27:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f6627fff640 1 -- 172.21.15.16:0/3923072768 learned_addr learned my addr 172.21.15.16:0/3923072768 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:44.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f6627fff640 1 -- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66280cf9a0 msgr2=0x7f66280cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f6627fff640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66280cf9a0 0x7f66280cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f6627fff640 1 -- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f662815d4b0 msgr2=0x7f662815b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f6627fff640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f662815d4b0 0x7f662815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.524+0000 7f6627fff640 1 -- 172.21.15.16:0/3923072768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618009400 con 0x7f66280b41e0 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.525+0000 7f6627fff640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66280b41e0 0x7f662815c260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6618000c00 tx=0x7f66180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.525+0000 7f66257fa640 1 -- 172.21.15.16:0/3923072768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661803aa60 con 0x7f66280b41e0 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.525+0000 7f66257fa640 1 -- 172.21.15.16:0/3923072768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661803abe0 con 0x7f66280b41e0 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.525+0000 7f66257fa640 1 -- 172.21.15.16:0/3923072768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66180443f0 con 0x7f66280b41e0 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.525+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662815da10 con 0x7f66280b41e0 2023-11-24T06:27:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.525+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66280cb950 con 0x7f66280b41e0 2023-11-24T06:27:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.526+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6628067450 con 0x7f66280b41e0 2023-11-24T06:27:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.527+0000 7f66257fa640 1 -- 172.21.15.16:0/3923072768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f661804d450 con 0x7f66280b41e0 2023-11-24T06:27:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.528+0000 7f66257fa640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660806ae40 0x7f660806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.528+0000 7f66257fa640 1 -- 172.21.15.16:0/3923072768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f66180ab830 con 0x7f66280b41e0 2023-11-24T06:27:44.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.529+0000 7f661f7fe640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660806ae40 0x7f660806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.530+0000 7f661f7fe640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660806ae40 0x7f660806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66280cc5f0 tx=0x7f6610007a40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.534+0000 7f66257fa640 1 -- 172.21.15.16:0/3923072768 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6618076850 con 0x7f66280b41e0 2023-11-24T06:27:44.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.730+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66280cd340 con 0x7f66280b41e0 2023-11-24T06:27:44.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.730+0000 7f66257fa640 1 -- 172.21.15.16:0/3923072768 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6618035020 con 0x7f66280b41e0 2023-11-24T06:27:44.734 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660806ae40 msgr2=0x7f660806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660806ae40 0x7f660806d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66280cc5f0 tx=0x7f6610007a40 comp rx=0 tx=0).stop 2023-11-24T06:27:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66280b41e0 msgr2=0x7f662815c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66280b41e0 0x7f662815c260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6618000c00 tx=0x7f66180046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 shutdown_connections 2023-11-24T06:27:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660806ae40 0x7f660806d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f662815d4b0 0x7f662815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66280cf9a0 0x7f66280cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 --2- 172.21.15.16:0/3923072768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66280b41e0 0x7f662815c260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 >> 172.21.15.16:0/3923072768 conn(0x7f66280d3000 msgr2=0x7f66280cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:44.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 shutdown_connections 2023-11-24T06:27:44.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.734+0000 7f662d1ef640 1 -- 172.21.15.16:0/3923072768 wait complete. 2023-11-24T06:27:44.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:27:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.983+0000 7f8a91ada640 1 Processor -- start 2023-11-24T06:27:44.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.983+0000 7f8a91ada640 1 -- start start 2023-11-24T06:27:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a91ada640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a8c0b34e0 0x7f8a8c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a91ada640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8a8c0b4140 0x7f8a8c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a91ada640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a8c0d1b90 0x7f8a8c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a91ada640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8c0d2020 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a91ada640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8c0d0420 con 0x7f8a8c0b4140 2023-11-24T06:27:44.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a91ada640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0d0e70 con 0x7f8a8c0d1b90 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a90ad8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a8c0b34e0 0x7f8a8c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a90ad8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a8c0b34e0 0x7f8a8c0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33652/0 (socket says 172.21.15.16:33652) 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a90ad8640 1 -- 172.21.15.16:0/258009174 learned_addr learned my addr 172.21.15.16:0/258009174 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a90ad8640 1 -- 172.21.15.16:0/258009174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a8c0d1b90 msgr2=0x7f8a8c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.985+0000 7f8a90ad8640 1 --2- 172.21.15.16:0/258009174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a8c0d1b90 0x7f8a8c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.986+0000 7f8a90ad8640 1 -- 172.21.15.16:0/258009174 >> v2:172.21.15.173:3300/0 conn(0x7f8a8c0b4140 msgr2=0x7f8a8c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.986+0000 7f8a8bfff640 1 --2- 172.21.15.16:0/258009174 >> v2:172.21.15.173:3300/0 conn(0x7f8a8c0b4140 0x7f8a8c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.986+0000 7f8a90ad8640 1 --2- 172.21.15.16:0/258009174 >> v2:172.21.15.173:3300/0 conn(0x7f8a8c0b4140 0x7f8a8c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.986+0000 7f8a90ad8640 1 -- 172.21.15.16:0/258009174 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c0d3d50 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.986+0000 7f8a8bfff640 1 --2- 172.21.15.16:0/258009174 >> v2:172.21.15.173:3300/0 conn(0x7f8a8c0b4140 0x7f8a8c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:44.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.987+0000 7f8a90ad8640 1 --2- 172.21.15.16:0/258009174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 0x7f8a8c0b3c00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0a0fa0 tx=0x7f8a8002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cad3e0cb0ed31a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.987+0000 7f8a8b7fe640 1 -- 172.21.15.16:0/258009174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80031860 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.987+0000 7f8a8b7fe640 1 -- 172.21.15.16:0/258009174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a800319c0 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.987+0000 7f8a8b7fe640 1 -- 172.21.15.16:0/258009174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80031d50 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.988+0000 7f8a91ada640 1 -- 172.21.15.16:0/258009174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 msgr2=0x7f8a8c0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.988+0000 7f8a91ada640 1 --2- 172.21.15.16:0/258009174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 0x7f8a8c0b3c00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0a0fa0 tx=0x7f8a8002b900 comp rx=0 tx=0).stop 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.988+0000 7f8a91ada640 1 -- 172.21.15.16:0/258009174 shutdown_connections 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.988+0000 7f8a91ada640 1 --2- 172.21.15.16:0/258009174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a8c0d1b90 0x7f8a8c0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.988+0000 7f8a91ada640 1 --2- 172.21.15.16:0/258009174 >> v2:172.21.15.173:3300/0 conn(0x7f8a8c0b4140 0x7f8a8c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.988+0000 7f8a91ada640 1 --2- 172.21.15.16:0/258009174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 0x7f8a8c0b3c00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.988+0000 7f8a91ada640 1 -- 172.21.15.16:0/258009174 >> 172.21.15.16:0/258009174 conn(0x7f8a8c0d3090 msgr2=0x7f8a8c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:44.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.988+0000 7f8a91ada640 1 -- 172.21.15.16:0/258009174 shutdown_connections 2023-11-24T06:27:44.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.988+0000 7f8a91ada640 1 -- 172.21.15.16:0/258009174 wait complete. 2023-11-24T06:27:44.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.990+0000 7f8a91ada640 1 Processor -- start 2023-11-24T06:27:44.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.990+0000 7f8a91ada640 1 -- start start 2023-11-24T06:27:44.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.991+0000 7f8a91ada640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 0x7f8a8c0b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.991+0000 7f8a91ada640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a8c0b4140 0x7f8a8c149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.991+0000 7f8a91ada640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a8c0d1b90 0x7f8a8c0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.991+0000 7f8a91ada640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0b7f70 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.991+0000 7f8a91ada640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c149f50 con 0x7f8a8c0d1b90 2023-11-24T06:27:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.991+0000 7f8a91ada640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0b7360 con 0x7f8a8c0b4140 2023-11-24T06:27:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.991+0000 7f8a90ad8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 0x7f8a8c0b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.991+0000 7f8a90ad8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 0x7f8a8c0b6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33664/0 (socket says 172.21.15.16:33664) 2023-11-24T06:27:44.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.992+0000 7f8a90ad8640 1 -- 172.21.15.16:0/456297928 learned_addr learned my addr 172.21.15.16:0/456297928 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:44.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.992+0000 7f8a90ad8640 1 -- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a8c0b4140 msgr2=0x7f8a8c149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.992+0000 7f8a90ad8640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a8c0b4140 0x7f8a8c149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.992+0000 7f8a912d9640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a8c0d1b90 0x7f8a8c0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.992+0000 7f8a90ad8640 1 -- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a8c0d1b90 msgr2=0x7f8a8c0c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.992+0000 7f8a90ad8640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a8c0d1b90 0x7f8a8c0c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.992+0000 7f8a90ad8640 1 -- 172.21.15.16:0/456297928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a800094d0 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.992+0000 7f8a90ad8640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 0x7f8a8c0b6cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009960 tx=0x7f8a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:44.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.993+0000 7f8a897fa640 1 -- 172.21.15.16:0/456297928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8003adf0 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.993+0000 7f8a897fa640 1 -- 172.21.15.16:0/456297928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a80045450 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.993+0000 7f8a897fa640 1 -- 172.21.15.16:0/456297928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80045800 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.993+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c0b7640 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.993+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c0b7c00 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.994+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c067450 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.995+0000 7f8a897fa640 1 -- 172.21.15.16:0/456297928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a80045960 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.996+0000 7f8a897fa640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8a7806ae40 0x7f8a7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.996+0000 7f8a897fa640 1 -- 172.21.15.16:0/456297928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f8a800abfe0 con 0x7f8a8c0b34e0 2023-11-24T06:27:44.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.997+0000 7f8a8bfff640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8a7806ae40 0x7f8a7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:44.998+0000 7f8a8bfff640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8a7806ae40 0x7f8a7806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c006fd0 tx=0x7f8a7c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.001+0000 7f8a897fa640 1 -- 172.21.15.16:0/456297928 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a80076f70 con 0x7f8a8c0b34e0 2023-11-24T06:27:45.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.211+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a8c15cf00 con 0x7f8a8c0b34e0 2023-11-24T06:27:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.212+0000 7f8a897fa640 1 -- 172.21.15.16:0/456297928 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8a8004a080 con 0x7f8a8c0b34e0 2023-11-24T06:27:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:27:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.214+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8a7806ae40 msgr2=0x7f8a7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.214+0000 7f8a91ada640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8a7806ae40 0x7f8a7806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c006fd0 tx=0x7f8a7c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.214+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 msgr2=0x7f8a8c0b6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.214+0000 7f8a91ada640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 0x7f8a8c0b6cd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009960 tx=0x7f8a800046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.215+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 shutdown_connections 2023-11-24T06:27:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.215+0000 7f8a91ada640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8a7806ae40 0x7f8a7806d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.215+0000 7f8a91ada640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a8c0d1b90 0x7f8a8c0c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.215+0000 7f8a91ada640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a8c0b4140 0x7f8a8c149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.215+0000 7f8a91ada640 1 --2- 172.21.15.16:0/456297928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8c0b34e0 0x7f8a8c0b6cd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.215+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 >> 172.21.15.16:0/456297928 conn(0x7f8a8c0d3090 msgr2=0x7f8a8c06c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.215+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 shutdown_connections 2023-11-24T06:27:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.215+0000 7f8a91ada640 1 -- 172.21.15.16:0/456297928 wait complete. 2023-11-24T06:27:45.235 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:27:45.235 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:45.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.495+0000 7f243cd7a640 1 Processor -- start 2023-11-24T06:27:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.496+0000 7f243cd7a640 1 -- start start 2023-11-24T06:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.497+0000 7f243cd7a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24380cc990 0x7f243806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.497+0000 7f243cd7a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2438070e50 con 0x7f2438143c40 2023-11-24T06:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.497+0000 7f243cd7a640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2438144020 con 0x7f24380cc500 2023-11-24T06:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.497+0000 7f243cd7a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24380723c0 con 0x7f24380cc990 2023-11-24T06:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.498+0000 7f24377fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2438143c40 0x7f2438071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42452/0 (socket says 172.21.15.16:42452) 2023-11-24T06:27:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.498+0000 7f24377fe640 1 -- 172.21.15.16:0/3666647046 learned_addr learned my addr 172.21.15.16:0/3666647046 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.498+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299876877 0 0) 0x7f2438070e50 con 0x7f2438143c40 2023-11-24T06:27:45.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.498+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2424003520 con 0x7f2438143c40 2023-11-24T06:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.498+0000 7f2437fff640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24380cc990 0x7f243806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 22341232 0 0) 0x7f2424003520 con 0x7f2438143c40 2023-11-24T06:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24380cc990 msgr2=0x7f243806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24380cc990 0x7f243806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 >> v1:172.21.15.173:6789/0 conn(0x7f24380cc500 legacy=0x7f243806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2438071e00 con 0x7f2438143c40 2023-11-24T06:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f2428001690 con 0x7f2438143c40 2023-11-24T06:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 >> v1:172.21.15.16:6789/0 conn(0x7f2438143c40 legacy=0x7f2438071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24380cc990 0x7f24240039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2424004000 0x7f24240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f2424005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24380723c0 con 0x7f2424004d00 2023-11-24T06:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242400bbe0 con 0x7f2424004000 2023-11-24T06:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242400bda0 con 0x7f24380cc990 2023-11-24T06:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2437fff640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f2424005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f242ffff640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2424004000 0x7f24240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2437fff640 1 -- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24380cc990 msgr2=0x7f24240039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2437fff640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24380cc990 0x7f24240039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2437fff640 1 -- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2424004000 msgr2=0x7f24240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2437fff640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2424004000 0x7f24240047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.499+0000 7f2437fff640 1 -- 172.21.15.16:0/3666647046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2424003520 con 0x7f2424004d00 2023-11-24T06:27:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f2437fff640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f2424005180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2430007360 tx=0x7f24240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00cf2f60415e57a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243002ccf0 con 0x7f2424004d00 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243002ce50 con 0x7f2424004d00 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f2436ffd640 1 -- 172.21.15.16:0/3666647046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430034660 con 0x7f2424004d00 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f243cd7a640 1 -- 172.21.15.16:0/3666647046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 msgr2=0x7f2424005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f243cd7a640 1 --2- 172.21.15.16:0/3666647046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f2424005180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2430007360 tx=0x7f24240057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f2437fff640 1 -- 172.21.15.16:0/3666647046 reap_dead start 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f243cd7a640 1 -- 172.21.15.16:0/3666647046 shutdown_connections 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f243cd7a640 1 -- 172.21.15.16:0/3666647046 >> 172.21.15.16:0/3666647046 conn(0x7f243806a800 msgr2=0x7f243806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f243cd7a640 1 -- 172.21.15.16:0/3666647046 shutdown_connections 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.501+0000 7f243cd7a640 1 -- 172.21.15.16:0/3666647046 wait complete. 2023-11-24T06:27:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.503+0000 7f243cd7a640 1 Processor -- start 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.503+0000 7f243cd7a640 1 -- start start 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.504+0000 7f243cd7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f24381f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.504+0000 7f243cd7a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f243813d470 0x7f24380adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.504+0000 7f243cd7a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24380ae0d0 0x7f24381e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.504+0000 7f243cd7a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24381e5730 con 0x7f2424004d00 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.504+0000 7f243cd7a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24381e4880 con 0x7f24380ae0d0 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.504+0000 7f243cd7a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24381e4b20 con 0x7f243813d470 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.504+0000 7f24377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f24381f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f24377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f24381f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33686/0 (socket says 172.21.15.16:33686) 2023-11-24T06:27:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f24377fe640 1 -- 172.21.15.16:0/3319742432 learned_addr learned my addr 172.21.15.16:0/3319742432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f24377fe640 1 -- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f243813d470 msgr2=0x7f24380adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f24377fe640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f243813d470 0x7f24380adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f24377fe640 1 -- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24380ae0d0 msgr2=0x7f24381e5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f24377fe640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24380ae0d0 0x7f24381e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f24377fe640 1 -- 172.21.15.16:0/3319742432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24381e4e00 con 0x7f2424004d00 2023-11-24T06:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f24377fe640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f24381f2ea0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f242800c770 tx=0x7f242800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f2434ff9640 1 -- 172.21.15.16:0/3319742432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24280074e0 con 0x7f2424004d00 2023-11-24T06:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f2434ff9640 1 -- 172.21.15.16:0/3319742432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2428007660 con 0x7f2424004d00 2023-11-24T06:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f2434ff9640 1 -- 172.21.15.16:0/3319742432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428004a20 con 0x7f2424004d00 2023-11-24T06:27:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.505+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24381e50e0 con 0x7f2424004d00 2023-11-24T06:27:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.506+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24381f33e0 con 0x7f2424004d00 2023-11-24T06:27:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.507+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2438067450 con 0x7f2424004d00 2023-11-24T06:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.507+0000 7f2434ff9640 1 -- 172.21.15.16:0/3319742432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2428004b80 con 0x7f2424004d00 2023-11-24T06:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.508+0000 7f2434ff9640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241806ae40 0x7f241806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.508+0000 7f2434ff9640 1 -- 172.21.15.16:0/3319742432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f2428088b10 con 0x7f2424004d00 2023-11-24T06:27:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.509+0000 7f242ffff640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241806ae40 0x7f241806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.510+0000 7f242ffff640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241806ae40 0x7f241806d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f243813dd80 tx=0x7f2420009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.513+0000 7f2434ff9640 1 -- 172.21.15.16:0/3319742432 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2428053aa0 con 0x7f2424004d00 2023-11-24T06:27:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.706+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24380ceb30 con 0x7f2424004d00 2023-11-24T06:27:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.709+0000 7f2434ff9640 1 -- 172.21.15.16:0/3319742432 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7f2428018020 con 0x7f2424004d00 2023-11-24T06:27:45.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:45.712 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241806ae40 msgr2=0x7f241806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241806ae40 0x7f241806d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f243813dd80 tx=0x7f2420009040 comp rx=0 tx=0).stop 2023-11-24T06:27:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 msgr2=0x7f24381f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f24381f2ea0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f242800c770 tx=0x7f242800cc30 comp rx=0 tx=0).stop 2023-11-24T06:27:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 shutdown_connections 2023-11-24T06:27:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241806ae40 0x7f241806d300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24380ae0d0 0x7f24381e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f243813d470 0x7f24380adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 --2- 172.21.15.16:0/3319742432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2424004d00 0x7f24381f2ea0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 >> 172.21.15.16:0/3319742432 conn(0x7f243806a800 msgr2=0x7f243813dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 shutdown_connections 2023-11-24T06:27:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.712+0000 7f243cd7a640 1 -- 172.21.15.16:0/3319742432 wait complete. 2023-11-24T06:27:45.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:45.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.964+0000 7fc895b68640 1 Processor -- start 2023-11-24T06:27:45.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.965+0000 7fc895b68640 1 -- start start 2023-11-24T06:27:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.966+0000 7fc895b68640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc89013df80 0x7fc89015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.966+0000 7fc895b68640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8900cf9a0 0x7fc8900cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.967+0000 7fc895b68640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89006c470 con 0x7fc89013e360 2023-11-24T06:27:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.967+0000 7fc895b68640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89006c750 con 0x7fc89013df80 2023-11-24T06:27:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.967+0000 7fc895b68640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89006ca30 con 0x7fc8900cf9a0 2023-11-24T06:27:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.967+0000 7fc887fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc89013e360 0x7fc8900cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42460/0 (socket says 172.21.15.16:42460) 2023-11-24T06:27:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.967+0000 7fc887fff640 1 -- 172.21.15.16:0/464695325 learned_addr learned my addr 172.21.15.16:0/464695325 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.967+0000 7fc88f7fe640 1 --2- 172.21.15.16:0/464695325 >> v2:172.21.15.173:3300/0 conn(0x7fc89013df80 0x7fc89015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.967+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680981281 0 0) 0x7fc89006c470 con 0x7fc89013e360 2023-11-24T06:27:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.967+0000 7fc88ffff640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8900cf9a0 0x7fc8900cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.967+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc87c003520 con 0x7fc89013e360 2023-11-24T06:27:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1836125915 0 0) 0x7fc87c003520 con 0x7fc89013e360 2023-11-24T06:27:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8900cf9a0 msgr2=0x7fc8900cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8900cf9a0 0x7fc8900cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 >> v2:172.21.15.173:3300/0 conn(0x7fc89013df80 msgr2=0x7fc89015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 --2- 172.21.15.16:0/464695325 >> v2:172.21.15.173:3300/0 conn(0x7fc89013df80 0x7fc89015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89006cd10 con 0x7fc89013e360 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fc878001670 con 0x7fc89013e360 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 >> v1:172.21.15.16:6789/0 conn(0x7fc89013e360 legacy=0x7fc8900cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc87c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc87c0047c0 0x7fc87c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc87c005180 0x7fc87c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89006c750 con 0x7fc87c003970 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89006ca30 con 0x7fc87c005180 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c00bef0 con 0x7fc87c0047c0 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88ffff640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8900cf9a0 0x7fc8900cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88f7fe640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc87c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88f7fe640 1 -- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc87c0047c0 msgr2=0x7fc87c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88f7fe640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc87c0047c0 0x7fc87c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88f7fe640 1 -- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc87c005180 msgr2=0x7fc87c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88f7fe640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc87c005180 0x7fc87c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.968+0000 7fc88f7fe640 1 -- 172.21.15.16:0/464695325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c003520 con 0x7fc87c003970 2023-11-24T06:27:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.969+0000 7fc88f7fe640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc87c003d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ac0 tx=0x7fc87c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18a811cb21dab0a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.970+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88002c710 con 0x7fc87c003970 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.970+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88002c870 con 0x7fc87c003970 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.970+0000 7fc88effd640 1 -- 172.21.15.16:0/464695325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88002cc00 con 0x7fc87c003970 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.970+0000 7fc895b68640 1 -- 172.21.15.16:0/464695325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 msgr2=0x7fc87c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.970+0000 7fc895b68640 1 --2- 172.21.15.16:0/464695325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc87c003d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ac0 tx=0x7fc87c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.970+0000 7fc88ffff640 1 -- 172.21.15.16:0/464695325 reap_dead start 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.970+0000 7fc895b68640 1 -- 172.21.15.16:0/464695325 shutdown_connections 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.970+0000 7fc895b68640 1 -- 172.21.15.16:0/464695325 >> 172.21.15.16:0/464695325 conn(0x7fc89006a800 msgr2=0x7fc89006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.971+0000 7fc895b68640 1 -- 172.21.15.16:0/464695325 shutdown_connections 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.971+0000 7fc895b68640 1 -- 172.21.15.16:0/464695325 wait complete. 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.972+0000 7fc895b68640 1 Processor -- start 2023-11-24T06:27:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.972+0000 7fc895b68640 1 -- start start 2023-11-24T06:27:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.973+0000 7fc895b68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc8901500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.973+0000 7fc895b68640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8901e5150 0x7fc8901e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.973+0000 7fc895b68640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc890151f40 0x7fc8901523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.973+0000 7fc895b68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8901e5af0 con 0x7fc87c003970 2023-11-24T06:27:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.973+0000 7fc895b68640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc890152c50 con 0x7fc8901e5150 2023-11-24T06:27:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.973+0000 7fc895b68640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc890152f30 con 0x7fc890151f40 2023-11-24T06:27:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.974+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc8901500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.974+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc8901500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33710/0 (socket says 172.21.15.16:33710) 2023-11-24T06:27:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.974+0000 7fc88f7fe640 1 -- 172.21.15.16:0/29088433 learned_addr learned my addr 172.21.15.16:0/29088433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.974+0000 7fc88f7fe640 1 -- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc890151f40 msgr2=0x7fc8901523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.974+0000 7fc88f7fe640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc890151f40 0x7fc8901523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.974+0000 7fc887fff640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8901e5150 0x7fc8901e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.974+0000 7fc88f7fe640 1 -- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8901e5150 msgr2=0x7fc8901e55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.974+0000 7fc88f7fe640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8901e5150 0x7fc8901e55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.974+0000 7fc88f7fe640 1 -- 172.21.15.16:0/29088433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc890153210 con 0x7fc87c003970 2023-11-24T06:27:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.975+0000 7fc88f7fe640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc8901500d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc880000c00 tx=0x7fc880004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.975+0000 7fc88cff9640 1 -- 172.21.15.16:0/29088433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88002ce80 con 0x7fc87c003970 2023-11-24T06:27:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.975+0000 7fc88cff9640 1 -- 172.21.15.16:0/29088433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc880004500 con 0x7fc87c003970 2023-11-24T06:27:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.975+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8901534f0 con 0x7fc87c003970 2023-11-24T06:27:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.975+0000 7fc88cff9640 1 -- 172.21.15.16:0/29088433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc880003810 con 0x7fc87c003970 2023-11-24T06:27:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.975+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc890152900 con 0x7fc87c003970 2023-11-24T06:27:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.976+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8900675d0 con 0x7fc87c003970 2023-11-24T06:27:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.977+0000 7fc88cff9640 1 -- 172.21.15.16:0/29088433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc88003b070 con 0x7fc87c003970 2023-11-24T06:27:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.978+0000 7fc88cff9640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc87006ae40 0x7fc87006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.978+0000 7fc88cff9640 1 -- 172.21.15.16:0/29088433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fc8800ab190 con 0x7fc87c003970 2023-11-24T06:27:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.978+0000 7fc887fff640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc87006ae40 0x7fc87006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.979+0000 7fc887fff640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc87006ae40 0x7fc87006d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc8901f6630 tx=0x7fc878009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:45.982+0000 7fc88cff9640 1 -- 172.21.15.16:0/29088433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc880076140 con 0x7fc87c003970 2023-11-24T06:27:46.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.180+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc89006df00 con 0x7fc87c003970 2023-11-24T06:27:46.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.181+0000 7fc88cff9640 1 -- 172.21.15.16:0/29088433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc880079760 con 0x7fc87c003970 2023-11-24T06:27:46.184 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc87006ae40 msgr2=0x7fc87006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc87006ae40 0x7fc87006d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc8901f6630 tx=0x7fc878009040 comp rx=0 tx=0).stop 2023-11-24T06:27:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 msgr2=0x7fc8901500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc8901500d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc880000c00 tx=0x7fc880004730 comp rx=0 tx=0).stop 2023-11-24T06:27:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 shutdown_connections 2023-11-24T06:27:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc87006ae40 0x7fc87006d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc890151f40 0x7fc8901523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8901e5150 0x7fc8901e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 --2- 172.21.15.16:0/29088433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87c003970 0x7fc8901500d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 >> 172.21.15.16:0/29088433 conn(0x7fc89006a800 msgr2=0x7fc890070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 shutdown_connections 2023-11-24T06:27:46.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.184+0000 7fc895b68640 1 -- 172.21.15.16:0/29088433 wait complete. 2023-11-24T06:27:46.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:46.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.424+0000 7fb92ffff640 1 Processor -- start 2023-11-24T06:27:46.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.425+0000 7fb92ffff640 1 -- start start 2023-11-24T06:27:46.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.427+0000 7fb92ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb930070a30 0x7fb93015bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.427+0000 7fb92ffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb93015f4e0 con 0x7fb9300adf20 2023-11-24T06:27:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.427+0000 7fb92ffff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb930070ea0 con 0x7fb9300ccb50 2023-11-24T06:27:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.427+0000 7fb92ffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93015c450 con 0x7fb930070a30 2023-11-24T06:27:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.427+0000 7fb92effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9300adf20 0x7fb9300cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42470/0 (socket says 172.21.15.16:42470) 2023-11-24T06:27:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.427+0000 7fb92effd640 1 -- 172.21.15.16:0/66106378 learned_addr learned my addr 172.21.15.16:0/66106378 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.427+0000 7fb92f7fe640 1 --2- 172.21.15.16:0/66106378 >> v2:172.21.15.16:3300/0 conn(0x7fb930070a30 0x7fb93015bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.428+0000 7fb92f7fe640 1 -- 172.21.15.16:0/66106378 >> v1:172.21.15.173:6789/0 conn(0x7fb9300ccb50 legacy=0x7fb930143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.428+0000 7fb92f7fe640 1 -- 172.21.15.16:0/66106378 >> v1:172.21.15.16:6789/0 conn(0x7fb9300adf20 legacy=0x7fb9300cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.428+0000 7fb92f7fe640 1 -- 172.21.15.16:0/66106378 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930071130 con 0x7fb930070a30 2023-11-24T06:27:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.429+0000 7fb92f7fe640 1 --2- 172.21.15.16:0/66106378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 0x7fb93015bf10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb928009d90 tx=0x7fb92802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41647e77d3c4feb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.429+0000 7fb92e7fc640 1 -- 172.21.15.16:0/66106378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92802cbd0 con 0x7fb930070a30 2023-11-24T06:27:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.429+0000 7fb92e7fc640 1 -- 172.21.15.16:0/66106378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92802cd30 con 0x7fb930070a30 2023-11-24T06:27:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.429+0000 7fb92e7fc640 1 -- 172.21.15.16:0/66106378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92802cea0 con 0x7fb930070a30 2023-11-24T06:27:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.430+0000 7fb92ffff640 1 -- 172.21.15.16:0/66106378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 msgr2=0x7fb93015bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.430+0000 7fb92ffff640 1 --2- 172.21.15.16:0/66106378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 0x7fb93015bf10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb928009d90 tx=0x7fb92802be20 comp rx=0 tx=0).stop 2023-11-24T06:27:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.430+0000 7fb92ffff640 1 -- 172.21.15.16:0/66106378 shutdown_connections 2023-11-24T06:27:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.430+0000 7fb92ffff640 1 --2- 172.21.15.16:0/66106378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 0x7fb93015bf10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.430+0000 7fb92ffff640 1 -- 172.21.15.16:0/66106378 >> 172.21.15.16:0/66106378 conn(0x7fb93006a7d0 msgr2=0x7fb93006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.430+0000 7fb92ffff640 1 -- 172.21.15.16:0/66106378 shutdown_connections 2023-11-24T06:27:46.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.430+0000 7fb92ffff640 1 -- 172.21.15.16:0/66106378 wait complete. 2023-11-24T06:27:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.432+0000 7fb92ffff640 1 Processor -- start 2023-11-24T06:27:46.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.432+0000 7fb92ffff640 1 -- start start 2023-11-24T06:27:46.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.433+0000 7fb92ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 0x7fb9300acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.433+0000 7fb92ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9300adf20 0x7fb93013e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.433+0000 7fb92ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9300ccb50 0x7fb93013a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.433+0000 7fb92ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93013df30 con 0x7fb930070a30 2023-11-24T06:27:46.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.433+0000 7fb92ffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93013d230 con 0x7fb9300adf20 2023-11-24T06:27:46.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.433+0000 7fb92ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93013d8e0 con 0x7fb9300ccb50 2023-11-24T06:27:46.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.433+0000 7fb92effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 0x7fb9300acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.433+0000 7fb92effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 0x7fb9300acc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33742/0 (socket says 172.21.15.16:33742) 2023-11-24T06:27:46.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.433+0000 7fb92effd640 1 -- 172.21.15.16:0/1104246243 learned_addr learned my addr 172.21.15.16:0/1104246243 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:46.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.434+0000 7fb927fff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9300adf20 0x7fb93013e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.434+0000 7fb92effd640 1 -- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9300ccb50 msgr2=0x7fb93013a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.434+0000 7fb92effd640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9300ccb50 0x7fb93013a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.434+0000 7fb92effd640 1 -- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9300adf20 msgr2=0x7fb93013e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.434+0000 7fb92effd640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9300adf20 0x7fb93013e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.434+0000 7fb92effd640 1 -- 172.21.15.16:0/1104246243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb928009a70 con 0x7fb930070a30 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.434+0000 7fb927fff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9300adf20 0x7fb93013e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.434+0000 7fb92effd640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 0x7fb9300acc60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c4a0 tx=0x7fb92000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.434+0000 7fb92cff9640 1 -- 172.21.15.16:0/1104246243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920005ce0 con 0x7fb930070a30 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.435+0000 7fb92cff9640 1 -- 172.21.15.16:0/1104246243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920005e40 con 0x7fb930070a30 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.435+0000 7fb92cff9640 1 -- 172.21.15.16:0/1104246243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920007270 con 0x7fb930070a30 2023-11-24T06:27:46.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.435+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93013dbc0 con 0x7fb930070a30 2023-11-24T06:27:46.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.435+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9300ad380 con 0x7fb930070a30 2023-11-24T06:27:46.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.436+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb930067450 con 0x7fb930070a30 2023-11-24T06:27:46.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.437+0000 7fb92cff9640 1 -- 172.21.15.16:0/1104246243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb920007490 con 0x7fb930070a30 2023-11-24T06:27:46.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.438+0000 7fb92cff9640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb91006ae40 0x7fb91006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.438+0000 7fb92cff9640 1 -- 172.21.15.16:0/1104246243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fb920088b70 con 0x7fb930070a30 2023-11-24T06:27:46.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.438+0000 7fb927fff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb91006ae40 0x7fb91006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.440+0000 7fb927fff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb91006ae40 0x7fb91006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb93015f8b0 tx=0x7fb918009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.442+0000 7fb92cff9640 1 -- 172.21.15.16:0/1104246243 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb920053bb0 con 0x7fb930070a30 2023-11-24T06:27:46.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.636+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9300ad740 con 0x7fb930070a30 2023-11-24T06:27:46.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.637+0000 7fb92cff9640 1 -- 172.21.15.16:0/1104246243 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9200170a0 con 0x7fb930070a30 2023-11-24T06:27:46.639 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb91006ae40 msgr2=0x7fb91006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb91006ae40 0x7fb91006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb93015f8b0 tx=0x7fb918009040 comp rx=0 tx=0).stop 2023-11-24T06:27:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 msgr2=0x7fb9300acc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 0x7fb9300acc60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb92000c4a0 tx=0x7fb92000c960 comp rx=0 tx=0).stop 2023-11-24T06:27:46.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 shutdown_connections 2023-11-24T06:27:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb91006ae40 0x7fb91006d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9300ccb50 0x7fb93013a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9300adf20 0x7fb93013e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 --2- 172.21.15.16:0/1104246243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930070a30 0x7fb9300acc60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 >> 172.21.15.16:0/1104246243 conn(0x7fb93006a7d0 msgr2=0x7fb93015cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 shutdown_connections 2023-11-24T06:27:46.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.640+0000 7fb92ffff640 1 -- 172.21.15.16:0/1104246243 wait complete. 2023-11-24T06:27:46.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.892+0000 7f704bb2c640 1 Processor -- start 2023-11-24T06:27:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.892+0000 7f704bb2c640 1 -- start start 2023-11-24T06:27:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.894+0000 7f704bb2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f704413df80 0x7f704415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.894+0000 7f704bb2c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70440cf9a0 0x7f70440cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.894+0000 7f704bb2c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704406c470 con 0x7f704413e360 2023-11-24T06:27:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.894+0000 7f704bb2c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704406c750 con 0x7f704413df80 2023-11-24T06:27:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.894+0000 7f704bb2c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704406ca30 con 0x7f70440cf9a0 2023-11-24T06:27:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.894+0000 7f70490c7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f704413e360 0x7f70440cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42486/0 (socket says 172.21.15.16:42486) 2023-11-24T06:27:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.894+0000 7f70490c7640 1 -- 172.21.15.16:0/960070363 learned_addr learned my addr 172.21.15.16:0/960070363 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.894+0000 7f70498c8640 1 --2- 172.21.15.16:0/960070363 >> v2:172.21.15.16:3300/0 conn(0x7f704413df80 0x7f704415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.895+0000 7f704a0c9640 1 --2- 172.21.15.16:0/960070363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70440cf9a0 0x7f70440cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.895+0000 7f70488c6640 1 -- 172.21.15.16:0/960070363 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967719033 0 0) 0x7f704406c470 con 0x7f704413e360 2023-11-24T06:27:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.895+0000 7f70488c6640 1 -- 172.21.15.16:0/960070363 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7030003520 con 0x7f704413e360 2023-11-24T06:27:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.895+0000 7f70498c8640 1 -- 172.21.15.16:0/960070363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70440cf9a0 msgr2=0x7f70440cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:46.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.895+0000 7f70498c8640 1 --2- 172.21.15.16:0/960070363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70440cf9a0 0x7f70440cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.895+0000 7f70498c8640 1 -- 172.21.15.16:0/960070363 >> v1:172.21.15.16:6789/0 conn(0x7f704413e360 legacy=0x7f70440cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.895+0000 7f70498c8640 1 -- 172.21.15.16:0/960070363 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704406cd10 con 0x7f704413df80 2023-11-24T06:27:46.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.896+0000 7f70498c8640 1 --2- 172.21.15.16:0/960070363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704413df80 0x7f704415bee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f70400097a0 tx=0x7f704002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7793aee9235ddd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.896+0000 7f70488c6640 1 -- 172.21.15.16:0/960070363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704002d040 con 0x7f704413df80 2023-11-24T06:27:46.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.896+0000 7f70488c6640 1 -- 172.21.15.16:0/960070363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7040031810 con 0x7f704413df80 2023-11-24T06:27:46.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.896+0000 7f70488c6640 1 -- 172.21.15.16:0/960070363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040031ba0 con 0x7f704413df80 2023-11-24T06:27:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.897+0000 7f704bb2c640 1 -- 172.21.15.16:0/960070363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704413df80 msgr2=0x7f704415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.897+0000 7f704bb2c640 1 --2- 172.21.15.16:0/960070363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704413df80 0x7f704415bee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f70400097a0 tx=0x7f704002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.897+0000 7f704bb2c640 1 -- 172.21.15.16:0/960070363 shutdown_connections 2023-11-24T06:27:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.897+0000 7f704bb2c640 1 --2- 172.21.15.16:0/960070363 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70440cf9a0 0x7f70440cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.897+0000 7f704bb2c640 1 --2- 172.21.15.16:0/960070363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704413df80 0x7f704415bee0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.897+0000 7f704bb2c640 1 -- 172.21.15.16:0/960070363 >> 172.21.15.16:0/960070363 conn(0x7f704406a800 msgr2=0x7f704406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.897+0000 7f704bb2c640 1 -- 172.21.15.16:0/960070363 shutdown_connections 2023-11-24T06:27:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.897+0000 7f704bb2c640 1 -- 172.21.15.16:0/960070363 wait complete. 2023-11-24T06:27:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.899+0000 7f704bb2c640 1 Processor -- start 2023-11-24T06:27:46.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.899+0000 7f704bb2c640 1 -- start start 2023-11-24T06:27:46.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.900+0000 7f704bb2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70440cf9a0 0x7f704414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.900+0000 7f704bb2c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704413df80 0x7f70440b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.900+0000 7f704bb2c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704413e360 0x7f7044131a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.900+0000 7f704bb2c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704414a9b0 con 0x7f70440cf9a0 2023-11-24T06:27:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.900+0000 7f704bb2c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440b39b0 con 0x7f704413df80 2023-11-24T06:27:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.900+0000 7f704bb2c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7044132510 con 0x7f704413e360 2023-11-24T06:27:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.900+0000 7f70498c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70440cf9a0 0x7f704414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f70498c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70440cf9a0 0x7f704414b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33762/0 (socket says 172.21.15.16:33762) 2023-11-24T06:27:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f70498c8640 1 -- 172.21.15.16:0/2547969807 learned_addr learned my addr 172.21.15.16:0/2547969807 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f704a0c9640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704413e360 0x7f7044131a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f70490c7640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704413df80 0x7f70440b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f70498c8640 1 -- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704413e360 msgr2=0x7f7044131a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f70498c8640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704413e360 0x7f7044131a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f70498c8640 1 -- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704413df80 msgr2=0x7f70440b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f70498c8640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704413df80 0x7f70440b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f70498c8640 1 -- 172.21.15.16:0/2547969807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009400 con 0x7f70440cf9a0 2023-11-24T06:27:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.901+0000 7f70498c8640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70440cf9a0 0x7f704414b3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70400098c0 tx=0x7f70400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.902+0000 7f703a7fc640 1 -- 172.21.15.16:0/2547969807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040006e10 con 0x7f70440cf9a0 2023-11-24T06:27:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.902+0000 7f703a7fc640 1 -- 172.21.15.16:0/2547969807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704003ad60 con 0x7f70440cf9a0 2023-11-24T06:27:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.902+0000 7f703a7fc640 1 -- 172.21.15.16:0/2547969807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040044620 con 0x7f70440cf9a0 2023-11-24T06:27:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.902+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70441327f0 con 0x7f70440cf9a0 2023-11-24T06:27:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.902+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704414aea0 con 0x7f70440cf9a0 2023-11-24T06:27:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.903+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704406d920 con 0x7f70440cf9a0 2023-11-24T06:27:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.904+0000 7f703a7fc640 1 -- 172.21.15.16:0/2547969807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f704004d440 con 0x7f70440cf9a0 2023-11-24T06:27:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.905+0000 7f703a7fc640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f702806ae40 0x7f702806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.906+0000 7f703a7fc640 1 -- 172.21.15.16:0/2547969807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f70400aba40 con 0x7f70440cf9a0 2023-11-24T06:27:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.906+0000 7f70490c7640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f702806ae40 0x7f702806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.907+0000 7f70490c7640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f702806ae40 0x7f702806d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70440b45b0 tx=0x7f7034009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:46.910+0000 7f703a7fc640 1 -- 172.21.15.16:0/2547969807 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7040076a60 con 0x7f70440cf9a0 2023-11-24T06:27:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.105+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70440675d0 con 0x7f70440cf9a0 2023-11-24T06:27:47.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.106+0000 7f703a7fc640 1 -- 172.21.15.16:0/2547969807 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7040035020 con 0x7f70440cf9a0 2023-11-24T06:27:47.108 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f702806ae40 msgr2=0x7f702806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f702806ae40 0x7f702806d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70440b45b0 tx=0x7f7034009040 comp rx=0 tx=0).stop 2023-11-24T06:27:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70440cf9a0 msgr2=0x7f704414b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70440cf9a0 0x7f704414b3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f70400098c0 tx=0x7f70400046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 shutdown_connections 2023-11-24T06:27:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f702806ae40 0x7f702806d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704413e360 0x7f7044131a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704413df80 0x7f70440b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 --2- 172.21.15.16:0/2547969807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70440cf9a0 0x7f704414b3c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 >> 172.21.15.16:0/2547969807 conn(0x7f704406a800 msgr2=0x7f70440ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 shutdown_connections 2023-11-24T06:27:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.109+0000 7f704bb2c640 1 -- 172.21.15.16:0/2547969807 wait complete. 2023-11-24T06:27:47.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:27:47.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.354+0000 7f7c7af81640 1 Processor -- start 2023-11-24T06:27:47.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.355+0000 7f7c7af81640 1 -- start start 2023-11-24T06:27:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.356+0000 7f7c7af81640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c74162810 0x7f7c74071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.356+0000 7f7c7af81640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7c74071780 0x7f7c7406c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.356+0000 7f7c7af81640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c74071c10 con 0x7f7c74162430 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.356+0000 7f7c7af81640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740702e0 con 0x7f7c74162810 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.356+0000 7f7c7af81640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740705c0 con 0x7f7c74071780 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.357+0000 7f7c7977e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c74162810 0x7f7c74071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.357+0000 7f7c7977e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c74162810 0x7f7c74071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33772/0 (socket says 172.21.15.16:33772) 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.357+0000 7f7c7977e640 1 -- 172.21.15.16:0/237031389 learned_addr learned my addr 172.21.15.16:0/237031389 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.357+0000 7f7c7977e640 1 -- 172.21.15.16:0/237031389 >> v2:172.21.15.173:3300/0 conn(0x7f7c74071780 msgr2=0x7f7c7406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.357+0000 7f7c7977e640 1 --2- 172.21.15.16:0/237031389 >> v2:172.21.15.173:3300/0 conn(0x7f7c74071780 0x7f7c7406c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.357+0000 7f7c7977e640 1 -- 172.21.15.16:0/237031389 >> v1:172.21.15.173:6789/0 conn(0x7f7c74162430 legacy=0x7f7c740b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:47.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.357+0000 7f7c7977e640 1 -- 172.21.15.16:0/237031389 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740708a0 con 0x7f7c74162810 2023-11-24T06:27:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.359+0000 7f7c7977e640 1 --2- 172.21.15.16:0/237031389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74162810 0x7f7c74071240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7c74161ad0 tx=0x7f7c6402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584d938138e9491c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.359+0000 7f7c78f7d640 1 -- 172.21.15.16:0/237031389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64031870 con 0x7f7c74162810 2023-11-24T06:27:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.359+0000 7f7c78f7d640 1 -- 172.21.15.16:0/237031389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c640319d0 con 0x7f7c74162810 2023-11-24T06:27:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.359+0000 7f7c78f7d640 1 -- 172.21.15.16:0/237031389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64031d60 con 0x7f7c74162810 2023-11-24T06:27:47.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.360+0000 7f7c7af81640 1 -- 172.21.15.16:0/237031389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74162810 msgr2=0x7f7c74071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.360+0000 7f7c7af81640 1 --2- 172.21.15.16:0/237031389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74162810 0x7f7c74071240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7c74161ad0 tx=0x7f7c6402b910 comp rx=0 tx=0).stop 2023-11-24T06:27:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.360+0000 7f7c7af81640 1 -- 172.21.15.16:0/237031389 shutdown_connections 2023-11-24T06:27:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.360+0000 7f7c7af81640 1 --2- 172.21.15.16:0/237031389 >> v2:172.21.15.173:3300/0 conn(0x7f7c74071780 0x7f7c7406c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.360+0000 7f7c7af81640 1 --2- 172.21.15.16:0/237031389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74162810 0x7f7c74071240 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.360+0000 7f7c7af81640 1 -- 172.21.15.16:0/237031389 >> 172.21.15.16:0/237031389 conn(0x7f7c7406a800 msgr2=0x7f7c7406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.360+0000 7f7c7af81640 1 -- 172.21.15.16:0/237031389 shutdown_connections 2023-11-24T06:27:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.360+0000 7f7c7af81640 1 -- 172.21.15.16:0/237031389 wait complete. 2023-11-24T06:27:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.362+0000 7f7c7af81640 1 Processor -- start 2023-11-24T06:27:47.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.362+0000 7f7c7af81640 1 -- start start 2023-11-24T06:27:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c7af81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74071780 0x7f7c741403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c7af81640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c74162430 0x7f7c74140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c7af81640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c74141260 0x7f7c740b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c7af81640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741424a0 con 0x7f7c74071780 2023-11-24T06:27:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c7af81640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741eb1a0 con 0x7f7c74162430 2023-11-24T06:27:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c7af81640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74141890 con 0x7f7c74141260 2023-11-24T06:27:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c79f7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74071780 0x7f7c741403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c79f7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74071780 0x7f7c741403b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33774/0 (socket says 172.21.15.16:33774) 2023-11-24T06:27:47.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c79f7f640 1 -- 172.21.15.16:0/1121092382 learned_addr learned my addr 172.21.15.16:0/1121092382 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c79f7f640 1 -- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c74141260 msgr2=0x7f7c740b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c79f7f640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c74141260 0x7f7c740b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c79f7f640 1 -- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c74162430 msgr2=0x7f7c74140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c79f7f640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c74162430 0x7f7c74140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.364+0000 7f7c79f7f640 1 -- 172.21.15.16:0/1121092382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640094d0 con 0x7f7c74071780 2023-11-24T06:27:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.365+0000 7f7c79f7f640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74071780 0x7f7c741403b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c410 tx=0x7f7c7000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.365+0000 7f7c6a7fc640 1 -- 172.21.15.16:0/1121092382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700035f0 con 0x7f7c74071780 2023-11-24T06:27:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.365+0000 7f7c6a7fc640 1 -- 172.21.15.16:0/1121092382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70003750 con 0x7f7c74071780 2023-11-24T06:27:47.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.365+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c74141b70 con 0x7f7c74071780 2023-11-24T06:27:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.365+0000 7f7c6a7fc640 1 -- 172.21.15.16:0/1121092382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70003b40 con 0x7f7c74071780 2023-11-24T06:27:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.365+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c74142130 con 0x7f7c74071780 2023-11-24T06:27:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.366+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c74067450 con 0x7f7c74071780 2023-11-24T06:27:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.367+0000 7f7c6a7fc640 1 -- 172.21.15.16:0/1121092382 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c70003ca0 con 0x7f7c74071780 2023-11-24T06:27:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.368+0000 7f7c6a7fc640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c5806ae70 0x7f7c5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.369+0000 7f7c6a7fc640 1 -- 172.21.15.16:0/1121092382 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f7c70088a00 con 0x7f7c74071780 2023-11-24T06:27:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.369+0000 7f7c7977e640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c5806ae70 0x7f7c5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.370+0000 7f7c7977e640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c5806ae70 0x7f7c5806d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7c64036000 tx=0x7f7c64030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:47.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.373+0000 7f7c6a7fc640 1 -- 172.21.15.16:0/1121092382 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c70053900 con 0x7f7c74071780 2023-11-24T06:27:47.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.580+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7c740b2ef0 con 0x7f7c74071780 2023-11-24T06:27:47.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.581+0000 7f7c6a7fc640 1 -- 172.21.15.16:0/1121092382 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7c70017080 con 0x7f7c74071780 2023-11-24T06:27:47.583 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:27:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.583+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c5806ae70 msgr2=0x7f7c5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.583+0000 7f7c7af81640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c5806ae70 0x7f7c5806d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7c64036000 tx=0x7f7c64030000 comp rx=0 tx=0).stop 2023-11-24T06:27:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.583+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74071780 msgr2=0x7f7c741403b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.584+0000 7f7c7af81640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74071780 0x7f7c741403b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c410 tx=0x7f7c7000c8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.584+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 shutdown_connections 2023-11-24T06:27:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.584+0000 7f7c7af81640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c5806ae70 0x7f7c5806d330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.584+0000 7f7c7af81640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c74141260 0x7f7c740b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.584+0000 7f7c7af81640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c74162430 0x7f7c74140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.584+0000 7f7c7af81640 1 --2- 172.21.15.16:0/1121092382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c74071780 0x7f7c741403b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.584+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 >> 172.21.15.16:0/1121092382 conn(0x7f7c7406a800 msgr2=0x7f7c7406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.584+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 shutdown_connections 2023-11-24T06:27:47.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:47.584+0000 7f7c7af81640 1 -- 172.21.15.16:0/1121092382 wait complete. 2023-11-24T06:27:47.607 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-24T06:27:47.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:47.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:27:47.712 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:27:47.712 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:27:47.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:47.713 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:27:47.776 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:27:47.776 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:47.835 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:27:47.835 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:47.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.123+0000 7fd1ae444640 1 Processor -- start 2023-11-24T06:27:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.123+0000 7fd1ae444640 1 -- start start 2023-11-24T06:27:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.125+0000 7fd1ae444640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071410 0x7fd1a8146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.125+0000 7fd1ae444640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1a806c7c0 0x7fd1a80717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.125+0000 7fd1ae444640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a806cc20 con 0x7fd1a8071d30 2023-11-24T06:27:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.125+0000 7fd1ae444640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a806cec0 con 0x7fd1a806c7c0 2023-11-24T06:27:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.125+0000 7fd1ae444640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a806d1a0 con 0x7fd1a8071410 2023-11-24T06:27:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd1acc41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1a806c7c0 0x7fd1a80717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd1adc43640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd1a8071d30 0x7fd1a80b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42500/0 (socket says 172.21.15.16:42500) 2023-11-24T06:27:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd1adc43640 1 -- 172.21.15.16:0/2925689482 learned_addr learned my addr 172.21.15.16:0/2925689482 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd1ad442640 1 --2- 172.21.15.16:0/2925689482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071410 0x7fd1a8146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd1acc41640 1 -- 172.21.15.16:0/2925689482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071410 msgr2=0x7fd1a8146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd1acc41640 1 --2- 172.21.15.16:0/2925689482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071410 0x7fd1a8146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd1acc41640 1 -- 172.21.15.16:0/2925689482 >> v1:172.21.15.16:6789/0 conn(0x7fd1a8071d30 legacy=0x7fd1a80b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd1acc41640 1 -- 172.21.15.16:0/2925689482 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a806d480 con 0x7fd1a806c7c0 2023-11-24T06:27:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd19ffff640 1 -- 172.21.15.16:0/2925689482 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102650041 0 0) 0x7fd1a806cc20 con 0x7fd1a8071d30 2023-11-24T06:27:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.126+0000 7fd1ad442640 1 --2- 172.21.15.16:0/2925689482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071410 0x7fd1a8146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:48.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.127+0000 7fd1acc41640 1 --2- 172.21.15.16:0/2925689482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a806c7c0 0x7fd1a80717f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd1980097a0 tx=0x7fd19802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3180fd675120bfd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.128+0000 7fd19ffff640 1 -- 172.21.15.16:0/2925689482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19802d040 con 0x7fd1a806c7c0 2023-11-24T06:27:48.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.128+0000 7fd19ffff640 1 -- 172.21.15.16:0/2925689482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd198031810 con 0x7fd1a806c7c0 2023-11-24T06:27:48.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.128+0000 7fd19ffff640 1 -- 172.21.15.16:0/2925689482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198031ba0 con 0x7fd1a806c7c0 2023-11-24T06:27:48.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.128+0000 7fd1ae444640 1 -- 172.21.15.16:0/2925689482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a806c7c0 msgr2=0x7fd1a80717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.128+0000 7fd1ae444640 1 --2- 172.21.15.16:0/2925689482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a806c7c0 0x7fd1a80717f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd1980097a0 tx=0x7fd19802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.128+0000 7fd1ae444640 1 -- 172.21.15.16:0/2925689482 shutdown_connections 2023-11-24T06:27:48.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.128+0000 7fd1ae444640 1 --2- 172.21.15.16:0/2925689482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a806c7c0 0x7fd1a80717f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.128+0000 7fd1ae444640 1 --2- 172.21.15.16:0/2925689482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071410 0x7fd1a8146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.128+0000 7fd1ae444640 1 -- 172.21.15.16:0/2925689482 >> 172.21.15.16:0/2925689482 conn(0x7fd1a806a800 msgr2=0x7fd1a806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.129+0000 7fd1ae444640 1 -- 172.21.15.16:0/2925689482 shutdown_connections 2023-11-24T06:27:48.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.129+0000 7fd1ae444640 1 -- 172.21.15.16:0/2925689482 wait complete. 2023-11-24T06:27:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.130+0000 7fd1ae444640 1 Processor -- start 2023-11-24T06:27:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.130+0000 7fd1ae444640 1 -- start start 2023-11-24T06:27:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.131+0000 7fd1ae444640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1a806c7c0 0x7fd1a80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.131+0000 7fd1ae444640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a8071410 0x7fd1a80c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.131+0000 7fd1ae444640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071d30 0x7fd1a81330e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.131+0000 7fd1ae444640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8160620 con 0x7fd1a8071410 2023-11-24T06:27:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.131+0000 7fd1ae444640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80c3e10 con 0x7fd1a806c7c0 2023-11-24T06:27:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.131+0000 7fd1ae444640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80c40f0 con 0x7fd1a8071d30 2023-11-24T06:27:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1acc41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a8071410 0x7fd1a80c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1acc41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a8071410 0x7fd1a80c4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33782/0 (socket says 172.21.15.16:33782) 2023-11-24T06:27:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1acc41640 1 -- 172.21.15.16:0/3655693349 learned_addr learned my addr 172.21.15.16:0/3655693349 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1adc43640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071d30 0x7fd1a81330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1ad442640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1a806c7c0 0x7fd1a80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1acc41640 1 -- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071d30 msgr2=0x7fd1a81330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1acc41640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071d30 0x7fd1a81330e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1acc41640 1 -- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1a806c7c0 msgr2=0x7fd1a80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1acc41640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1a806c7c0 0x7fd1a80b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.132+0000 7fd1acc41640 1 -- 172.21.15.16:0/3655693349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd198009400 con 0x7fd1a8071410 2023-11-24T06:27:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.133+0000 7fd1acc41640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a8071410 0x7fd1a80c4bd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd1980098c0 tx=0x7fd1980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.133+0000 7fd19dffb640 1 -- 172.21.15.16:0/3655693349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198044400 con 0x7fd1a8071410 2023-11-24T06:27:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.133+0000 7fd19dffb640 1 -- 172.21.15.16:0/3655693349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd198044580 con 0x7fd1a8071410 2023-11-24T06:27:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.133+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a80c43d0 con 0x7fd1a8071410 2023-11-24T06:27:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.133+0000 7fd19dffb640 1 -- 172.21.15.16:0/3655693349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198044930 con 0x7fd1a8071410 2023-11-24T06:27:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.133+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a80b3ed0 con 0x7fd1a8071410 2023-11-24T06:27:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.134+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a80c48f0 con 0x7fd1a8071410 2023-11-24T06:27:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.135+0000 7fd19dffb640 1 -- 172.21.15.16:0/3655693349 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd19804d440 con 0x7fd1a8071410 2023-11-24T06:27:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.136+0000 7fd19dffb640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd18c06ae40 0x7fd18c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.136+0000 7fd19dffb640 1 -- 172.21.15.16:0/3655693349 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fd1980aba70 con 0x7fd1a8071410 2023-11-24T06:27:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.137+0000 7fd1ad442640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd18c06ae40 0x7fd18c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.138+0000 7fd1ad442640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd18c06ae40 0x7fd18c06d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4006fd0 tx=0x7fd1a4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.140+0000 7fd19dffb640 1 -- 172.21.15.16:0/3655693349 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1a80c48f0 con 0x7fd1a8071410 2023-11-24T06:27:48.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.336+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1a80c48f0 con 0x7fd1a8071410 2023-11-24T06:27:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.339+0000 7fd19dffb640 1 -- 172.21.15.16:0/3655693349 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7fd1a80c48f0 con 0x7fd1a8071410 2023-11-24T06:27:48.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:48.342 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.341+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd18c06ae40 msgr2=0x7fd18c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.341+0000 7fd1ae444640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd18c06ae40 0x7fd18c06d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4006fd0 tx=0x7fd1a4008040 comp rx=0 tx=0).stop 2023-11-24T06:27:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.341+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a8071410 msgr2=0x7fd1a80c4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.342+0000 7fd1ae444640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a8071410 0x7fd1a80c4bd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd1980098c0 tx=0x7fd1980046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.342+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 shutdown_connections 2023-11-24T06:27:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.342+0000 7fd1ae444640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd18c06ae40 0x7fd18c06d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.342+0000 7fd1ae444640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1a8071d30 0x7fd1a81330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.342+0000 7fd1ae444640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1a8071410 0x7fd1a80c4bd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.342+0000 7fd1ae444640 1 --2- 172.21.15.16:0/3655693349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1a806c7c0 0x7fd1a80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.342+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 >> 172.21.15.16:0/3655693349 conn(0x7fd1a806a800 msgr2=0x7fd1a81408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.342+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 shutdown_connections 2023-11-24T06:27:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.342+0000 7fd1ae444640 1 -- 172.21.15.16:0/3655693349 wait complete. 2023-11-24T06:27:48.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:48.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.587+0000 7fa261a2e640 1 Processor -- start 2023-11-24T06:27:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.587+0000 7fa261a2e640 1 -- start start 2023-11-24T06:27:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.589+0000 7fa261a2e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 0x7fa25c06e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.589+0000 7fa261a2e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa25c0b4480 0x7fa25c0ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.589+0000 7fa261a2e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c0cda70 con 0x7fa25c15a070 2023-11-24T06:27:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.589+0000 7fa261a2e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25c0cdd50 con 0x7fa25c0b4480 2023-11-24T06:27:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.589+0000 7fa261a2e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0ce030 con 0x7fa25c159c90 2023-11-24T06:27:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.590+0000 7fa26122d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa25c0b4480 0x7fa25c0ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.590+0000 7fa25bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa25c15a070 0x7fa25c0b3d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42512/0 (socket says 172.21.15.16:42512) 2023-11-24T06:27:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.590+0000 7fa25bfff640 1 -- 172.21.15.16:0/1517053421 learned_addr learned my addr 172.21.15.16:0/1517053421 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.590+0000 7fa260a2c640 1 --2- 172.21.15.16:0/1517053421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 0x7fa25c06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.590+0000 7fa25b7fe640 1 -- 172.21.15.16:0/1517053421 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851839562 0 0) 0x7fa25c0cda70 con 0x7fa25c15a070 2023-11-24T06:27:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.590+0000 7fa26122d640 1 -- 172.21.15.16:0/1517053421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 msgr2=0x7fa25c06e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.590+0000 7fa26122d640 1 --2- 172.21.15.16:0/1517053421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 0x7fa25c06e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.590+0000 7fa26122d640 1 -- 172.21.15.16:0/1517053421 >> v1:172.21.15.16:6789/0 conn(0x7fa25c15a070 legacy=0x7fa25c0b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.590+0000 7fa26122d640 1 -- 172.21.15.16:0/1517053421 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0ce310 con 0x7fa25c0b4480 2023-11-24T06:27:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.591+0000 7fa26122d640 1 --2- 172.21.15.16:0/1517053421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c0b4480 0x7fa25c0ceb10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa254009d90 tx=0x7fa25402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccb0724575b8314 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa25b7fe640 1 -- 172.21.15.16:0/1517053421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25402cc50 con 0x7fa25c0b4480 2023-11-24T06:27:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa25b7fe640 1 -- 172.21.15.16:0/1517053421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25402cdb0 con 0x7fa25c0b4480 2023-11-24T06:27:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa25b7fe640 1 -- 172.21.15.16:0/1517053421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254034660 con 0x7fa25c0b4480 2023-11-24T06:27:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa261a2e640 1 -- 172.21.15.16:0/1517053421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c0b4480 msgr2=0x7fa25c0ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa261a2e640 1 --2- 172.21.15.16:0/1517053421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c0b4480 0x7fa25c0ceb10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa254009d90 tx=0x7fa25402bea0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa261a2e640 1 -- 172.21.15.16:0/1517053421 shutdown_connections 2023-11-24T06:27:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa261a2e640 1 --2- 172.21.15.16:0/1517053421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c0b4480 0x7fa25c0ceb10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa261a2e640 1 --2- 172.21.15.16:0/1517053421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 0x7fa25c06e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa261a2e640 1 -- 172.21.15.16:0/1517053421 >> 172.21.15.16:0/1517053421 conn(0x7fa25c06b4f0 msgr2=0x7fa25c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa261a2e640 1 -- 172.21.15.16:0/1517053421 shutdown_connections 2023-11-24T06:27:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.592+0000 7fa261a2e640 1 -- 172.21.15.16:0/1517053421 wait complete. 2023-11-24T06:27:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.594+0000 7fa261a2e640 1 Processor -- start 2023-11-24T06:27:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.594+0000 7fa261a2e640 1 -- start start 2023-11-24T06:27:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.595+0000 7fa261a2e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c0b4480 0x7fa25c14e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.595+0000 7fa261a2e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 0x7fa25c1e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.595+0000 7fa261a2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c15a070 0x7fa25c1e7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.595+0000 7fa261a2e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c15b5e0 con 0x7fa25c15a070 2023-11-24T06:27:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.595+0000 7fa261a2e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c15b8c0 con 0x7fa25c0b4480 2023-11-24T06:27:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.595+0000 7fa261a2e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c15bba0 con 0x7fa25c159c90 2023-11-24T06:27:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa26122d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c15a070 0x7fa25c1e7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa26122d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c15a070 0x7fa25c1e7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33802/0 (socket says 172.21.15.16:33802) 2023-11-24T06:27:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa26122d640 1 -- 172.21.15.16:0/3639912679 learned_addr learned my addr 172.21.15.16:0/3639912679 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa25bfff640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 0x7fa25c1e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa260a2c640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c0b4480 0x7fa25c14e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa26122d640 1 -- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 msgr2=0x7fa25c1e6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa26122d640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 0x7fa25c1e6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa26122d640 1 -- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c0b4480 msgr2=0x7fa25c14e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa26122d640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c0b4480 0x7fa25c14e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa26122d640 1 -- 172.21.15.16:0/3639912679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa254009a70 con 0x7fa25c15a070 2023-11-24T06:27:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.596+0000 7fa25bfff640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 0x7fa25c1e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:48.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.597+0000 7fa26122d640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c15a070 0x7fa25c1e7000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa254009d90 tx=0x7fa254004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.597+0000 7fa2597fa640 1 -- 172.21.15.16:0/3639912679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254003b90 con 0x7fa25c15a070 2023-11-24T06:27:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.597+0000 7fa2597fa640 1 -- 172.21.15.16:0/3639912679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa254003cf0 con 0x7fa25c15a070 2023-11-24T06:27:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.597+0000 7fa2597fa640 1 -- 172.21.15.16:0/3639912679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254004270 con 0x7fa25c15a070 2023-11-24T06:27:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.597+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c15be80 con 0x7fa25c15a070 2023-11-24T06:27:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.597+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c14e990 con 0x7fa25c15a070 2023-11-24T06:27:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.598+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c15c3a0 con 0x7fa25c15a070 2023-11-24T06:27:48.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.599+0000 7fa2597fa640 1 -- 172.21.15.16:0/3639912679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa25403b050 con 0x7fa25c15a070 2023-11-24T06:27:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.600+0000 7fa2597fa640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa24806ae40 0x7fa24806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:48.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.600+0000 7fa2597fa640 1 -- 172.21.15.16:0/3639912679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fa2540abc00 con 0x7fa25c15a070 2023-11-24T06:27:48.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.601+0000 7fa260a2c640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa24806ae40 0x7fa24806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:48.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.602+0000 7fa260a2c640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa24806ae40 0x7fa24806d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa250006fd0 tx=0x7fa250008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:48.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.605+0000 7fa2597fa640 1 -- 172.21.15.16:0/3639912679 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa25c15c3a0 con 0x7fa25c15a070 2023-11-24T06:27:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.798+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa25c15c3a0 con 0x7fa25c15a070 2023-11-24T06:27:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.799+0000 7fa2597fa640 1 -- 172.21.15.16:0/3639912679 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa25c15c3a0 con 0x7fa25c15a070 2023-11-24T06:27:48.801 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa24806ae40 msgr2=0x7fa24806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa24806ae40 0x7fa24806d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa250006fd0 tx=0x7fa250008040 comp rx=0 tx=0).stop 2023-11-24T06:27:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c15a070 msgr2=0x7fa25c1e7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c15a070 0x7fa25c1e7000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa254009d90 tx=0x7fa254004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 shutdown_connections 2023-11-24T06:27:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa24806ae40 0x7fa24806d300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa25c15a070 0x7fa25c1e7000 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c159c90 0x7fa25c1e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 --2- 172.21.15.16:0/3639912679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c0b4480 0x7fa25c14e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.801+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 >> 172.21.15.16:0/3639912679 conn(0x7fa25c06b4f0 msgr2=0x7fa25c158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.802+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 shutdown_connections 2023-11-24T06:27:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:48.802+0000 7fa261a2e640 1 -- 172.21.15.16:0/3639912679 wait complete. 2023-11-24T06:27:48.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:49.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.047+0000 7f181a02e640 1 Processor -- start 2023-11-24T06:27:49.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.047+0000 7f181a02e640 1 -- start start 2023-11-24T06:27:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.049+0000 7f181a02e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f18140ccb50 0x7f1814070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.049+0000 7f181a02e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f181415bf10 0x7f1814143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.049+0000 7f181a02e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181415f4e0 con 0x7f18140adf20 2023-11-24T06:27:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.049+0000 7f181a02e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181415f970 con 0x7f181415bf10 2023-11-24T06:27:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.049+0000 7f181a02e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1814071130 con 0x7f18140ccb50 2023-11-24T06:27:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.049+0000 7f181902c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f18140adf20 0x7f18140cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42520/0 (socket says 172.21.15.16:42520) 2023-11-24T06:27:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.049+0000 7f181902c640 1 -- 172.21.15.16:0/1854476779 learned_addr learned my addr 172.21.15.16:0/1854476779 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.050+0000 7f181982d640 1 --2- 172.21.15.16:0/1854476779 >> v2:172.21.15.16:3300/0 conn(0x7f181415bf10 0x7f1814143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.050+0000 7f181882b640 1 --2- 172.21.15.16:0/1854476779 >> v2:172.21.15.173:3300/0 conn(0x7f18140ccb50 0x7f1814070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.050+0000 7f1813fff640 1 -- 172.21.15.16:0/1854476779 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128131764 0 0) 0x7f181415f4e0 con 0x7f18140adf20 2023-11-24T06:27:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.050+0000 7f1813fff640 1 -- 172.21.15.16:0/1854476779 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17fc003520 con 0x7f18140adf20 2023-11-24T06:27:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.050+0000 7f181982d640 1 -- 172.21.15.16:0/1854476779 >> v2:172.21.15.173:3300/0 conn(0x7f18140ccb50 msgr2=0x7f1814070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.050+0000 7f181982d640 1 --2- 172.21.15.16:0/1854476779 >> v2:172.21.15.173:3300/0 conn(0x7f18140ccb50 0x7f1814070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.050+0000 7f181982d640 1 -- 172.21.15.16:0/1854476779 >> v1:172.21.15.16:6789/0 conn(0x7f18140adf20 legacy=0x7f18140cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.050+0000 7f181982d640 1 -- 172.21.15.16:0/1854476779 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181415ebb0 con 0x7f181415bf10 2023-11-24T06:27:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.051+0000 7f181982d640 1 --2- 172.21.15.16:0/1854476779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 0x7f1814143d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f181415e9e0 tx=0x7f180c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688ddbb55070d4cf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.051+0000 7f1813fff640 1 -- 172.21.15.16:0/1854476779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c02cbf0 con 0x7f181415bf10 2023-11-24T06:27:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.051+0000 7f1813fff640 1 -- 172.21.15.16:0/1854476779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c02cd50 con 0x7f181415bf10 2023-11-24T06:27:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.051+0000 7f1813fff640 1 -- 172.21.15.16:0/1854476779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c034660 con 0x7f181415bf10 2023-11-24T06:27:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.052+0000 7f181a02e640 1 -- 172.21.15.16:0/1854476779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 msgr2=0x7f1814143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.052+0000 7f181a02e640 1 --2- 172.21.15.16:0/1854476779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 0x7f1814143d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f181415e9e0 tx=0x7f180c02be40 comp rx=0 tx=0).stop 2023-11-24T06:27:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.052+0000 7f181a02e640 1 -- 172.21.15.16:0/1854476779 shutdown_connections 2023-11-24T06:27:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.052+0000 7f181a02e640 1 --2- 172.21.15.16:0/1854476779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 0x7f1814143d30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.052+0000 7f181a02e640 1 --2- 172.21.15.16:0/1854476779 >> v2:172.21.15.173:3300/0 conn(0x7f18140ccb50 0x7f1814070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.052+0000 7f181a02e640 1 -- 172.21.15.16:0/1854476779 >> 172.21.15.16:0/1854476779 conn(0x7f181406a7d0 msgr2=0x7f181406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.052+0000 7f181a02e640 1 -- 172.21.15.16:0/1854476779 shutdown_connections 2023-11-24T06:27:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.052+0000 7f181a02e640 1 -- 172.21.15.16:0/1854476779 wait complete. 2023-11-24T06:27:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.054+0000 7f181a02e640 1 Processor -- start 2023-11-24T06:27:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.054+0000 7f181a02e640 1 -- start start 2023-11-24T06:27:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.055+0000 7f181a02e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18140adf20 0x7f18140c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.055+0000 7f181a02e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18140ccb50 0x7f181415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.055+0000 7f181a02e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 0x7f18140cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.055+0000 7f181a02e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140c4ab0 con 0x7f181415bf10 2023-11-24T06:27:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.055+0000 7f181a02e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140c3e30 con 0x7f18140adf20 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.055+0000 7f181a02e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18140c4110 con 0x7f18140ccb50 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.055+0000 7f181982d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 0x7f18140cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.055+0000 7f181982d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 0x7f18140cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33824/0 (socket says 172.21.15.16:33824) 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.055+0000 7f181982d640 1 -- 172.21.15.16:0/1740031221 learned_addr learned my addr 172.21.15.16:0/1740031221 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f181902c640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18140adf20 0x7f18140c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f181882b640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18140ccb50 0x7f181415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f181982d640 1 -- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18140ccb50 msgr2=0x7f181415aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f181982d640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18140ccb50 0x7f181415aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f181982d640 1 -- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18140adf20 msgr2=0x7f18140c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f181982d640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18140adf20 0x7f18140c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f181982d640 1 -- 172.21.15.16:0/1740031221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c009a70 con 0x7f181415bf10 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f181902c640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18140adf20 0x7f18140c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f181982d640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 0x7f18140cbf10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f180c009e80 tx=0x7f180c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.056+0000 7f1811ffb640 1 -- 172.21.15.16:0/1740031221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c040650 con 0x7f181415bf10 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.057+0000 7f1811ffb640 1 -- 172.21.15.16:0/1740031221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c0407b0 con 0x7f181415bf10 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.057+0000 7f1811ffb640 1 -- 172.21.15.16:0/1740031221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c040b60 con 0x7f181415bf10 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.057+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18140c43f0 con 0x7f181415bf10 2023-11-24T06:27:49.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.057+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18140cc450 con 0x7f181415bf10 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.058+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18140c4bf0 con 0x7f181415bf10 2023-11-24T06:27:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.059+0000 7f1811ffb640 1 -- 172.21.15.16:0/1740031221 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f180c040cc0 con 0x7f181415bf10 2023-11-24T06:27:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.060+0000 7f1811ffb640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f180006ae40 0x7f180006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.060+0000 7f1811ffb640 1 -- 172.21.15.16:0/1740031221 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f180c0ac420 con 0x7f181415bf10 2023-11-24T06:27:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.060+0000 7f181902c640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f180006ae40 0x7f180006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.061+0000 7f181902c640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f180006ae40 0x7f180006d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f18040011e0 tx=0x7f1804009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.064+0000 7f1811ffb640 1 -- 172.21.15.16:0/1740031221 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18140c4bf0 con 0x7f181415bf10 2023-11-24T06:27:49.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.257+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18140c4bf0 con 0x7f181415bf10 2023-11-24T06:27:49.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.258+0000 7f1811ffb640 1 -- 172.21.15.16:0/1740031221 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18140c4bf0 con 0x7f181415bf10 2023-11-24T06:27:49.261 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f180006ae40 msgr2=0x7f180006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f180006ae40 0x7f180006d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f18040011e0 tx=0x7f1804009040 comp rx=0 tx=0).stop 2023-11-24T06:27:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 msgr2=0x7f18140cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 0x7f18140cbf10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f180c009e80 tx=0x7f180c004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 shutdown_connections 2023-11-24T06:27:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f180006ae40 0x7f180006d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f181415bf10 0x7f18140cbf10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18140ccb50 0x7f181415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 --2- 172.21.15.16:0/1740031221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18140adf20 0x7f18140c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 >> 172.21.15.16:0/1740031221 conn(0x7f181406a7d0 msgr2=0x7f181406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 shutdown_connections 2023-11-24T06:27:49.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.261+0000 7f181a02e640 1 -- 172.21.15.16:0/1740031221 wait complete. 2023-11-24T06:27:49.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:49.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.504+0000 7f5dbbd53640 1 Processor -- start 2023-11-24T06:27:49.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.504+0000 7f5dbbd53640 1 -- start start 2023-11-24T06:27:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.505+0000 7f5dbbd53640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db40cd240 0x7f5db40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.505+0000 7f5dbbd53640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5db406c6b0 0x7f5db4099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.505+0000 7f5dbbd53640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db406cea0 con 0x7f5db4099fa0 2023-11-24T06:27:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.505+0000 7f5dbbd53640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db409a430 con 0x7f5db406c6b0 2023-11-24T06:27:49.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.505+0000 7f5dbbd53640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db406cbc0 con 0x7f5db40cd240 2023-11-24T06:27:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.506+0000 7f5db9aef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db40cd240 0x7f5db40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.506+0000 7f5db9aef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db40cd240 0x7f5db40b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:50892/0 (socket says 172.21.15.16:50892) 2023-11-24T06:27:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.506+0000 7f5db9aef640 1 -- 172.21.15.16:0/2577295182 learned_addr learned my addr 172.21.15.16:0/2577295182 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:49.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.506+0000 7f5db92ee640 1 --2- 172.21.15.16:0/2577295182 >> v2:172.21.15.173:3300/0 conn(0x7f5db406c6b0 0x7f5db4099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.507+0000 7f5db9aef640 1 -- 172.21.15.16:0/2577295182 >> v2:172.21.15.173:3300/0 conn(0x7f5db406c6b0 msgr2=0x7f5db4099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.507+0000 7f5db9aef640 1 --2- 172.21.15.16:0/2577295182 >> v2:172.21.15.173:3300/0 conn(0x7f5db406c6b0 0x7f5db4099a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.507+0000 7f5db9aef640 1 -- 172.21.15.16:0/2577295182 >> v1:172.21.15.173:6789/0 conn(0x7f5db4099fa0 legacy=0x7f5db406b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.507+0000 7f5db9aef640 1 -- 172.21.15.16:0/2577295182 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db406f350 con 0x7f5db40cd240 2023-11-24T06:27:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.509+0000 7f5db9aef640 1 --2- 172.21.15.16:0/2577295182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db40cd240 0x7f5db40b3820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009e50 tx=0x7f5da402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c448b05d6f462d5b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.509+0000 7f5db8aed640 1 -- 172.21.15.16:0/2577295182 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da402ca70 con 0x7f5db40cd240 2023-11-24T06:27:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.510+0000 7f5db8aed640 1 -- 172.21.15.16:0/2577295182 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da402cbd0 con 0x7f5db40cd240 2023-11-24T06:27:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.510+0000 7f5db8aed640 1 -- 172.21.15.16:0/2577295182 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da40343f0 con 0x7f5db40cd240 2023-11-24T06:27:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.510+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2577295182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db40cd240 msgr2=0x7f5db40b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.510+0000 7f5dbbd53640 1 --2- 172.21.15.16:0/2577295182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db40cd240 0x7f5db40b3820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009e50 tx=0x7f5da402bed0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.510+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2577295182 shutdown_connections 2023-11-24T06:27:49.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.510+0000 7f5dbbd53640 1 --2- 172.21.15.16:0/2577295182 >> v2:172.21.15.173:3300/0 conn(0x7f5db406c6b0 0x7f5db4099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.510+0000 7f5dbbd53640 1 --2- 172.21.15.16:0/2577295182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db40cd240 0x7f5db40b3820 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.510+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2577295182 >> 172.21.15.16:0/2577295182 conn(0x7f5db40b2d00 msgr2=0x7f5db40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.511+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2577295182 shutdown_connections 2023-11-24T06:27:49.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.511+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2577295182 wait complete. 2023-11-24T06:27:49.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.513+0000 7f5dbbd53640 1 Processor -- start 2023-11-24T06:27:49.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.513+0000 7f5dbbd53640 1 -- start start 2023-11-24T06:27:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5dbbd53640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db406c6b0 0x7f5db4156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5dbbd53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4099fa0 0x7f5db40c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5dbbd53640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5db40cd240 0x7f5db40cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5dbbd53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b5680 con 0x7f5db4099fa0 2023-11-24T06:27:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5dbbd53640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b4a70 con 0x7f5db40cd240 2023-11-24T06:27:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5dbbd53640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b4d50 con 0x7f5db406c6b0 2023-11-24T06:27:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5db92ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4099fa0 0x7f5db40c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5db92ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4099fa0 0x7f5db40c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33832/0 (socket says 172.21.15.16:33832) 2023-11-24T06:27:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5db92ee640 1 -- 172.21.15.16:0/2357034127 learned_addr learned my addr 172.21.15.16:0/2357034127 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5db9aef640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db406c6b0 0x7f5db4156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.515+0000 7f5dba2f0640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5db40cd240 0x7f5db40cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5db92ee640 1 -- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db406c6b0 msgr2=0x7f5db4156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5db92ee640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db406c6b0 0x7f5db4156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5db92ee640 1 -- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5db40cd240 msgr2=0x7f5db40cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5db92ee640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5db40cd240 0x7f5db40cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5db92ee640 1 -- 172.21.15.16:0/2357034127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da4009b30 con 0x7f5db4099fa0 2023-11-24T06:27:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5db92ee640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4099fa0 0x7f5db40c9a70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5da800e840 tx=0x7f5da800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5db27fc640 1 -- 172.21.15.16:0/2357034127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80094f0 con 0x7f5db4099fa0 2023-11-24T06:27:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5db27fc640 1 -- 172.21.15.16:0/2357034127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da800c940 con 0x7f5db4099fa0 2023-11-24T06:27:49.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5db27fc640 1 -- 172.21.15.16:0/2357034127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da800ccd0 con 0x7f5db4099fa0 2023-11-24T06:27:49.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.516+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db40b5030 con 0x7f5db4099fa0 2023-11-24T06:27:49.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.517+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db4156e60 con 0x7f5db4099fa0 2023-11-24T06:27:49.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.518+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db40b5830 con 0x7f5db4099fa0 2023-11-24T06:27:49.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.519+0000 7f5db27fc640 1 -- 172.21.15.16:0/2357034127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5da8013470 con 0x7f5db4099fa0 2023-11-24T06:27:49.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.520+0000 7f5db27fc640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5da006ae40 0x7f5da006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.520+0000 7f5db27fc640 1 -- 172.21.15.16:0/2357034127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f5da8089690 con 0x7f5db4099fa0 2023-11-24T06:27:49.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.521+0000 7f5db9aef640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5da006ae40 0x7f5da006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.522+0000 7f5db9aef640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5da006ae40 0x7f5da006d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5da4035000 tx=0x7f5da4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.524+0000 7f5db27fc640 1 -- 172.21.15.16:0/2357034127 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5db40b5830 con 0x7f5db4099fa0 2023-11-24T06:27:49.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.721+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5db40675d0 con 0x7f5db4099fa0 2023-11-24T06:27:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.722+0000 7f5db27fc640 1 -- 172.21.15.16:0/2357034127 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5da8054760 con 0x7f5db4099fa0 2023-11-24T06:27:49.724 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.724+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5da006ae40 msgr2=0x7f5da006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.724+0000 7f5dbbd53640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5da006ae40 0x7f5da006d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5da4035000 tx=0x7f5da4030000 comp rx=0 tx=0).stop 2023-11-24T06:27:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.724+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4099fa0 msgr2=0x7f5db40c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.724+0000 7f5dbbd53640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4099fa0 0x7f5db40c9a70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5da800e840 tx=0x7f5da800ed00 comp rx=0 tx=0).stop 2023-11-24T06:27:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.725+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 shutdown_connections 2023-11-24T06:27:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.725+0000 7f5dbbd53640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5da006ae40 0x7f5da006d300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.725+0000 7f5dbbd53640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5db40cd240 0x7f5db40cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.725+0000 7f5dbbd53640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4099fa0 0x7f5db40c9a70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.725+0000 7f5dbbd53640 1 --2- 172.21.15.16:0/2357034127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5db406c6b0 0x7f5db4156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.725+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 >> 172.21.15.16:0/2357034127 conn(0x7f5db40b2d00 msgr2=0x7f5db40b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.725+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 shutdown_connections 2023-11-24T06:27:49.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.725+0000 7f5dbbd53640 1 -- 172.21.15.16:0/2357034127 wait complete. 2023-11-24T06:27:49.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:27:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.970+0000 7fb4f8c78640 1 Processor -- start 2023-11-24T06:27:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.971+0000 7fb4f8c78640 1 -- start start 2023-11-24T06:27:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.972+0000 7fb4f8c78640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f4160920 0x7fb4f40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.972+0000 7fb4f8c78640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb4f40cfe70 0x7fb4f4161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.972+0000 7fb4f8c78640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f40d0a10 con 0x7fb4f40cf9f0 2023-11-24T06:27:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.972+0000 7fb4f8c78640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f40d1f50 con 0x7fb4f40cfe70 2023-11-24T06:27:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.972+0000 7fb4f8c78640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40cf080 con 0x7fb4f4160920 2023-11-24T06:27:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.972+0000 7fb4f2ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb4f40cf9f0 0x7fb4f4161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42522/0 (socket says 172.21.15.16:42522) 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.972+0000 7fb4f2ffd640 1 -- 172.21.15.16:0/218069860 learned_addr learned my addr 172.21.15.16:0/218069860 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f3fff640 1 --2- 172.21.15.16:0/218069860 >> v2:172.21.15.173:3300/0 conn(0x7fb4f40cfe70 0x7fb4f4161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f4160920 0x7fb4f40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940627168 0 0) 0x7fb4f40d0a10 con 0x7fb4f40cf9f0 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e0003520 con 0x7fb4f40cf9f0 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2048638681 0 0) 0x7fb4e0003520 con 0x7fb4f40cf9f0 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f4160920 msgr2=0x7fb4f40b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f4160920 0x7fb4f40b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 >> v2:172.21.15.173:3300/0 conn(0x7fb4f40cfe70 msgr2=0x7fb4f4161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 --2- 172.21.15.16:0/218069860 >> v2:172.21.15.173:3300/0 conn(0x7fb4f40cfe70 0x7fb4f4161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f40c6be0 con 0x7fb4f40cf9f0 2023-11-24T06:27:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fb4dc001670 con 0x7fb4f40cf9f0 2023-11-24T06:27:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.973+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 >> v1:172.21.15.16:6789/0 conn(0x7fb4f40cf9f0 legacy=0x7fb4f4161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f27fc640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f4160920 0x7fb4e0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f27fc640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4e0003ef0 0x7fb4e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f27fc640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40d1f50 con 0x7fb4e0004d00 2023-11-24T06:27:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40cf080 con 0x7fb4e0003ef0 2023-11-24T06:27:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e000b980 con 0x7fb4f4160920 2023-11-24T06:27:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f3fff640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f4160920 0x7fb4e0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4e0003ef0 0x7fb4e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f3fff640 1 -- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f4160920 msgr2=0x7fb4e0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f3fff640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f4160920 0x7fb4e0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f3fff640 1 -- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4e0003ef0 msgr2=0x7fb4e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f3fff640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4e0003ef0 0x7fb4e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f3fff640 1 -- 172.21.15.16:0/218069860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0003520 con 0x7fb4e0004d00 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4e0003ef0 0x7fb4e00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.974+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f4160920 0x7fb4e0003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.976+0000 7fb4f3fff640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4e0005180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009ac0 tx=0x7fb4e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328e680f7c6aba23 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.976+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec02c6d0 con 0x7fb4e0004d00 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.976+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec02c830 con 0x7fb4e0004d00 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.976+0000 7fb4f27fc640 1 -- 172.21.15.16:0/218069860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec02cbc0 con 0x7fb4e0004d00 2023-11-24T06:27:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.977+0000 7fb4f8c78640 1 -- 172.21.15.16:0/218069860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 msgr2=0x7fb4e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.977+0000 7fb4f8c78640 1 --2- 172.21.15.16:0/218069860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4e0005180 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009ac0 tx=0x7fb4e00057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.977+0000 7fb4f3fff640 1 -- 172.21.15.16:0/218069860 reap_dead start 2023-11-24T06:27:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.977+0000 7fb4f8c78640 1 -- 172.21.15.16:0/218069860 shutdown_connections 2023-11-24T06:27:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.977+0000 7fb4f8c78640 1 -- 172.21.15.16:0/218069860 >> 172.21.15.16:0/218069860 conn(0x7fb4f40d3090 msgr2=0x7fb4f40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.977+0000 7fb4f8c78640 1 -- 172.21.15.16:0/218069860 shutdown_connections 2023-11-24T06:27:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.977+0000 7fb4f8c78640 1 -- 172.21.15.16:0/218069860 wait complete. 2023-11-24T06:27:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.979+0000 7fb4f8c78640 1 Processor -- start 2023-11-24T06:27:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.979+0000 7fb4f8c78640 1 -- start start 2023-11-24T06:27:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f8c78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4f40c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f8c78640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f41573e0 0x7fb4f4157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f8c78640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4f40c8650 0x7fb4f40c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f8c78640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40c9010 con 0x7fb4e0004d00 2023-11-24T06:27:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f8c78640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f4158b70 con 0x7fb4f40c8650 2023-11-24T06:27:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f8c78640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40c6120 con 0x7fb4f41573e0 2023-11-24T06:27:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4f40c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4f40c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33846/0 (socket says 172.21.15.16:33846) 2023-11-24T06:27:49.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f37fe640 1 -- 172.21.15.16:0/7423001 learned_addr learned my addr 172.21.15.16:0/7423001 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:49.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f3fff640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4f40c8650 0x7fb4f40c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.980+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f41573e0 0x7fb4f4157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:49.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.981+0000 7fb4f37fe640 1 -- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f41573e0 msgr2=0x7fb4f4157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.981+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f41573e0 0x7fb4f4157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.981+0000 7fb4f37fe640 1 -- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4f40c8650 msgr2=0x7fb4f40c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:49.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.981+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4f40c8650 0x7fb4f40c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:49.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.981+0000 7fb4f37fe640 1 -- 172.21.15.16:0/7423001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4157f60 con 0x7fb4e0004d00 2023-11-24T06:27:49.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.981+0000 7fb4f37fe640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4f40c5be0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400f380 tx=0x7fb4e400f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:49.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.982+0000 7fb4ebfff640 1 -- 172.21.15.16:0/7423001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4003b90 con 0x7fb4e0004d00 2023-11-24T06:27:49.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.982+0000 7fb4ebfff640 1 -- 172.21.15.16:0/7423001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e4011040 con 0x7fb4e0004d00 2023-11-24T06:27:49.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.982+0000 7fb4ebfff640 1 -- 172.21.15.16:0/7423001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4004020 con 0x7fb4e0004d00 2023-11-24T06:27:49.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.982+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f4158240 con 0x7fb4e0004d00 2023-11-24T06:27:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.982+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f4158800 con 0x7fb4e0004d00 2023-11-24T06:27:50.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.983+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f41589c0 con 0x7fb4e0004d00 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.984+0000 7fb4ebfff640 1 -- 172.21.15.16:0/7423001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb4e4004180 con 0x7fb4e0004d00 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.985+0000 7fb4ebfff640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb4d406ae70 0x7fb4d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.985+0000 7fb4ebfff640 1 -- 172.21.15.16:0/7423001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fb4e4089d40 con 0x7fb4e0004d00 2023-11-24T06:27:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.986+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb4d406ae70 0x7fb4d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.987+0000 7fb4f2ffd640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb4d406ae70 0x7fb4d406d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4145220 tx=0x7fb4dc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:49.989+0000 7fb4ebfff640 1 -- 172.21.15.16:0/7423001 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4e4054d80 con 0x7fb4e0004d00 2023-11-24T06:27:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.196+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb4f4067450 con 0x7fb4e0004d00 2023-11-24T06:27:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.197+0000 7fb4ebfff640 1 -- 172.21.15.16:0/7423001 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb4e40583a0 con 0x7fb4e0004d00 2023-11-24T06:27:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:27:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.199+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb4d406ae70 msgr2=0x7fb4d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.199+0000 7fb4f8c78640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb4d406ae70 0x7fb4d406d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4145220 tx=0x7fb4dc009040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.199+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 msgr2=0x7fb4f40c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.199+0000 7fb4f8c78640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4f40c5be0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400f380 tx=0x7fb4e400f840 comp rx=0 tx=0).stop 2023-11-24T06:27:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.200+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 shutdown_connections 2023-11-24T06:27:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.200+0000 7fb4f8c78640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb4d406ae70 0x7fb4d406d330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.200+0000 7fb4f8c78640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4f40c8650 0x7fb4f40c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.200+0000 7fb4f8c78640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4f41573e0 0x7fb4f4157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.200+0000 7fb4f8c78640 1 --2- 172.21.15.16:0/7423001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e0004d00 0x7fb4f40c5be0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.200+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 >> 172.21.15.16:0/7423001 conn(0x7fb4f40d3090 msgr2=0x7fb4f406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.200+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 shutdown_connections 2023-11-24T06:27:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.200+0000 7fb4f8c78640 1 -- 172.21.15.16:0/7423001 wait complete. 2023-11-24T06:27:50.222 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:27:50.222 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:50.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:50.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.511+0000 7fd5e2ec2640 1 Processor -- start 2023-11-24T06:27:50.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.511+0000 7fd5e2ec2640 1 -- start start 2023-11-24T06:27:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e2ec2640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd5dc143ae0 0x7fd5dc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e2ec2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5dc15e3b0 0x7fd5dc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e2ec2640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5dc0cfc30 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e2ec2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5dc0b1ee0 con 0x7fd5dc15e3b0 2023-11-24T06:27:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e2ec2640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5dc0b2780 con 0x7fd5dc143ae0 2023-11-24T06:27:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e145f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5dc15e3b0 0x7fd5dc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e145f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5dc15e3b0 0x7fd5dc0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33862/0 (socket says 172.21.15.16:33862) 2023-11-24T06:27:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e145f640 1 -- 172.21.15.16:0/2505312 learned_addr learned my addr 172.21.15.16:0/2505312 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e145f640 1 -- 172.21.15.16:0/2505312 >> v2:172.21.15.173:3300/0 conn(0x7fd5dc143ae0 msgr2=0x7fd5dc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.513+0000 7fd5e0c5e640 1 --2- 172.21.15.16:0/2505312 >> v2:172.21.15.173:3300/0 conn(0x7fd5dc143ae0 0x7fd5dc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.514+0000 7fd5e145f640 1 --2- 172.21.15.16:0/2505312 >> v2:172.21.15.173:3300/0 conn(0x7fd5dc143ae0 0x7fd5dc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.514+0000 7fd5e145f640 1 -- 172.21.15.16:0/2505312 >> v1:172.21.15.173:6789/0 conn(0x7fd5dc0cf7d0 legacy=0x7fd5dc15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.514+0000 7fd5e145f640 1 -- 172.21.15.16:0/2505312 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc15e990 con 0x7fd5dc15e3b0 2023-11-24T06:27:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.514+0000 7fd5e0c5e640 1 --2- 172.21.15.16:0/2505312 >> v2:172.21.15.173:3300/0 conn(0x7fd5dc143ae0 0x7fd5dc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.515+0000 7fd5e145f640 1 --2- 172.21.15.16:0/2505312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc15e3b0 0x7fd5dc0702c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0cfea0 tx=0x7fd5d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ee7372940240d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.515+0000 7fd5d37fe640 1 -- 172.21.15.16:0/2505312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d402ca80 con 0x7fd5dc15e3b0 2023-11-24T06:27:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.515+0000 7fd5d37fe640 1 -- 172.21.15.16:0/2505312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d402cbe0 con 0x7fd5dc15e3b0 2023-11-24T06:27:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.515+0000 7fd5d37fe640 1 -- 172.21.15.16:0/2505312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40343f0 con 0x7fd5dc15e3b0 2023-11-24T06:27:50.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.516+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/2505312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc15e3b0 msgr2=0x7fd5dc0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.516+0000 7fd5e2ec2640 1 --2- 172.21.15.16:0/2505312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc15e3b0 0x7fd5dc0702c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0cfea0 tx=0x7fd5d402bee0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.516+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/2505312 shutdown_connections 2023-11-24T06:27:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.516+0000 7fd5e2ec2640 1 --2- 172.21.15.16:0/2505312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc15e3b0 0x7fd5dc0702c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.516+0000 7fd5e2ec2640 1 --2- 172.21.15.16:0/2505312 >> v2:172.21.15.173:3300/0 conn(0x7fd5dc143ae0 0x7fd5dc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.516+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/2505312 >> 172.21.15.16:0/2505312 conn(0x7fd5dc06a800 msgr2=0x7fd5dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.516+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/2505312 shutdown_connections 2023-11-24T06:27:50.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.516+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/2505312 wait complete. 2023-11-24T06:27:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.518+0000 7fd5e2ec2640 1 Processor -- start 2023-11-24T06:27:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.518+0000 7fd5e2ec2640 1 -- start start 2023-11-24T06:27:50.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5e2ec2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc0cf7d0 0x7fd5dc13f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5e2ec2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5dc143ae0 0x7fd5dc0aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5e2ec2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5dc15e3b0 0x7fd5dc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5e2ec2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc0ac780 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5e2ec2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc130bb0 con 0x7fd5dc143ae0 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5e2ec2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc130e90 con 0x7fd5dc15e3b0 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5e0c5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc0cf7d0 0x7fd5dc13f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5e0c5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc0cf7d0 0x7fd5dc13f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33866/0 (socket says 172.21.15.16:33866) 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5e0c5e640 1 -- 172.21.15.16:0/1223525466 learned_addr learned my addr 172.21.15.16:0/1223525466 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.519+0000 7fd5d3fff640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5dc143ae0 0x7fd5dc0aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5e0c5e640 1 -- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5dc15e3b0 msgr2=0x7fd5dc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5e0c5e640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5dc15e3b0 0x7fd5dc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5e0c5e640 1 -- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5dc143ae0 msgr2=0x7fd5dc0aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5e0c5e640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5dc143ae0 0x7fd5dc0aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5e0c5e640 1 -- 172.21.15.16:0/1223525466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d4009b30 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5d3fff640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5dc143ae0 0x7fd5dc0aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5e0c5e640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc0cf7d0 0x7fd5dc13f860 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c7f0 tx=0x7fd5d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5d17fa640 1 -- 172.21.15.16:0/1223525466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8007a30 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5d17fa640 1 -- 172.21.15.16:0/1223525466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d8007b90 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5dc131170 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5d17fa640 1 -- 172.21.15.16:0/1223525466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800ce40 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.520+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc12e2d0 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.522+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5dc0675d0 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.523+0000 7fd5d17fa640 1 -- 172.21.15.16:0/1223525466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd5d80113f0 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.524+0000 7fd5d17fa640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd5c806ae70 0x7fd5c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.524+0000 7fd5d17fa640 1 -- 172.21.15.16:0/1223525466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fd5d8089780 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.525+0000 7fd5d3fff640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd5c806ae70 0x7fd5c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.526+0000 7fd5d3fff640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd5c806ae70 0x7fd5c806d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc003eb0 tx=0x7fd5cc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.529+0000 7fd5d17fa640 1 -- 172.21.15.16:0/1223525466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5d80547a0 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.722+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5dc0ce620 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.725+0000 7fd5d17fa640 1 -- 172.21.15.16:0/1223525466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7fd5d8057dc0 con 0x7fd5dc0cf7d0 2023-11-24T06:27:50.728 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:50.728 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:50.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd5c806ae70 msgr2=0x7fd5c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd5c806ae70 0x7fd5c806d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc003eb0 tx=0x7fd5cc009040 comp rx=0 tx=0).stop 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc0cf7d0 msgr2=0x7fd5dc13f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc0cf7d0 0x7fd5dc13f860 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c7f0 tx=0x7fd5d800cbf0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 shutdown_connections 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd5c806ae70 0x7fd5c806d330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5dc15e3b0 0x7fd5dc0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5dc143ae0 0x7fd5dc0aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 --2- 172.21.15.16:0/1223525466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5dc0cf7d0 0x7fd5dc13f860 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 >> 172.21.15.16:0/1223525466 conn(0x7fd5dc06a800 msgr2=0x7fd5dc140b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 shutdown_connections 2023-11-24T06:27:50.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.728+0000 7fd5e2ec2640 1 -- 172.21.15.16:0/1223525466 wait complete. 2023-11-24T06:27:50.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:50.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:50.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.985+0000 7f8ecc231640 1 Processor -- start 2023-11-24T06:27:50.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.985+0000 7f8ecc231640 1 -- start start 2023-11-24T06:27:50.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.986+0000 7f8ecc231640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec4161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.986+0000 7f8ecc231640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8ec40ced50 0x7f8ec40d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.986+0000 7f8ecc231640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec40cf1b0 con 0x7f8ec40ce8f0 2023-11-24T06:27:50.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.986+0000 7f8ecc231640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec41618a0 con 0x7f8ec40ced50 2023-11-24T06:27:50.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.986+0000 7f8ecc231640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40d0de0 con 0x7f8ec4160f80 2023-11-24T06:27:50.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.987+0000 7f8ec9fcd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec4161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.987+0000 7f8eca7ce640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8ec40ced50 0x7f8ec40d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.987+0000 7f8ec9fcd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec4161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:50942/0 (socket says 172.21.15.16:50942) 2023-11-24T06:27:50.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.987+0000 7f8ec9fcd640 1 -- 172.21.15.16:0/3862611583 learned_addr learned my addr 172.21.15.16:0/3862611583 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:50.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.987+0000 7f8ec9fcd640 1 -- 172.21.15.16:0/3862611583 >> v2:172.21.15.173:3300/0 conn(0x7f8ec40ced50 msgr2=0x7f8ec40d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:50.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.987+0000 7f8ec9fcd640 1 --2- 172.21.15.16:0/3862611583 >> v2:172.21.15.173:3300/0 conn(0x7f8ec40ced50 0x7f8ec40d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.987+0000 7f8ec9fcd640 1 -- 172.21.15.16:0/3862611583 >> v1:172.21.15.173:6789/0 conn(0x7f8ec40ce8f0 legacy=0x7f8ec40cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.987+0000 7f8ec9fcd640 1 -- 172.21.15.16:0/3862611583 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40d10c0 con 0x7f8ec4160f80 2023-11-24T06:27:50.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.989+0000 7f8ec9fcd640 1 --2- 172.21.15.16:0/3862611583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec4161360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009870 tx=0x7f8ec002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33bd2a3ec14d2b95 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:50.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.989+0000 7f8ec8fcb640 1 -- 172.21.15.16:0/3862611583 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0030840 con 0x7f8ec4160f80 2023-11-24T06:27:50.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.989+0000 7f8ec8fcb640 1 -- 172.21.15.16:0/3862611583 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec00309a0 con 0x7f8ec4160f80 2023-11-24T06:27:50.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.989+0000 7f8ec8fcb640 1 -- 172.21.15.16:0/3862611583 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0030d30 con 0x7f8ec4160f80 2023-11-24T06:27:50.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.990+0000 7f8ecc231640 1 -- 172.21.15.16:0/3862611583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 msgr2=0x7f8ec4161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:50.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.990+0000 7f8ecc231640 1 --2- 172.21.15.16:0/3862611583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec4161360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009870 tx=0x7f8ec002b980 comp rx=0 tx=0).stop 2023-11-24T06:27:50.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.990+0000 7f8ecc231640 1 -- 172.21.15.16:0/3862611583 shutdown_connections 2023-11-24T06:27:50.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.990+0000 7f8ecc231640 1 --2- 172.21.15.16:0/3862611583 >> v2:172.21.15.173:3300/0 conn(0x7f8ec40ced50 0x7f8ec40d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.990+0000 7f8ecc231640 1 --2- 172.21.15.16:0/3862611583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec4161360 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:50.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.990+0000 7f8ecc231640 1 -- 172.21.15.16:0/3862611583 >> 172.21.15.16:0/3862611583 conn(0x7f8ec409c550 msgr2=0x7f8ec40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:50.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.990+0000 7f8ecc231640 1 -- 172.21.15.16:0/3862611583 shutdown_connections 2023-11-24T06:27:50.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.990+0000 7f8ecc231640 1 -- 172.21.15.16:0/3862611583 wait complete. 2023-11-24T06:27:50.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.992+0000 7f8ecc231640 1 Processor -- start 2023-11-24T06:27:50.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.992+0000 7f8ecc231640 1 -- start start 2023-11-24T06:27:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.993+0000 7f8ecc231640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ec40ce8f0 0x7f8ec4159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.993+0000 7f8ecc231640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40ced50 0x7f8ec40c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.993+0000 7f8ecc231640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec40c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.993+0000 7f8ecc231640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40cba40 con 0x7f8ec40ced50 2023-11-24T06:27:50.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.993+0000 7f8ecc231640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40cbd20 con 0x7f8ec40ce8f0 2023-11-24T06:27:50.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.993+0000 7f8ecc231640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40cc000 con 0x7f8ec4160f80 2023-11-24T06:27:50.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec97cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40ced50 0x7f8ec40c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec97cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40ced50 0x7f8ec40c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33880/0 (socket says 172.21.15.16:33880) 2023-11-24T06:27:50.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec97cc640 1 -- 172.21.15.16:0/882006255 learned_addr learned my addr 172.21.15.16:0/882006255 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:50.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec9fcd640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ec40ce8f0 0x7f8ec4159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8eca7ce640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec40c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec97cc640 1 -- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 msgr2=0x7f8ec40c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec97cc640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec40c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec97cc640 1 -- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ec40ce8f0 msgr2=0x7f8ec4159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec97cc640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ec40ce8f0 0x7f8ec4159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec97cc640 1 -- 172.21.15.16:0/882006255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec00094d0 con 0x7f8ec40ced50 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.994+0000 7f8ec9fcd640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ec40ce8f0 0x7f8ec4159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.995+0000 7f8ec97cc640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40ced50 0x7f8ec40c6d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400c400 tx=0x7f8eb400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.995+0000 7f8eba7fc640 1 -- 172.21.15.16:0/882006255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40052d0 con 0x7f8ec40ced50 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.995+0000 7f8eba7fc640 1 -- 172.21.15.16:0/882006255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb4005430 con 0x7f8ec40ced50 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.995+0000 7f8eba7fc640 1 -- 172.21.15.16:0/882006255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4003830 con 0x7f8ec40ced50 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.995+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec40cc2e0 con 0x7f8ec40ced50 2023-11-24T06:27:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.995+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec4159bc0 con 0x7f8ec40ced50 2023-11-24T06:27:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.996+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec4160cc0 con 0x7f8ec40ced50 2023-11-24T06:27:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.997+0000 7f8eba7fc640 1 -- 172.21.15.16:0/882006255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8eb4003990 con 0x7f8ec40ced50 2023-11-24T06:27:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.998+0000 7f8eba7fc640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ea806ae40 0x7f8ea806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.998+0000 7f8eba7fc640 1 -- 172.21.15.16:0/882006255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f8eb4088cb0 con 0x7f8ec40ced50 2023-11-24T06:27:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:50.999+0000 7f8ec9fcd640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ea806ae40 0x7f8ea806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.000+0000 7f8ec9fcd640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ea806ae40 0x7f8ea806d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0036000 tx=0x7f8ec0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.002+0000 7f8eba7fc640 1 -- 172.21.15.16:0/882006255 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8eb4053c40 con 0x7f8ec40ced50 2023-11-24T06:27:51.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.196+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ec40675d0 con 0x7f8ec40ced50 2023-11-24T06:27:51.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.197+0000 7f8eba7fc640 1 -- 172.21.15.16:0/882006255 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8eb40170a0 con 0x7f8ec40ced50 2023-11-24T06:27:51.199 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:51.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ea806ae40 msgr2=0x7f8ea806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ea806ae40 0x7f8ea806d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0036000 tx=0x7f8ec0008030 comp rx=0 tx=0).stop 2023-11-24T06:27:51.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40ced50 msgr2=0x7f8ec40c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40ced50 0x7f8ec40c6d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400c400 tx=0x7f8eb400c8c0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 shutdown_connections 2023-11-24T06:27:51.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ea806ae40 0x7f8ea806d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ec4160f80 0x7f8ec40c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec40ced50 0x7f8ec40c6d40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 --2- 172.21.15.16:0/882006255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ec40ce8f0 0x7f8ec4159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 >> 172.21.15.16:0/882006255 conn(0x7f8ec409c550 msgr2=0x7f8ec40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 shutdown_connections 2023-11-24T06:27:51.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.201+0000 7f8ecc231640 1 -- 172.21.15.16:0/882006255 wait complete. 2023-11-24T06:27:51.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:51.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.436+0000 7f4bfe010640 1 Processor -- start 2023-11-24T06:27:51.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.436+0000 7f4bfe010640 1 -- start start 2023-11-24T06:27:51.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.437+0000 7f4bfe010640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf80cf9a0 0x7f4bf80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.437+0000 7f4bfe010640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4bf80b31d0 0x7f4bf80b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.437+0000 7f4bfe010640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf80d0320 con 0x7f4bf815d4b0 2023-11-24T06:27:51.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.437+0000 7f4bfe010640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf80d1c60 con 0x7f4bf80b31d0 2023-11-24T06:27:51.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.437+0000 7f4bfe010640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80b2730 con 0x7f4bf80cf9a0 2023-11-24T06:27:51.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.437+0000 7f4bfd00e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4bf815d4b0 0x7f4bf80d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42536/0 (socket says 172.21.15.16:42536) 2023-11-24T06:27:51.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.437+0000 7f4bfd00e640 1 -- 172.21.15.16:0/2979041976 learned_addr learned my addr 172.21.15.16:0/2979041976 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4bfc80d640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf80cf9a0 0x7f4bf80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4bfd80f640 1 --2- 172.21.15.16:0/2979041976 >> v2:172.21.15.173:3300/0 conn(0x7f4bf80b31d0 0x7f4bf80b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465872209 0 0) 0x7f4bf80d0320 con 0x7f4bf815d4b0 2023-11-24T06:27:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be4003520 con 0x7f4bf815d4b0 2023-11-24T06:27:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1883024849 0 0) 0x7f4be4003520 con 0x7f4bf815d4b0 2023-11-24T06:27:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf80cf9a0 msgr2=0x7f4bf80cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf80cf9a0 0x7f4bf80cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 >> v2:172.21.15.173:3300/0 conn(0x7f4bf80b31d0 msgr2=0x7f4bf80b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 --2- 172.21.15.16:0/2979041976 >> v2:172.21.15.173:3300/0 conn(0x7f4bf80b31d0 0x7f4bf80b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf80d1ec0 con 0x7f4bf815d4b0 2023-11-24T06:27:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f4bf4001670 con 0x7f4bf815d4b0 2023-11-24T06:27:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 >> v1:172.21.15.16:6789/0 conn(0x7f4bf815d4b0 legacy=0x7f4bf80d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4be4003970 0x7f4be4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be40047c0 0x7f4be4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4be4005180 0x7f4be400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80d1c60 con 0x7f4be40047c0 2023-11-24T06:27:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80b2730 con 0x7f4be4005180 2023-11-24T06:27:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be400bfc0 con 0x7f4be4003970 2023-11-24T06:27:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.438+0000 7f4bfd80f640 1 --2- 172.21.15.16:0/2979041976 >> v2:172.21.15.173:3300/0 conn(0x7f4bf80b31d0 0x7f4bf80b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.439+0000 7f4bfc80d640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf80cf9a0 0x7f4bf80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.439+0000 7f4bfc80d640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be40047c0 0x7f4be4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.439+0000 7f4bfd80f640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4be4005180 0x7f4be400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.439+0000 7f4bfd00e640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4be4003970 0x7f4be4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.439+0000 7f4bfc80d640 1 -- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4be4003970 msgr2=0x7f4be4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.439+0000 7f4bfc80d640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4be4003970 0x7f4be4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.439+0000 7f4bfc80d640 1 -- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4be4005180 msgr2=0x7f4be400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.439+0000 7f4bfc80d640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4be4005180 0x7f4be400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.439+0000 7f4bfc80d640 1 -- 172.21.15.16:0/2979041976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be4003520 con 0x7f4be40047c0 2023-11-24T06:27:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.440+0000 7f4bfc80d640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be40047c0 0x7f4be4004c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4bf808b580 tx=0x7f4be40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380f8466a53db794 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8033600 con 0x7f4be40047c0 2023-11-24T06:27:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be802f050 con 0x7f4be40047c0 2023-11-24T06:27:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4beffff640 1 -- 172.21.15.16:0/2979041976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8033980 con 0x7f4be40047c0 2023-11-24T06:27:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4bfe010640 1 -- 172.21.15.16:0/2979041976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be40047c0 msgr2=0x7f4be4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4bfe010640 1 --2- 172.21.15.16:0/2979041976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be40047c0 0x7f4be4004c40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4bf808b580 tx=0x7f4be40057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4bfd80f640 1 -- 172.21.15.16:0/2979041976 reap_dead start 2023-11-24T06:27:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4bfe010640 1 -- 172.21.15.16:0/2979041976 shutdown_connections 2023-11-24T06:27:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4bfe010640 1 -- 172.21.15.16:0/2979041976 >> 172.21.15.16:0/2979041976 conn(0x7f4bf80d3000 msgr2=0x7f4bf80d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4bfe010640 1 -- 172.21.15.16:0/2979041976 shutdown_connections 2023-11-24T06:27:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.441+0000 7f4bfe010640 1 -- 172.21.15.16:0/2979041976 wait complete. 2023-11-24T06:27:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.443+0000 7f4bfe010640 1 Processor -- start 2023-11-24T06:27:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.443+0000 7f4bfe010640 1 -- start start 2023-11-24T06:27:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.444+0000 7f4bfe010640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4be40047c0 0x7f4bf81f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.444+0000 7f4bfe010640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bf81f62d0 0x7f4bf80b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.444+0000 7f4bfe010640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf815a180 0x7f4bf815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.444+0000 7f4bfe010640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf81f3ca0 con 0x7f4bf81f62d0 2023-11-24T06:27:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.444+0000 7f4bfe010640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf815ab70 con 0x7f4be40047c0 2023-11-24T06:27:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.444+0000 7f4bfe010640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf81f67b0 con 0x7f4bf815a180 2023-11-24T06:27:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfc80d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bf81f62d0 0x7f4bf80b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfc80d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bf81f62d0 0x7f4bf80b03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33898/0 (socket says 172.21.15.16:33898) 2023-11-24T06:27:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfc80d640 1 -- 172.21.15.16:0/1806447087 learned_addr learned my addr 172.21.15.16:0/1806447087 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfd00e640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4be40047c0 0x7f4bf81f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfd80f640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf815a180 0x7f4bf815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfc80d640 1 -- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf815a180 msgr2=0x7f4bf815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfc80d640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf815a180 0x7f4bf815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfc80d640 1 -- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4be40047c0 msgr2=0x7f4bf81f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfc80d640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4be40047c0 0x7f4bf81f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.445+0000 7f4bfc80d640 1 -- 172.21.15.16:0/1806447087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf81f6a90 con 0x7f4bf81f62d0 2023-11-24T06:27:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.446+0000 7f4bfc80d640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bf81f62d0 0x7f4bf80b03c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4be80097c0 tx=0x7f4be80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.446+0000 7f4bedffb640 1 -- 172.21.15.16:0/1806447087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be803d710 con 0x7f4bf81f62d0 2023-11-24T06:27:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.446+0000 7f4bedffb640 1 -- 172.21.15.16:0/1806447087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be803d870 con 0x7f4bf81f62d0 2023-11-24T06:27:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.446+0000 7f4bedffb640 1 -- 172.21.15.16:0/1806447087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be803dc00 con 0x7f4bf81f62d0 2023-11-24T06:27:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.446+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf81f6d70 con 0x7f4bf81f62d0 2023-11-24T06:27:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.446+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf809d570 con 0x7f4bf81f62d0 2023-11-24T06:27:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.447+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf8067450 con 0x7f4bf81f62d0 2023-11-24T06:27:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.448+0000 7f4bedffb640 1 -- 172.21.15.16:0/1806447087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4be8032420 con 0x7f4bf81f62d0 2023-11-24T06:27:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.449+0000 7f4bedffb640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4bdc06ae40 0x7f4bdc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.449+0000 7f4bedffb640 1 -- 172.21.15.16:0/1806447087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f4be80ab220 con 0x7f4bf81f62d0 2023-11-24T06:27:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.450+0000 7f4bfd00e640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4bdc06ae40 0x7f4bdc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.451+0000 7f4bfd00e640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4bdc06ae40 0x7f4bdc06d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000ff0 tx=0x7f4bf4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.453+0000 7f4bedffb640 1 -- 172.21.15.16:0/1806447087 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4be80761b0 con 0x7f4bf81f62d0 2023-11-24T06:27:51.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.646+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4bf8067d00 con 0x7f4bf81f62d0 2023-11-24T06:27:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.648+0000 7f4bedffb640 1 -- 172.21.15.16:0/1806447087 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4be800f040 con 0x7f4bf81f62d0 2023-11-24T06:27:51.650 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:51.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4bdc06ae40 msgr2=0x7f4bdc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4bdc06ae40 0x7f4bdc06d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000ff0 tx=0x7f4bf4009040 comp rx=0 tx=0).stop 2023-11-24T06:27:51.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bf81f62d0 msgr2=0x7f4bf80b03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bf81f62d0 0x7f4bf80b03c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4be80097c0 tx=0x7f4be80046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 shutdown_connections 2023-11-24T06:27:51.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4bdc06ae40 0x7f4bdc06d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4bf815a180 0x7f4bf815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bf81f62d0 0x7f4bf80b03c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 --2- 172.21.15.16:0/1806447087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4be40047c0 0x7f4bf81f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.650+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 >> 172.21.15.16:0/1806447087 conn(0x7f4bf80d3000 msgr2=0x7f4bf80b7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.651+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 shutdown_connections 2023-11-24T06:27:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.651+0000 7f4bfe010640 1 -- 172.21.15.16:0/1806447087 wait complete. 2023-11-24T06:27:51.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.905+0000 7fde871db640 1 Processor -- start 2023-11-24T06:27:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.905+0000 7fde871db640 1 -- start start 2023-11-24T06:27:51.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.907+0000 7fde871db640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde80143910 0x7fde8015d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.907+0000 7fde871db640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fde8015ded0 0x7fde800ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.907+0000 7fde871db640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8015e7e0 con 0x7fde80070b70 2023-11-24T06:27:51.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.907+0000 7fde871db640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8015e360 con 0x7fde8015ded0 2023-11-24T06:27:51.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.907+0000 7fde871db640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800cea80 con 0x7fde80143910 2023-11-24T06:27:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.907+0000 7fde859d8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fde80070b70 0x7fde800b2350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42544/0 (socket says 172.21.15.16:42544) 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.907+0000 7fde859d8640 1 -- 172.21.15.16:0/971231828 learned_addr learned my addr 172.21.15.16:0/971231828 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.908+0000 7fde869da640 1 --2- 172.21.15.16:0/971231828 >> v2:172.21.15.173:3300/0 conn(0x7fde8015ded0 0x7fde800ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.908+0000 7fde861d9640 1 --2- 172.21.15.16:0/971231828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde80143910 0x7fde8015d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.908+0000 7fde851d7640 1 -- 172.21.15.16:0/971231828 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362072311 0 0) 0x7fde8015e7e0 con 0x7fde80070b70 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.908+0000 7fde851d7640 1 -- 172.21.15.16:0/971231828 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde6c003520 con 0x7fde80070b70 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.908+0000 7fde869da640 1 -- 172.21.15.16:0/971231828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde80143910 msgr2=0x7fde8015d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.908+0000 7fde869da640 1 --2- 172.21.15.16:0/971231828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde80143910 0x7fde8015d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.908+0000 7fde869da640 1 -- 172.21.15.16:0/971231828 >> v1:172.21.15.16:6789/0 conn(0x7fde80070b70 legacy=0x7fde800b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.908+0000 7fde869da640 1 -- 172.21.15.16:0/971231828 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde800710e0 con 0x7fde8015ded0 2023-11-24T06:27:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.910+0000 7fde869da640 1 --2- 172.21.15.16:0/971231828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde8015ded0 0x7fde800ce540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fde78009d90 tx=0x7fde7802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a0b02b329b3df0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.910+0000 7fde851d7640 1 -- 172.21.15.16:0/971231828 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7802cbc0 con 0x7fde8015ded0 2023-11-24T06:27:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.910+0000 7fde851d7640 1 -- 172.21.15.16:0/971231828 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7802cd20 con 0x7fde8015ded0 2023-11-24T06:27:51.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.910+0000 7fde851d7640 1 -- 172.21.15.16:0/971231828 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78034660 con 0x7fde8015ded0 2023-11-24T06:27:51.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.911+0000 7fde871db640 1 -- 172.21.15.16:0/971231828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde8015ded0 msgr2=0x7fde800ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.911+0000 7fde871db640 1 --2- 172.21.15.16:0/971231828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde8015ded0 0x7fde800ce540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fde78009d90 tx=0x7fde7802be10 comp rx=0 tx=0).stop 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.911+0000 7fde871db640 1 -- 172.21.15.16:0/971231828 shutdown_connections 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.911+0000 7fde871db640 1 --2- 172.21.15.16:0/971231828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde8015ded0 0x7fde800ce540 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.911+0000 7fde871db640 1 --2- 172.21.15.16:0/971231828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde80143910 0x7fde8015d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.911+0000 7fde871db640 1 -- 172.21.15.16:0/971231828 >> 172.21.15.16:0/971231828 conn(0x7fde8006a7d0 msgr2=0x7fde8006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.911+0000 7fde871db640 1 -- 172.21.15.16:0/971231828 shutdown_connections 2023-11-24T06:27:51.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.912+0000 7fde871db640 1 -- 172.21.15.16:0/971231828 wait complete. 2023-11-24T06:27:51.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.914+0000 7fde871db640 1 Processor -- start 2023-11-24T06:27:51.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.914+0000 7fde871db640 1 -- start start 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.915+0000 7fde871db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde80070b70 0x7fde80139550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.915+0000 7fde871db640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde80143910 0x7fde8015c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.915+0000 7fde871db640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde8015ded0 0x7fde8015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.915+0000 7fde871db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8013bdd0 con 0x7fde80070b70 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.915+0000 7fde871db640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde801f2d40 con 0x7fde80143910 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.915+0000 7fde871db640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8013b530 con 0x7fde8015ded0 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde861d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde80070b70 0x7fde80139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde861d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde80070b70 0x7fde80139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33904/0 (socket says 172.21.15.16:33904) 2023-11-24T06:27:51.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde861d9640 1 -- 172.21.15.16:0/1644882018 learned_addr learned my addr 172.21.15.16:0/1644882018 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde859d8640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde80143910 0x7fde8015c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde869da640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde8015ded0 0x7fde8015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde861d9640 1 -- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde8015ded0 msgr2=0x7fde8015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde861d9640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde8015ded0 0x7fde8015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde861d9640 1 -- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde80143910 msgr2=0x7fde8015c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde861d9640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde80143910 0x7fde8015c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde861d9640 1 -- 172.21.15.16:0/1644882018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde78009a70 con 0x7fde80070b70 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.916+0000 7fde869da640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde8015ded0 0x7fde8015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.917+0000 7fde861d9640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde80070b70 0x7fde80139550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0089b0 tx=0x7fde7c008aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.917+0000 7fde76ffd640 1 -- 172.21.15.16:0/1644882018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c010040 con 0x7fde80070b70 2023-11-24T06:27:51.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.917+0000 7fde76ffd640 1 -- 172.21.15.16:0/1644882018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c009d20 con 0x7fde80070b70 2023-11-24T06:27:51.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.917+0000 7fde76ffd640 1 -- 172.21.15.16:0/1644882018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00bea0 con 0x7fde80070b70 2023-11-24T06:27:51.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.917+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8013b810 con 0x7fde80070b70 2023-11-24T06:27:51.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.917+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8015a590 con 0x7fde80070b70 2023-11-24T06:27:51.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.919+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde80067450 con 0x7fde80070b70 2023-11-24T06:27:51.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.920+0000 7fde76ffd640 1 -- 172.21.15.16:0/1644882018 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde7c016630 con 0x7fde80070b70 2023-11-24T06:27:51.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.921+0000 7fde76ffd640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6406ae40 0x7fde6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:51.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.921+0000 7fde76ffd640 1 -- 172.21.15.16:0/1644882018 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fde7c0893c0 con 0x7fde80070b70 2023-11-24T06:27:51.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.922+0000 7fde859d8640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6406ae40 0x7fde6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:51.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.923+0000 7fde859d8640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6406ae40 0x7fde6406d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde80140840 tx=0x7fde70009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:51.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:51.926+0000 7fde76ffd640 1 -- 172.21.15.16:0/1644882018 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde7c0544a0 con 0x7fde80070b70 2023-11-24T06:27:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.125+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde800b3f20 con 0x7fde80070b70 2023-11-24T06:27:52.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.126+0000 7fde76ffd640 1 -- 172.21.15.16:0/1644882018 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde7c057ac0 con 0x7fde80070b70 2023-11-24T06:27:52.128 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6406ae40 msgr2=0x7fde6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6406ae40 0x7fde6406d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde80140840 tx=0x7fde70009040 comp rx=0 tx=0).stop 2023-11-24T06:27:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde80070b70 msgr2=0x7fde80139550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde80070b70 0x7fde80139550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0089b0 tx=0x7fde7c008aa0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 shutdown_connections 2023-11-24T06:27:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde6406ae40 0x7fde6406d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde8015ded0 0x7fde8015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde80143910 0x7fde8015c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 --2- 172.21.15.16:0/1644882018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde80070b70 0x7fde80139550 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 >> 172.21.15.16:0/1644882018 conn(0x7fde8006a7d0 msgr2=0x7fde800adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 shutdown_connections 2023-11-24T06:27:52.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.129+0000 7fde871db640 1 -- 172.21.15.16:0/1644882018 wait complete. 2023-11-24T06:27:52.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:27:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.382+0000 7f61d0cea640 1 Processor -- start 2023-11-24T06:27:52.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.382+0000 7f61d0cea640 1 -- start start 2023-11-24T06:27:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.383+0000 7f61d0cea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61cc160920 0x7f61cc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.383+0000 7f61d0cea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 0x7f61cc161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.383+0000 7f61d0cea640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61cc0d0a10 con 0x7f61cc0cf9f0 2023-11-24T06:27:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.383+0000 7f61d0cea640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61cc0d0330 con 0x7f61cc160920 2023-11-24T06:27:52.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.383+0000 7f61d0cea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc0d1f50 con 0x7f61cc0cfea0 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.384+0000 7f61c2ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f61cc0cf9f0 0x7f61cc161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42558/0 (socket says 172.21.15.16:42558) 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.384+0000 7f61cb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61cc160920 0x7f61cc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.384+0000 7f61c2ffd640 1 -- 172.21.15.16:0/463845921 learned_addr learned my addr 172.21.15.16:0/463845921 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.384+0000 7f61cbfff640 1 --2- 172.21.15.16:0/463845921 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 0x7f61cc161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.384+0000 7f61cb7fe640 1 -- 172.21.15.16:0/463845921 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 msgr2=0x7f61cc161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.384+0000 7f61cb7fe640 1 --2- 172.21.15.16:0/463845921 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 0x7f61cc161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.384+0000 7f61cb7fe640 1 -- 172.21.15.16:0/463845921 >> v1:172.21.15.16:6789/0 conn(0x7f61cc0cf9f0 legacy=0x7f61cc161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.384+0000 7f61caffd640 1 -- 172.21.15.16:0/463845921 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221582054 0 0) 0x7f61cc0d0a10 con 0x7f61cc0cf9f0 2023-11-24T06:27:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.384+0000 7f61cb7fe640 1 -- 172.21.15.16:0/463845921 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc0cf080 con 0x7f61cc160920 2023-11-24T06:27:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.386+0000 7f61cb7fe640 1 --2- 172.21.15.16:0/463845921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 0x7f61cc0b3f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0097a0 tx=0x7f61bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f888183dfbc28cfd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.386+0000 7f61caffd640 1 -- 172.21.15.16:0/463845921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc031830 con 0x7f61cc160920 2023-11-24T06:27:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.386+0000 7f61caffd640 1 -- 172.21.15.16:0/463845921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc031990 con 0x7f61cc160920 2023-11-24T06:27:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.386+0000 7f61caffd640 1 -- 172.21.15.16:0/463845921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc031d20 con 0x7f61cc160920 2023-11-24T06:27:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.387+0000 7f61d0cea640 1 -- 172.21.15.16:0/463845921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 msgr2=0x7f61cc0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.387+0000 7f61d0cea640 1 --2- 172.21.15.16:0/463845921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 0x7f61cc0b3f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0097a0 tx=0x7f61bc02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.387+0000 7f61d0cea640 1 -- 172.21.15.16:0/463845921 shutdown_connections 2023-11-24T06:27:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.387+0000 7f61d0cea640 1 --2- 172.21.15.16:0/463845921 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 0x7f61cc161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.387+0000 7f61d0cea640 1 --2- 172.21.15.16:0/463845921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 0x7f61cc0b3f30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.387+0000 7f61d0cea640 1 -- 172.21.15.16:0/463845921 >> 172.21.15.16:0/463845921 conn(0x7f61cc0d3090 msgr2=0x7f61cc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.387+0000 7f61d0cea640 1 -- 172.21.15.16:0/463845921 shutdown_connections 2023-11-24T06:27:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.387+0000 7f61d0cea640 1 -- 172.21.15.16:0/463845921 wait complete. 2023-11-24T06:27:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.389+0000 7f61d0cea640 1 Processor -- start 2023-11-24T06:27:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.389+0000 7f61d0cea640 1 -- start start 2023-11-24T06:27:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61d0cea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61cc0cf9f0 0x7f61cc151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61d0cea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 0x7f61cc153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61d0cea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 0x7f61cc0c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61d0cea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc153330 con 0x7f61cc160920 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61d0cea640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc152290 con 0x7f61cc0cf9f0 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61d0cea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc152570 con 0x7f61cc0cfea0 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 0x7f61cc0c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 0x7f61cc0c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33918/0 (socket says 172.21.15.16:33918) 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61cbfff640 1 -- 172.21.15.16:0/982112918 learned_addr learned my addr 172.21.15.16:0/982112918 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.390+0000 7f61c2ffd640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 0x7f61cc153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61cb7fe640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61cc0cf9f0 0x7f61cc151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61cbfff640 1 -- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 msgr2=0x7f61cc153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61cbfff640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 0x7f61cc153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61cbfff640 1 -- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61cc0cf9f0 msgr2=0x7f61cc151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61cbfff640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61cc0cf9f0 0x7f61cc151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61cbfff640 1 -- 172.21.15.16:0/982112918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc009400 con 0x7f61cc160920 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61c2ffd640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 0x7f61cc153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61cb7fe640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61cc0cf9f0 0x7f61cc151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61cbfff640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 0x7f61cc0c8990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f61c4007e40 tx=0x7f61c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.391+0000 7f61c8ff9640 1 -- 172.21.15.16:0/982112918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4014620 con 0x7f61cc160920 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.392+0000 7f61c8ff9640 1 -- 172.21.15.16:0/982112918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c400ee90 con 0x7f61cc160920 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.392+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61cc152850 con 0x7f61cc160920 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.392+0000 7f61c8ff9640 1 -- 172.21.15.16:0/982112918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40149e0 con 0x7f61cc160920 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.392+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61cc152e10 con 0x7f61cc160920 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.393+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61cc067450 con 0x7f61cc160920 2023-11-24T06:27:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.394+0000 7f61c8ff9640 1 -- 172.21.15.16:0/982112918 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61c4014b40 con 0x7f61cc160920 2023-11-24T06:27:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.395+0000 7f61c8ff9640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61ac06ae70 0x7f61ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.396+0000 7f61c8ff9640 1 -- 172.21.15.16:0/982112918 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f61c408a0b0 con 0x7f61cc160920 2023-11-24T06:27:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.396+0000 7f61cb7fe640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61ac06ae70 0x7f61ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.397+0000 7f61cb7fe640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61ac06ae70 0x7f61ac06d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61bc036000 tx=0x7f61bc03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.400+0000 7f61c8ff9640 1 -- 172.21.15.16:0/982112918 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61c4055040 con 0x7f61cc160920 2023-11-24T06:27:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.609+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f61cc08c7e0 con 0x7f61cc160920 2023-11-24T06:27:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.610+0000 7f61c8ff9640 1 -- 172.21.15.16:0/982112918 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f61c4058660 con 0x7f61cc160920 2023-11-24T06:27:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:27:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.612+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61ac06ae70 msgr2=0x7f61ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.612+0000 7f61d0cea640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61ac06ae70 0x7f61ac06d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61bc036000 tx=0x7f61bc03c040 comp rx=0 tx=0).stop 2023-11-24T06:27:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.612+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 msgr2=0x7f61cc0c8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.612+0000 7f61d0cea640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 0x7f61cc0c8990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f61c4007e40 tx=0x7f61c400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.613+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 shutdown_connections 2023-11-24T06:27:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.613+0000 7f61d0cea640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61ac06ae70 0x7f61ac06d330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.613+0000 7f61d0cea640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61cc160920 0x7f61cc0c8990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.613+0000 7f61d0cea640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61cc0cfea0 0x7f61cc153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.613+0000 7f61d0cea640 1 --2- 172.21.15.16:0/982112918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61cc0cf9f0 0x7f61cc151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.613+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 >> 172.21.15.16:0/982112918 conn(0x7f61cc0d3090 msgr2=0x7f61cc06aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.613+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 shutdown_connections 2023-11-24T06:27:52.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:52.613+0000 7f61d0cea640 1 -- 172.21.15.16:0/982112918 wait complete. 2023-11-24T06:27:52.634 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi173.front.sepia.ceph.com... 2023-11-24T06:27:52.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:52.634 DEBUG:teuthology.orchestra.run.smithi173:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:27:52.733 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:27:52.733 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:27:52.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:52.734 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:27:52.791 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:27:52.791 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:52.852 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:27:52.852 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:52.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:52.910 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:27:52.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.157+0000 7fdd35e85640 1 Processor -- start 2023-11-24T06:27:53.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.157+0000 7fdd35e85640 1 -- start start 2023-11-24T06:27:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.159+0000 7fdd35e85640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd30071410 0x7fdd30146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.159+0000 7fdd35e85640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdd3006c7c0 0x7fdd300717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.159+0000 7fdd35e85640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd30071d30 0x7fdd300b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.159+0000 7fdd35e85640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3006cc20 con 0x7fdd30071d30 2023-11-24T06:27:53.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.159+0000 7fdd35e85640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3006cec0 con 0x7fdd3006c7c0 2023-11-24T06:27:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.159+0000 7fdd35e85640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3006d1a0 con 0x7fdd30071410 2023-11-24T06:27:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.159+0000 7fdd35684640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd30071d30 0x7fdd300b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.159+0000 7fdd35684640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd30071d30 0x7fdd300b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55246/0 (socket says 172.21.15.16:55246) 2023-11-24T06:27:53.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.159+0000 7fdd35684640 1 -- 172.21.15.16:0/386776689 learned_addr learned my addr 172.21.15.16:0/386776689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.160+0000 7fdd27fff640 1 --2- 172.21.15.16:0/386776689 >> v2:172.21.15.173:3300/0 conn(0x7fdd3006c7c0 0x7fdd300717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.160+0000 7fdd35684640 1 -- 172.21.15.16:0/386776689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd30071410 msgr2=0x7fdd30146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.160+0000 7fdd35684640 1 --2- 172.21.15.16:0/386776689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd30071410 0x7fdd30146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.160+0000 7fdd35684640 1 -- 172.21.15.16:0/386776689 >> v2:172.21.15.173:3300/0 conn(0x7fdd3006c7c0 msgr2=0x7fdd300717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:53.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.160+0000 7fdd35684640 1 --2- 172.21.15.16:0/386776689 >> v2:172.21.15.173:3300/0 conn(0x7fdd3006c7c0 0x7fdd300717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.160+0000 7fdd35684640 1 -- 172.21.15.16:0/386776689 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3006d480 con 0x7fdd30071d30 2023-11-24T06:27:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.160+0000 7fdd27fff640 1 --2- 172.21.15.16:0/386776689 >> v2:172.21.15.173:3300/0 conn(0x7fdd3006c7c0 0x7fdd300717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.161+0000 7fdd35684640 1 --2- 172.21.15.16:0/386776689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071d30 0x7fdd300b1e00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd3006e440 tx=0x7fdd2802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a194b8bbd0b2cd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.161+0000 7fdd277fe640 1 -- 172.21.15.16:0/386776689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2802ca70 con 0x7fdd30071d30 2023-11-24T06:27:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.161+0000 7fdd277fe640 1 -- 172.21.15.16:0/386776689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2802cbd0 con 0x7fdd30071d30 2023-11-24T06:27:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.161+0000 7fdd277fe640 1 -- 172.21.15.16:0/386776689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd280343f0 con 0x7fdd30071d30 2023-11-24T06:27:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.161+0000 7fdd35e85640 1 -- 172.21.15.16:0/386776689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071d30 msgr2=0x7fdd300b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.161+0000 7fdd35e85640 1 --2- 172.21.15.16:0/386776689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071d30 0x7fdd300b1e00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd3006e440 tx=0x7fdd2802bed0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.162+0000 7fdd35e85640 1 -- 172.21.15.16:0/386776689 shutdown_connections 2023-11-24T06:27:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.162+0000 7fdd35e85640 1 --2- 172.21.15.16:0/386776689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071d30 0x7fdd300b1e00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.162+0000 7fdd35e85640 1 --2- 172.21.15.16:0/386776689 >> v2:172.21.15.173:3300/0 conn(0x7fdd3006c7c0 0x7fdd300717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.162+0000 7fdd35e85640 1 --2- 172.21.15.16:0/386776689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd30071410 0x7fdd30146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.162+0000 7fdd35e85640 1 -- 172.21.15.16:0/386776689 >> 172.21.15.16:0/386776689 conn(0x7fdd3006a800 msgr2=0x7fdd3006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.162+0000 7fdd35e85640 1 -- 172.21.15.16:0/386776689 shutdown_connections 2023-11-24T06:27:53.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.162+0000 7fdd35e85640 1 -- 172.21.15.16:0/386776689 wait complete. 2023-11-24T06:27:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.163+0000 7fdd35e85640 1 Processor -- start 2023-11-24T06:27:53.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.163+0000 7fdd35e85640 1 -- start start 2023-11-24T06:27:53.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.164+0000 7fdd35e85640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3006c7c0 0x7fdd30136cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.164+0000 7fdd35e85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071410 0x7fdd3015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.164+0000 7fdd35e85640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd30071d30 0x7fdd300b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.164+0000 7fdd35e85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3013b890 con 0x7fdd30071410 2023-11-24T06:27:53.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.164+0000 7fdd35e85640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd30137210 con 0x7fdd3006c7c0 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.164+0000 7fdd35e85640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3015dcf0 con 0x7fdd30071d30 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071410 0x7fdd3015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071410 0x7fdd3015d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55262/0 (socket says 172.21.15.16:55262) 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd27fff640 1 -- 172.21.15.16:0/854085643 learned_addr learned my addr 172.21.15.16:0/854085643 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd27fff640 1 -- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd30071d30 msgr2=0x7fdd300b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd27fff640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd30071d30 0x7fdd300b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd27fff640 1 -- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3006c7c0 msgr2=0x7fdd30136cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd34e83640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3006c7c0 0x7fdd30136cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd27fff640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3006c7c0 0x7fdd30136cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd27fff640 1 -- 172.21.15.16:0/854085643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd28009b30 con 0x7fdd30071410 2023-11-24T06:27:53.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.165+0000 7fdd27fff640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071410 0x7fdd3015d4a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c7f0 tx=0x7fdd2000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.166+0000 7fdd257fa640 1 -- 172.21.15.16:0/854085643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20007a30 con 0x7fdd30071410 2023-11-24T06:27:53.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.166+0000 7fdd257fa640 1 -- 172.21.15.16:0/854085643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd20007b90 con 0x7fdd30071410 2023-11-24T06:27:53.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.166+0000 7fdd257fa640 1 -- 172.21.15.16:0/854085643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000ce40 con 0x7fdd30071410 2023-11-24T06:27:53.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.166+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3015dfd0 con 0x7fdd30071410 2023-11-24T06:27:53.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.166+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3015e590 con 0x7fdd30071410 2023-11-24T06:27:53.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.167+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3006abe0 con 0x7fdd30071410 2023-11-24T06:27:53.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.168+0000 7fdd257fa640 1 -- 172.21.15.16:0/854085643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd200113f0 con 0x7fdd30071410 2023-11-24T06:27:53.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.168+0000 7fdd257fa640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd1c06ae70 0x7fdd1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.169+0000 7fdd257fa640 1 -- 172.21.15.16:0/854085643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fdd20089880 con 0x7fdd30071410 2023-11-24T06:27:53.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.169+0000 7fdd34e83640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd1c06ae70 0x7fdd1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.170+0000 7fdd34e83640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd1c06ae70 0x7fdd1c06d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00a810 tx=0x7fdd2c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.176+0000 7fdd257fa640 1 -- 172.21.15.16:0/854085643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd20054810 con 0x7fdd30071410 2023-11-24T06:27:53.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.372+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd30067450 con 0x7fdd30071410 2023-11-24T06:27:53.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.375+0000 7fdd257fa640 1 -- 172.21.15.16:0/854085643 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7fdd20057e30 con 0x7fdd30071410 2023-11-24T06:27:53.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:53.378 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:53.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd1c06ae70 msgr2=0x7fdd1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd1c06ae70 0x7fdd1c06d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00a810 tx=0x7fdd2c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:53.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071410 msgr2=0x7fdd3015d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071410 0x7fdd3015d4a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000c7f0 tx=0x7fdd2000cbf0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 shutdown_connections 2023-11-24T06:27:53.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd1c06ae70 0x7fdd1c06d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd30071d30 0x7fdd300b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd30071410 0x7fdd3015d4a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 --2- 172.21.15.16:0/854085643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd3006c7c0 0x7fdd30136cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 >> 172.21.15.16:0/854085643 conn(0x7fdd3006a800 msgr2=0x7fdd300b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:53.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 shutdown_connections 2023-11-24T06:27:53.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.378+0000 7fdd35e85640 1 -- 172.21.15.16:0/854085643 wait complete. 2023-11-24T06:27:53.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:53.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.624+0000 7fdf2411d640 1 Processor -- start 2023-11-24T06:27:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.624+0000 7fdf2411d640 1 -- start start 2023-11-24T06:27:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.626+0000 7fdf2411d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdf1c0ae5a0 0x7fdf1c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.626+0000 7fdf2411d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0cf290 0x7fdf1c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.626+0000 7fdf2411d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1c070c40 con 0x7fdf1c0b3ae0 2023-11-24T06:27:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.626+0000 7fdf2411d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c071450 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.626+0000 7fdf2411d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0cfc60 con 0x7fdf1c0cf290 2023-11-24T06:27:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.627+0000 7fdf21eb9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdf1c0ae5a0 0x7fdf1c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.627+0000 7fdf21eb9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdf1c0ae5a0 0x7fdf1c15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:41522/0 (socket says 172.21.15.16:41522) 2023-11-24T06:27:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.627+0000 7fdf226ba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0cf290 0x7fdf1c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.627+0000 7fdf21eb9640 1 -- 172.21.15.16:0/4016794012 learned_addr learned my addr 172.21.15.16:0/4016794012 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.628+0000 7fdf21eb9640 1 -- 172.21.15.16:0/4016794012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0cf290 msgr2=0x7fdf1c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.628+0000 7fdf21eb9640 1 --2- 172.21.15.16:0/4016794012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0cf290 0x7fdf1c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.628+0000 7fdf21eb9640 1 -- 172.21.15.16:0/4016794012 >> v1:172.21.15.173:6789/0 conn(0x7fdf1c0b3ae0 legacy=0x7fdf1c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.628+0000 7fdf21eb9640 1 -- 172.21.15.16:0/4016794012 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c06c470 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.629+0000 7fdf21eb9640 1 --2- 172.21.15.16:0/4016794012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0ae5a0 0x7fdf1c15e430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf1802ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a45ef31019eec6b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.630+0000 7fdf20eb7640 1 -- 172.21.15.16:0/4016794012 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1802f830 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.630+0000 7fdf20eb7640 1 -- 172.21.15.16:0/4016794012 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1802f990 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.630+0000 7fdf20eb7640 1 -- 172.21.15.16:0/4016794012 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1802fd20 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.630+0000 7fdf2411d640 1 -- 172.21.15.16:0/4016794012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0ae5a0 msgr2=0x7fdf1c15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.630+0000 7fdf2411d640 1 --2- 172.21.15.16:0/4016794012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0ae5a0 0x7fdf1c15e430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009870 tx=0x7fdf1802ba10 comp rx=0 tx=0).stop 2023-11-24T06:27:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.631+0000 7fdf2411d640 1 -- 172.21.15.16:0/4016794012 shutdown_connections 2023-11-24T06:27:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.631+0000 7fdf2411d640 1 --2- 172.21.15.16:0/4016794012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0cf290 0x7fdf1c0cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.631+0000 7fdf2411d640 1 --2- 172.21.15.16:0/4016794012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0ae5a0 0x7fdf1c15e430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.631+0000 7fdf2411d640 1 -- 172.21.15.16:0/4016794012 >> 172.21.15.16:0/4016794012 conn(0x7fdf1c06a800 msgr2=0x7fdf1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:53.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.631+0000 7fdf2411d640 1 -- 172.21.15.16:0/4016794012 shutdown_connections 2023-11-24T06:27:53.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.631+0000 7fdf2411d640 1 -- 172.21.15.16:0/4016794012 wait complete. 2023-11-24T06:27:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.632+0000 7fdf2411d640 1 Processor -- start 2023-11-24T06:27:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.633+0000 7fdf2411d640 1 -- start start 2023-11-24T06:27:53.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf2411d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1c0ae5a0 0x7fdf1c158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf2411d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0b3ae0 0x7fdf1c153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf2411d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0cf290 0x7fdf1c145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf2411d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c157a10 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf2411d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c157cf0 con 0x7fdf1c0cf290 2023-11-24T06:27:53.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf2411d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c157fd0 con 0x7fdf1c0b3ae0 2023-11-24T06:27:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf21eb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1c0ae5a0 0x7fdf1c158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf21eb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1c0ae5a0 0x7fdf1c158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55270/0 (socket says 172.21.15.16:55270) 2023-11-24T06:27:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf21eb9640 1 -- 172.21.15.16:0/1204902174 learned_addr learned my addr 172.21.15.16:0/1204902174 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf226ba640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0cf290 0x7fdf1c145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf21eb9640 1 -- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0b3ae0 msgr2=0x7fdf1c153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf216b8640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0b3ae0 0x7fdf1c153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf21eb9640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0b3ae0 0x7fdf1c153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf21eb9640 1 -- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0cf290 msgr2=0x7fdf1c145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf21eb9640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0cf290 0x7fdf1c145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.634+0000 7fdf21eb9640 1 -- 172.21.15.16:0/1204902174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180094d0 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.635+0000 7fdf226ba640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0cf290 0x7fdf1c145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.635+0000 7fdf216b8640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0b3ae0 0x7fdf1c153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.635+0000 7fdf21eb9640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1c0ae5a0 0x7fdf1c158ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdf18000c00 tx=0x7fdf180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.635+0000 7fdf127fc640 1 -- 172.21.15.16:0/1204902174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1803b040 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.635+0000 7fdf127fc640 1 -- 172.21.15.16:0/1204902174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18037be0 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.635+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c1582b0 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.635+0000 7fdf127fc640 1 -- 172.21.15.16:0/1204902174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18042480 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.635+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c150620 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.637+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c1587d0 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.637+0000 7fdf127fc640 1 -- 172.21.15.16:0/1204902174 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf180425e0 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.638+0000 7fdf127fc640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf0006ae40 0x7fdf0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.638+0000 7fdf127fc640 1 -- 172.21.15.16:0/1204902174 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fdf180ac840 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.639+0000 7fdf216b8640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf0006ae40 0x7fdf0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:53.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.640+0000 7fdf216b8640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf0006ae40 0x7fdf0006d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c160bb0 tx=0x7fdf0c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:53.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.642+0000 7fdf127fc640 1 -- 172.21.15.16:0/1204902174 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf1c1587d0 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.839+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf1c1587d0 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.840+0000 7fdf127fc640 1 -- 172.21.15.16:0/1204902174 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf1c1587d0 con 0x7fdf1c0ae5a0 2023-11-24T06:27:53.843 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf0006ae40 msgr2=0x7fdf0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf0006ae40 0x7fdf0006d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c160bb0 tx=0x7fdf0c009040 comp rx=0 tx=0).stop 2023-11-24T06:27:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1c0ae5a0 msgr2=0x7fdf1c158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1c0ae5a0 0x7fdf1c158ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdf18000c00 tx=0x7fdf180046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 shutdown_connections 2023-11-24T06:27:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf0006ae40 0x7fdf0006d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf1c0cf290 0x7fdf1c145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1c0b3ae0 0x7fdf1c153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 --2- 172.21.15.16:0/1204902174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf1c0ae5a0 0x7fdf1c158ab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 >> 172.21.15.16:0/1204902174 conn(0x7fdf1c06a800 msgr2=0x7fdf1c160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 shutdown_connections 2023-11-24T06:27:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:53.843+0000 7fdf2411d640 1 -- 172.21.15.16:0/1204902174 wait complete. 2023-11-24T06:27:53.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:54.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.096+0000 7f434f59e640 1 Processor -- start 2023-11-24T06:27:54.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.096+0000 7f434f59e640 1 -- start start 2023-11-24T06:27:54.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.098+0000 7f434f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43500ccb50 0x7f4350070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.098+0000 7f434f59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f435015bf10 0x7f4350143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.098+0000 7f434f59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435015f4e0 con 0x7f43500adf20 2023-11-24T06:27:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.098+0000 7f434f59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435015f970 con 0x7f43500ccb50 2023-11-24T06:27:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.098+0000 7f434f59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435015c3c0 con 0x7f435015bf10 2023-11-24T06:27:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.098+0000 7f434e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f43500adf20 0x7f43500cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39640/0 (socket says 172.21.15.16:39640) 2023-11-24T06:27:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.098+0000 7f434e59c640 1 -- 172.21.15.16:0/3073602429 learned_addr learned my addr 172.21.15.16:0/3073602429 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:54.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.098+0000 7f434dd9b640 1 --2- 172.21.15.16:0/3073602429 >> v2:172.21.15.16:3300/0 conn(0x7f43500ccb50 0x7f4350070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.099+0000 7f434ed9d640 1 --2- 172.21.15.16:0/3073602429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f435015bf10 0x7f4350143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.099+0000 7f434dd9b640 1 -- 172.21.15.16:0/3073602429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f435015bf10 msgr2=0x7f4350143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.099+0000 7f434dd9b640 1 --2- 172.21.15.16:0/3073602429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f435015bf10 0x7f4350143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.099+0000 7f434dd9b640 1 -- 172.21.15.16:0/3073602429 >> v1:172.21.15.16:6789/0 conn(0x7f43500adf20 legacy=0x7f43500cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.099+0000 7f434dd9b640 1 -- 172.21.15.16:0/3073602429 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350071130 con 0x7f43500ccb50 2023-11-24T06:27:54.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.099+0000 7f434d59a640 1 -- 172.21.15.16:0/3073602429 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453683756 0 0) 0x7f435015f4e0 con 0x7f43500adf20 2023-11-24T06:27:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.101+0000 7f434dd9b640 1 --2- 172.21.15.16:0/3073602429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 0x7f4350070a30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f43380097a0 tx=0x7f433802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0246628eacff941 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.101+0000 7f434d59a640 1 -- 172.21.15.16:0/3073602429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433802e070 con 0x7f43500ccb50 2023-11-24T06:27:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.101+0000 7f434d59a640 1 -- 172.21.15.16:0/3073602429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43380315a0 con 0x7f43500ccb50 2023-11-24T06:27:54.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.101+0000 7f434d59a640 1 -- 172.21.15.16:0/3073602429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338031930 con 0x7f43500ccb50 2023-11-24T06:27:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.102+0000 7f434f59e640 1 -- 172.21.15.16:0/3073602429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 msgr2=0x7f4350070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.102+0000 7f434f59e640 1 --2- 172.21.15.16:0/3073602429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 0x7f4350070a30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f43380097a0 tx=0x7f433802b840 comp rx=0 tx=0).stop 2023-11-24T06:27:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.102+0000 7f434f59e640 1 -- 172.21.15.16:0/3073602429 shutdown_connections 2023-11-24T06:27:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.102+0000 7f434f59e640 1 --2- 172.21.15.16:0/3073602429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f435015bf10 0x7f4350143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.102+0000 7f434f59e640 1 --2- 172.21.15.16:0/3073602429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 0x7f4350070a30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.102+0000 7f434f59e640 1 -- 172.21.15.16:0/3073602429 >> 172.21.15.16:0/3073602429 conn(0x7f435006a7d0 msgr2=0x7f435006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.102+0000 7f434f59e640 1 -- 172.21.15.16:0/3073602429 shutdown_connections 2023-11-24T06:27:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.102+0000 7f434f59e640 1 -- 172.21.15.16:0/3073602429 wait complete. 2023-11-24T06:27:54.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.104+0000 7f434f59e640 1 Processor -- start 2023-11-24T06:27:54.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.104+0000 7f434f59e640 1 -- start start 2023-11-24T06:27:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.105+0000 7f434f59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43500adf20 0x7f435006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.105+0000 7f434f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 0x7f43501416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.105+0000 7f434f59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f435015bf10 0x7f43500c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.105+0000 7f434f59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350141c30 con 0x7f43500ccb50 2023-11-24T06:27:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.105+0000 7f434f59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500c3e30 con 0x7f435015bf10 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.105+0000 7f434f59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43501f6370 con 0x7f43500adf20 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.105+0000 7f434dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 0x7f43501416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.105+0000 7f434dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 0x7f43501416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55288/0 (socket says 172.21.15.16:55288) 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.105+0000 7f434dd9b640 1 -- 172.21.15.16:0/852907012 learned_addr learned my addr 172.21.15.16:0/852907012 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.106+0000 7f434e59c640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43500adf20 0x7f435006ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.106+0000 7f434ed9d640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f435015bf10 0x7f43500c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.106+0000 7f434dd9b640 1 -- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43500adf20 msgr2=0x7f435006ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.106+0000 7f434dd9b640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43500adf20 0x7f435006ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.106+0000 7f434dd9b640 1 -- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f435015bf10 msgr2=0x7f43500c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.106+0000 7f434dd9b640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f435015bf10 0x7f43500c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.106+0000 7f434dd9b640 1 -- 172.21.15.16:0/852907012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4338009400 con 0x7f43500ccb50 2023-11-24T06:27:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.106+0000 7f434e59c640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43500adf20 0x7f435006ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.107+0000 7f434dd9b640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 0x7f43501416f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f43380096f0 tx=0x7f43380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.107+0000 7f4346ffd640 1 -- 172.21.15.16:0/852907012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338032840 con 0x7f43500ccb50 2023-11-24T06:27:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.107+0000 7f4346ffd640 1 -- 172.21.15.16:0/852907012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43380329c0 con 0x7f43500ccb50 2023-11-24T06:27:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.107+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43501f6650 con 0x7f43500ccb50 2023-11-24T06:27:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.107+0000 7f4346ffd640 1 -- 172.21.15.16:0/852907012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338032d70 con 0x7f43500ccb50 2023-11-24T06:27:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.107+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43501f6c10 con 0x7f43500ccb50 2023-11-24T06:27:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.108+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4350067450 con 0x7f43500ccb50 2023-11-24T06:27:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.109+0000 7f4346ffd640 1 -- 172.21.15.16:0/852907012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f433803b3f0 con 0x7f43500ccb50 2023-11-24T06:27:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.110+0000 7f4346ffd640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f433006ae40 0x7f433006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.110+0000 7f4346ffd640 1 -- 172.21.15.16:0/852907012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f43380ab5c0 con 0x7f43500ccb50 2023-11-24T06:27:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.110+0000 7f434e59c640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f433006ae40 0x7f433006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.112+0000 7f434e59c640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f433006ae40 0x7f433006d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4340007fd0 tx=0x7f4340005b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.114+0000 7f4346ffd640 1 -- 172.21.15.16:0/852907012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4338076550 con 0x7f43500ccb50 2023-11-24T06:27:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.310+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f435006ace0 con 0x7f43500ccb50 2023-11-24T06:27:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.312+0000 7f4346ffd640 1 -- 172.21.15.16:0/852907012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4338079b70 con 0x7f43500ccb50 2023-11-24T06:27:54.314 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f433006ae40 msgr2=0x7f433006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f433006ae40 0x7f433006d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4340007fd0 tx=0x7f4340005b70 comp rx=0 tx=0).stop 2023-11-24T06:27:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 msgr2=0x7f43501416f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 0x7f43501416f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f43380096f0 tx=0x7f43380046c0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 shutdown_connections 2023-11-24T06:27:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f433006ae40 0x7f433006d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f435015bf10 0x7f43500c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43500ccb50 0x7f43501416f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 --2- 172.21.15.16:0/852907012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43500adf20 0x7f435006ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 >> 172.21.15.16:0/852907012 conn(0x7f435006a7d0 msgr2=0x7f4350143850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.314+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 shutdown_connections 2023-11-24T06:27:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.315+0000 7f434f59e640 1 -- 172.21.15.16:0/852907012 wait complete. 2023-11-24T06:27:54.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:54.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.558+0000 7fcd3759e640 1 Processor -- start 2023-11-24T06:27:54.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.559+0000 7fcd3759e640 1 -- start start 2023-11-24T06:27:54.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.561+0000 7fcd3759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd380ccb50 0x7fcd38070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.561+0000 7fcd3759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3815f4e0 con 0x7fcd380adf20 2023-11-24T06:27:54.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.561+0000 7fcd3759e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3815c3c0 con 0x7fcd3815bf10 2023-11-24T06:27:54.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.561+0000 7fcd3759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3815f970 con 0x7fcd380ccb50 2023-11-24T06:27:54.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.561+0000 7fcd3659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd380adf20 0x7fcd380cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39644/0 (socket says 172.21.15.16:39644) 2023-11-24T06:27:54.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.561+0000 7fcd3659c640 1 -- 172.21.15.16:0/3856229447 learned_addr learned my addr 172.21.15.16:0/3856229447 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.561+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717984908 0 0) 0x7fcd3815f4e0 con 0x7fcd380adf20 2023-11-24T06:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.561+0000 7fcd2ffff640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd380ccb50 0x7fcd38070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.561+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd24003520 con 0x7fcd380adf20 2023-11-24T06:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3124280332 0 0) 0x7fcd24003520 con 0x7fcd380adf20 2023-11-24T06:27:54.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd380ccb50 msgr2=0x7fcd38070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd380ccb50 0x7fcd38070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 >> v1:172.21.15.173:6789/0 conn(0x7fcd3815bf10 legacy=0x7fcd38143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd38071130 con 0x7fcd380adf20 2023-11-24T06:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fcd28001670 con 0x7fcd380adf20 2023-11-24T06:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 >> v1:172.21.15.16:6789/0 conn(0x7fcd380adf20 legacy=0x7fcd380cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240039a0 0x7fcd24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd380ccb50 0x7fcd240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd24004d00 0x7fcd24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3815f970 con 0x7fcd240039a0 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2400bb60 con 0x7fcd24004d00 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2400bd20 con 0x7fcd380ccb50 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.562+0000 7fcd3659c640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240039a0 0x7fcd24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.563+0000 7fcd2ffff640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd380ccb50 0x7fcd240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.563+0000 7fcd36d9d640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd24004d00 0x7fcd24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.563+0000 7fcd3659c640 1 -- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd380ccb50 msgr2=0x7fcd240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.563+0000 7fcd3659c640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd380ccb50 0x7fcd240047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.563+0000 7fcd3659c640 1 -- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd24004d00 msgr2=0x7fcd24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:54.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.563+0000 7fcd3659c640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd24004d00 0x7fcd24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.563+0000 7fcd3659c640 1 -- 172.21.15.16:0/3856229447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24003520 con 0x7fcd240039a0 2023-11-24T06:27:54.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.563+0000 7fcd36d9d640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd24004d00 0x7fcd24005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.564+0000 7fcd3659c640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240039a0 0x7fcd24003dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800c210 tx=0x7fcd240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b4e434ab13dff3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.564+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28005ad0 con 0x7fcd240039a0 2023-11-24T06:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.564+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28005c30 con 0x7fcd240039a0 2023-11-24T06:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.564+0000 7fcd35d9b640 1 -- 172.21.15.16:0/3856229447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2802c890 con 0x7fcd240039a0 2023-11-24T06:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.565+0000 7fcd3759e640 1 -- 172.21.15.16:0/3856229447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240039a0 msgr2=0x7fcd24003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.565+0000 7fcd3759e640 1 --2- 172.21.15.16:0/3856229447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd240039a0 0x7fcd24003dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800c210 tx=0x7fcd240057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.565+0000 7fcd36d9d640 1 -- 172.21.15.16:0/3856229447 reap_dead start 2023-11-24T06:27:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.565+0000 7fcd3759e640 1 -- 172.21.15.16:0/3856229447 shutdown_connections 2023-11-24T06:27:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.565+0000 7fcd3759e640 1 -- 172.21.15.16:0/3856229447 >> 172.21.15.16:0/3856229447 conn(0x7fcd3806a7d0 msgr2=0x7fcd3806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.565+0000 7fcd3759e640 1 -- 172.21.15.16:0/3856229447 shutdown_connections 2023-11-24T06:27:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.565+0000 7fcd3759e640 1 -- 172.21.15.16:0/3856229447 wait complete. 2023-11-24T06:27:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.567+0000 7fcd3759e640 1 Processor -- start 2023-11-24T06:27:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.567+0000 7fcd3759e640 1 -- start start 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.568+0000 7fcd3759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd240039a0 0x7fcd3815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.568+0000 7fcd3759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd380abf00 0x7fcd380ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.568+0000 7fcd3759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd38131610 0x7fcd38131a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.568+0000 7fcd3759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd381325e0 con 0x7fcd380abf00 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.568+0000 7fcd3759e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380ac8a0 con 0x7fcd38131610 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.568+0000 7fcd3759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd38132300 con 0x7fcd240039a0 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.568+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd380abf00 0x7fcd380ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.568+0000 7fcd2ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd380abf00 0x7fcd380ac360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55310/0 (socket says 172.21.15.16:55310) 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.568+0000 7fcd2ffff640 1 -- 172.21.15.16:0/3728006967 learned_addr learned my addr 172.21.15.16:0/3728006967 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd3659c640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd240039a0 0x7fcd3815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd36d9d640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd38131610 0x7fcd38131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd2ffff640 1 -- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd240039a0 msgr2=0x7fcd3815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd2ffff640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd240039a0 0x7fcd3815a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd2ffff640 1 -- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd38131610 msgr2=0x7fcd38131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd2ffff640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd38131610 0x7fcd38131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd2ffff640 1 -- 172.21.15.16:0/3728006967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3812f700 con 0x7fcd380abf00 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd2ffff640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd380abf00 0x7fcd380ac360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000e5d0 tx=0x7fcd2000ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd2effd640 1 -- 172.21.15.16:0/3728006967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20013610 con 0x7fcd380abf00 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd2effd640 1 -- 172.21.15.16:0/3728006967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd20013770 con 0x7fcd380abf00 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd2effd640 1 -- 172.21.15.16:0/3728006967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20013b00 con 0x7fcd380abf00 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.569+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3812f9e0 con 0x7fcd380abf00 2023-11-24T06:27:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.570+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd38131fd0 con 0x7fcd380abf00 2023-11-24T06:27:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.571+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd38067450 con 0x7fcd380abf00 2023-11-24T06:27:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.572+0000 7fcd2effd640 1 -- 172.21.15.16:0/3728006967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd20013c60 con 0x7fcd380abf00 2023-11-24T06:27:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.573+0000 7fcd2effd640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcd1806ae70 0x7fcd1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.573+0000 7fcd2effd640 1 -- 172.21.15.16:0/3728006967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fcd200893a0 con 0x7fcd380abf00 2023-11-24T06:27:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.573+0000 7fcd3659c640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcd1806ae70 0x7fcd1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:54.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.575+0000 7fcd3659c640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcd1806ae70 0x7fcd1806d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcd28034000 tx=0x7fcd28007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:54.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.577+0000 7fcd2effd640 1 -- 172.21.15.16:0/3728006967 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd200542a0 con 0x7fcd380abf00 2023-11-24T06:27:54.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.769+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd38070c00 con 0x7fcd380abf00 2023-11-24T06:27:54.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.770+0000 7fcd2effd640 1 -- 172.21.15.16:0/3728006967 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd200578c0 con 0x7fcd380abf00 2023-11-24T06:27:54.772 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:54.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcd1806ae70 msgr2=0x7fcd1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcd1806ae70 0x7fcd1806d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcd28034000 tx=0x7fcd28007000 comp rx=0 tx=0).stop 2023-11-24T06:27:54.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd380abf00 msgr2=0x7fcd380ac360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:54.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd380abf00 0x7fcd380ac360 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000e5d0 tx=0x7fcd2000ea90 comp rx=0 tx=0).stop 2023-11-24T06:27:54.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 shutdown_connections 2023-11-24T06:27:54.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcd1806ae70 0x7fcd1806d330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd38131610 0x7fcd38131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd380abf00 0x7fcd380ac360 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 --2- 172.21.15.16:0/3728006967 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd240039a0 0x7fcd3815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:54.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 >> 172.21.15.16:0/3728006967 conn(0x7fcd3806a7d0 msgr2=0x7fcd3815dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:54.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 shutdown_connections 2023-11-24T06:27:54.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:54.773+0000 7fcd3759e640 1 -- 172.21.15.16:0/3728006967 wait complete. 2023-11-24T06:27:54.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:27:55.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.028+0000 7fde06a15640 1 Processor -- start 2023-11-24T06:27:55.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.028+0000 7fde06a15640 1 -- start start 2023-11-24T06:27:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.030+0000 7fde06a15640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013e360 0x7fde000cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.030+0000 7fde06a15640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fde0015f9c0 0x7fde000cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.030+0000 7fde06a15640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0015c5f0 con 0x7fde0013df80 2023-11-24T06:27:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.030+0000 7fde06a15640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0006c470 con 0x7fde0015f9c0 2023-11-24T06:27:55.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.030+0000 7fde06a15640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0006c750 con 0x7fde0013e360 2023-11-24T06:27:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.031+0000 7fde04fb2640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fde0015f9c0 0x7fde000cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.031+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013e360 0x7fde000cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.031+0000 7fde04fb2640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fde0015f9c0 0x7fde000cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:41562/0 (socket says 172.21.15.16:41562) 2023-11-24T06:27:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.031+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013e360 0x7fde000cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:39552/0 (socket says 172.21.15.16:39552) 2023-11-24T06:27:55.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.031+0000 7fde04fb2640 1 -- 172.21.15.16:0/1960793412 learned_addr learned my addr 172.21.15.16:0/1960793412 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.031+0000 7fddfffff640 1 --1- 172.21.15.16:0/1960793412 >> v1:172.21.15.173:6789/0 conn(0x7fde0013df80 0x7fde0015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:50454/0 (socket says 172.21.15.16:50454) 2023-11-24T06:27:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.032+0000 7fde04fb2640 1 -- 172.21.15.16:0/1960793412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013e360 msgr2=0x7fde000cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.032+0000 7fde04fb2640 1 --2- 172.21.15.16:0/1960793412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013e360 0x7fde000cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.032+0000 7fde04fb2640 1 -- 172.21.15.16:0/1960793412 >> v1:172.21.15.173:6789/0 conn(0x7fde0013df80 legacy=0x7fde0015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.032+0000 7fde04fb2640 1 -- 172.21.15.16:0/1960793412 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0006ca30 con 0x7fde0015f9c0 2023-11-24T06:27:55.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.033+0000 7fde04fb2640 1 --2- 172.21.15.16:0/1960793412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 0x7fde000cf7d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009e50 tx=0x7fddf802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c03fdaa23f596e84 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.034+0000 7fddff7fe640 1 -- 172.21.15.16:0/1960793412 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf802cb00 con 0x7fde0015f9c0 2023-11-24T06:27:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.034+0000 7fddff7fe640 1 -- 172.21.15.16:0/1960793412 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf802cc60 con 0x7fde0015f9c0 2023-11-24T06:27:55.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.034+0000 7fddff7fe640 1 -- 172.21.15.16:0/1960793412 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf80343f0 con 0x7fde0015f9c0 2023-11-24T06:27:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.034+0000 7fde06a15640 1 -- 172.21.15.16:0/1960793412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 msgr2=0x7fde000cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.034+0000 7fde06a15640 1 --2- 172.21.15.16:0/1960793412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 0x7fde000cf7d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009e50 tx=0x7fddf802bf60 comp rx=0 tx=0).stop 2023-11-24T06:27:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.035+0000 7fde06a15640 1 -- 172.21.15.16:0/1960793412 shutdown_connections 2023-11-24T06:27:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.035+0000 7fde06a15640 1 --2- 172.21.15.16:0/1960793412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 0x7fde000cf7d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.035+0000 7fde06a15640 1 --2- 172.21.15.16:0/1960793412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013e360 0x7fde000cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.035+0000 7fde06a15640 1 -- 172.21.15.16:0/1960793412 >> 172.21.15.16:0/1960793412 conn(0x7fde0006a800 msgr2=0x7fde0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.035+0000 7fde06a15640 1 -- 172.21.15.16:0/1960793412 shutdown_connections 2023-11-24T06:27:55.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.035+0000 7fde06a15640 1 -- 172.21.15.16:0/1960793412 wait complete. 2023-11-24T06:27:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.037+0000 7fde06a15640 1 Processor -- start 2023-11-24T06:27:55.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.037+0000 7fde06a15640 1 -- start start 2023-11-24T06:27:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.038+0000 7fde06a15640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013df80 0x7fde00150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.038+0000 7fde06a15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0013e360 0x7fde0013d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.038+0000 7fde06a15640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 0x7fde001f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.038+0000 7fde06a15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde001f68f0 con 0x7fde0013e360 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.038+0000 7fde06a15640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde001f6580 con 0x7fde0015f9c0 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.038+0000 7fde06a15640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde001f3790 con 0x7fde0013df80 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.038+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0013e360 0x7fde0013d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.038+0000 7fddf77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0013e360 0x7fde0013d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55324/0 (socket says 172.21.15.16:55324) 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.038+0000 7fddf77fe640 1 -- 172.21.15.16:0/689230340 learned_addr learned my addr 172.21.15.16:0/689230340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fddfffff640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013df80 0x7fde00150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fde04fb2640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 0x7fde001f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fddf77fe640 1 -- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013df80 msgr2=0x7fde00150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fddf77fe640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013df80 0x7fde00150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fddf77fe640 1 -- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 msgr2=0x7fde001f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fddf77fe640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 0x7fde001f2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fddf77fe640 1 -- 172.21.15.16:0/689230340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf8009b30 con 0x7fde0013e360 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fddfffff640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013df80 0x7fde00150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fde04fb2640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 0x7fde001f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.039+0000 7fddf77fe640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0013e360 0x7fde0013d7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdde800e9e0 tx=0x7fdde800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.040+0000 7fddfd7fa640 1 -- 172.21.15.16:0/689230340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8009670 con 0x7fde0013e360 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.040+0000 7fddfd7fa640 1 -- 172.21.15.16:0/689230340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde800c9d0 con 0x7fde0013e360 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.040+0000 7fddfd7fa640 1 -- 172.21.15.16:0/689230340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde800cd80 con 0x7fde0013e360 2023-11-24T06:27:55.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.040+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde001f3a70 con 0x7fde0013e360 2023-11-24T06:27:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.040+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde001f3d50 con 0x7fde0013e360 2023-11-24T06:27:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.041+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde001f6a30 con 0x7fde0013e360 2023-11-24T06:27:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.042+0000 7fddfd7fa640 1 -- 172.21.15.16:0/689230340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdde8013470 con 0x7fde0013e360 2023-11-24T06:27:55.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.043+0000 7fddfd7fa640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdde006ae40 0x7fdde006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.043+0000 7fddfd7fa640 1 -- 172.21.15.16:0/689230340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fdde8089880 con 0x7fde0013e360 2023-11-24T06:27:55.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.043+0000 7fddfffff640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdde006ae40 0x7fdde006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.045+0000 7fddfffff640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdde006ae40 0x7fdde006d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fddf0007fd0 tx=0x7fddf0007d40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.047+0000 7fddfd7fa640 1 -- 172.21.15.16:0/689230340 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde001f6a30 con 0x7fde0013e360 2023-11-24T06:27:55.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.259+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fde001f6a30 con 0x7fde0013e360 2023-11-24T06:27:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.260+0000 7fddfd7fa640 1 -- 172.21.15.16:0/689230340 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fde001f6a30 con 0x7fde0013e360 2023-11-24T06:27:55.262 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdde006ae40 msgr2=0x7fdde006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdde006ae40 0x7fdde006d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fddf0007fd0 tx=0x7fddf0007d40 comp rx=0 tx=0).stop 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0013e360 msgr2=0x7fde0013d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0013e360 0x7fde0013d7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdde800e9e0 tx=0x7fdde800eea0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 shutdown_connections 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdde006ae40 0x7fdde006d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde0015f9c0 0x7fde001f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0013e360 0x7fde0013d7d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 --2- 172.21.15.16:0/689230340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde0013df80 0x7fde00150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 >> 172.21.15.16:0/689230340 conn(0x7fde0006a800 msgr2=0x7fde0015caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 shutdown_connections 2023-11-24T06:27:55.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.263+0000 7fde06a15640 1 -- 172.21.15.16:0/689230340 wait complete. 2023-11-24T06:27:55.284 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:27:55.284 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:55.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.566+0000 7fb46ffff640 1 Processor -- start 2023-11-24T06:27:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.566+0000 7fb46ffff640 1 -- start start 2023-11-24T06:27:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.569+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb470162430 0x7fb4700b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.569+0000 7fb46ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb470162810 0x7fb470071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.569+0000 7fb46ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb4700b3650 0x7fb470071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.569+0000 7fb46ffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb470071cc0 con 0x7fb470162810 2023-11-24T06:27:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.569+0000 7fb46ffff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4700702e0 con 0x7fb4700b3650 2023-11-24T06:27:55.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.569+0000 7fb46ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4700705c0 con 0x7fb470162430 2023-11-24T06:27:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.569+0000 7fb467fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb470162810 0x7fb470071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.570+0000 7fb467fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb470162810 0x7fb470071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55336/0 (socket says 172.21.15.16:55336) 2023-11-24T06:27:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.570+0000 7fb46f7fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb4700b3650 0x7fb470071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.570+0000 7fb467fff640 1 -- 172.21.15.16:0/4151535059 learned_addr learned my addr 172.21.15.16:0/4151535059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.570+0000 7fb46effd640 1 --2- 172.21.15.16:0/4151535059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb470162430 0x7fb4700b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.570+0000 7fb467fff640 1 -- 172.21.15.16:0/4151535059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb470162430 msgr2=0x7fb4700b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.570+0000 7fb467fff640 1 --2- 172.21.15.16:0/4151535059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb470162430 0x7fb4700b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.570+0000 7fb467fff640 1 -- 172.21.15.16:0/4151535059 >> v2:172.21.15.173:3300/0 conn(0x7fb4700b3650 msgr2=0x7fb470071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.570+0000 7fb467fff640 1 --2- 172.21.15.16:0/4151535059 >> v2:172.21.15.173:3300/0 conn(0x7fb4700b3650 0x7fb470071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.570+0000 7fb467fff640 1 -- 172.21.15.16:0/4151535059 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4700708a0 con 0x7fb470162810 2023-11-24T06:27:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.571+0000 7fb467fff640 1 --2- 172.21.15.16:0/4151535059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470162810 0x7fb470071240 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb4700b2bc0 tx=0x7fb45802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48431026b4a7af59 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46e7fc640 1 -- 172.21.15.16:0/4151535059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458031860 con 0x7fb470162810 2023-11-24T06:27:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46e7fc640 1 -- 172.21.15.16:0/4151535059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580319c0 con 0x7fb470162810 2023-11-24T06:27:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46e7fc640 1 -- 172.21.15.16:0/4151535059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458031d50 con 0x7fb470162810 2023-11-24T06:27:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46ffff640 1 -- 172.21.15.16:0/4151535059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470162810 msgr2=0x7fb470071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46ffff640 1 --2- 172.21.15.16:0/4151535059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470162810 0x7fb470071240 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb4700b2bc0 tx=0x7fb45802b900 comp rx=0 tx=0).stop 2023-11-24T06:27:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46ffff640 1 -- 172.21.15.16:0/4151535059 shutdown_connections 2023-11-24T06:27:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46ffff640 1 --2- 172.21.15.16:0/4151535059 >> v2:172.21.15.173:3300/0 conn(0x7fb4700b3650 0x7fb470071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46ffff640 1 --2- 172.21.15.16:0/4151535059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470162810 0x7fb470071240 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46ffff640 1 --2- 172.21.15.16:0/4151535059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb470162430 0x7fb4700b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46ffff640 1 -- 172.21.15.16:0/4151535059 >> 172.21.15.16:0/4151535059 conn(0x7fb47006a800 msgr2=0x7fb47006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46ffff640 1 -- 172.21.15.16:0/4151535059 shutdown_connections 2023-11-24T06:27:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.572+0000 7fb46ffff640 1 -- 172.21.15.16:0/4151535059 wait complete. 2023-11-24T06:27:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.574+0000 7fb46ffff640 1 Processor -- start 2023-11-24T06:27:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.574+0000 7fb46ffff640 1 -- start start 2023-11-24T06:27:55.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.575+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4700b3650 0x7fb470143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.575+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb470162430 0x7fb470162c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.575+0000 7fb46ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470158a10 0x7fb470163140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.575+0000 7fb46ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47015bb90 con 0x7fb470158a10 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.575+0000 7fb46ffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4700af170 con 0x7fb470162430 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.575+0000 7fb46ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47015b5d0 con 0x7fb4700b3650 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb46f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470158a10 0x7fb470163140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb46f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470158a10 0x7fb470163140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55344/0 (socket says 172.21.15.16:55344) 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb46f7fe640 1 -- 172.21.15.16:0/1887468819 learned_addr learned my addr 172.21.15.16:0/1887468819 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb46f7fe640 1 -- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4700b3650 msgr2=0x7fb470143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb46f7fe640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4700b3650 0x7fb470143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb467fff640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb470162430 0x7fb470162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb46f7fe640 1 -- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb470162430 msgr2=0x7fb470162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb46f7fe640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb470162430 0x7fb470162c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb46f7fe640 1 -- 172.21.15.16:0/1887468819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580094d0 con 0x7fb470158a10 2023-11-24T06:27:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.576+0000 7fb46f7fe640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470158a10 0x7fb470163140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb468009dd0 tx=0x7fb46800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.577+0000 7fb4677fe640 1 -- 172.21.15.16:0/1887468819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468014480 con 0x7fb470158a10 2023-11-24T06:27:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.577+0000 7fb4677fe640 1 -- 172.21.15.16:0/1887468819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46800f040 con 0x7fb470158a10 2023-11-24T06:27:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.577+0000 7fb4677fe640 1 -- 172.21.15.16:0/1887468819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468014840 con 0x7fb470158a10 2023-11-24T06:27:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.577+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47015b8b0 con 0x7fb470158a10 2023-11-24T06:27:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.577+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4701441b0 con 0x7fb470158a10 2023-11-24T06:27:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.578+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb470067450 con 0x7fb470158a10 2023-11-24T06:27:55.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.579+0000 7fb4677fe640 1 -- 172.21.15.16:0/1887468819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb4680149a0 con 0x7fb470158a10 2023-11-24T06:27:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.580+0000 7fb4677fe640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb45006ae70 0x7fb45006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:55.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.580+0000 7fb4677fe640 1 -- 172.21.15.16:0/1887468819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fb46808a310 con 0x7fb470158a10 2023-11-24T06:27:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.581+0000 7fb46effd640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb45006ae70 0x7fb45006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.582+0000 7fb46effd640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb45006ae70 0x7fb45006d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb460009870 tx=0x7fb460008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.585+0000 7fb4677fe640 1 -- 172.21.15.16:0/1887468819 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4680552a0 con 0x7fb470158a10 2023-11-24T06:27:55.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.780+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4700b2620 con 0x7fb470158a10 2023-11-24T06:27:55.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.782+0000 7fb4677fe640 1 -- 172.21.15.16:0/1887468819 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7fb4680588c0 con 0x7fb470158a10 2023-11-24T06:27:55.786 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:55.787 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.784+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb45006ae70 msgr2=0x7fb45006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.784+0000 7fb46ffff640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb45006ae70 0x7fb45006d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb460009870 tx=0x7fb460008040 comp rx=0 tx=0).stop 2023-11-24T06:27:55.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.784+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470158a10 msgr2=0x7fb470163140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:55.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.784+0000 7fb46ffff640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470158a10 0x7fb470163140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb468009dd0 tx=0x7fb46800c490 comp rx=0 tx=0).stop 2023-11-24T06:27:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.784+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 shutdown_connections 2023-11-24T06:27:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.784+0000 7fb46ffff640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb45006ae70 0x7fb45006d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.784+0000 7fb46ffff640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb470158a10 0x7fb470163140 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.784+0000 7fb46ffff640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb470162430 0x7fb470162c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.784+0000 7fb46ffff640 1 --2- 172.21.15.16:0/1887468819 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4700b3650 0x7fb470143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.785+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 >> 172.21.15.16:0/1887468819 conn(0x7fb47006a800 msgr2=0x7fb47006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.785+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 shutdown_connections 2023-11-24T06:27:55.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:55.785+0000 7fb46ffff640 1 -- 172.21.15.16:0/1887468819 wait complete. 2023-11-24T06:27:55.808 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:55.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.039+0000 7f2062472640 1 Processor -- start 2023-11-24T06:27:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.039+0000 7f2062472640 1 -- start start 2023-11-24T06:27:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.041+0000 7f2062472640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 0x7f205c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.041+0000 7f2062472640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f205c15f9c0 0x7f205c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.041+0000 7f2062472640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c15c5f0 con 0x7f205c13df80 2023-11-24T06:27:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.041+0000 7f2062472640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c06c470 con 0x7f205c15f9c0 2023-11-24T06:27:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.041+0000 7f2062472640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c06c750 con 0x7f205c13e360 2023-11-24T06:27:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.041+0000 7f2060a0f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f205c15f9c0 0x7f205c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.041+0000 7f205bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f205c13df80 0x7f205c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39648/0 (socket says 172.21.15.16:39648) 2023-11-24T06:27:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.041+0000 7f205bfff640 1 -- 172.21.15.16:0/2027605316 learned_addr learned my addr 172.21.15.16:0/2027605316 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:56.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.042+0000 7f205b7fe640 1 --2- 172.21.15.16:0/2027605316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 0x7f205c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.042+0000 7f2060a0f640 1 -- 172.21.15.16:0/2027605316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 msgr2=0x7f205c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.042+0000 7f2060a0f640 1 --2- 172.21.15.16:0/2027605316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 0x7f205c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.042+0000 7f205affd640 1 -- 172.21.15.16:0/2027605316 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757122162 0 0) 0x7f205c15c5f0 con 0x7f205c13df80 2023-11-24T06:27:56.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.042+0000 7f2060a0f640 1 -- 172.21.15.16:0/2027605316 >> v1:172.21.15.16:6789/0 conn(0x7f205c13df80 legacy=0x7f205c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.042+0000 7f2060a0f640 1 -- 172.21.15.16:0/2027605316 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c06ca30 con 0x7f205c15f9c0 2023-11-24T06:27:56.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.043+0000 7f2060a0f640 1 --2- 172.21.15.16:0/2027605316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 0x7f205c0cf7d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2054009d90 tx=0x7f205402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2387eb8e52d468ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.043+0000 7f205affd640 1 -- 172.21.15.16:0/2027605316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205402cc50 con 0x7f205c15f9c0 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.043+0000 7f205affd640 1 -- 172.21.15.16:0/2027605316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205402cdb0 con 0x7f205c15f9c0 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.043+0000 7f205affd640 1 -- 172.21.15.16:0/2027605316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054034660 con 0x7f205c15f9c0 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.044+0000 7f2062472640 1 -- 172.21.15.16:0/2027605316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 msgr2=0x7f205c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.044+0000 7f2062472640 1 --2- 172.21.15.16:0/2027605316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 0x7f205c0cf7d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2054009d90 tx=0x7f205402bea0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.044+0000 7f2062472640 1 -- 172.21.15.16:0/2027605316 shutdown_connections 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.044+0000 7f2062472640 1 --2- 172.21.15.16:0/2027605316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 0x7f205c0cf7d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.044+0000 7f2062472640 1 --2- 172.21.15.16:0/2027605316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 0x7f205c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.044+0000 7f2062472640 1 -- 172.21.15.16:0/2027605316 >> 172.21.15.16:0/2027605316 conn(0x7f205c06a800 msgr2=0x7f205c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.044+0000 7f2062472640 1 -- 172.21.15.16:0/2027605316 shutdown_connections 2023-11-24T06:27:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.044+0000 7f2062472640 1 -- 172.21.15.16:0/2027605316 wait complete. 2023-11-24T06:27:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.046+0000 7f2062472640 1 Processor -- start 2023-11-24T06:27:56.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.046+0000 7f2062472640 1 -- start start 2023-11-24T06:27:56.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.047+0000 7f2062472640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c13df80 0x7f205c1521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.047+0000 7f2062472640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 0x7f205c153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.047+0000 7f2062472640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 0x7f205c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.047+0000 7f2062472640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c132e50 con 0x7f205c15f9c0 2023-11-24T06:27:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.047+0000 7f2062472640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c154610 con 0x7f205c13df80 2023-11-24T06:27:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.047+0000 7f2062472640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c1324b0 con 0x7f205c13e360 2023-11-24T06:27:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f2060a0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 0x7f205c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f2060a0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 0x7f205c156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55364/0 (socket says 172.21.15.16:55364) 2023-11-24T06:27:56.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f2060a0f640 1 -- 172.21.15.16:0/2748981506 learned_addr learned my addr 172.21.15.16:0/2748981506 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:56.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f205bfff640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c13df80 0x7f205c1521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f205b7fe640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 0x7f205c153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f2060a0f640 1 -- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 msgr2=0x7f205c153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f2060a0f640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 0x7f205c153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f2060a0f640 1 -- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c13df80 msgr2=0x7f205c1521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f2060a0f640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c13df80 0x7f205c1521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f2060a0f640 1 -- 172.21.15.16:0/2748981506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2054009a70 con 0x7f205c15f9c0 2023-11-24T06:27:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.048+0000 7f2060a0f640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 0x7f205c156bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2054009d90 tx=0x7f2054004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.049+0000 7f2058ff9640 1 -- 172.21.15.16:0/2748981506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054040650 con 0x7f205c15f9c0 2023-11-24T06:27:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.049+0000 7f2058ff9640 1 -- 172.21.15.16:0/2748981506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20540407d0 con 0x7f205c15f9c0 2023-11-24T06:27:56.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.049+0000 7f2058ff9640 1 -- 172.21.15.16:0/2748981506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054040b80 con 0x7f205c15f9c0 2023-11-24T06:27:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.049+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c132790 con 0x7f205c15f9c0 2023-11-24T06:27:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.049+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c152710 con 0x7f205c15f9c0 2023-11-24T06:27:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.050+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c132f90 con 0x7f205c15f9c0 2023-11-24T06:27:56.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.051+0000 7f2058ff9640 1 -- 172.21.15.16:0/2748981506 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2054040ce0 con 0x7f205c15f9c0 2023-11-24T06:27:56.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.052+0000 7f2058ff9640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204806ae40 0x7f204806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.052+0000 7f2058ff9640 1 -- 172.21.15.16:0/2748981506 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f20540ac420 con 0x7f205c15f9c0 2023-11-24T06:27:56.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.053+0000 7f205bfff640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204806ae40 0x7f204806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.054+0000 7f205bfff640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204806ae40 0x7f204806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f204c000f60 tx=0x7f204c007c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.058+0000 7f2058ff9640 1 -- 172.21.15.16:0/2748981506 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f205c132f90 con 0x7f205c15f9c0 2023-11-24T06:27:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.254+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f205c132f90 con 0x7f205c15f9c0 2023-11-24T06:27:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.255+0000 7f2058ff9640 1 -- 172.21.15.16:0/2748981506 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f205c132f90 con 0x7f205c15f9c0 2023-11-24T06:27:56.258 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204806ae40 msgr2=0x7f204806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204806ae40 0x7f204806d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f204c000f60 tx=0x7f204c007c00 comp rx=0 tx=0).stop 2023-11-24T06:27:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 msgr2=0x7f205c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 0x7f205c156bc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2054009d90 tx=0x7f2054004a40 comp rx=0 tx=0).stop 2023-11-24T06:27:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 shutdown_connections 2023-11-24T06:27:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204806ae40 0x7f204806d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205c15f9c0 0x7f205c156bc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205c13e360 0x7f205c153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 --2- 172.21.15.16:0/2748981506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205c13df80 0x7f205c1521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.258+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 >> 172.21.15.16:0/2748981506 conn(0x7f205c06a800 msgr2=0x7f205c0b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.259+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 shutdown_connections 2023-11-24T06:27:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.259+0000 7f2062472640 1 -- 172.21.15.16:0/2748981506 wait complete. 2023-11-24T06:27:56.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.507+0000 7f89c162e640 1 Processor -- start 2023-11-24T06:27:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.507+0000 7f89c162e640 1 -- start start 2023-11-24T06:27:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.509+0000 7f89c162e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f89bc0ccb50 0x7f89bc070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.509+0000 7f89c162e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89bc15bf10 0x7f89bc143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.509+0000 7f89c162e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89bc15f4e0 con 0x7f89bc0adf20 2023-11-24T06:27:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.509+0000 7f89c162e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89bc15f970 con 0x7f89bc15bf10 2023-11-24T06:27:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.509+0000 7f89c162e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89bc071130 con 0x7f89bc0ccb50 2023-11-24T06:27:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.509+0000 7f89c0e2d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89bc15bf10 0x7f89bc143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.509+0000 7f89c0e2d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89bc15bf10 0x7f89bc143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55366/0 (socket says 172.21.15.16:55366) 2023-11-24T06:27:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.509+0000 7f89c0e2d640 1 -- 172.21.15.16:0/3480929110 learned_addr learned my addr 172.21.15.16:0/3480929110 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.509+0000 7f89b3fff640 1 --2- 172.21.15.16:0/3480929110 >> v2:172.21.15.173:3300/0 conn(0x7f89bc0ccb50 0x7f89bc070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.510+0000 7f89c0e2d640 1 -- 172.21.15.16:0/3480929110 >> v2:172.21.15.173:3300/0 conn(0x7f89bc0ccb50 msgr2=0x7f89bc070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.510+0000 7f89c0e2d640 1 --2- 172.21.15.16:0/3480929110 >> v2:172.21.15.173:3300/0 conn(0x7f89bc0ccb50 0x7f89bc070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.510+0000 7f89c0e2d640 1 -- 172.21.15.16:0/3480929110 >> v1:172.21.15.173:6789/0 conn(0x7f89bc0adf20 legacy=0x7f89bc0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.510+0000 7f89c0e2d640 1 -- 172.21.15.16:0/3480929110 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc15ebb0 con 0x7f89bc15bf10 2023-11-24T06:27:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.511+0000 7f89c0e2d640 1 --2- 172.21.15.16:0/3480929110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc15bf10 0x7f89bc143d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f89bc15e9e0 tx=0x7f89b402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b15f9b69323b4b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.511+0000 7f89bb7fe640 1 -- 172.21.15.16:0/3480929110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b402c9f0 con 0x7f89bc15bf10 2023-11-24T06:27:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.511+0000 7f89bb7fe640 1 -- 172.21.15.16:0/3480929110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b402cb50 con 0x7f89bc15bf10 2023-11-24T06:27:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.511+0000 7f89bb7fe640 1 -- 172.21.15.16:0/3480929110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b40343f0 con 0x7f89bc15bf10 2023-11-24T06:27:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.512+0000 7f89c162e640 1 -- 172.21.15.16:0/3480929110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc15bf10 msgr2=0x7f89bc143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.512+0000 7f89c162e640 1 --2- 172.21.15.16:0/3480929110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc15bf10 0x7f89bc143d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f89bc15e9e0 tx=0x7f89b402be50 comp rx=0 tx=0).stop 2023-11-24T06:27:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.512+0000 7f89c162e640 1 -- 172.21.15.16:0/3480929110 shutdown_connections 2023-11-24T06:27:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.512+0000 7f89c162e640 1 --2- 172.21.15.16:0/3480929110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc15bf10 0x7f89bc143d30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.512+0000 7f89c162e640 1 --2- 172.21.15.16:0/3480929110 >> v2:172.21.15.173:3300/0 conn(0x7f89bc0ccb50 0x7f89bc070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.512+0000 7f89c162e640 1 -- 172.21.15.16:0/3480929110 >> 172.21.15.16:0/3480929110 conn(0x7f89bc06a7d0 msgr2=0x7f89bc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.512+0000 7f89c162e640 1 -- 172.21.15.16:0/3480929110 shutdown_connections 2023-11-24T06:27:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.512+0000 7f89c162e640 1 -- 172.21.15.16:0/3480929110 wait complete. 2023-11-24T06:27:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.514+0000 7f89c162e640 1 Processor -- start 2023-11-24T06:27:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.514+0000 7f89c162e640 1 -- start start 2023-11-24T06:27:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89c162e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89bc0adf20 0x7f89bc1e6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89c162e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc0ccb50 0x7f89bc140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89c162e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89bc15bf10 0x7f89bc1e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89c162e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc1e66b0 con 0x7f89bc0ccb50 2023-11-24T06:27:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89c162e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc141530 con 0x7f89bc15bf10 2023-11-24T06:27:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89c162e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc1e50d0 con 0x7f89bc0adf20 2023-11-24T06:27:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc0ccb50 0x7f89bc140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc0ccb50 0x7f89bc140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55382/0 (socket says 172.21.15.16:55382) 2023-11-24T06:27:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89b3fff640 1 -- 172.21.15.16:0/3601757375 learned_addr learned my addr 172.21.15.16:0/3601757375 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89b3fff640 1 -- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89bc0adf20 msgr2=0x7f89bc1e6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89b3fff640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89bc0adf20 0x7f89bc1e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89b3fff640 1 -- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89bc15bf10 msgr2=0x7f89bc1e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89c0e2d640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89bc15bf10 0x7f89bc1e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89b3fff640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89bc15bf10 0x7f89bc1e48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.515+0000 7f89b3fff640 1 -- 172.21.15.16:0/3601757375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b4009b30 con 0x7f89bc0ccb50 2023-11-24T06:27:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.516+0000 7f89b3fff640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc0ccb50 0x7f89bc140ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d7d0 tx=0x7f89a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.516+0000 7f89b97fa640 1 -- 172.21.15.16:0/3601757375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a400b6c0 con 0x7f89bc0ccb50 2023-11-24T06:27:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.516+0000 7f89b97fa640 1 -- 172.21.15.16:0/3601757375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a400b820 con 0x7f89bc0ccb50 2023-11-24T06:27:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.516+0000 7f89b97fa640 1 -- 172.21.15.16:0/3601757375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a400bbb0 con 0x7f89bc0ccb50 2023-11-24T06:27:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.516+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89bc1e53b0 con 0x7f89bc0ccb50 2023-11-24T06:27:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.516+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc1e5970 con 0x7f89bc0ccb50 2023-11-24T06:27:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.517+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89bc067450 con 0x7f89bc0ccb50 2023-11-24T06:27:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.518+0000 7f89b97fa640 1 -- 172.21.15.16:0/3601757375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89a400bd10 con 0x7f89bc0ccb50 2023-11-24T06:27:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.519+0000 7f89b97fa640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f899c06ae40 0x7f899c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.519+0000 7f89b97fa640 1 -- 172.21.15.16:0/3601757375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f89a4089c90 con 0x7f89bc0ccb50 2023-11-24T06:27:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.520+0000 7f89bbfff640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f899c06ae40 0x7f899c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.521+0000 7f89bbfff640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f899c06ae40 0x7f899c06d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89ac001560 tx=0x7f89ac007990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.524+0000 7f89b97fa640 1 -- 172.21.15.16:0/3601757375 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89a4054c20 con 0x7f89bc0ccb50 2023-11-24T06:27:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.717+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89bc0cae50 con 0x7f89bc0ccb50 2023-11-24T06:27:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.719+0000 7f89b97fa640 1 -- 172.21.15.16:0/3601757375 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89a4058240 con 0x7f89bc0ccb50 2023-11-24T06:27:56.722 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f899c06ae40 msgr2=0x7f899c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f899c06ae40 0x7f899c06d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89ac001560 tx=0x7f89ac007990 comp rx=0 tx=0).stop 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc0ccb50 msgr2=0x7f89bc140ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc0ccb50 0x7f89bc140ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d7d0 tx=0x7f89a400dc90 comp rx=0 tx=0).stop 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 shutdown_connections 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f899c06ae40 0x7f899c06d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89bc15bf10 0x7f89bc1e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89bc0ccb50 0x7f89bc140ff0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 --2- 172.21.15.16:0/3601757375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89bc0adf20 0x7f89bc1e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 >> 172.21.15.16:0/3601757375 conn(0x7f89bc06a7d0 msgr2=0x7f89bc06abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 shutdown_connections 2023-11-24T06:27:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.721+0000 7f89c162e640 1 -- 172.21.15.16:0/3601757375 wait complete. 2023-11-24T06:27:56.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:56.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.972+0000 7f9830f6c640 1 Processor -- start 2023-11-24T06:27:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.972+0000 7f9830f6c640 1 -- start start 2023-11-24T06:27:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.974+0000 7f9830f6c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f982c15d4b0 0x7f982c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.974+0000 7f9830f6c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f982c0cf9a0 0x7f982c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.974+0000 7f9830f6c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f982c0d0320 con 0x7f982c0cddf0 2023-11-24T06:27:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.974+0000 7f9830f6c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f982c0b2730 con 0x7f982c0cf9a0 2023-11-24T06:27:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.974+0000 7f9830f6c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982c0d1ec0 con 0x7f982c15d4b0 2023-11-24T06:27:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.974+0000 7f982bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f982c0cddf0 0x7f982c0ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39650/0 (socket says 172.21.15.16:39650) 2023-11-24T06:27:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982bfff640 1 -- 172.21.15.16:0/936540215 learned_addr learned my addr 172.21.15.16:0/936540215 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:56.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982b7fe640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f982c15d4b0 0x7f982c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f9823fff640 1 --2- 172.21.15.16:0/936540215 >> v2:172.21.15.173:3300/0 conn(0x7f982c0cf9a0 0x7f982c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672432906 0 0) 0x7f982c0d0320 con 0x7f982c0cddf0 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9818003520 con 0x7f982c0cddf0 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2867669521 0 0) 0x7f9818003520 con 0x7f982c0cddf0 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f982c15d4b0 msgr2=0x7f982c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f982c15d4b0 0x7f982c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 >> v2:172.21.15.173:3300/0 conn(0x7f982c0cf9a0 msgr2=0x7f982c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 --2- 172.21.15.16:0/936540215 >> v2:172.21.15.173:3300/0 conn(0x7f982c0cf9a0 0x7f982c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c0b29d0 con 0x7f982c0cddf0 2023-11-24T06:27:56.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f98240019f0 con 0x7f982c0cddf0 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.975+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 >> v1:172.21.15.16:6789/0 conn(0x7f982c0cddf0 legacy=0x7f982c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f982affd640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9818003970 0x7f9818003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f982affd640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98180047c0 0x7f9818004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f982affd640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f982c15d4b0 0x7f9818005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c0b2730 con 0x7f98180047c0 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c0d1ec0 con 0x7f9818003970 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981800b980 con 0x7f982c15d4b0 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f9823fff640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98180047c0 0x7f9818004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f9823fff640 1 -- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f982c15d4b0 msgr2=0x7f9818005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f9823fff640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f982c15d4b0 0x7f9818005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f982b7fe640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9818003970 0x7f9818003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f9823fff640 1 -- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9818003970 msgr2=0x7f9818003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f9823fff640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9818003970 0x7f9818003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.976+0000 7f9823fff640 1 -- 172.21.15.16:0/936540215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9818003520 con 0x7f98180047c0 2023-11-24T06:27:56.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.978+0000 7f9823fff640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98180047c0 0x7f9818004c40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9814009ac0 tx=0x7f98180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a5fced146c2e8a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.978+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981402c710 con 0x7f98180047c0 2023-11-24T06:27:56.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.978+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981402c870 con 0x7f98180047c0 2023-11-24T06:27:56.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.978+0000 7f982affd640 1 -- 172.21.15.16:0/936540215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981402cc00 con 0x7f98180047c0 2023-11-24T06:27:56.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.979+0000 7f9830f6c640 1 -- 172.21.15.16:0/936540215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98180047c0 msgr2=0x7f9818004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.979+0000 7f9830f6c640 1 --2- 172.21.15.16:0/936540215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98180047c0 0x7f9818004c40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9814009ac0 tx=0x7f98180057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.979+0000 7f982bfff640 1 -- 172.21.15.16:0/936540215 reap_dead start 2023-11-24T06:27:56.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.979+0000 7f9830f6c640 1 -- 172.21.15.16:0/936540215 shutdown_connections 2023-11-24T06:27:56.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.979+0000 7f9830f6c640 1 -- 172.21.15.16:0/936540215 >> 172.21.15.16:0/936540215 conn(0x7f982c0d3000 msgr2=0x7f982c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:56.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.979+0000 7f9830f6c640 1 -- 172.21.15.16:0/936540215 shutdown_connections 2023-11-24T06:27:56.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.979+0000 7f9830f6c640 1 -- 172.21.15.16:0/936540215 wait complete. 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.981+0000 7f9830f6c640 1 Processor -- start 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.981+0000 7f9830f6c640 1 -- start start 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f9830f6c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98180047c0 0x7f982c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f9830f6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c157460 0x7f982c0cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f9830f6c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f982c160020 0x7f982c155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f9830f6c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c15b140 con 0x7f982c157460 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f9830f6c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c0caba0 con 0x7f982c160020 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f9830f6c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982c0ca270 con 0x7f98180047c0 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c157460 0x7f982c0cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f9823fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c157460 0x7f982c0cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55398/0 (socket says 172.21.15.16:55398) 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f9823fff640 1 -- 172.21.15.16:0/2369174387 learned_addr learned my addr 172.21.15.16:0/2369174387 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.982+0000 7f982b7fe640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98180047c0 0x7f982c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f982bfff640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f982c160020 0x7f982c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9823fff640 1 -- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98180047c0 msgr2=0x7f982c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9823fff640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98180047c0 0x7f982c15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9823fff640 1 -- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f982c160020 msgr2=0x7f982c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9823fff640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f982c160020 0x7f982c155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9823fff640 1 -- 172.21.15.16:0/2369174387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982c15da10 con 0x7f982c157460 2023-11-24T06:27:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f982bfff640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f982c160020 0x7f982c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9823fff640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c157460 0x7f982c0cb030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f981402c2b0 tx=0x7f9814004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9828ff9640 1 -- 172.21.15.16:0/2369174387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814003810 con 0x7f982c157460 2023-11-24T06:27:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9828ff9640 1 -- 172.21.15.16:0/2369174387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981402de90 con 0x7f982c157460 2023-11-24T06:27:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982c0ca550 con 0x7f982c157460 2023-11-24T06:27:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.983+0000 7f9828ff9640 1 -- 172.21.15.16:0/2369174387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814003b70 con 0x7f982c157460 2023-11-24T06:27:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.984+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982c15ae50 con 0x7f982c157460 2023-11-24T06:27:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.985+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982c0ca3b0 con 0x7f982c157460 2023-11-24T06:27:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.986+0000 7f9828ff9640 1 -- 172.21.15.16:0/2369174387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9814003cd0 con 0x7f982c157460 2023-11-24T06:27:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.987+0000 7f9828ff9640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f980c06ae70 0x7f980c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:56.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.987+0000 7f9828ff9640 1 -- 172.21.15.16:0/2369174387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f98140ab8f0 con 0x7f982c157460 2023-11-24T06:27:56.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.987+0000 7f982b7fe640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f980c06ae70 0x7f980c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:56.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.988+0000 7f982b7fe640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f980c06ae70 0x7f980c06d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f981c009930 tx=0x7f981c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:56.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:56.991+0000 7f9828ff9640 1 -- 172.21.15.16:0/2369174387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f982c0ca3b0 con 0x7f982c157460 2023-11-24T06:27:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.186+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f982c0ca3b0 con 0x7f982c157460 2023-11-24T06:27:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.187+0000 7f9828ff9640 1 -- 172.21.15.16:0/2369174387 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f982c0ca3b0 con 0x7f982c157460 2023-11-24T06:27:57.190 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.190+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f980c06ae70 msgr2=0x7f980c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.190+0000 7f9830f6c640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f980c06ae70 0x7f980c06d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f981c009930 tx=0x7f981c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.190+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c157460 msgr2=0x7f982c0cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.190+0000 7f9830f6c640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c157460 0x7f982c0cb030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f981402c2b0 tx=0x7f9814004730 comp rx=0 tx=0).stop 2023-11-24T06:27:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.191+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 shutdown_connections 2023-11-24T06:27:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.191+0000 7f9830f6c640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f980c06ae70 0x7f980c06d330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.191+0000 7f9830f6c640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f982c160020 0x7f982c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.191+0000 7f9830f6c640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f982c157460 0x7f982c0cb030 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.191+0000 7f9830f6c640 1 --2- 172.21.15.16:0/2369174387 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98180047c0 0x7f982c15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.191+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 >> 172.21.15.16:0/2369174387 conn(0x7f982c0d3000 msgr2=0x7f982c0ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.191+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 shutdown_connections 2023-11-24T06:27:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.191+0000 7f9830f6c640 1 -- 172.21.15.16:0/2369174387 wait complete. 2023-11-24T06:27:57.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:27:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.435+0000 7f343f44b640 1 Processor -- start 2023-11-24T06:27:57.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.436+0000 7f343f44b640 1 -- start start 2023-11-24T06:27:57.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.437+0000 7f343f44b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3438143c40 0x7f3438071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.437+0000 7f343f44b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f34380717d0 0x7f34380cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.437+0000 7f343f44b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3438070e50 con 0x7f34380cca40 2023-11-24T06:27:57.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.437+0000 7f343f44b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34380723c0 con 0x7f34380717d0 2023-11-24T06:27:57.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.437+0000 7f343f44b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34380cce70 con 0x7f3438143c40 2023-11-24T06:27:57.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.438+0000 7f343dc48640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f34380717d0 0x7f34380cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.438+0000 7f343e449640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3438143c40 0x7f3438071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.438+0000 7f343dc48640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f34380717d0 0x7f34380cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:41672/0 (socket says 172.21.15.16:41672) 2023-11-24T06:27:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.438+0000 7f343dc48640 1 -- 172.21.15.16:0/78764641 learned_addr learned my addr 172.21.15.16:0/78764641 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.439+0000 7f343dc48640 1 -- 172.21.15.16:0/78764641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3438143c40 msgr2=0x7f3438071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.439+0000 7f343dc48640 1 --2- 172.21.15.16:0/78764641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3438143c40 0x7f3438071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.439+0000 7f343dc48640 1 -- 172.21.15.16:0/78764641 >> v1:172.21.15.173:6789/0 conn(0x7f34380cca40 legacy=0x7f343815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.439+0000 7f343dc48640 1 -- 172.21.15.16:0/78764641 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343806e170 con 0x7f34380717d0 2023-11-24T06:27:57.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.440+0000 7f343dc48640 1 --2- 172.21.15.16:0/78764641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380717d0 0x7f34380cc500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3428009870 tx=0x7f342802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e53b9255bf27892 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.441+0000 7f343d447640 1 -- 172.21.15.16:0/78764641 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428031640 con 0x7f34380717d0 2023-11-24T06:27:57.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.441+0000 7f343d447640 1 -- 172.21.15.16:0/78764641 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34280317a0 con 0x7f34380717d0 2023-11-24T06:27:57.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.441+0000 7f343d447640 1 -- 172.21.15.16:0/78764641 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428031b30 con 0x7f34380717d0 2023-11-24T06:27:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.442+0000 7f343f44b640 1 -- 172.21.15.16:0/78764641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380717d0 msgr2=0x7f34380cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.442+0000 7f343f44b640 1 --2- 172.21.15.16:0/78764641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380717d0 0x7f34380cc500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3428009870 tx=0x7f342802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.442+0000 7f343f44b640 1 -- 172.21.15.16:0/78764641 shutdown_connections 2023-11-24T06:27:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.442+0000 7f343f44b640 1 --2- 172.21.15.16:0/78764641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380717d0 0x7f34380cc500 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.442+0000 7f343f44b640 1 --2- 172.21.15.16:0/78764641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3438143c40 0x7f3438071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.442+0000 7f343f44b640 1 -- 172.21.15.16:0/78764641 >> 172.21.15.16:0/78764641 conn(0x7f343806a800 msgr2=0x7f343806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:57.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.442+0000 7f343f44b640 1 -- 172.21.15.16:0/78764641 shutdown_connections 2023-11-24T06:27:57.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.442+0000 7f343f44b640 1 -- 172.21.15.16:0/78764641 wait complete. 2023-11-24T06:27:57.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.444+0000 7f343f44b640 1 Processor -- start 2023-11-24T06:27:57.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.445+0000 7f343f44b640 1 -- start start 2023-11-24T06:27:57.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343f44b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380717d0 0x7f34381540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343f44b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cca40 0x7f34380b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343f44b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3438143c40 0x7f34380b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343f44b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3438142100 con 0x7f3438143c40 2023-11-24T06:27:57.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343f44b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3438141230 con 0x7f34380cca40 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343f44b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3438141480 con 0x7f34380717d0 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343ec4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3438143c40 0x7f34380b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343ec4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3438143c40 0x7f34380b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55414/0 (socket says 172.21.15.16:55414) 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343ec4a640 1 -- 172.21.15.16:0/2417630226 learned_addr learned my addr 172.21.15.16:0/2417630226 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343dc48640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cca40 0x7f34380b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.446+0000 7f343e449640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380717d0 0x7f34381540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f343ec4a640 1 -- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380717d0 msgr2=0x7f34381540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f343ec4a640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380717d0 0x7f34381540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f343ec4a640 1 -- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cca40 msgr2=0x7f34380b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f343ec4a640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cca40 0x7f34380b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f343ec4a640 1 -- 172.21.15.16:0/2417630226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280094d0 con 0x7f3438143c40 2023-11-24T06:27:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f343ec4a640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3438143c40 0x7f34380b3b30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3430001eb0 tx=0x7f343000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f342effd640 1 -- 172.21.15.16:0/2417630226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34300052d0 con 0x7f3438143c40 2023-11-24T06:27:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f342effd640 1 -- 172.21.15.16:0/2417630226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3430005430 con 0x7f3438143c40 2023-11-24T06:27:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f342effd640 1 -- 172.21.15.16:0/2417630226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430003810 con 0x7f3438143c40 2023-11-24T06:27:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3438141760 con 0x7f3438143c40 2023-11-24T06:27:57.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.447+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3438141d20 con 0x7f3438143c40 2023-11-24T06:27:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.449+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3438141eb0 con 0x7f3438143c40 2023-11-24T06:27:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.450+0000 7f342effd640 1 -- 172.21.15.16:0/2417630226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3430003970 con 0x7f3438143c40 2023-11-24T06:27:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.450+0000 7f342effd640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f341c06ae40 0x7f341c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.451+0000 7f342effd640 1 -- 172.21.15.16:0/2417630226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f3430088d70 con 0x7f3438143c40 2023-11-24T06:27:57.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.451+0000 7f343e449640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f341c06ae40 0x7f341c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:57.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.453+0000 7f343e449640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f341c06ae40 0x7f341c06d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f34380717a0 tx=0x7f343400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.456+0000 7f342effd640 1 -- 172.21.15.16:0/2417630226 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3430053c80 con 0x7f3438143c40 2023-11-24T06:27:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.662+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3438067450 con 0x7f3438143c40 2023-11-24T06:27:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.663+0000 7f342effd640 1 -- 172.21.15.16:0/2417630226 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f34300572a0 con 0x7f3438143c40 2023-11-24T06:27:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:27:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f341c06ae40 msgr2=0x7f341c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f341c06ae40 0x7f341c06d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f34380717a0 tx=0x7f343400b040 comp rx=0 tx=0).stop 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3438143c40 msgr2=0x7f34380b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3438143c40 0x7f34380b3b30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3430001eb0 tx=0x7f343000c890 comp rx=0 tx=0).stop 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 shutdown_connections 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f341c06ae40 0x7f341c06d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3438143c40 0x7f34380b3b30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34380cca40 0x7f34380b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 --2- 172.21.15.16:0/2417630226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34380717d0 0x7f34381540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 >> 172.21.15.16:0/2417630226 conn(0x7f343806a800 msgr2=0x7f343813a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 shutdown_connections 2023-11-24T06:27:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:57.665+0000 7f343f44b640 1 -- 172.21.15.16:0/2417630226 wait complete. 2023-11-24T06:27:57.686 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-24T06:27:57.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-24T06:27:57.789 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:27:57.789 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:27:57.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:27:57.789 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:27:57.846 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:27:57.846 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:57.906 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:27:57.906 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:27:57.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:27:58.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.193+0000 7f43b2478640 1 Processor -- start 2023-11-24T06:27:58.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.193+0000 7f43b2478640 1 -- start start 2023-11-24T06:27:58.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.195+0000 7f43b2478640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f43ac13cfa0 0x7f43ac15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.195+0000 7f43b2478640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43ac13fcb0 0x7f43ac0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.195+0000 7f43b2478640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac0cdec0 con 0x7f43ac0cd530 2023-11-24T06:27:58.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.195+0000 7f43b2478640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac0ce1a0 con 0x7f43ac13fcb0 2023-11-24T06:27:58.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.195+0000 7f43b2478640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac0ce480 con 0x7f43ac13cfa0 2023-11-24T06:27:58.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.195+0000 7f43b1c77640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43ac13fcb0 0x7f43ac0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.195+0000 7f43b0c75640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f43ac0cd530 0x7f43ac13f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39656/0 (socket says 172.21.15.16:39656) 2023-11-24T06:27:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.195+0000 7f43b0c75640 1 -- 172.21.15.16:0/430581186 learned_addr learned my addr 172.21.15.16:0/430581186 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.196+0000 7f43b1476640 1 --2- 172.21.15.16:0/430581186 >> v2:172.21.15.173:3300/0 conn(0x7f43ac13cfa0 0x7f43ac15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.196+0000 7f43b1c77640 1 -- 172.21.15.16:0/430581186 >> v2:172.21.15.173:3300/0 conn(0x7f43ac13cfa0 msgr2=0x7f43ac15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.196+0000 7f43b1c77640 1 --2- 172.21.15.16:0/430581186 >> v2:172.21.15.173:3300/0 conn(0x7f43ac13cfa0 0x7f43ac15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.196+0000 7f43b1c77640 1 -- 172.21.15.16:0/430581186 >> v1:172.21.15.16:6789/0 conn(0x7f43ac0cd530 legacy=0x7f43ac13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.196+0000 7f43a3fff640 1 -- 172.21.15.16:0/430581186 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020461062 0 0) 0x7f43ac0cdec0 con 0x7f43ac0cd530 2023-11-24T06:27:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.196+0000 7f43b1c77640 1 -- 172.21.15.16:0/430581186 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0ce760 con 0x7f43ac13fcb0 2023-11-24T06:27:58.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.196+0000 7f43b1476640 1 --2- 172.21.15.16:0/430581186 >> v2:172.21.15.173:3300/0 conn(0x7f43ac13cfa0 0x7f43ac15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:58.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.197+0000 7f43b1c77640 1 --2- 172.21.15.16:0/430581186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac13fcb0 0x7f43ac0b2040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f43ac06cf00 tx=0x7f43a402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323bf4f533aee4a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.197+0000 7f43a3fff640 1 -- 172.21.15.16:0/430581186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a402cbf0 con 0x7f43ac13fcb0 2023-11-24T06:27:58.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.197+0000 7f43a3fff640 1 -- 172.21.15.16:0/430581186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a402cd50 con 0x7f43ac13fcb0 2023-11-24T06:27:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.197+0000 7f43a3fff640 1 -- 172.21.15.16:0/430581186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4034660 con 0x7f43ac13fcb0 2023-11-24T06:27:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.198+0000 7f43b2478640 1 -- 172.21.15.16:0/430581186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac13fcb0 msgr2=0x7f43ac0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.198+0000 7f43b2478640 1 --2- 172.21.15.16:0/430581186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac13fcb0 0x7f43ac0b2040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f43ac06cf00 tx=0x7f43a402be40 comp rx=0 tx=0).stop 2023-11-24T06:27:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.198+0000 7f43b2478640 1 -- 172.21.15.16:0/430581186 shutdown_connections 2023-11-24T06:27:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.198+0000 7f43b2478640 1 --2- 172.21.15.16:0/430581186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac13fcb0 0x7f43ac0b2040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.198+0000 7f43b2478640 1 --2- 172.21.15.16:0/430581186 >> v2:172.21.15.173:3300/0 conn(0x7f43ac13cfa0 0x7f43ac15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.198+0000 7f43b2478640 1 -- 172.21.15.16:0/430581186 >> 172.21.15.16:0/430581186 conn(0x7f43ac06a800 msgr2=0x7f43ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.198+0000 7f43b2478640 1 -- 172.21.15.16:0/430581186 shutdown_connections 2023-11-24T06:27:58.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.198+0000 7f43b2478640 1 -- 172.21.15.16:0/430581186 wait complete. 2023-11-24T06:27:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.200+0000 7f43b2478640 1 Processor -- start 2023-11-24T06:27:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.200+0000 7f43b2478640 1 -- start start 2023-11-24T06:27:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.201+0000 7f43b2478640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac0cd530 0x7f43ac13b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.201+0000 7f43b2478640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43ac13cfa0 0x7f43ac1f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.201+0000 7f43b2478640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43ac13fcb0 0x7f43ac138760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.201+0000 7f43b2478640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac135070 con 0x7f43ac0cd530 2023-11-24T06:27:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.201+0000 7f43b2478640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac13a6d0 con 0x7f43ac13cfa0 2023-11-24T06:27:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.201+0000 7f43b2478640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac13a9b0 con 0x7f43ac13fcb0 2023-11-24T06:27:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.201+0000 7f43b1476640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac0cd530 0x7f43ac13b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.201+0000 7f43b1476640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac0cd530 0x7f43ac13b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55442/0 (socket says 172.21.15.16:55442) 2023-11-24T06:27:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.201+0000 7f43b1476640 1 -- 172.21.15.16:0/3198488511 learned_addr learned my addr 172.21.15.16:0/3198488511 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b0c75640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43ac13cfa0 0x7f43ac1f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b1c77640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43ac13fcb0 0x7f43ac138760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b1476640 1 -- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43ac13fcb0 msgr2=0x7f43ac138760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b1476640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43ac13fcb0 0x7f43ac138760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b1476640 1 -- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43ac13cfa0 msgr2=0x7f43ac1f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b1476640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43ac13cfa0 0x7f43ac1f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b1476640 1 -- 172.21.15.16:0/3198488511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a4009a70 con 0x7f43ac0cd530 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b1c77640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43ac13fcb0 0x7f43ac138760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b1476640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac0cd530 0x7f43ac13b770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c7f0 tx=0x7f43a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43a1ffb640 1 -- 172.21.15.16:0/3198488511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8007a30 con 0x7f43ac0cd530 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43a1ffb640 1 -- 172.21.15.16:0/3198488511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8007b90 con 0x7f43ac0cd530 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43a1ffb640 1 -- 172.21.15.16:0/3198488511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800ce40 con 0x7f43ac0cd530 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.202+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac13ac90 con 0x7f43ac0cd530 2023-11-24T06:27:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.203+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac13b250 con 0x7f43ac0cd530 2023-11-24T06:27:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.204+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ac067450 con 0x7f43ac0cd530 2023-11-24T06:27:58.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.205+0000 7f43a1ffb640 1 -- 172.21.15.16:0/3198488511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43a80113f0 con 0x7f43ac0cd530 2023-11-24T06:27:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.206+0000 7f43a1ffb640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f439006ae40 0x7f439006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.206+0000 7f43a1ffb640 1 -- 172.21.15.16:0/3198488511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f43a8089880 con 0x7f43ac0cd530 2023-11-24T06:27:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.206+0000 7f43b0c75640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f439006ae40 0x7f439006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.207+0000 7f43b0c75640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f439006ae40 0x7f439006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0b02f0 tx=0x7f439c007b50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.210+0000 7f43a1ffb640 1 -- 172.21.15.16:0/3198488511 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43a80548b0 con 0x7f43ac0cd530 2023-11-24T06:27:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.403+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43ac0cc270 con 0x7f43ac0cd530 2023-11-24T06:27:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.406+0000 7f43a1ffb640 1 -- 172.21.15.16:0/3198488511 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7f43a801f090 con 0x7f43ac0cd530 2023-11-24T06:27:58.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:27:58.409 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:27:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f439006ae40 msgr2=0x7f439006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f439006ae40 0x7f439006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0b02f0 tx=0x7f439c007b50 comp rx=0 tx=0).stop 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac0cd530 msgr2=0x7f43ac13b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac0cd530 0x7f43ac13b770 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c7f0 tx=0x7f43a800cbf0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 shutdown_connections 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f439006ae40 0x7f439006d300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f43ac13fcb0 0x7f43ac138760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43ac13cfa0 0x7f43ac1f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 --2- 172.21.15.16:0/3198488511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43ac0cd530 0x7f43ac13b770 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 >> 172.21.15.16:0/3198488511 conn(0x7f43ac06a800 msgr2=0x7f43ac0aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 shutdown_connections 2023-11-24T06:27:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.409+0000 7f43b2478640 1 -- 172.21.15.16:0/3198488511 wait complete. 2023-11-24T06:27:58.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:27:58.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:27:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.653+0000 7f6a393f5640 1 Processor -- start 2023-11-24T06:27:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.654+0000 7f6a393f5640 1 -- start start 2023-11-24T06:27:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.656+0000 7f6a393f5640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6a3413e360 0x7f6a340cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.656+0000 7f6a393f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3415c5f0 con 0x7f6a3415f9c0 2023-11-24T06:27:58.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.656+0000 7f6a393f5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3406c470 con 0x7f6a3413df80 2023-11-24T06:27:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.656+0000 7f6a393f5640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3406c750 con 0x7f6a3413e360 2023-11-24T06:27:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.656+0000 7f6a337fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a3415f9c0 0x7f6a340cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39664/0 (socket says 172.21.15.16:39664) 2023-11-24T06:27:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.656+0000 7f6a337fe640 1 -- 172.21.15.16:0/1421876017 learned_addr learned my addr 172.21.15.16:0/1421876017 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a2bfff640 1 --2- 172.21.15.16:0/1421876017 >> v2:172.21.15.173:3300/0 conn(0x7f6a3413e360 0x7f6a340cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034499266 0 0) 0x7f6a3415c5f0 con 0x7f6a3415f9c0 2023-11-24T06:27:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a20003520 con 0x7f6a3415f9c0 2023-11-24T06:27:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1098259690 0 0) 0x7f6a20003520 con 0x7f6a3415f9c0 2023-11-24T06:27:58.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 >> v2:172.21.15.173:3300/0 conn(0x7f6a3413e360 msgr2=0x7f6a340cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a327fc640 1 --2- 172.21.15.16:0/1421876017 >> v2:172.21.15.173:3300/0 conn(0x7f6a3413e360 0x7f6a340cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 >> v1:172.21.15.173:6789/0 conn(0x7f6a3413df80 legacy=0x7f6a3415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3406ca30 con 0x7f6a3415f9c0 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f6a2c001a10 con 0x7f6a3415f9c0 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.657+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 >> v1:172.21.15.16:6789/0 conn(0x7f6a3415f9c0 legacy=0x7f6a340cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a327fc640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a327fc640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a200047c0 0x7f6a20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a327fc640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a20005180 0x7f6a2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3406c750 con 0x7f6a200039c0 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2000c080 con 0x7f6a200047c0 2023-11-24T06:27:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2000c240 con 0x7f6a20005180 2023-11-24T06:27:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a32ffd640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a2bfff640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a200047c0 0x7f6a20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a32ffd640 1 -- 172.21.15.16:0/1421876017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a20005180 msgr2=0x7f6a2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a32ffd640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a20005180 0x7f6a2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a32ffd640 1 -- 172.21.15.16:0/1421876017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a200047c0 msgr2=0x7f6a20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a32ffd640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a200047c0 0x7f6a20004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a32ffd640 1 -- 172.21.15.16:0/1421876017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a20003520 con 0x7f6a200039c0 2023-11-24T06:27:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.658+0000 7f6a337fe640 1 -- 172.21.15.16:0/1421876017 reap_dead start 2023-11-24T06:27:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.659+0000 7f6a32ffd640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a20003de0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a24001750 tx=0x7f6a200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178273b08ce91c49 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.660+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a240052d0 con 0x7f6a200039c0 2023-11-24T06:27:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.660+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2402e050 con 0x7f6a200039c0 2023-11-24T06:27:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.660+0000 7f6a327fc640 1 -- 172.21.15.16:0/1421876017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24003830 con 0x7f6a200039c0 2023-11-24T06:27:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.660+0000 7f6a393f5640 1 -- 172.21.15.16:0/1421876017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 msgr2=0x7f6a20003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.660+0000 7f6a393f5640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a20003de0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a24001750 tx=0x7f6a200057f0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.660+0000 7f6a393f5640 1 -- 172.21.15.16:0/1421876017 shutdown_connections 2023-11-24T06:27:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.660+0000 7f6a393f5640 1 --2- 172.21.15.16:0/1421876017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a20003de0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.660+0000 7f6a393f5640 1 -- 172.21.15.16:0/1421876017 >> 172.21.15.16:0/1421876017 conn(0x7f6a3406a800 msgr2=0x7f6a3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.661+0000 7f6a393f5640 1 -- 172.21.15.16:0/1421876017 shutdown_connections 2023-11-24T06:27:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.661+0000 7f6a393f5640 1 -- 172.21.15.16:0/1421876017 wait complete. 2023-11-24T06:27:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.662+0000 7f6a393f5640 1 Processor -- start 2023-11-24T06:27:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.662+0000 7f6a393f5640 1 -- start start 2023-11-24T06:27:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.663+0000 7f6a393f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a341f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.663+0000 7f6a393f5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a340c4840 0x7f6a34156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.663+0000 7f6a393f5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a34157740 0x7f6a3415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.663+0000 7f6a393f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340c5070 con 0x7f6a200039c0 2023-11-24T06:27:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.663+0000 7f6a393f5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340c4190 con 0x7f6a340c4840 2023-11-24T06:27:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.663+0000 7f6a393f5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341f3e20 con 0x7f6a34157740 2023-11-24T06:27:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a32ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a341f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a32ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a341f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55460/0 (socket says 172.21.15.16:55460) 2023-11-24T06:27:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a32ffd640 1 -- 172.21.15.16:0/790705701 learned_addr learned my addr 172.21.15.16:0/790705701 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a2bfff640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a340c4840 0x7f6a34156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a32ffd640 1 -- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a34157740 msgr2=0x7f6a3415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a337fe640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a34157740 0x7f6a3415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a32ffd640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a34157740 0x7f6a3415d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a32ffd640 1 -- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a340c4840 msgr2=0x7f6a34156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a32ffd640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a340c4840 0x7f6a34156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a32ffd640 1 -- 172.21.15.16:0/790705701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340c4d90 con 0x7f6a200039c0 2023-11-24T06:27:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.664+0000 7f6a2bfff640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a340c4840 0x7f6a34156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.665+0000 7f6a32ffd640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a341f38e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.665+0000 7f6a2b7fe640 1 -- 172.21.15.16:0/790705701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a240076c0 con 0x7f6a200039c0 2023-11-24T06:27:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.665+0000 7f6a2b7fe640 1 -- 172.21.15.16:0/790705701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a24007a30 con 0x7f6a200039c0 2023-11-24T06:27:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.665+0000 7f6a2b7fe640 1 -- 172.21.15.16:0/790705701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24006be0 con 0x7f6a200039c0 2023-11-24T06:27:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.665+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a34156c60 con 0x7f6a200039c0 2023-11-24T06:27:58.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.665+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a34157220 con 0x7f6a200039c0 2023-11-24T06:27:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.666+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a340675d0 con 0x7f6a200039c0 2023-11-24T06:27:58.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.667+0000 7f6a2b7fe640 1 -- 172.21.15.16:0/790705701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a24006d40 con 0x7f6a200039c0 2023-11-24T06:27:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.668+0000 7f6a2b7fe640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6a1406ae40 0x7f6a1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.668+0000 7f6a2b7fe640 1 -- 172.21.15.16:0/790705701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f6a240ab1e0 con 0x7f6a200039c0 2023-11-24T06:27:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.669+0000 7f6a2bfff640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6a1406ae40 0x7f6a1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.670+0000 7f6a2bfff640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6a1406ae40 0x7f6a1406d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6a3415ab20 tx=0x7f6a1c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:58.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.673+0000 7f6a2b7fe640 1 -- 172.21.15.16:0/790705701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a240762a0 con 0x7f6a200039c0 2023-11-24T06:27:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.866+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a3406df00 con 0x7f6a200039c0 2023-11-24T06:27:58.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.867+0000 7f6a2b7fe640 1 -- 172.21.15.16:0/790705701 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a240798c0 con 0x7f6a200039c0 2023-11-24T06:27:58.869 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:27:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6a1406ae40 msgr2=0x7f6a1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6a1406ae40 0x7f6a1406d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6a3415ab20 tx=0x7f6a1c008040 comp rx=0 tx=0).stop 2023-11-24T06:27:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 msgr2=0x7f6a341f38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a341f38e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24001120 comp rx=0 tx=0).stop 2023-11-24T06:27:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 shutdown_connections 2023-11-24T06:27:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6a1406ae40 0x7f6a1406d300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a34157740 0x7f6a3415d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a340c4840 0x7f6a34156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 --2- 172.21.15.16:0/790705701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a200039c0 0x7f6a341f38e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 >> 172.21.15.16:0/790705701 conn(0x7f6a3406a800 msgr2=0x7f6a34070350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.870+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 shutdown_connections 2023-11-24T06:27:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:58.871+0000 7f6a393f5640 1 -- 172.21.15.16:0/790705701 wait complete. 2023-11-24T06:27:58.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:27:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.109+0000 7f57c359e640 1 Processor -- start 2023-11-24T06:27:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.109+0000 7f57c359e640 1 -- start start 2023-11-24T06:27:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.111+0000 7f57c359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57c4070a30 0x7f57c415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.111+0000 7f57c359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c415f4e0 con 0x7f57c40ccb50 2023-11-24T06:27:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.111+0000 7f57c359e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c4070ea0 con 0x7f57c40adf20 2023-11-24T06:27:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.111+0000 7f57c359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c415c450 con 0x7f57c4070a30 2023-11-24T06:27:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.112+0000 7f57bbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f57c40ccb50 0x7f57c4143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39666/0 (socket says 172.21.15.16:39666) 2023-11-24T06:27:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.112+0000 7f57bbfff640 1 -- 172.21.15.16:0/2630207109 learned_addr learned my addr 172.21.15.16:0/2630207109 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:27:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.112+0000 7f57c2d9d640 1 --2- 172.21.15.16:0/2630207109 >> v2:172.21.15.16:3300/0 conn(0x7f57c4070a30 0x7f57c415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.112+0000 7f57c1d9b640 1 -- 172.21.15.16:0/2630207109 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023359400 0 0) 0x7f57c415f4e0 con 0x7f57c40ccb50 2023-11-24T06:27:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.112+0000 7f57c1d9b640 1 -- 172.21.15.16:0/2630207109 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b0003520 con 0x7f57c40ccb50 2023-11-24T06:27:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.112+0000 7f57c2d9d640 1 -- 172.21.15.16:0/2630207109 >> v1:172.21.15.173:6789/0 conn(0x7f57c40adf20 legacy=0x7f57c40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.112+0000 7f57c2d9d640 1 -- 172.21.15.16:0/2630207109 >> v1:172.21.15.16:6789/0 conn(0x7f57c40ccb50 legacy=0x7f57c4143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.112+0000 7f57c2d9d640 1 -- 172.21.15.16:0/2630207109 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4071130 con 0x7f57c4070a30 2023-11-24T06:27:59.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.114+0000 7f57c2d9d640 1 --2- 172.21.15.16:0/2630207109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c4070a30 0x7f57c415bf10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009d90 tx=0x7f57bc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3708130c8a15a2dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.114+0000 7f57c1d9b640 1 -- 172.21.15.16:0/2630207109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc02cc60 con 0x7f57c4070a30 2023-11-24T06:27:59.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.114+0000 7f57c1d9b640 1 -- 172.21.15.16:0/2630207109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc02cdc0 con 0x7f57c4070a30 2023-11-24T06:27:59.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.114+0000 7f57c1d9b640 1 -- 172.21.15.16:0/2630207109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc034660 con 0x7f57c4070a30 2023-11-24T06:27:59.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.114+0000 7f57c359e640 1 -- 172.21.15.16:0/2630207109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c4070a30 msgr2=0x7f57c415bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.114+0000 7f57c359e640 1 --2- 172.21.15.16:0/2630207109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c4070a30 0x7f57c415bf10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009d90 tx=0x7f57bc02beb0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.115+0000 7f57c359e640 1 -- 172.21.15.16:0/2630207109 shutdown_connections 2023-11-24T06:27:59.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.115+0000 7f57c359e640 1 --2- 172.21.15.16:0/2630207109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c4070a30 0x7f57c415bf10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.115+0000 7f57c359e640 1 -- 172.21.15.16:0/2630207109 >> 172.21.15.16:0/2630207109 conn(0x7f57c406a7d0 msgr2=0x7f57c406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.115+0000 7f57c359e640 1 -- 172.21.15.16:0/2630207109 shutdown_connections 2023-11-24T06:27:59.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.115+0000 7f57c359e640 1 -- 172.21.15.16:0/2630207109 wait complete. 2023-11-24T06:27:59.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.117+0000 7f57c359e640 1 Processor -- start 2023-11-24T06:27:59.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.117+0000 7f57c359e640 1 -- start start 2023-11-24T06:27:59.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57c359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57c4070a30 0x7f57c40ab610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57c359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40adf20 0x7f57c40abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57c359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57c40ccb50 0x7f57c41436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57c359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40ac180 con 0x7f57c40adf20 2023-11-24T06:27:59.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57c359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40a9050 con 0x7f57c40ccb50 2023-11-24T06:27:59.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57c359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40a9330 con 0x7f57c4070a30 2023-11-24T06:27:59.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40adf20 0x7f57c40abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40adf20 0x7f57c40abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55480/0 (socket says 172.21.15.16:55480) 2023-11-24T06:27:59.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57bbfff640 1 -- 172.21.15.16:0/3225477377 learned_addr learned my addr 172.21.15.16:0/3225477377 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57bbfff640 1 -- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57c4070a30 msgr2=0x7f57c40ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57bbfff640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57c4070a30 0x7f57c40ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57bbfff640 1 -- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57c40ccb50 msgr2=0x7f57c41436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57c2d9d640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57c40ccb50 0x7f57c41436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57bbfff640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57c40ccb50 0x7f57c41436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.118+0000 7f57bbfff640 1 -- 172.21.15.16:0/3225477377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc009a70 con 0x7f57c40adf20 2023-11-24T06:27:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.119+0000 7f57bbfff640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40adf20 0x7f57c40abb50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00c4d0 tx=0x7f57ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.119+0000 7f57baffd640 1 -- 172.21.15.16:0/3225477377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac005ce0 con 0x7f57c40adf20 2023-11-24T06:27:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.119+0000 7f57baffd640 1 -- 172.21.15.16:0/3225477377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac005e40 con 0x7f57c40adf20 2023-11-24T06:27:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.119+0000 7f57baffd640 1 -- 172.21.15.16:0/3225477377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac007270 con 0x7f57c40adf20 2023-11-24T06:27:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.119+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c40a9610 con 0x7f57c40adf20 2023-11-24T06:27:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.119+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c40b0150 con 0x7f57c40adf20 2023-11-24T06:27:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.120+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c40a9b30 con 0x7f57c40adf20 2023-11-24T06:27:59.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.121+0000 7f57baffd640 1 -- 172.21.15.16:0/3225477377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57ac007490 con 0x7f57c40adf20 2023-11-24T06:27:59.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.122+0000 7f57baffd640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f57a406ae40 0x7f57a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.122+0000 7f57baffd640 1 -- 172.21.15.16:0/3225477377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f57ac088c20 con 0x7f57c40adf20 2023-11-24T06:27:59.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.123+0000 7f57c259c640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f57a406ae40 0x7f57a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.124+0000 7f57c259c640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f57a406ae40 0x7f57a406d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f57b4000f60 tx=0x7f57b40079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.127+0000 7f57baffd640 1 -- 172.21.15.16:0/3225477377 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57c40a9b30 con 0x7f57c40adf20 2023-11-24T06:27:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.323+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57c40a9b30 con 0x7f57c40adf20 2023-11-24T06:27:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.324+0000 7f57baffd640 1 -- 172.21.15.16:0/3225477377 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57c40a9b30 con 0x7f57c40adf20 2023-11-24T06:27:59.326 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:27:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f57a406ae40 msgr2=0x7f57a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f57a406ae40 0x7f57a406d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f57b4000f60 tx=0x7f57b40079b0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40adf20 msgr2=0x7f57c40abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40adf20 0x7f57c40abb50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00c4d0 tx=0x7f57ac00c990 comp rx=0 tx=0).stop 2023-11-24T06:27:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 shutdown_connections 2023-11-24T06:27:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f57a406ae40 0x7f57a406d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57c40ccb50 0x7f57c41436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40adf20 0x7f57c40abb50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 --2- 172.21.15.16:0/3225477377 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57c4070a30 0x7f57c40ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 >> 172.21.15.16:0/3225477377 conn(0x7f57c406a7d0 msgr2=0x7f57c40cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 shutdown_connections 2023-11-24T06:27:59.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.327+0000 7f57c359e640 1 -- 172.21.15.16:0/3225477377 wait complete. 2023-11-24T06:27:59.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:27:59.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.575+0000 7fd3d22ad640 1 Processor -- start 2023-11-24T06:27:59.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.576+0000 7fd3d22ad640 1 -- start start 2023-11-24T06:27:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.577+0000 7fd3d22ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3cc15d4b0 0x7fd3cc0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.577+0000 7fd3d22ad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 0x7fd3cc0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.577+0000 7fd3d22ad640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0d0320 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.577+0000 7fd3d22ad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc0b2730 con 0x7fd3cc15d4b0 2023-11-24T06:27:59.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d22ad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0d1ec0 con 0x7fd3cc0b41e0 2023-11-24T06:27:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d12ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3cc15d4b0 0x7fd3cc0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d12ab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3cc15d4b0 0x7fd3cc0d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55490/0 (socket says 172.21.15.16:55490) 2023-11-24T06:27:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d12ab640 1 -- 172.21.15.16:0/4122889037 learned_addr learned my addr 172.21.15.16:0/4122889037 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d12ab640 1 -- 172.21.15.16:0/4122889037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 msgr2=0x7fd3cc0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d1aac640 1 --2- 172.21.15.16:0/4122889037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 0x7fd3cc0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d12ab640 1 --2- 172.21.15.16:0/4122889037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 0x7fd3cc0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d12ab640 1 -- 172.21.15.16:0/4122889037 >> v1:172.21.15.173:6789/0 conn(0x7fd3cc0cf9a0 legacy=0x7fd3cc0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:27:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d12ab640 1 -- 172.21.15.16:0/4122889037 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0b29d0 con 0x7fd3cc15d4b0 2023-11-24T06:27:59.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.578+0000 7fd3d1aac640 1 --2- 172.21.15.16:0/4122889037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 0x7fd3cc0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:27:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.580+0000 7fd3d12ab640 1 --2- 172.21.15.16:0/4122889037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc15d4b0 0x7fd3cc0d1550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009870 tx=0x7fd3c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d45cf328f66cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.580+0000 7fd3c3fff640 1 -- 172.21.15.16:0/4122889037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8031640 con 0x7fd3cc15d4b0 2023-11-24T06:27:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.580+0000 7fd3c3fff640 1 -- 172.21.15.16:0/4122889037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c80317a0 con 0x7fd3cc15d4b0 2023-11-24T06:27:59.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.580+0000 7fd3c3fff640 1 -- 172.21.15.16:0/4122889037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8031b30 con 0x7fd3cc15d4b0 2023-11-24T06:27:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.581+0000 7fd3d22ad640 1 -- 172.21.15.16:0/4122889037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc15d4b0 msgr2=0x7fd3cc0d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.581+0000 7fd3d22ad640 1 --2- 172.21.15.16:0/4122889037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc15d4b0 0x7fd3cc0d1550 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009870 tx=0x7fd3c802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.581+0000 7fd3d22ad640 1 -- 172.21.15.16:0/4122889037 shutdown_connections 2023-11-24T06:27:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.581+0000 7fd3d22ad640 1 --2- 172.21.15.16:0/4122889037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 0x7fd3cc0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.581+0000 7fd3d22ad640 1 --2- 172.21.15.16:0/4122889037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc15d4b0 0x7fd3cc0d1550 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.581+0000 7fd3d22ad640 1 -- 172.21.15.16:0/4122889037 >> 172.21.15.16:0/4122889037 conn(0x7fd3cc0d3000 msgr2=0x7fd3cc0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.581+0000 7fd3d22ad640 1 -- 172.21.15.16:0/4122889037 shutdown_connections 2023-11-24T06:27:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.581+0000 7fd3d22ad640 1 -- 172.21.15.16:0/4122889037 wait complete. 2023-11-24T06:27:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.583+0000 7fd3d22ad640 1 Processor -- start 2023-11-24T06:27:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.583+0000 7fd3d22ad640 1 -- start start 2023-11-24T06:27:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.584+0000 7fd3d22ad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 0x7fd3cc148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.584+0000 7fd3d22ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc0cf9a0 0x7fd3cc0a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.584+0000 7fd3d22ad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3cc15d4b0 0x7fd3cc1479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.584+0000 7fd3d22ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc15ad80 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.584+0000 7fd3d22ad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc149380 con 0x7fd3cc15d4b0 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.584+0000 7fd3d22ad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc146920 con 0x7fd3cc0b41e0 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.584+0000 7fd3d0aaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc0cf9a0 0x7fd3cc0a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.584+0000 7fd3d0aaa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc0cf9a0 0x7fd3cc0a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55494/0 (socket says 172.21.15.16:55494) 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.584+0000 7fd3d0aaa640 1 -- 172.21.15.16:0/1954079785 learned_addr learned my addr 172.21.15.16:0/1954079785 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3d12ab640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 0x7fd3cc148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3d0aaa640 1 -- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 msgr2=0x7fd3cc148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3d0aaa640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 0x7fd3cc148e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3d0aaa640 1 -- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3cc15d4b0 msgr2=0x7fd3cc1479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3d1aac640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3cc15d4b0 0x7fd3cc1479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3d0aaa640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3cc15d4b0 0x7fd3cc1479c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3d0aaa640 1 -- 172.21.15.16:0/1954079785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c80094d0 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3d0aaa640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc0cf9a0 0x7fd3cc0a0730 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c430 tx=0x7fd3bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3c1ffb640 1 -- 172.21.15.16:0/1954079785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc011790 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3c1ffb640 1 -- 172.21.15.16:0/1954079785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc0118f0 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3c1ffb640 1 -- 172.21.15.16:0/1954079785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc011ca0 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.585+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc15da10 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.586+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc146ee0 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.587+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc0d2cd0 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.588+0000 7fd3c1ffb640 1 -- 172.21.15.16:0/1954079785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3bc01c460 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.589+0000 7fd3c1ffb640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b006ae40 0x7fd3b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:27:59.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.589+0000 7fd3c1ffb640 1 -- 172.21.15.16:0/1954079785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fd3bc089680 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.590+0000 7fd3d12ab640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b006ae40 0x7fd3b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:27:59.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.591+0000 7fd3d12ab640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b006ae40 0x7fd3b006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8036000 tx=0x7fd3c803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:27:59.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.594+0000 7fd3c1ffb640 1 -- 172.21.15.16:0/1954079785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3bc054610 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.788+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3cc067450 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.789+0000 7fd3c1ffb640 1 -- 172.21.15.16:0/1954079785 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3bc057c30 con 0x7fd3cc0cf9a0 2023-11-24T06:27:59.792 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:27:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.792+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b006ae40 msgr2=0x7fd3b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.792+0000 7fd3d22ad640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b006ae40 0x7fd3b006d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8036000 tx=0x7fd3c803c040 comp rx=0 tx=0).stop 2023-11-24T06:27:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.792+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc0cf9a0 msgr2=0x7fd3cc0a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:27:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.792+0000 7fd3d22ad640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc0cf9a0 0x7fd3cc0a0730 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c430 tx=0x7fd3bc00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.793+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 shutdown_connections 2023-11-24T06:27:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.793+0000 7fd3d22ad640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b006ae40 0x7fd3b006d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.793+0000 7fd3d22ad640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3cc15d4b0 0x7fd3cc1479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.793+0000 7fd3d22ad640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3cc0cf9a0 0x7fd3cc0a0730 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.793+0000 7fd3d22ad640 1 --2- 172.21.15.16:0/1954079785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3cc0b41e0 0x7fd3cc148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:27:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.793+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 >> 172.21.15.16:0/1954079785 conn(0x7fd3cc0d3000 msgr2=0x7fd3cc0cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:27:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.793+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 shutdown_connections 2023-11-24T06:27:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:27:59.793+0000 7fd3d22ad640 1 -- 172.21.15.16:0/1954079785 wait complete. 2023-11-24T06:27:59.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.040+0000 7f132053e640 1 Processor -- start 2023-11-24T06:28:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.041+0000 7f132053e640 1 -- start start 2023-11-24T06:28:00.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.043+0000 7f132053e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13180d05d0 0x7f13180b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.043+0000 7f132053e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180b2130 con 0x7f1318144b70 2023-11-24T06:28:00.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.043+0000 7f132053e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1318144f50 con 0x7f131815b7a0 2023-11-24T06:28:00.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.043+0000 7f132053e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131815edc0 con 0x7f13180d05d0 2023-11-24T06:28:00.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.043+0000 7f131e2da640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1318144b70 0x7f131815f530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39670/0 (socket says 172.21.15.16:39670) 2023-11-24T06:28:00.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.043+0000 7f131e2da640 1 -- 172.21.15.16:0/591612810 learned_addr learned my addr 172.21.15.16:0/591612810 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:00.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.043+0000 7f131eadb640 1 --2- 172.21.15.16:0/591612810 >> v2:172.21.15.16:3300/0 conn(0x7f13180d05d0 0x7f13180b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.043+0000 7f131d2d8640 1 -- 172.21.15.16:0/591612810 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527555286 0 0) 0x7f13180b2130 con 0x7f1318144b70 2023-11-24T06:28:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.043+0000 7f131d2d8640 1 -- 172.21.15.16:0/591612810 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1304003520 con 0x7f1318144b70 2023-11-24T06:28:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.044+0000 7f131eadb640 1 -- 172.21.15.16:0/591612810 >> v1:172.21.15.173:6789/0 conn(0x7f131815b7a0 legacy=0x7f13180cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.044+0000 7f131eadb640 1 -- 172.21.15.16:0/591612810 >> v1:172.21.15.16:6789/0 conn(0x7f1318144b70 legacy=0x7f131815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.044+0000 7f131eadb640 1 -- 172.21.15.16:0/591612810 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180b2360 con 0x7f13180d05d0 2023-11-24T06:28:00.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.045+0000 7f131eadb640 1 --2- 172.21.15.16:0/591612810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13180d05d0 0x7f13180b1bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1310009d90 tx=0x7f131002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89dc04ab6ffff86 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.045+0000 7f131d2d8640 1 -- 172.21.15.16:0/591612810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131002cbd0 con 0x7f13180d05d0 2023-11-24T06:28:00.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.045+0000 7f131d2d8640 1 -- 172.21.15.16:0/591612810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131002cd30 con 0x7f13180d05d0 2023-11-24T06:28:00.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.045+0000 7f131d2d8640 1 -- 172.21.15.16:0/591612810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131002cea0 con 0x7f13180d05d0 2023-11-24T06:28:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.046+0000 7f132053e640 1 -- 172.21.15.16:0/591612810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13180d05d0 msgr2=0x7f13180b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.046+0000 7f132053e640 1 --2- 172.21.15.16:0/591612810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13180d05d0 0x7f13180b1bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1310009d90 tx=0x7f131002be20 comp rx=0 tx=0).stop 2023-11-24T06:28:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.046+0000 7f132053e640 1 -- 172.21.15.16:0/591612810 shutdown_connections 2023-11-24T06:28:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.046+0000 7f132053e640 1 --2- 172.21.15.16:0/591612810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13180d05d0 0x7f13180b1bf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.046+0000 7f132053e640 1 -- 172.21.15.16:0/591612810 >> 172.21.15.16:0/591612810 conn(0x7f131806b4f0 msgr2=0x7f131806b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.046+0000 7f132053e640 1 -- 172.21.15.16:0/591612810 shutdown_connections 2023-11-24T06:28:00.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.046+0000 7f132053e640 1 -- 172.21.15.16:0/591612810 wait complete. 2023-11-24T06:28:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.048+0000 7f132053e640 1 Processor -- start 2023-11-24T06:28:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.048+0000 7f132053e640 1 -- start start 2023-11-24T06:28:00.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.049+0000 7f132053e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13180d05d0 0x7f1318133d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.049+0000 7f132053e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318144b70 0x7f131815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.049+0000 7f132053e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131815b7a0 0x7f13180b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.049+0000 7f132053e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13181342d0 con 0x7f1318144b70 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.049+0000 7f132053e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131815b110 con 0x7f131815b7a0 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.049+0000 7f132053e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13180a8640 con 0x7f13180d05d0 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.049+0000 7f131dad9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318144b70 0x7f131815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.049+0000 7f131dad9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318144b70 0x7f131815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55500/0 (socket says 172.21.15.16:55500) 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131dad9640 1 -- 172.21.15.16:0/721159555 learned_addr learned my addr 172.21.15.16:0/721159555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131e2da640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13180d05d0 0x7f1318133d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131eadb640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131815b7a0 0x7f13180b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131dad9640 1 -- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13180d05d0 msgr2=0x7f1318133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131dad9640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13180d05d0 0x7f1318133d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131dad9640 1 -- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131815b7a0 msgr2=0x7f13180b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131dad9640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131815b7a0 0x7f13180b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131dad9640 1 -- 172.21.15.16:0/721159555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310009a70 con 0x7f1318144b70 2023-11-24T06:28:00.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131eadb640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131815b7a0 0x7f13180b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:00.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.050+0000 7f131dad9640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318144b70 0x7f131815abd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f130800b420 tx=0x7f130800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.051+0000 7f130effd640 1 -- 172.21.15.16:0/721159555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308011830 con 0x7f1318144b70 2023-11-24T06:28:00.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.051+0000 7f130effd640 1 -- 172.21.15.16:0/721159555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1308011990 con 0x7f1318144b70 2023-11-24T06:28:00.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.051+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13180a8920 con 0x7f1318144b70 2023-11-24T06:28:00.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.051+0000 7f130effd640 1 -- 172.21.15.16:0/721159555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308011d40 con 0x7f1318144b70 2023-11-24T06:28:00.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.051+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1318131670 con 0x7f1318144b70 2023-11-24T06:28:00.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.052+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131815fbe0 con 0x7f1318144b70 2023-11-24T06:28:00.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.053+0000 7f130effd640 1 -- 172.21.15.16:0/721159555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f130801b460 con 0x7f1318144b70 2023-11-24T06:28:00.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.054+0000 7f130effd640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12fc06ae40 0x7f12fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.054+0000 7f130effd640 1 -- 172.21.15.16:0/721159555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f1308089330 con 0x7f1318144b70 2023-11-24T06:28:00.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.055+0000 7f131e2da640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12fc06ae40 0x7f12fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.056+0000 7f131e2da640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12fc06ae40 0x7f12fc06d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f13140011e0 tx=0x7f1314009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.059+0000 7f130effd640 1 -- 172.21.15.16:0/721159555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13080542c0 con 0x7f1318144b70 2023-11-24T06:28:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.268+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f13180675d0 con 0x7f1318144b70 2023-11-24T06:28:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.268+0000 7f130effd640 1 -- 172.21.15.16:0/721159555 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f13080578e0 con 0x7f1318144b70 2023-11-24T06:28:00.271 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:28:00.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12fc06ae40 msgr2=0x7f12fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12fc06ae40 0x7f12fc06d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f13140011e0 tx=0x7f1314009040 comp rx=0 tx=0).stop 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318144b70 msgr2=0x7f131815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318144b70 0x7f131815abd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f130800b420 tx=0x7f130800b8e0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 shutdown_connections 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12fc06ae40 0x7f12fc06d300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131815b7a0 0x7f13180b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1318144b70 0x7f131815abd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 --2- 172.21.15.16:0/721159555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13180d05d0 0x7f1318133d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 >> 172.21.15.16:0/721159555 conn(0x7f131806b4f0 msgr2=0x7f1318142810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 shutdown_connections 2023-11-24T06:28:00.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.271+0000 7f132053e640 1 -- 172.21.15.16:0/721159555 wait complete. 2023-11-24T06:28:00.292 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:28:00.292 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:00.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:00.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.593+0000 7fce57fff640 1 Processor -- start 2023-11-24T06:28:00.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.593+0000 7fce57fff640 1 -- start start 2023-11-24T06:28:00.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.594+0000 7fce57fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce58143c40 0x7fce58071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.594+0000 7fce57fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce580717d0 0x7fce580cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.594+0000 7fce57fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fce580cca40 0x7fce5815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.594+0000 7fce57fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce580723c0 con 0x7fce58143c40 2023-11-24T06:28:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce57fff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce58070e50 con 0x7fce580cca40 2023-11-24T06:28:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce57fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce580cce70 con 0x7fce580717d0 2023-11-24T06:28:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce56ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce58143c40 0x7fce58071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce56ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce58143c40 0x7fce58071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55512/0 (socket says 172.21.15.16:55512) 2023-11-24T06:28:00.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce56ffd640 1 -- 172.21.15.16:0/2812828551 learned_addr learned my addr 172.21.15.16:0/2812828551 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce56ffd640 1 -- 172.21.15.16:0/2812828551 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce580717d0 msgr2=0x7fce580cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce56ffd640 1 --2- 172.21.15.16:0/2812828551 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce580717d0 0x7fce580cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce56ffd640 1 -- 172.21.15.16:0/2812828551 >> v2:172.21.15.173:3300/0 conn(0x7fce580cca40 msgr2=0x7fce5815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce56ffd640 1 --2- 172.21.15.16:0/2812828551 >> v2:172.21.15.173:3300/0 conn(0x7fce580cca40 0x7fce5815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.595+0000 7fce56ffd640 1 -- 172.21.15.16:0/2812828551 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58144020 con 0x7fce58143c40 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.596+0000 7fce56ffd640 1 --2- 172.21.15.16:0/2812828551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce58143c40 0x7fce58071290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce580cc3f0 tx=0x7fce4802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97524d80db9f9dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce55ffb640 1 -- 172.21.15.16:0/2812828551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4802c9e0 con 0x7fce58143c40 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce55ffb640 1 -- 172.21.15.16:0/2812828551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4802cb40 con 0x7fce58143c40 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce55ffb640 1 -- 172.21.15.16:0/2812828551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce480343f0 con 0x7fce58143c40 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce57fff640 1 -- 172.21.15.16:0/2812828551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce58143c40 msgr2=0x7fce58071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce57fff640 1 --2- 172.21.15.16:0/2812828551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce58143c40 0x7fce58071290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce580cc3f0 tx=0x7fce4802be40 comp rx=0 tx=0).stop 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce57fff640 1 -- 172.21.15.16:0/2812828551 shutdown_connections 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce57fff640 1 --2- 172.21.15.16:0/2812828551 >> v2:172.21.15.173:3300/0 conn(0x7fce580cca40 0x7fce5815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce57fff640 1 --2- 172.21.15.16:0/2812828551 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce580717d0 0x7fce580cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce57fff640 1 --2- 172.21.15.16:0/2812828551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce58143c40 0x7fce58071290 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce57fff640 1 -- 172.21.15.16:0/2812828551 >> 172.21.15.16:0/2812828551 conn(0x7fce5806a800 msgr2=0x7fce5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce57fff640 1 -- 172.21.15.16:0/2812828551 shutdown_connections 2023-11-24T06:28:00.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.597+0000 7fce57fff640 1 -- 172.21.15.16:0/2812828551 wait complete. 2023-11-24T06:28:00.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.599+0000 7fce57fff640 1 Processor -- start 2023-11-24T06:28:00.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.599+0000 7fce57fff640 1 -- start start 2023-11-24T06:28:00.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.600+0000 7fce57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce580717d0 0x7fce580c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.600+0000 7fce57fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce580cca40 0x7fce580c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.600+0000 7fce57fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce58143c40 0x7fce580c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.600+0000 7fce57fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce580c4830 con 0x7fce580717d0 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.600+0000 7fce57fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce580c4b10 con 0x7fce58143c40 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.600+0000 7fce57fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce580c4df0 con 0x7fce580cca40 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.601+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce580717d0 0x7fce580c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.601+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce580717d0 0x7fce580c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55518/0 (socket says 172.21.15.16:55518) 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.601+0000 7fce56ffd640 1 -- 172.21.15.16:0/1223862371 learned_addr learned my addr 172.21.15.16:0/1223862371 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.601+0000 7fce56ffd640 1 -- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce580cca40 msgr2=0x7fce580c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.601+0000 7fce56ffd640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce580cca40 0x7fce580c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.601+0000 7fce56ffd640 1 -- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce58143c40 msgr2=0x7fce580c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.601+0000 7fce56ffd640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce58143c40 0x7fce580c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.601+0000 7fce56ffd640 1 -- 172.21.15.16:0/1223862371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce48009b30 con 0x7fce580717d0 2023-11-24T06:28:00.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.601+0000 7fce56ffd640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce580717d0 0x7fce580c6a30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fce48000f80 tx=0x7fce48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.602+0000 7fce4f7fe640 1 -- 172.21.15.16:0/1223862371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4803e450 con 0x7fce580717d0 2023-11-24T06:28:00.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.602+0000 7fce4f7fe640 1 -- 172.21.15.16:0/1223862371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4803e5f0 con 0x7fce580717d0 2023-11-24T06:28:00.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.602+0000 7fce4f7fe640 1 -- 172.21.15.16:0/1223862371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4803e9a0 con 0x7fce580717d0 2023-11-24T06:28:00.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.602+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce580c50d0 con 0x7fce580717d0 2023-11-24T06:28:00.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.602+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce580adb10 con 0x7fce580717d0 2023-11-24T06:28:00.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.603+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce580c55f0 con 0x7fce580717d0 2023-11-24T06:28:00.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.604+0000 7fce4f7fe640 1 -- 172.21.15.16:0/1223862371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce4803eb00 con 0x7fce580717d0 2023-11-24T06:28:00.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.605+0000 7fce4f7fe640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce3806ae40 0x7fce3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:00.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.605+0000 7fce4f7fe640 1 -- 172.21.15.16:0/1223862371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fce480ac970 con 0x7fce580717d0 2023-11-24T06:28:00.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.606+0000 7fce567fc640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce3806ae40 0x7fce3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:00.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.607+0000 7fce567fc640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce3806ae40 0x7fce3806d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce5813dd80 tx=0x7fce40006bf0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:00.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.610+0000 7fce4f7fe640 1 -- 172.21.15.16:0/1223862371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce580c55f0 con 0x7fce580717d0 2023-11-24T06:28:00.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.804+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce580c55f0 con 0x7fce580717d0 2023-11-24T06:28:00.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.807+0000 7fce4f7fe640 1 -- 172.21.15.16:0/1223862371 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7fce580c55f0 con 0x7fce580717d0 2023-11-24T06:28:00.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:00.810 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:00.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.809+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce3806ae40 msgr2=0x7fce3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.809+0000 7fce57fff640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce3806ae40 0x7fce3806d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce5813dd80 tx=0x7fce40006bf0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce580717d0 msgr2=0x7fce580c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce580717d0 0x7fce580c6a30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fce48000f80 tx=0x7fce48004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 shutdown_connections 2023-11-24T06:28:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce3806ae40 0x7fce3806d300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce58143c40 0x7fce580c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce580cca40 0x7fce580c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 --2- 172.21.15.16:0/1223862371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce580717d0 0x7fce580c6a30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 >> 172.21.15.16:0/1223862371 conn(0x7fce5806a800 msgr2=0x7fce5813f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 shutdown_connections 2023-11-24T06:28:00.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:00.810+0000 7fce57fff640 1 -- 172.21.15.16:0/1223862371 wait complete. 2023-11-24T06:28:00.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:00.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.071+0000 7f6e6e2e6640 1 Processor -- start 2023-11-24T06:28:01.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.071+0000 7f6e6e2e6640 1 -- start start 2023-11-24T06:28:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.073+0000 7f6e6e2e6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e68071e40 0x7f6e680ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.073+0000 7f6e6e2e6640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6e6815d770 0x7f6e6815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.073+0000 7f6e6e2e6640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6806c730 con 0x7f6e68143810 2023-11-24T06:28:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.073+0000 7f6e6e2e6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e68143bf0 con 0x7f6e68071e40 2023-11-24T06:28:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.073+0000 7f6e6e2e6640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e680722f0 con 0x7f6e6815d770 2023-11-24T06:28:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.073+0000 7f6e6cae3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e68071e40 0x7f6e680ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.073+0000 7f6e6cae3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e68071e40 0x7f6e680ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55532/0 (socket says 172.21.15.16:55532) 2023-11-24T06:28:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.073+0000 7f6e6cae3640 1 -- 172.21.15.16:0/801889781 learned_addr learned my addr 172.21.15.16:0/801889781 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.074+0000 7f6e6dae5640 1 --2- 172.21.15.16:0/801889781 >> v2:172.21.15.173:3300/0 conn(0x7f6e6815d770 0x7f6e6815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.074+0000 7f6e6cae3640 1 -- 172.21.15.16:0/801889781 >> v2:172.21.15.173:3300/0 conn(0x7f6e6815d770 msgr2=0x7f6e6815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.074+0000 7f6e6cae3640 1 --2- 172.21.15.16:0/801889781 >> v2:172.21.15.173:3300/0 conn(0x7f6e6815d770 0x7f6e6815dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.074+0000 7f6e6cae3640 1 -- 172.21.15.16:0/801889781 >> v1:172.21.15.173:6789/0 conn(0x7f6e68143810 legacy=0x7f6e6815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.074+0000 7f6e6cae3640 1 -- 172.21.15.16:0/801889781 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e680724d0 con 0x7f6e68071e40 2023-11-24T06:28:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.075+0000 7f6e6cae3640 1 --2- 172.21.15.16:0/801889781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e68071e40 0x7f6e680ae170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6e680b4440 tx=0x7f6e5802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dc7c9b4d8200d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.075+0000 7f6e5ffff640 1 -- 172.21.15.16:0/801889781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58031750 con 0x7f6e68071e40 2023-11-24T06:28:01.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.075+0000 7f6e5ffff640 1 -- 172.21.15.16:0/801889781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e580318b0 con 0x7f6e68071e40 2023-11-24T06:28:01.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.075+0000 7f6e5ffff640 1 -- 172.21.15.16:0/801889781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e58031c40 con 0x7f6e68071e40 2023-11-24T06:28:01.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.076+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/801889781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e68071e40 msgr2=0x7f6e680ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.076+0000 7f6e6e2e6640 1 --2- 172.21.15.16:0/801889781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e68071e40 0x7f6e680ae170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6e680b4440 tx=0x7f6e5802b7f0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.076+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/801889781 shutdown_connections 2023-11-24T06:28:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.076+0000 7f6e6e2e6640 1 --2- 172.21.15.16:0/801889781 >> v2:172.21.15.173:3300/0 conn(0x7f6e6815d770 0x7f6e6815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.076+0000 7f6e6e2e6640 1 --2- 172.21.15.16:0/801889781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e68071e40 0x7f6e680ae170 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.076+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/801889781 >> 172.21.15.16:0/801889781 conn(0x7f6e6806a7d0 msgr2=0x7f6e6806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.076+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/801889781 shutdown_connections 2023-11-24T06:28:01.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.076+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/801889781 wait complete. 2023-11-24T06:28:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.078+0000 7f6e6e2e6640 1 Processor -- start 2023-11-24T06:28:01.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.078+0000 7f6e6e2e6640 1 -- start start 2023-11-24T06:28:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.079+0000 7f6e6e2e6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e68071e40 0x7f6e680b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.079+0000 7f6e6e2e6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e68143810 0x7f6e681440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.079+0000 7f6e6e2e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6815d770 0x7f6e6815f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.079+0000 7f6e6e2e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e680b1940 con 0x7f6e6815d770 2023-11-24T06:28:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.079+0000 7f6e6e2e6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e681e6dd0 con 0x7f6e68071e40 2023-11-24T06:28:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.079+0000 7f6e6e2e6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e680b10c0 con 0x7f6e68143810 2023-11-24T06:28:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.079+0000 7f6e6dae5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6815d770 0x7f6e6815f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6dae5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6815d770 0x7f6e6815f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55538/0 (socket says 172.21.15.16:55538) 2023-11-24T06:28:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6dae5640 1 -- 172.21.15.16:0/2202783702 learned_addr learned my addr 172.21.15.16:0/2202783702 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:01.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6d2e4640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e68071e40 0x7f6e680b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6dae5640 1 -- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e68143810 msgr2=0x7f6e681440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6dae5640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e68143810 0x7f6e681440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6dae5640 1 -- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e68071e40 msgr2=0x7f6e680b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6dae5640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e68071e40 0x7f6e680b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6dae5640 1 -- 172.21.15.16:0/2202783702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580094d0 con 0x7f6e6815d770 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6d2e4640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e68071e40 0x7f6e680b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e6dae5640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6815d770 0x7f6e6815f040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6e60007e60 tx=0x7f6e6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.080+0000 7f6e5dffb640 1 -- 172.21.15.16:0/2202783702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60014420 con 0x7f6e6815d770 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.081+0000 7f6e5dffb640 1 -- 172.21.15.16:0/2202783702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e600145a0 con 0x7f6e6815d770 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.081+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e680b1380 con 0x7f6e6815d770 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.081+0000 7f6e5dffb640 1 -- 172.21.15.16:0/2202783702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60014950 con 0x7f6e6815d770 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.081+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e680b0de0 con 0x7f6e6815d770 2023-11-24T06:28:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.082+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e681e6ba0 con 0x7f6e6815d770 2023-11-24T06:28:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.083+0000 7f6e5dffb640 1 -- 172.21.15.16:0/2202783702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e60014ab0 con 0x7f6e6815d770 2023-11-24T06:28:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.084+0000 7f6e5dffb640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6e4c06ae70 0x7f6e4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.084+0000 7f6e5dffb640 1 -- 172.21.15.16:0/2202783702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f6e6008a030 con 0x7f6e6815d770 2023-11-24T06:28:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.085+0000 7f6e6d2e4640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6e4c06ae70 0x7f6e4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.086+0000 7f6e6d2e4640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6e4c06ae70 0x7f6e4c06d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e64003eb0 tx=0x7f6e64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.089+0000 7f6e5dffb640 1 -- 172.21.15.16:0/2202783702 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e600bce90 con 0x7f6e6815d770 2023-11-24T06:28:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.286+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e68067450 con 0x7f6e6815d770 2023-11-24T06:28:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.287+0000 7f6e5dffb640 1 -- 172.21.15.16:0/2202783702 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e681e6ba0 con 0x7f6e6815d770 2023-11-24T06:28:01.289 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:28:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.289+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6e4c06ae70 msgr2=0x7f6e4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.289+0000 7f6e6e2e6640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6e4c06ae70 0x7f6e4c06d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6e64003eb0 tx=0x7f6e64009040 comp rx=0 tx=0).stop 2023-11-24T06:28:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.289+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6815d770 msgr2=0x7f6e6815f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.289+0000 7f6e6e2e6640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6815d770 0x7f6e6815f040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6e60007e60 tx=0x7f6e6000e4b0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.289+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 shutdown_connections 2023-11-24T06:28:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.289+0000 7f6e6e2e6640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6e4c06ae70 0x7f6e4c06d330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.290+0000 7f6e6e2e6640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e6815d770 0x7f6e6815f040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.290+0000 7f6e6e2e6640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e68143810 0x7f6e681440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.290+0000 7f6e6e2e6640 1 --2- 172.21.15.16:0/2202783702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e68071e40 0x7f6e680b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.290+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 >> 172.21.15.16:0/2202783702 conn(0x7f6e6806a7d0 msgr2=0x7f6e680b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.290+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 shutdown_connections 2023-11-24T06:28:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.290+0000 7f6e6e2e6640 1 -- 172.21.15.16:0/2202783702 wait complete. 2023-11-24T06:28:01.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.539+0000 7fb70ca39640 1 Processor -- start 2023-11-24T06:28:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.539+0000 7fb70ca39640 1 -- start start 2023-11-24T06:28:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.540+0000 7fb70ca39640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 0x7fb7080d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.540+0000 7fb70ca39640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7080b41e0 0x7fb7080cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.540+0000 7fb70ca39640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7080d0320 con 0x7fb7080cf9a0 2023-11-24T06:28:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.540+0000 7fb70ca39640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7080b2730 con 0x7fb7080b41e0 2023-11-24T06:28:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.540+0000 7fb70ca39640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080d1ec0 con 0x7fb70815d4b0 2023-11-24T06:28:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.541+0000 7fb706ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb7080cf9a0 0x7fb7080cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39682/0 (socket says 172.21.15.16:39682) 2023-11-24T06:28:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.541+0000 7fb706ffd640 1 -- 172.21.15.16:0/1653484840 learned_addr learned my addr 172.21.15.16:0/1653484840 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.541+0000 7fb707fff640 1 --2- 172.21.15.16:0/1653484840 >> v2:172.21.15.16:3300/0 conn(0x7fb7080b41e0 0x7fb7080cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.541+0000 7fb7077fe640 1 --2- 172.21.15.16:0/1653484840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 0x7fb7080d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.541+0000 7fb7067fc640 1 -- 172.21.15.16:0/1653484840 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930312260 0 0) 0x7fb7080d0320 con 0x7fb7080cf9a0 2023-11-24T06:28:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.542+0000 7fb7067fc640 1 -- 172.21.15.16:0/1653484840 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6f0003520 con 0x7fb7080cf9a0 2023-11-24T06:28:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.542+0000 7fb707fff640 1 -- 172.21.15.16:0/1653484840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 msgr2=0x7fb7080d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.542+0000 7fb707fff640 1 --2- 172.21.15.16:0/1653484840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 0x7fb7080d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.542+0000 7fb707fff640 1 -- 172.21.15.16:0/1653484840 >> v1:172.21.15.16:6789/0 conn(0x7fb7080cf9a0 legacy=0x7fb7080cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.542+0000 7fb707fff640 1 -- 172.21.15.16:0/1653484840 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7080b29d0 con 0x7fb7080b41e0 2023-11-24T06:28:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.543+0000 7fb707fff640 1 --2- 172.21.15.16:0/1653484840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080b41e0 0x7fb7080cfde0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb7000097d0 tx=0x7fb70002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5997d28c16a1e2f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.543+0000 7fb7067fc640 1 -- 172.21.15.16:0/1653484840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70002d040 con 0x7fb7080b41e0 2023-11-24T06:28:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.543+0000 7fb7067fc640 1 -- 172.21.15.16:0/1653484840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700031810 con 0x7fb7080b41e0 2023-11-24T06:28:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.543+0000 7fb7067fc640 1 -- 172.21.15.16:0/1653484840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700031b80 con 0x7fb7080b41e0 2023-11-24T06:28:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.544+0000 7fb70ca39640 1 -- 172.21.15.16:0/1653484840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080b41e0 msgr2=0x7fb7080cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.544+0000 7fb70ca39640 1 --2- 172.21.15.16:0/1653484840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080b41e0 0x7fb7080cfde0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb7000097d0 tx=0x7fb70002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.544+0000 7fb70ca39640 1 -- 172.21.15.16:0/1653484840 shutdown_connections 2023-11-24T06:28:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.544+0000 7fb70ca39640 1 --2- 172.21.15.16:0/1653484840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080b41e0 0x7fb7080cfde0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.544+0000 7fb70ca39640 1 --2- 172.21.15.16:0/1653484840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 0x7fb7080d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.544+0000 7fb70ca39640 1 -- 172.21.15.16:0/1653484840 >> 172.21.15.16:0/1653484840 conn(0x7fb7080d3000 msgr2=0x7fb7080d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.544+0000 7fb70ca39640 1 -- 172.21.15.16:0/1653484840 shutdown_connections 2023-11-24T06:28:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.544+0000 7fb70ca39640 1 -- 172.21.15.16:0/1653484840 wait complete. 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.546+0000 7fb70ca39640 1 Processor -- start 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.546+0000 7fb70ca39640 1 -- start start 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.547+0000 7fb70ca39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7080b41e0 0x7fb7080cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.547+0000 7fb70ca39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080cf9a0 0x7fb7080b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.547+0000 7fb70ca39640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 0x7fb7080b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.547+0000 7fb70ca39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080cb700 con 0x7fb7080cf9a0 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.547+0000 7fb70ca39640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70815ef80 con 0x7fb7080b41e0 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.547+0000 7fb70ca39640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080cbb50 con 0x7fb70815d4b0 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.547+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080cf9a0 0x7fb7080b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.547+0000 7fb706ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080cf9a0 0x7fb7080b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55554/0 (socket says 172.21.15.16:55554) 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.547+0000 7fb706ffd640 1 -- 172.21.15.16:0/955770426 learned_addr learned my addr 172.21.15.16:0/955770426 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb7077fe640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7080b41e0 0x7fb7080cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb707fff640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 0x7fb7080b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb706ffd640 1 -- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 msgr2=0x7fb7080b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb706ffd640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 0x7fb7080b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb706ffd640 1 -- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7080b41e0 msgr2=0x7fb7080cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb706ffd640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7080b41e0 0x7fb7080cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb706ffd640 1 -- 172.21.15.16:0/955770426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb700009400 con 0x7fb7080cf9a0 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb7077fe640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7080b41e0 0x7fb7080cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb706ffd640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080cf9a0 0x7fb7080b1000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00c460 tx=0x7fb6fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb6effff640 1 -- 172.21.15.16:0/955770426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc005ce0 con 0x7fb7080cf9a0 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb6effff640 1 -- 172.21.15.16:0/955770426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc005e40 con 0x7fb7080cf9a0 2023-11-24T06:28:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.548+0000 7fb6effff640 1 -- 172.21.15.16:0/955770426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc007290 con 0x7fb7080cf9a0 2023-11-24T06:28:01.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.549+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70815da10 con 0x7fb7080cf9a0 2023-11-24T06:28:01.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.549+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7080cc110 con 0x7fb7080cf9a0 2023-11-24T06:28:01.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.550+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb708067450 con 0x7fb7080cf9a0 2023-11-24T06:28:01.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.551+0000 7fb6effff640 1 -- 172.21.15.16:0/955770426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6fc0073f0 con 0x7fb7080cf9a0 2023-11-24T06:28:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.552+0000 7fb6effff640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb6f406ae40 0x7fb6f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.552+0000 7fb6effff640 1 -- 172.21.15.16:0/955770426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fb6fc088c60 con 0x7fb7080cf9a0 2023-11-24T06:28:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.552+0000 7fb7077fe640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb6f406ae40 0x7fb6f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:01.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.553+0000 7fb7077fe640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb6f406ae40 0x7fb6f406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8007440 tx=0x7fb6f800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:01.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.556+0000 7fb6effff640 1 -- 172.21.15.16:0/955770426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6fc053c80 con 0x7fb7080cf9a0 2023-11-24T06:28:01.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.751+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb70806c050 con 0x7fb7080cf9a0 2023-11-24T06:28:01.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.752+0000 7fb6effff640 1 -- 172.21.15.16:0/955770426 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb6fc017020 con 0x7fb7080cf9a0 2023-11-24T06:28:01.755 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:28:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.756+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb6f406ae40 msgr2=0x7fb6f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb6f406ae40 0x7fb6f406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8007440 tx=0x7fb6f800b040 comp rx=0 tx=0).stop 2023-11-24T06:28:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080cf9a0 msgr2=0x7fb7080b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080cf9a0 0x7fb7080b1000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00c460 tx=0x7fb6fc00c920 comp rx=0 tx=0).stop 2023-11-24T06:28:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 shutdown_connections 2023-11-24T06:28:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb6f406ae40 0x7fb6f406d300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb70815d4b0 0x7fb7080b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7080cf9a0 0x7fb7080b1000 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 --2- 172.21.15.16:0/955770426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7080b41e0 0x7fb7080cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 >> 172.21.15.16:0/955770426 conn(0x7fb7080d3000 msgr2=0x7fb70806ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 shutdown_connections 2023-11-24T06:28:01.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:01.757+0000 7fb70ca39640 1 -- 172.21.15.16:0/955770426 wait complete. 2023-11-24T06:28:01.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:02.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.007+0000 7f231359e640 1 Processor -- start 2023-11-24T06:28:02.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.007+0000 7f231359e640 1 -- start start 2023-11-24T06:28:02.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.009+0000 7f231359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2314070220 0x7f231406f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.009+0000 7f231359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23140d1a80 con 0x7f231406e450 2023-11-24T06:28:02.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.009+0000 7f231359e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231406f020 con 0x7f231415eaa0 2023-11-24T06:28:02.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.009+0000 7f231359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231406f300 con 0x7f2314070220 2023-11-24T06:28:02.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.009+0000 7f2311d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2314070220 0x7f231406f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.009+0000 7f2312d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f231406e450 0x7f23140af5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39686/0 (socket says 172.21.15.16:39686) 2023-11-24T06:28:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.009+0000 7f2312d9d640 1 -- 172.21.15.16:0/1459571248 learned_addr learned my addr 172.21.15.16:0/1459571248 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.010+0000 7f231159a640 1 -- 172.21.15.16:0/1459571248 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208043970 0 0) 0x7f23140d1a80 con 0x7f231406e450 2023-11-24T06:28:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.010+0000 7f231159a640 1 -- 172.21.15.16:0/1459571248 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2300003520 con 0x7f231406e450 2023-11-24T06:28:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.010+0000 7f2311d9b640 1 -- 172.21.15.16:0/1459571248 >> v1:172.21.15.173:6789/0 conn(0x7f231415eaa0 legacy=0x7f23140cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.010+0000 7f2311d9b640 1 -- 172.21.15.16:0/1459571248 >> v1:172.21.15.16:6789/0 conn(0x7f231406e450 legacy=0x7f23140af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.010+0000 7f2311d9b640 1 -- 172.21.15.16:0/1459571248 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23140d1d60 con 0x7f2314070220 2023-11-24T06:28:02.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.011+0000 7f2311d9b640 1 --2- 172.21.15.16:0/1459571248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2314070220 0x7f231406f670 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2304009d90 tx=0x7f230402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6447a07723d2f681 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.011+0000 7f231159a640 1 -- 172.21.15.16:0/1459571248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230402cc60 con 0x7f2314070220 2023-11-24T06:28:02.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.011+0000 7f231159a640 1 -- 172.21.15.16:0/1459571248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230402cdc0 con 0x7f2314070220 2023-11-24T06:28:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.011+0000 7f231159a640 1 -- 172.21.15.16:0/1459571248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304034660 con 0x7f2314070220 2023-11-24T06:28:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.012+0000 7f231359e640 1 -- 172.21.15.16:0/1459571248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2314070220 msgr2=0x7f231406f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.012+0000 7f231359e640 1 --2- 172.21.15.16:0/1459571248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2314070220 0x7f231406f670 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2304009d90 tx=0x7f230402beb0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.012+0000 7f231359e640 1 -- 172.21.15.16:0/1459571248 shutdown_connections 2023-11-24T06:28:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.012+0000 7f231359e640 1 --2- 172.21.15.16:0/1459571248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2314070220 0x7f231406f670 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.012+0000 7f231359e640 1 -- 172.21.15.16:0/1459571248 >> 172.21.15.16:0/1459571248 conn(0x7f23140d0ec0 msgr2=0x7f23140d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.012+0000 7f231359e640 1 -- 172.21.15.16:0/1459571248 shutdown_connections 2023-11-24T06:28:02.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.012+0000 7f231359e640 1 -- 172.21.15.16:0/1459571248 wait complete. 2023-11-24T06:28:02.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.014+0000 7f231359e640 1 Processor -- start 2023-11-24T06:28:02.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.014+0000 7f231359e640 1 -- start start 2023-11-24T06:28:02.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.015+0000 7f231359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f231406e450 0x7f2314158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.015+0000 7f231359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314070220 0x7f23141f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.015+0000 7f231359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f231415eaa0 0x7f23140c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.015+0000 7f231359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23140ccaf0 con 0x7f231406e450 2023-11-24T06:28:02.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.015+0000 7f231359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141f3f20 con 0x7f231415eaa0 2023-11-24T06:28:02.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.015+0000 7f231359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23141f6e10 con 0x7f2314070220 2023-11-24T06:28:02.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.015+0000 7f231259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f231406e450 0x7f2314158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.015+0000 7f231259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f231406e450 0x7f2314158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55570/0 (socket says 172.21.15.16:55570) 2023-11-24T06:28:02.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.015+0000 7f231259c640 1 -- 172.21.15.16:0/3388761014 learned_addr learned my addr 172.21.15.16:0/3388761014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:02.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f2311d9b640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314070220 0x7f23141f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f2312d9d640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f231415eaa0 0x7f23140c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f231259c640 1 -- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314070220 msgr2=0x7f23141f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f231259c640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314070220 0x7f23141f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f231259c640 1 -- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f231415eaa0 msgr2=0x7f23140c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f231259c640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f231415eaa0 0x7f23140c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f231259c640 1 -- 172.21.15.16:0/3388761014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2304009a70 con 0x7f231406e450 2023-11-24T06:28:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f2312d9d640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f231415eaa0 0x7f23140c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f231259c640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f231406e450 0x7f2314158420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00c480 tx=0x7f22fc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.016+0000 7f230affd640 1 -- 172.21.15.16:0/3388761014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0052d0 con 0x7f231406e450 2023-11-24T06:28:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.017+0000 7f230affd640 1 -- 172.21.15.16:0/3388761014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22fc005430 con 0x7f231406e450 2023-11-24T06:28:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.017+0000 7f230affd640 1 -- 172.21.15.16:0/3388761014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc012870 con 0x7f231406e450 2023-11-24T06:28:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.017+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23141f70f0 con 0x7f231406e450 2023-11-24T06:28:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.017+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23140ccfe0 con 0x7f231406e450 2023-11-24T06:28:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.018+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23140cd200 con 0x7f231406e450 2023-11-24T06:28:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.019+0000 7f230affd640 1 -- 172.21.15.16:0/3388761014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22fc0129d0 con 0x7f231406e450 2023-11-24T06:28:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.020+0000 7f230affd640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f22f406ae40 0x7f22f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.020+0000 7f230affd640 1 -- 172.21.15.16:0/3388761014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f22fc089260 con 0x7f231406e450 2023-11-24T06:28:02.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.021+0000 7f2311d9b640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f22f406ae40 0x7f22f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.022+0000 7f2311d9b640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f22f406ae40 0x7f22f406d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2304036000 tx=0x7f2304030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.024+0000 7f230affd640 1 -- 172.21.15.16:0/3388761014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22fc054280 con 0x7f231406e450 2023-11-24T06:28:02.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.221+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2314067b60 con 0x7f231406e450 2023-11-24T06:28:02.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.222+0000 7f230affd640 1 -- 172.21.15.16:0/3388761014 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f22fc0578a0 con 0x7f231406e450 2023-11-24T06:28:02.225 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:28:02.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f22f406ae40 msgr2=0x7f22f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f22f406ae40 0x7f22f406d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2304036000 tx=0x7f2304030000 comp rx=0 tx=0).stop 2023-11-24T06:28:02.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f231406e450 msgr2=0x7f2314158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f231406e450 0x7f2314158420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00c480 tx=0x7f22fc00c940 comp rx=0 tx=0).stop 2023-11-24T06:28:02.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 shutdown_connections 2023-11-24T06:28:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f22f406ae40 0x7f22f406d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f231415eaa0 0x7f23140c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2314070220 0x7f23141f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 --2- 172.21.15.16:0/3388761014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f231406e450 0x7f2314158420 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 >> 172.21.15.16:0/3388761014 conn(0x7f23140d0ec0 msgr2=0x7f23140c3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 shutdown_connections 2023-11-24T06:28:02.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.225+0000 7f231359e640 1 -- 172.21.15.16:0/3388761014 wait complete. 2023-11-24T06:28:02.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.482+0000 7fca9a839640 1 Processor -- start 2023-11-24T06:28:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.483+0000 7fca9a839640 1 -- start start 2023-11-24T06:28:02.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.484+0000 7fca9a839640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 0x7fca940b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.485+0000 7fca9a839640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca940d0a10 con 0x7fca940d0f40 2023-11-24T06:28:02.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.485+0000 7fca9a839640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca940cf080 con 0x7fca94160920 2023-11-24T06:28:02.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.485+0000 7fca9a839640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca940c6be0 con 0x7fca940d0100 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.485+0000 7fca9a038640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fca940d0f40 0x7fca940d13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39694/0 (socket says 172.21.15.16:39694) 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.485+0000 7fca9a038640 1 -- 172.21.15.16:0/3171412524 learned_addr learned my addr 172.21.15.16:0/3171412524 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.485+0000 7fca99036640 1 --2- 172.21.15.16:0/3171412524 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 0x7fca940b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.486+0000 7fca98835640 1 -- 172.21.15.16:0/3171412524 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155742927 0 0) 0x7fca940d0a10 con 0x7fca940d0f40 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.486+0000 7fca98835640 1 -- 172.21.15.16:0/3171412524 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca80003520 con 0x7fca940d0f40 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.486+0000 7fca98835640 1 -- 172.21.15.16:0/3171412524 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1380719632 0 0) 0x7fca80003520 con 0x7fca940d0f40 2023-11-24T06:28:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.486+0000 7fca99036640 1 -- 172.21.15.16:0/3171412524 >> v1:172.21.15.173:6789/0 conn(0x7fca94160920 legacy=0x7fca940cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.486+0000 7fca99036640 1 -- 172.21.15.16:0/3171412524 >> v1:172.21.15.16:6789/0 conn(0x7fca940d0f40 legacy=0x7fca940d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.486+0000 7fca99036640 1 -- 172.21.15.16:0/3171412524 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca940ce7e0 con 0x7fca940d0100 2023-11-24T06:28:02.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.487+0000 7fca99036640 1 --2- 172.21.15.16:0/3171412524 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 0x7fca940b3f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca840097d0 tx=0x7fca8402b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca280ff66fc3209b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.488+0000 7fca98835640 1 -- 172.21.15.16:0/3171412524 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0019f0 con 0x7fca940d0100 2023-11-24T06:28:02.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.488+0000 7fca98835640 1 -- 172.21.15.16:0/3171412524 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8402e050 con 0x7fca940d0100 2023-11-24T06:28:02.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.488+0000 7fca98835640 1 -- 172.21.15.16:0/3171412524 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84030730 con 0x7fca940d0100 2023-11-24T06:28:02.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.489+0000 7fca9a839640 1 -- 172.21.15.16:0/3171412524 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 msgr2=0x7fca940b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.489+0000 7fca9a839640 1 --2- 172.21.15.16:0/3171412524 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 0x7fca940b3f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fca840097d0 tx=0x7fca8402b660 comp rx=0 tx=0).stop 2023-11-24T06:28:02.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.489+0000 7fca9a839640 1 -- 172.21.15.16:0/3171412524 shutdown_connections 2023-11-24T06:28:02.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.489+0000 7fca9a839640 1 --2- 172.21.15.16:0/3171412524 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 0x7fca940b3f30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.489+0000 7fca9a839640 1 -- 172.21.15.16:0/3171412524 >> 172.21.15.16:0/3171412524 conn(0x7fca940d3090 msgr2=0x7fca940d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.489+0000 7fca9a839640 1 -- 172.21.15.16:0/3171412524 shutdown_connections 2023-11-24T06:28:02.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.489+0000 7fca9a839640 1 -- 172.21.15.16:0/3171412524 wait complete. 2023-11-24T06:28:02.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.491+0000 7fca9a839640 1 Processor -- start 2023-11-24T06:28:02.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.492+0000 7fca9a839640 1 -- start start 2023-11-24T06:28:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.493+0000 7fca9a839640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 0x7fca940cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.493+0000 7fca9a839640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940d0f40 0x7fca941f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.493+0000 7fca9a839640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca94160920 0x7fca940b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.493+0000 7fca9a839640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca940b75a0 con 0x7fca940d0f40 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.493+0000 7fca9a839640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca940b7880 con 0x7fca94160920 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.493+0000 7fca9a839640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca940b7b60 con 0x7fca940d0100 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.493+0000 7fca99036640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940d0f40 0x7fca941f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.493+0000 7fca99036640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940d0f40 0x7fca941f24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55578/0 (socket says 172.21.15.16:55578) 2023-11-24T06:28:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.493+0000 7fca99036640 1 -- 172.21.15.16:0/2073483446 learned_addr learned my addr 172.21.15.16:0/2073483446 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.494+0000 7fca9a038640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca94160920 0x7fca940b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.494+0000 7fca99837640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 0x7fca940cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.494+0000 7fca99036640 1 -- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 msgr2=0x7fca940cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.494+0000 7fca99036640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 0x7fca940cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.494+0000 7fca99036640 1 -- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca94160920 msgr2=0x7fca940b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.494+0000 7fca99036640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca94160920 0x7fca940b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.494+0000 7fca99036640 1 -- 172.21.15.16:0/2073483446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca84009400 con 0x7fca940d0f40 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.494+0000 7fca99036640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940d0f40 0x7fca941f24d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fca84000c00 tx=0x7fca840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.495+0000 7fca8a7fc640 1 -- 172.21.15.16:0/2073483446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84004230 con 0x7fca940d0f40 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.495+0000 7fca8a7fc640 1 -- 172.21.15.16:0/2073483446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca84004390 con 0x7fca940d0f40 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.495+0000 7fca8a7fc640 1 -- 172.21.15.16:0/2073483446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8403c8c0 con 0x7fca940d0f40 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.495+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca940b7e40 con 0x7fca940d0f40 2023-11-24T06:28:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.495+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca941f2a10 con 0x7fca940d0f40 2023-11-24T06:28:02.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.496+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca94067450 con 0x7fca940d0f40 2023-11-24T06:28:02.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.497+0000 7fca8a7fc640 1 -- 172.21.15.16:0/2073483446 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca84044030 con 0x7fca940d0f40 2023-11-24T06:28:02.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.498+0000 7fca8a7fc640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fca7806ae70 0x7fca7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:02.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.498+0000 7fca8a7fc640 1 -- 172.21.15.16:0/2073483446 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fca84042070 con 0x7fca940d0f40 2023-11-24T06:28:02.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.499+0000 7fca99837640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fca7806ae70 0x7fca7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:02.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.500+0000 7fca99837640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fca7806ae70 0x7fca7806d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca90000f60 tx=0x7fca90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:02.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.502+0000 7fca8a7fc640 1 -- 172.21.15.16:0/2073483446 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca840dde90 con 0x7fca940d0f40 2023-11-24T06:28:02.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.710+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fca9408c7e0 con 0x7fca940d0f40 2023-11-24T06:28:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.711+0000 7fca8a7fc640 1 -- 172.21.15.16:0/2073483446 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fca84037080 con 0x7fca940d0f40 2023-11-24T06:28:02.713 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:28:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fca7806ae70 msgr2=0x7fca7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fca7806ae70 0x7fca7806d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca90000f60 tx=0x7fca90009040 comp rx=0 tx=0).stop 2023-11-24T06:28:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940d0f40 msgr2=0x7fca941f24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940d0f40 0x7fca941f24d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fca84000c00 tx=0x7fca840046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 shutdown_connections 2023-11-24T06:28:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fca7806ae70 0x7fca7806d330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca94160920 0x7fca940b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940d0f40 0x7fca941f24d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 --2- 172.21.15.16:0/2073483446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca940d0100 0x7fca940cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 >> 172.21.15.16:0/2073483446 conn(0x7fca940d3090 msgr2=0x7fca9406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 shutdown_connections 2023-11-24T06:28:02.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:02.713+0000 7fca9a839640 1 -- 172.21.15.16:0/2073483446 wait complete. 2023-11-24T06:28:02.734 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi173.front.sepia.ceph.com... 2023-11-24T06:28:02.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:02.734 DEBUG:teuthology.orchestra.run.smithi173:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-24T06:28:02.839 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:02.839 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:28:02.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:02.839 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:28:02.888 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:28:02.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.122+0000 7f85e6b7c640 1 Processor -- start 2023-11-24T06:28:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.123+0000 7f85e6b7c640 1 -- start start 2023-11-24T06:28:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.124+0000 7f85e6b7c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f85e013df80 0x7f85e015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.124+0000 7f85e6b7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85e00cf9a0 0x7f85e00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.124+0000 7f85e6b7c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e006c470 con 0x7f85e013e360 2023-11-24T06:28:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e6b7c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e006c750 con 0x7f85e00cf9a0 2023-11-24T06:28:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e6b7c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e006ca30 con 0x7f85e013df80 2023-11-24T06:28:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e5119640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85e00cf9a0 0x7f85e00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e5119640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85e00cf9a0 0x7f85e00cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39308/0 (socket says 172.21.15.16:39308) 2023-11-24T06:28:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e5119640 1 -- 172.21.15.16:0/3459568364 learned_addr learned my addr 172.21.15.16:0/3459568364 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e4918640 1 --2- 172.21.15.16:0/3459568364 >> v2:172.21.15.173:3300/0 conn(0x7f85e013df80 0x7f85e015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e5119640 1 -- 172.21.15.16:0/3459568364 >> v2:172.21.15.173:3300/0 conn(0x7f85e013df80 msgr2=0x7f85e015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e5119640 1 --2- 172.21.15.16:0/3459568364 >> v2:172.21.15.173:3300/0 conn(0x7f85e013df80 0x7f85e015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e5119640 1 -- 172.21.15.16:0/3459568364 >> v1:172.21.15.173:6789/0 conn(0x7f85e013e360 legacy=0x7f85e00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.125+0000 7f85e5119640 1 -- 172.21.15.16:0/3459568364 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e006cd10 con 0x7f85e00cf9a0 2023-11-24T06:28:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.127+0000 7f85e5119640 1 --2- 172.21.15.16:0/3459568364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e00cf9a0 0x7f85e00cd840 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f85e0072770 tx=0x7f85d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9314825202c846f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.127+0000 7f85df7fe640 1 -- 172.21.15.16:0/3459568364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8031870 con 0x7f85e00cf9a0 2023-11-24T06:28:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.127+0000 7f85df7fe640 1 -- 172.21.15.16:0/3459568364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d80319d0 con 0x7f85e00cf9a0 2023-11-24T06:28:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.127+0000 7f85df7fe640 1 -- 172.21.15.16:0/3459568364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8031d60 con 0x7f85e00cf9a0 2023-11-24T06:28:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.128+0000 7f85e6b7c640 1 -- 172.21.15.16:0/3459568364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e00cf9a0 msgr2=0x7f85e00cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.128+0000 7f85e6b7c640 1 --2- 172.21.15.16:0/3459568364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e00cf9a0 0x7f85e00cd840 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f85e0072770 tx=0x7f85d802b910 comp rx=0 tx=0).stop 2023-11-24T06:28:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.128+0000 7f85e6b7c640 1 -- 172.21.15.16:0/3459568364 shutdown_connections 2023-11-24T06:28:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.128+0000 7f85e6b7c640 1 --2- 172.21.15.16:0/3459568364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e00cf9a0 0x7f85e00cd840 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.128+0000 7f85e6b7c640 1 --2- 172.21.15.16:0/3459568364 >> v2:172.21.15.173:3300/0 conn(0x7f85e013df80 0x7f85e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.128+0000 7f85e6b7c640 1 -- 172.21.15.16:0/3459568364 >> 172.21.15.16:0/3459568364 conn(0x7f85e006a800 msgr2=0x7f85e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.128+0000 7f85e6b7c640 1 -- 172.21.15.16:0/3459568364 shutdown_connections 2023-11-24T06:28:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.128+0000 7f85e6b7c640 1 -- 172.21.15.16:0/3459568364 wait complete. 2023-11-24T06:28:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.130+0000 7f85e6b7c640 1 Processor -- start 2023-11-24T06:28:03.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.130+0000 7f85e6b7c640 1 -- start start 2023-11-24T06:28:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.131+0000 7f85e6b7c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85e00cf9a0 0x7f85e01f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.131+0000 7f85e6b7c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85e013df80 0x7f85e01342c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.131+0000 7f85e6b7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e013e360 0x7f85e00a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.131+0000 7f85e6b7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0134fd0 con 0x7f85e013e360 2023-11-24T06:28:03.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.131+0000 7f85e6b7c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0133220 con 0x7f85e013df80 2023-11-24T06:28:03.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.131+0000 7f85e6b7c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e0133500 con 0x7f85e00cf9a0 2023-11-24T06:28:03.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.131+0000 7f85e5119640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e013e360 0x7f85e00a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.131+0000 7f85e5119640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e013e360 0x7f85e00a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39310/0 (socket says 172.21.15.16:39310) 2023-11-24T06:28:03.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.131+0000 7f85e5119640 1 -- 172.21.15.16:0/142546225 learned_addr learned my addr 172.21.15.16:0/142546225 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85dffff640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85e013df80 0x7f85e01342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85e4918640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85e00cf9a0 0x7f85e01f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85e5119640 1 -- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85e00cf9a0 msgr2=0x7f85e01f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85e5119640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85e00cf9a0 0x7f85e01f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85e5119640 1 -- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85e013df80 msgr2=0x7f85e01342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85e5119640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85e013df80 0x7f85e01342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85e5119640 1 -- 172.21.15.16:0/142546225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d80094d0 con 0x7f85e013e360 2023-11-24T06:28:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85dffff640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85e013df80 0x7f85e01342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85e5119640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e013e360 0x7f85e00a84c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f85d80090b0 tx=0x7f85d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85dd7fa640 1 -- 172.21.15.16:0/142546225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d803ac60 con 0x7f85e013e360 2023-11-24T06:28:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85dd7fa640 1 -- 172.21.15.16:0/142546225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d803adc0 con 0x7f85e013e360 2023-11-24T06:28:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85dd7fa640 1 -- 172.21.15.16:0/142546225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8044630 con 0x7f85e013e360 2023-11-24T06:28:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.132+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e01337e0 con 0x7f85e013e360 2023-11-24T06:28:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.133+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e0133da0 con 0x7f85e013e360 2023-11-24T06:28:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.134+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e0133f90 con 0x7f85e013e360 2023-11-24T06:28:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.135+0000 7f85dd7fa640 1 -- 172.21.15.16:0/142546225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85d804d450 con 0x7f85e013e360 2023-11-24T06:28:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.135+0000 7f85dd7fa640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f85c006ae40 0x7f85c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.136+0000 7f85dd7fa640 1 -- 172.21.15.16:0/142546225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f85d802d040 con 0x7f85e013e360 2023-11-24T06:28:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.136+0000 7f85e4918640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f85c006ae40 0x7f85c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.137+0000 7f85e4918640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f85c006ae40 0x7f85c006d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009f30 tx=0x7f85d000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.140+0000 7f85dd7fa640 1 -- 172.21.15.16:0/142546225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85d8076c40 con 0x7f85e013e360 2023-11-24T06:28:03.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.375+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85e00675d0 con 0x7f85e013e360 2023-11-24T06:28:03.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.376+0000 7f85dd7fa640 1 -- 172.21.15.16:0/142546225 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f85d8035020 con 0x7f85e013e360 2023-11-24T06:28:03.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:03.378 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3019663874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3019663874},{"type":"v1","addr":"172.21.15.173:6837","nonce":3019663874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2239024365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2239024365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2239024365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:26:56.460479+0000","modified":"2023-11-24T06:26:59.516602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/915142029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":915142029},{"type":"v1","addr":"172.21.15.173:6835","nonce":915142029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:28:03.379 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-11-24T06:28:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f85c006ae40 msgr2=0x7f85c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f85c006ae40 0x7f85c006d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009f30 tx=0x7f85d000b040 comp rx=0 tx=0).stop 2023-11-24T06:28:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e013e360 msgr2=0x7f85e00a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e013e360 0x7f85e00a84c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f85d80090b0 tx=0x7f85d80046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 shutdown_connections 2023-11-24T06:28:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f85c006ae40 0x7f85c006d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85e013e360 0x7f85e00a84c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85e013df80 0x7f85e01342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 --2- 172.21.15.16:0/142546225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85e00cf9a0 0x7f85e01f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 >> 172.21.15.16:0/142546225 conn(0x7f85e006a800 msgr2=0x7f85e00702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 shutdown_connections 2023-11-24T06:28:03.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.379+0000 7f85e6b7c640 1 -- 172.21.15.16:0/142546225 wait complete. 2023-11-24T06:28:03.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.625+0000 7fbea1c64640 1 Processor -- start 2023-11-24T06:28:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.626+0000 7fbea1c64640 1 -- start start 2023-11-24T06:28:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.627+0000 7fbea1c64640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe9c160920 0x7fbe9c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.627+0000 7fbea1c64640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbe9c0cf9f0 0x7fbe9c0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.627+0000 7fbea1c64640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c161010 0x7fbe9c1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.627+0000 7fbea1c64640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c0d03e0 con 0x7fbe9c160920 2023-11-24T06:28:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.627+0000 7fbea1c64640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c0d0a10 con 0x7fbe9c0cf9f0 2023-11-24T06:28:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.627+0000 7fbea1c64640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0cf080 con 0x7fbe9c161010 2023-11-24T06:28:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.627+0000 7fbea0c62640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe9c160920 0x7fbe9c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbea0c62640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe9c160920 0x7fbe9c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39324/0 (socket says 172.21.15.16:39324) 2023-11-24T06:28:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbea0c62640 1 -- 172.21.15.16:0/3394812931 learned_addr learned my addr 172.21.15.16:0/3394812931 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbe9bfff640 1 --2- 172.21.15.16:0/3394812931 >> v2:172.21.15.173:3300/0 conn(0x7fbe9c0cf9f0 0x7fbe9c0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbea0c62640 1 -- 172.21.15.16:0/3394812931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c161010 msgr2=0x7fbe9c1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbea1463640 1 --2- 172.21.15.16:0/3394812931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c161010 0x7fbe9c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbea0c62640 1 --2- 172.21.15.16:0/3394812931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c161010 0x7fbe9c1614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbea0c62640 1 -- 172.21.15.16:0/3394812931 >> v2:172.21.15.173:3300/0 conn(0x7fbe9c0cf9f0 msgr2=0x7fbe9c0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbea0c62640 1 --2- 172.21.15.16:0/3394812931 >> v2:172.21.15.173:3300/0 conn(0x7fbe9c0cf9f0 0x7fbe9c0cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbea0c62640 1 -- 172.21.15.16:0/3394812931 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c0c6be0 con 0x7fbe9c160920 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.628+0000 7fbe9bfff640 1 --2- 172.21.15.16:0/3394812931 >> v2:172.21.15.173:3300/0 conn(0x7fbe9c0cf9f0 0x7fbe9c0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.629+0000 7fbea0c62640 1 --2- 172.21.15.16:0/3394812931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 0x7fbe9c0b3f30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c08c1f0 tx=0x7fbe9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e865467a805acf6d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbe9b7fe640 1 -- 172.21.15.16:0/3394812931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90031870 con 0x7fbe9c160920 2023-11-24T06:28:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbe9b7fe640 1 -- 172.21.15.16:0/3394812931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe900319d0 con 0x7fbe9c160920 2023-11-24T06:28:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbe9b7fe640 1 -- 172.21.15.16:0/3394812931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90031d60 con 0x7fbe9c160920 2023-11-24T06:28:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbea1c64640 1 -- 172.21.15.16:0/3394812931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 msgr2=0x7fbe9c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbea1c64640 1 --2- 172.21.15.16:0/3394812931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 0x7fbe9c0b3f30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c08c1f0 tx=0x7fbe9002b910 comp rx=0 tx=0).stop 2023-11-24T06:28:03.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbea1c64640 1 -- 172.21.15.16:0/3394812931 shutdown_connections 2023-11-24T06:28:03.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbea1c64640 1 --2- 172.21.15.16:0/3394812931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c161010 0x7fbe9c1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbea1c64640 1 --2- 172.21.15.16:0/3394812931 >> v2:172.21.15.173:3300/0 conn(0x7fbe9c0cf9f0 0x7fbe9c0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbea1c64640 1 --2- 172.21.15.16:0/3394812931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 0x7fbe9c0b3f30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbea1c64640 1 -- 172.21.15.16:0/3394812931 >> 172.21.15.16:0/3394812931 conn(0x7fbe9c0d3090 msgr2=0x7fbe9c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:03.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbea1c64640 1 -- 172.21.15.16:0/3394812931 shutdown_connections 2023-11-24T06:28:03.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.630+0000 7fbea1c64640 1 -- 172.21.15.16:0/3394812931 wait complete. 2023-11-24T06:28:03.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.632+0000 7fbea1c64640 1 Processor -- start 2023-11-24T06:28:03.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.632+0000 7fbea1c64640 1 -- start start 2023-11-24T06:28:03.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.633+0000 7fbea1c64640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c0cf9f0 0x7fbe9c1f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.633+0000 7fbea1c64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 0x7fbe9c155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.633+0000 7fbea1c64640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe9c161010 0x7fbe9c0b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.633+0000 7fbea1c64640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c155e40 con 0x7fbe9c160920 2023-11-24T06:28:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.633+0000 7fbea1c64640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0cb4a0 con 0x7fbe9c161010 2023-11-24T06:28:03.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.633+0000 7fbea1c64640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0b6eb0 con 0x7fbe9c0cf9f0 2023-11-24T06:28:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.634+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 0x7fbe9c155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.634+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 0x7fbe9c155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39328/0 (socket says 172.21.15.16:39328) 2023-11-24T06:28:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.634+0000 7fbe9bfff640 1 -- 172.21.15.16:0/543966502 learned_addr learned my addr 172.21.15.16:0/543966502 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.634+0000 7fbea0c62640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c0cf9f0 0x7fbe9c1f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.634+0000 7fbea1463640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe9c161010 0x7fbe9c0b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.634+0000 7fbe9bfff640 1 -- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c0cf9f0 msgr2=0x7fbe9c1f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.634+0000 7fbe9bfff640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c0cf9f0 0x7fbe9c1f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.634+0000 7fbe9bfff640 1 -- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe9c161010 msgr2=0x7fbe9c0b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbe9bfff640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe9c161010 0x7fbe9c0b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbe9bfff640 1 -- 172.21.15.16:0/543966502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe900094d0 con 0x7fbe9c160920 2023-11-24T06:28:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbea0c62640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c0cf9f0 0x7fbe9c1f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbea1463640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe9c161010 0x7fbe9c0b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbe9bfff640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 0x7fbe9c155900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800c7f0 tx=0x7fbe8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbe997fa640 1 -- 172.21.15.16:0/543966502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88007820 con 0x7fbe9c160920 2023-11-24T06:28:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbe997fa640 1 -- 172.21.15.16:0/543966502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe88007980 con 0x7fbe9c160920 2023-11-24T06:28:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c0b7190 con 0x7fbe9c160920 2023-11-24T06:28:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbe997fa640 1 -- 172.21.15.16:0/543966502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88007d30 con 0x7fbe9c160920 2023-11-24T06:28:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.635+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c0b7750 con 0x7fbe9c160920 2023-11-24T06:28:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.637+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c067450 con 0x7fbe9c160920 2023-11-24T06:28:03.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.638+0000 7fbe997fa640 1 -- 172.21.15.16:0/543966502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe880113f0 con 0x7fbe9c160920 2023-11-24T06:28:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.639+0000 7fbe997fa640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe8006ae40 0x7fbe8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.639+0000 7fbe997fa640 1 -- 172.21.15.16:0/543966502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fbe88089990 con 0x7fbe9c160920 2023-11-24T06:28:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.639+0000 7fbea0c62640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe8006ae40 0x7fbe8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:03.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.640+0000 7fbea0c62640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe8006ae40 0x7fbe8006d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbe90036000 tx=0x7fbe9003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:03.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.643+0000 7fbe997fa640 1 -- 172.21.15.16:0/543966502 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbe880549d0 con 0x7fbe9c160920 2023-11-24T06:28:03.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.836+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe9c15c4d0 con 0x7fbe9c160920 2023-11-24T06:28:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.839+0000 7fbe997fa640 1 -- 172.21.15.16:0/543966502 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7fbe88057ff0 con 0x7fbe9c160920 2023-11-24T06:28:03.841 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:03.842 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe8006ae40 msgr2=0x7fbe8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe8006ae40 0x7fbe8006d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbe90036000 tx=0x7fbe9003c040 comp rx=0 tx=0).stop 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 msgr2=0x7fbe9c155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 0x7fbe9c155900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800c7f0 tx=0x7fbe8800ccb0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 shutdown_connections 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe8006ae40 0x7fbe8006d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe9c161010 0x7fbe9c0b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe9c160920 0x7fbe9c155900 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 --2- 172.21.15.16:0/543966502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe9c0cf9f0 0x7fbe9c1f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 >> 172.21.15.16:0/543966502 conn(0x7fbe9c0d3090 msgr2=0x7fbe9c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 shutdown_connections 2023-11-24T06:28:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:03.842+0000 7fbea1c64640 1 -- 172.21.15.16:0/543966502 wait complete. 2023-11-24T06:28:03.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:26:54.688269+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:26:55.486606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:03.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:04.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.090+0000 7fe9631e5640 1 Processor -- start 2023-11-24T06:28:04.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.090+0000 7fe9631e5640 1 -- start start 2023-11-24T06:28:04.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.091+0000 7fe9631e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c15e880 0x7fe95c06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.091+0000 7fe9631e5640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe95c070490 0x7fe95c099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.091+0000 7fe9631e5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c0cfa70 con 0x7fe95c099fa0 2023-11-24T06:28:04.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.091+0000 7fe9631e5640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c09a410 con 0x7fe95c070490 2023-11-24T06:28:04.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.092+0000 7fe9631e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0cff00 con 0x7fe95c15e880 2023-11-24T06:28:04.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.092+0000 7fe961782640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe95c099fa0 0x7fe95c0cbb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51294/0 (socket says 172.21.15.16:51294) 2023-11-24T06:28:04.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.092+0000 7fe961782640 1 -- 172.21.15.16:0/407548282 learned_addr learned my addr 172.21.15.16:0/407548282 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.092+0000 7fe960f81640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c15e880 0x7fe95c06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.092+0000 7fe95bfff640 1 --2- 172.21.15.16:0/407548282 >> v2:172.21.15.173:3300/0 conn(0x7fe95c070490 0x7fe95c099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.092+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746495329 0 0) 0x7fe95c0cfa70 con 0x7fe95c099fa0 2023-11-24T06:28:04.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.092+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe948003520 con 0x7fe95c099fa0 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3634781056 0 0) 0x7fe948003520 con 0x7fe95c099fa0 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c15e880 msgr2=0x7fe95c06ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c15e880 0x7fe95c06ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 >> v2:172.21.15.173:3300/0 conn(0x7fe95c070490 msgr2=0x7fe95c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 --2- 172.21.15.16:0/407548282 >> v2:172.21.15.173:3300/0 conn(0x7fe95c070490 0x7fe95c099a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c06fc70 con 0x7fe95c099fa0 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fe954001670 con 0x7fe95c099fa0 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 >> v1:172.21.15.16:6789/0 conn(0x7fe95c099fa0 legacy=0x7fe95c0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c15e880 0x7fe948003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe948003f40 0x7fe9480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948004d00 0x7fe948005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c09a410 con 0x7fe948003f40 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0cff00 con 0x7fe948004d00 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94800ba00 con 0x7fe95c15e880 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.093+0000 7fe95bfff640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe948003f40 0x7fe9480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.094+0000 7fe960f81640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c15e880 0x7fe948003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.094+0000 7fe961782640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948004d00 0x7fe948005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.094+0000 7fe95bfff640 1 -- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c15e880 msgr2=0x7fe948003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.094+0000 7fe95bfff640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c15e880 0x7fe948003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.094+0000 7fe95bfff640 1 -- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948004d00 msgr2=0x7fe948005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.094+0000 7fe95bfff640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948004d00 0x7fe948005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.094+0000 7fe95bfff640 1 -- 172.21.15.16:0/407548282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948003520 con 0x7fe948003f40 2023-11-24T06:28:04.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.094+0000 7fe961782640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948004d00 0x7fe948005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.095+0000 7fe95bfff640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe948003f40 0x7fe9480047c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe944009ac0 tx=0x7fe9480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13638f048cf6fa57 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.095+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94402c710 con 0x7fe948003f40 2023-11-24T06:28:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.095+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94402c870 con 0x7fe948003f40 2023-11-24T06:28:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.095+0000 7fe95b7fe640 1 -- 172.21.15.16:0/407548282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94402cc00 con 0x7fe948003f40 2023-11-24T06:28:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.096+0000 7fe9631e5640 1 -- 172.21.15.16:0/407548282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe948003f40 msgr2=0x7fe9480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.096+0000 7fe9631e5640 1 --2- 172.21.15.16:0/407548282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe948003f40 0x7fe9480047c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe944009ac0 tx=0x7fe9480057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.096+0000 7fe961782640 1 -- 172.21.15.16:0/407548282 reap_dead start 2023-11-24T06:28:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.096+0000 7fe9631e5640 1 -- 172.21.15.16:0/407548282 shutdown_connections 2023-11-24T06:28:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.096+0000 7fe9631e5640 1 -- 172.21.15.16:0/407548282 >> 172.21.15.16:0/407548282 conn(0x7fe95c0b2d00 msgr2=0x7fe95c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.096+0000 7fe9631e5640 1 -- 172.21.15.16:0/407548282 shutdown_connections 2023-11-24T06:28:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.096+0000 7fe9631e5640 1 -- 172.21.15.16:0/407548282 wait complete. 2023-11-24T06:28:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.098+0000 7fe9631e5640 1 Processor -- start 2023-11-24T06:28:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.098+0000 7fe9631e5640 1 -- start start 2023-11-24T06:28:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.099+0000 7fe9631e5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948003f40 0x7fe95c06b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.099+0000 7fe9631e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95c1f3ae0 0x7fe95c0ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.099+0000 7fe9631e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c0cd410 0x7fe95c0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.099+0000 7fe9631e5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c06c490 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.099+0000 7fe9631e5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0cddd0 con 0x7fe948003f40 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.099+0000 7fe9631e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c06bbf0 con 0x7fe95c0cd410 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.099+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95c1f3ae0 0x7fe95c0ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.099+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95c1f3ae0 0x7fe95c0ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39346/0 (socket says 172.21.15.16:39346) 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.099+0000 7fe95bfff640 1 -- 172.21.15.16:0/327739625 learned_addr learned my addr 172.21.15.16:0/327739625 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.100+0000 7fe960f81640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948003f40 0x7fe95c06b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.100+0000 7fe961782640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c0cd410 0x7fe95c0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.100+0000 7fe95bfff640 1 -- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c0cd410 msgr2=0x7fe95c0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.100+0000 7fe95bfff640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c0cd410 0x7fe95c0cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.100+0000 7fe95bfff640 1 -- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948003f40 msgr2=0x7fe95c06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.100+0000 7fe95bfff640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948003f40 0x7fe95c06b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.100+0000 7fe95bfff640 1 -- 172.21.15.16:0/327739625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c06bed0 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.100+0000 7fe95bfff640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95c1f3ae0 0x7fe95c0ca7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe94402c2b0 tx=0x7fe944004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.101+0000 7fe9597fa640 1 -- 172.21.15.16:0/327739625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944003810 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.101+0000 7fe9597fa640 1 -- 172.21.15.16:0/327739625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94402de90 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.101+0000 7fe9597fa640 1 -- 172.21.15.16:0/327739625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944003b70 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.101+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c06c1b0 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.101+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0cace0 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.102+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c0d0fa0 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.103+0000 7fe9597fa640 1 -- 172.21.15.16:0/327739625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe944003cd0 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.104+0000 7fe9597fa640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe93c06ae70 0x7fe93c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.105+0000 7fe9597fa640 1 -- 172.21.15.16:0/327739625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fe9440ab610 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.105+0000 7fe960f81640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe93c06ae70 0x7fe93c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.106+0000 7fe960f81640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe93c06ae70 0x7fe93c06d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe95c1f6600 tx=0x7fe94c00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.109+0000 7fe9597fa640 1 -- 172.21.15.16:0/327739625 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9440765a0 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.310+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe95c0675d0 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.311+0000 7fe9597fa640 1 -- 172.21.15.16:0/327739625 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe944079bc0 con 0x7fe95c1f3ae0 2023-11-24T06:28:04.313 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:28:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe93c06ae70 msgr2=0x7fe93c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe93c06ae70 0x7fe93c06d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe95c1f6600 tx=0x7fe94c00b040 comp rx=0 tx=0).stop 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95c1f3ae0 msgr2=0x7fe95c0ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95c1f3ae0 0x7fe95c0ca7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe94402c2b0 tx=0x7fe944004730 comp rx=0 tx=0).stop 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 shutdown_connections 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe93c06ae70 0x7fe93c06d330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe95c0cd410 0x7fe95c0cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95c1f3ae0 0x7fe95c0ca7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 --2- 172.21.15.16:0/327739625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe948003f40 0x7fe95c06b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 >> 172.21.15.16:0/327739625 conn(0x7fe95c0b2d00 msgr2=0x7fe95c0c4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 shutdown_connections 2023-11-24T06:28:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.314+0000 7fe9631e5640 1 -- 172.21.15.16:0/327739625 wait complete. 2023-11-24T06:28:04.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.563+0000 7f024ef53640 1 Processor -- start 2023-11-24T06:28:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.563+0000 7f024ef53640 1 -- start start 2023-11-24T06:28:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.565+0000 7f024ef53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02480ae5a0 0x7f024815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.565+0000 7f024ef53640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0248070c40 con 0x7f02480b3ae0 2023-11-24T06:28:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.565+0000 7f024ef53640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024806c470 con 0x7f02480cf290 2023-11-24T06:28:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.565+0000 7f024ef53640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0248071450 con 0x7f02480ae5a0 2023-11-24T06:28:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.566+0000 7f0247fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f02480b3ae0 0x7f024806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51304/0 (socket says 172.21.15.16:51304) 2023-11-24T06:28:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.566+0000 7f0247fff640 1 -- 172.21.15.16:0/1851836620 learned_addr learned my addr 172.21.15.16:0/1851836620 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.566+0000 7f024ccef640 1 --2- 172.21.15.16:0/1851836620 >> v2:172.21.15.16:3300/0 conn(0x7f02480ae5a0 0x7f024815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.566+0000 7f02477fe640 1 -- 172.21.15.16:0/1851836620 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758387905 0 0) 0x7f0248070c40 con 0x7f02480b3ae0 2023-11-24T06:28:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.566+0000 7f02477fe640 1 -- 172.21.15.16:0/1851836620 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0238003520 con 0x7f02480b3ae0 2023-11-24T06:28:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.567+0000 7f024ccef640 1 -- 172.21.15.16:0/1851836620 >> v1:172.21.15.173:6789/0 conn(0x7f02480cf290 legacy=0x7f02480b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.567+0000 7f024ccef640 1 -- 172.21.15.16:0/1851836620 >> v1:172.21.15.16:6789/0 conn(0x7f02480b3ae0 legacy=0x7f024806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.567+0000 7f024ccef640 1 -- 172.21.15.16:0/1851836620 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024806c750 con 0x7f02480ae5a0 2023-11-24T06:28:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.568+0000 7f024ccef640 1 --2- 172.21.15.16:0/1851836620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480ae5a0 0x7f024815e430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f023c0097a0 tx=0x7f023c02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96129ff2e93dfab9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.568+0000 7f02477fe640 1 -- 172.21.15.16:0/1851836620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c02f810 con 0x7f02480ae5a0 2023-11-24T06:28:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.568+0000 7f02477fe640 1 -- 172.21.15.16:0/1851836620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c02f970 con 0x7f02480ae5a0 2023-11-24T06:28:04.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.568+0000 7f02477fe640 1 -- 172.21.15.16:0/1851836620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c02fd00 con 0x7f02480ae5a0 2023-11-24T06:28:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.569+0000 7f024ef53640 1 -- 172.21.15.16:0/1851836620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480ae5a0 msgr2=0x7f024815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.569+0000 7f024ef53640 1 --2- 172.21.15.16:0/1851836620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480ae5a0 0x7f024815e430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f023c0097a0 tx=0x7f023c02b9f0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.569+0000 7f024ef53640 1 -- 172.21.15.16:0/1851836620 shutdown_connections 2023-11-24T06:28:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.569+0000 7f024ef53640 1 --2- 172.21.15.16:0/1851836620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480ae5a0 0x7f024815e430 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.569+0000 7f024ef53640 1 -- 172.21.15.16:0/1851836620 >> 172.21.15.16:0/1851836620 conn(0x7f024806a800 msgr2=0x7f024806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.569+0000 7f024ef53640 1 -- 172.21.15.16:0/1851836620 shutdown_connections 2023-11-24T06:28:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.569+0000 7f024ef53640 1 -- 172.21.15.16:0/1851836620 wait complete. 2023-11-24T06:28:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.571+0000 7f024ef53640 1 Processor -- start 2023-11-24T06:28:04.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.571+0000 7f024ef53640 1 -- start start 2023-11-24T06:28:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.572+0000 7f024ef53640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02480ae5a0 0x7f02480cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.572+0000 7f024ef53640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02480b3ae0 0x7f024814a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.572+0000 7f024ef53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480cf290 0x7f02481f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.572+0000 7f024ef53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02481f6a10 con 0x7f02480cf290 2023-11-24T06:28:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.572+0000 7f024ef53640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024814a560 con 0x7f02480b3ae0 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.572+0000 7f024ef53640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02481f60e0 con 0x7f02480ae5a0 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.572+0000 7f024d4f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480cf290 0x7f02481f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.572+0000 7f024d4f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480cf290 0x7f02481f2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39372/0 (socket says 172.21.15.16:39372) 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.572+0000 7f024d4f0640 1 -- 172.21.15.16:0/1376204903 learned_addr learned my addr 172.21.15.16:0/1376204903 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f0247fff640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02480b3ae0 0x7f024814a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f024ccef640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02480ae5a0 0x7f02480cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f024d4f0640 1 -- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02480ae5a0 msgr2=0x7f02480cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f024d4f0640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02480ae5a0 0x7f02480cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f024d4f0640 1 -- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02480b3ae0 msgr2=0x7f024814a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f024d4f0640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02480b3ae0 0x7f024814a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f024d4f0640 1 -- 172.21.15.16:0/1376204903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023c009400 con 0x7f02480cf290 2023-11-24T06:28:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f024d4f0640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480cf290 0x7f02481f2380 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f024000c6b0 tx=0x7f024000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f02457fa640 1 -- 172.21.15.16:0/1376204903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240003b70 con 0x7f02480cf290 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f02457fa640 1 -- 172.21.15.16:0/1376204903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0240003cf0 con 0x7f02480cf290 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f02457fa640 1 -- 172.21.15.16:0/1376204903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240003ea0 con 0x7f02480cf290 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.573+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02481f63c0 con 0x7f02480cf290 2023-11-24T06:28:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.574+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02481f28c0 con 0x7f02480cf290 2023-11-24T06:28:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.575+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02481f6bc0 con 0x7f02480cf290 2023-11-24T06:28:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.576+0000 7f02457fa640 1 -- 172.21.15.16:0/1376204903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0240005310 con 0x7f02480cf290 2023-11-24T06:28:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.576+0000 7f02457fa640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f022c06ae40 0x7f022c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.577+0000 7f02457fa640 1 -- 172.21.15.16:0/1376204903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f02400894d0 con 0x7f02480cf290 2023-11-24T06:28:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.577+0000 7f024ccef640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f022c06ae40 0x7f022c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:04.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.578+0000 7f024ccef640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f022c06ae40 0x7f022c06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f023c036000 tx=0x7f023c039040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.581+0000 7f02457fa640 1 -- 172.21.15.16:0/1376204903 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0240054460 con 0x7f02480cf290 2023-11-24T06:28:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.776+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02480675d0 con 0x7f02480cf290 2023-11-24T06:28:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.777+0000 7f02457fa640 1 -- 172.21.15.16:0/1376204903 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0240057a80 con 0x7f02480cf290 2023-11-24T06:28:04.780 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:28:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f022c06ae40 msgr2=0x7f022c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f022c06ae40 0x7f022c06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f023c036000 tx=0x7f023c039040 comp rx=0 tx=0).stop 2023-11-24T06:28:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480cf290 msgr2=0x7f02481f2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480cf290 0x7f02481f2380 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f024000c6b0 tx=0x7f024000cb70 comp rx=0 tx=0).stop 2023-11-24T06:28:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 shutdown_connections 2023-11-24T06:28:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f022c06ae40 0x7f022c06d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02480cf290 0x7f02481f2380 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02480b3ae0 0x7f024814a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 --2- 172.21.15.16:0/1376204903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02480ae5a0 0x7f02480cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 >> 172.21.15.16:0/1376204903 conn(0x7f024806a800 msgr2=0x7f024806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 shutdown_connections 2023-11-24T06:28:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:04.780+0000 7f024ef53640 1 -- 172.21.15.16:0/1376204903 wait complete. 2023-11-24T06:28:04.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:05.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.021+0000 7f39ee3dc640 1 Processor -- start 2023-11-24T06:28:05.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.021+0000 7f39ee3dc640 1 -- start start 2023-11-24T06:28:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.022+0000 7f39ee3dc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80cfcf0 0x7f39e8160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ee3dc640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f39e80d3910 0x7f39e8160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ee3dc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39e80d00d0 0x7f39e80b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ee3dc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e80ceff0 con 0x7f39e80d00d0 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ee3dc640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e80cf410 con 0x7f39e80d3910 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ee3dc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80d4160 con 0x7f39e80cfcf0 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ec979640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39e80d00d0 0x7f39e80b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ec979640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39e80d00d0 0x7f39e80b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39386/0 (socket says 172.21.15.16:39386) 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ec979640 1 -- 172.21.15.16:0/243959090 learned_addr learned my addr 172.21.15.16:0/243959090 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39e7fff640 1 --2- 172.21.15.16:0/243959090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80cfcf0 0x7f39e8160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39e77fe640 1 --2- 172.21.15.16:0/243959090 >> v2:172.21.15.173:3300/0 conn(0x7f39e80d3910 0x7f39e8160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ec979640 1 -- 172.21.15.16:0/243959090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80cfcf0 msgr2=0x7f39e8160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.023+0000 7f39ec979640 1 --2- 172.21.15.16:0/243959090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80cfcf0 0x7f39e8160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.024+0000 7f39ec979640 1 -- 172.21.15.16:0/243959090 >> v2:172.21.15.173:3300/0 conn(0x7f39e80d3910 msgr2=0x7f39e8160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.024+0000 7f39ec979640 1 --2- 172.21.15.16:0/243959090 >> v2:172.21.15.173:3300/0 conn(0x7f39e80d3910 0x7f39e8160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.024+0000 7f39ec979640 1 -- 172.21.15.16:0/243959090 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80b2420 con 0x7f39e80d00d0 2023-11-24T06:28:05.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.024+0000 7f39e77fe640 1 --2- 172.21.15.16:0/243959090 >> v2:172.21.15.173:3300/0 conn(0x7f39e80d3910 0x7f39e8160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:05.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.025+0000 7f39ec979640 1 --2- 172.21.15.16:0/243959090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 0x7f39e80b2a50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f39e808b790 tx=0x7f39e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16957983048dd1b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.025+0000 7f39e6ffd640 1 -- 172.21.15.16:0/243959090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e002c9e0 con 0x7f39e80d00d0 2023-11-24T06:28:05.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.025+0000 7f39e6ffd640 1 -- 172.21.15.16:0/243959090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e002cb40 con 0x7f39e80d00d0 2023-11-24T06:28:05.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.025+0000 7f39e6ffd640 1 -- 172.21.15.16:0/243959090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e00343f0 con 0x7f39e80d00d0 2023-11-24T06:28:05.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.026+0000 7f39ee3dc640 1 -- 172.21.15.16:0/243959090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 msgr2=0x7f39e80b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.026+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/243959090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 0x7f39e80b2a50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f39e808b790 tx=0x7f39e002be40 comp rx=0 tx=0).stop 2023-11-24T06:28:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.026+0000 7f39ee3dc640 1 -- 172.21.15.16:0/243959090 shutdown_connections 2023-11-24T06:28:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.026+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/243959090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 0x7f39e80b2a50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.026+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/243959090 >> v2:172.21.15.173:3300/0 conn(0x7f39e80d3910 0x7f39e8160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.026+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/243959090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80cfcf0 0x7f39e8160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.026+0000 7f39ee3dc640 1 -- 172.21.15.16:0/243959090 >> 172.21.15.16:0/243959090 conn(0x7f39e809c530 msgr2=0x7f39e80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.026+0000 7f39ee3dc640 1 -- 172.21.15.16:0/243959090 shutdown_connections 2023-11-24T06:28:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.026+0000 7f39ee3dc640 1 -- 172.21.15.16:0/243959090 wait complete. 2023-11-24T06:28:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.028+0000 7f39ee3dc640 1 Processor -- start 2023-11-24T06:28:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.028+0000 7f39ee3dc640 1 -- start start 2023-11-24T06:28:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.029+0000 7f39ee3dc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e80cfcf0 0x7f39e80cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.029+0000 7f39ee3dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 0x7f39e80b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.029+0000 7f39ee3dc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80d3910 0x7f39e815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.029+0000 7f39ee3dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e8157050 con 0x7f39e80d00d0 2023-11-24T06:28:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.029+0000 7f39ee3dc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80b7840 con 0x7f39e80cfcf0 2023-11-24T06:28:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.029+0000 7f39ee3dc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e80b7b20 con 0x7f39e80d3910 2023-11-24T06:28:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.029+0000 7f39e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 0x7f39e80b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.029+0000 7f39e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 0x7f39e80b88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39390/0 (socket says 172.21.15.16:39390) 2023-11-24T06:28:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.029+0000 7f39e77fe640 1 -- 172.21.15.16:0/2367516238 learned_addr learned my addr 172.21.15.16:0/2367516238 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.030+0000 7f39e7fff640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e80cfcf0 0x7f39e80cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.030+0000 7f39ec979640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80d3910 0x7f39e815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.030+0000 7f39e77fe640 1 -- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80d3910 msgr2=0x7f39e815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.030+0000 7f39e77fe640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80d3910 0x7f39e815b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.030+0000 7f39e77fe640 1 -- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e80cfcf0 msgr2=0x7f39e80cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.030+0000 7f39e77fe640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e80cfcf0 0x7f39e80cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.030+0000 7f39e77fe640 1 -- 172.21.15.16:0/2367516238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0009b30 con 0x7f39e80d00d0 2023-11-24T06:28:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.030+0000 7f39e77fe640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 0x7f39e80b88e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c7f0 tx=0x7f39d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.031+0000 7f39e4ff9640 1 -- 172.21.15.16:0/2367516238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0007a30 con 0x7f39e80d00d0 2023-11-24T06:28:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.031+0000 7f39e4ff9640 1 -- 172.21.15.16:0/2367516238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0007b90 con 0x7f39e80d00d0 2023-11-24T06:28:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.031+0000 7f39e4ff9640 1 -- 172.21.15.16:0/2367516238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d000ce40 con 0x7f39e80d00d0 2023-11-24T06:28:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.031+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e80b7e00 con 0x7f39e80d00d0 2023-11-24T06:28:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.031+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e80b83c0 con 0x7f39e80d00d0 2023-11-24T06:28:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.032+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e80675d0 con 0x7f39e80d00d0 2023-11-24T06:28:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.033+0000 7f39e4ff9640 1 -- 172.21.15.16:0/2367516238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39d00113f0 con 0x7f39e80d00d0 2023-11-24T06:28:05.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.034+0000 7f39e4ff9640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f39c806ae40 0x7f39c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.034+0000 7f39e4ff9640 1 -- 172.21.15.16:0/2367516238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f39d00898d0 con 0x7f39e80d00d0 2023-11-24T06:28:05.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.034+0000 7f39e7fff640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f39c806ae40 0x7f39c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.036+0000 7f39e7fff640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f39c806ae40 0x7f39c806d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006fd0 tx=0x7f39d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.040+0000 7f39e4ff9640 1 -- 172.21.15.16:0/2367516238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f39d0054860 con 0x7f39e80d00d0 2023-11-24T06:28:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.236+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39e806c7b0 con 0x7f39e80d00d0 2023-11-24T06:28:05.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.237+0000 7f39e4ff9640 1 -- 172.21.15.16:0/2367516238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39d001f0a0 con 0x7f39e80d00d0 2023-11-24T06:28:05.240 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:28:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f39c806ae40 msgr2=0x7f39c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f39c806ae40 0x7f39c806d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f39d8006fd0 tx=0x7f39d8008040 comp rx=0 tx=0).stop 2023-11-24T06:28:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 msgr2=0x7f39e80b88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 0x7f39e80b88e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f39d000c7f0 tx=0x7f39d000cbf0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 shutdown_connections 2023-11-24T06:28:05.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f39c806ae40 0x7f39c806d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39e80d3910 0x7f39e815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e80d00d0 0x7f39e80b88e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 --2- 172.21.15.16:0/2367516238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39e80cfcf0 0x7f39e80cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 >> 172.21.15.16:0/2367516238 conn(0x7f39e809c530 msgr2=0x7f39e80b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.240+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 shutdown_connections 2023-11-24T06:28:05.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.241+0000 7f39ee3dc640 1 -- 172.21.15.16:0/2367516238 wait complete. 2023-11-24T06:28:05.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.465+0000 7fc9c49af640 1 Processor -- start 2023-11-24T06:28:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.465+0000 7fc9c49af640 1 -- start start 2023-11-24T06:28:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.466+0000 7fc9c49af640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c00cc500 0x7fc9c00cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.466+0000 7fc9c49af640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9c006d160 0x7fc9c006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.466+0000 7fc9c49af640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c0070e50 con 0x7fc9c0143c40 2023-11-24T06:28:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.466+0000 7fc9c49af640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c00723c0 con 0x7fc9c006d160 2023-11-24T06:28:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.466+0000 7fc9c49af640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0144020 con 0x7fc9c00cc500 2023-11-24T06:28:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.467+0000 7fc9bf59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9c006d160 0x7fc9c006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.467+0000 7fc9bf59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9c006d160 0x7fc9c006d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39396/0 (socket says 172.21.15.16:39396) 2023-11-24T06:28:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.467+0000 7fc9bf59e640 1 -- 172.21.15.16:0/87420116 learned_addr learned my addr 172.21.15.16:0/87420116 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.467+0000 7fc9be59c640 1 --2- 172.21.15.16:0/87420116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c00cc500 0x7fc9c00cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.467+0000 7fc9bf59e640 1 -- 172.21.15.16:0/87420116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c00cc500 msgr2=0x7fc9c00cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.467+0000 7fc9bf59e640 1 --2- 172.21.15.16:0/87420116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c00cc500 0x7fc9c00cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.467+0000 7fc9bf59e640 1 -- 172.21.15.16:0/87420116 >> v1:172.21.15.173:6789/0 conn(0x7fc9c0143c40 legacy=0x7fc9c0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.468+0000 7fc9bf59e640 1 -- 172.21.15.16:0/87420116 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c00ccea0 con 0x7fc9c006d160 2023-11-24T06:28:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.469+0000 7fc9bf59e640 1 --2- 172.21.15.16:0/87420116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c006d160 0x7fc9c006d590 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009e50 tx=0x7fc9b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f02c163bfebad6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.469+0000 7fc9bdd9b640 1 -- 172.21.15.16:0/87420116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b802ca70 con 0x7fc9c006d160 2023-11-24T06:28:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.469+0000 7fc9bdd9b640 1 -- 172.21.15.16:0/87420116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b802cbd0 con 0x7fc9c006d160 2023-11-24T06:28:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.469+0000 7fc9bdd9b640 1 -- 172.21.15.16:0/87420116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b80343f0 con 0x7fc9c006d160 2023-11-24T06:28:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.470+0000 7fc9c49af640 1 -- 172.21.15.16:0/87420116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c006d160 msgr2=0x7fc9c006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.470+0000 7fc9c49af640 1 --2- 172.21.15.16:0/87420116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c006d160 0x7fc9c006d590 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009e50 tx=0x7fc9b802bed0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.470+0000 7fc9c49af640 1 -- 172.21.15.16:0/87420116 shutdown_connections 2023-11-24T06:28:05.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.470+0000 7fc9c49af640 1 --2- 172.21.15.16:0/87420116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c006d160 0x7fc9c006d590 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.470+0000 7fc9c49af640 1 --2- 172.21.15.16:0/87420116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c00cc500 0x7fc9c00cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.470+0000 7fc9c49af640 1 -- 172.21.15.16:0/87420116 >> 172.21.15.16:0/87420116 conn(0x7fc9c006a800 msgr2=0x7fc9c006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.470+0000 7fc9c49af640 1 -- 172.21.15.16:0/87420116 shutdown_connections 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.470+0000 7fc9c49af640 1 -- 172.21.15.16:0/87420116 wait complete. 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.472+0000 7fc9c49af640 1 Processor -- start 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.472+0000 7fc9c49af640 1 -- start start 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.473+0000 7fc9c49af640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c006d160 0x7fc9c015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.473+0000 7fc9c49af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cc500 0x7fc9c0139520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.473+0000 7fc9c49af640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9c0143c40 0x7fc9c0159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.473+0000 7fc9c49af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c015aed0 con 0x7fc9c00cc500 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.473+0000 7fc9c49af640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0139a60 con 0x7fc9c0143c40 2023-11-24T06:28:05.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.473+0000 7fc9c49af640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0139d00 con 0x7fc9c006d160 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.473+0000 7fc9be59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cc500 0x7fc9c0139520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.473+0000 7fc9be59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cc500 0x7fc9c0139520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39402/0 (socket says 172.21.15.16:39402) 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.473+0000 7fc9be59c640 1 -- 172.21.15.16:0/2969701979 learned_addr learned my addr 172.21.15.16:0/2969701979 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9bf59e640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9c0143c40 0x7fc9c0159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9bed9d640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c006d160 0x7fc9c015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9be59c640 1 -- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c006d160 msgr2=0x7fc9c015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9be59c640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c006d160 0x7fc9c015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9be59c640 1 -- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9c0143c40 msgr2=0x7fc9c0159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9be59c640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9c0143c40 0x7fc9c0159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9be59c640 1 -- 172.21.15.16:0/2969701979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b8009b30 con 0x7fc9c00cc500 2023-11-24T06:28:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9bed9d640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c006d160 0x7fc9c015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9be59c640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cc500 0x7fc9c0139520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400d800 tx=0x7fc9b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9a77fe640 1 -- 172.21.15.16:0/2969701979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400b6c0 con 0x7fc9c00cc500 2023-11-24T06:28:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.474+0000 7fc9a77fe640 1 -- 172.21.15.16:0/2969701979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b400b820 con 0x7fc9c00cc500 2023-11-24T06:28:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.475+0000 7fc9a77fe640 1 -- 172.21.15.16:0/2969701979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400bbb0 con 0x7fc9c00cc500 2023-11-24T06:28:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.475+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c0139fe0 con 0x7fc9c00cc500 2023-11-24T06:28:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.475+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c013a880 con 0x7fc9c00cc500 2023-11-24T06:28:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.476+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c013aa40 con 0x7fc9c00cc500 2023-11-24T06:28:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.477+0000 7fc9a77fe640 1 -- 172.21.15.16:0/2969701979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9b400bd10 con 0x7fc9c00cc500 2023-11-24T06:28:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.478+0000 7fc9a77fe640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc9ac06ae40 0x7fc9ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.478+0000 7fc9a77fe640 1 -- 172.21.15.16:0/2969701979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fc9b4089c90 con 0x7fc9c00cc500 2023-11-24T06:28:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.479+0000 7fc9bed9d640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc9ac06ae40 0x7fc9ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.480+0000 7fc9bed9d640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc9ac06ae40 0x7fc9ac06d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0001680 tx=0x7fc9b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.482+0000 7fc9a77fe640 1 -- 172.21.15.16:0/2969701979 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9b4054c20 con 0x7fc9c00cc500 2023-11-24T06:28:05.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.712+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9c0067450 con 0x7fc9c00cc500 2023-11-24T06:28:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.713+0000 7fc9a77fe640 1 -- 172.21.15.16:0/2969701979 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fc9b4058240 con 0x7fc9c00cc500 2023-11-24T06:28:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:05.716 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3019663874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3019663874},{"type":"v1","addr":"172.21.15.173:6837","nonce":3019663874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2239024365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2239024365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2239024365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:26:56.460479+0000","modified":"2023-11-24T06:26:59.516602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/915142029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":915142029},{"type":"v1","addr":"172.21.15.173:6835","nonce":915142029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:28:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-11-24T06:28:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc9ac06ae40 msgr2=0x7fc9ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc9ac06ae40 0x7fc9ac06d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0001680 tx=0x7fc9b0009040 comp rx=0 tx=0).stop 2023-11-24T06:28:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cc500 msgr2=0x7fc9c0139520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cc500 0x7fc9c0139520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400d800 tx=0x7fc9b400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 shutdown_connections 2023-11-24T06:28:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc9ac06ae40 0x7fc9ac06d300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9c0143c40 0x7fc9c0159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cc500 0x7fc9c0139520 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 --2- 172.21.15.16:0/2969701979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9c006d160 0x7fc9c015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 >> 172.21.15.16:0/2969701979 conn(0x7fc9c006a800 msgr2=0x7fc9c006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 shutdown_connections 2023-11-24T06:28:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.716+0000 7fc9c49af640 1 -- 172.21.15.16:0/2969701979 wait complete. 2023-11-24T06:28:05.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.958+0000 7f18ce221640 1 Processor -- start 2023-11-24T06:28:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.959+0000 7f18ce221640 1 -- start start 2023-11-24T06:28:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.960+0000 7f18ce221640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813cfa0 0x7f18c815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.960+0000 7f18ce221640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18c813fcb0 0x7f18c80b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.960+0000 7f18ce221640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c80cdec0 con 0x7f18c80cd530 2023-11-24T06:28:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.960+0000 7f18ce221640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c80ce1a0 con 0x7f18c813fcb0 2023-11-24T06:28:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.960+0000 7f18ce221640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80ce480 con 0x7f18c813cfa0 2023-11-24T06:28:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.961+0000 7f18cda20640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18c813fcb0 0x7f18c80b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.961+0000 7f18cda20640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18c813fcb0 0x7f18c80b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39418/0 (socket says 172.21.15.16:39418) 2023-11-24T06:28:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.961+0000 7f18cda20640 1 -- 172.21.15.16:0/422786622 learned_addr learned my addr 172.21.15.16:0/422786622 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.961+0000 7f18cd21f640 1 --2- 172.21.15.16:0/422786622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813cfa0 0x7f18c815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.961+0000 7f18cda20640 1 -- 172.21.15.16:0/422786622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813cfa0 msgr2=0x7f18c815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.961+0000 7f18cda20640 1 --2- 172.21.15.16:0/422786622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813cfa0 0x7f18c815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.961+0000 7f18cda20640 1 -- 172.21.15.16:0/422786622 >> v1:172.21.15.173:6789/0 conn(0x7f18c80cd530 legacy=0x7f18c813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.962+0000 7f18cda20640 1 -- 172.21.15.16:0/422786622 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c80ce760 con 0x7f18c813fcb0 2023-11-24T06:28:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.963+0000 7f18cda20640 1 --2- 172.21.15.16:0/422786622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813fcb0 0x7f18c80b2040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009870 tx=0x7f18c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23036142d9ea1bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.963+0000 7f18bffff640 1 -- 172.21.15.16:0/422786622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0031640 con 0x7f18c813fcb0 2023-11-24T06:28:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.963+0000 7f18bffff640 1 -- 172.21.15.16:0/422786622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c00317a0 con 0x7f18c813fcb0 2023-11-24T06:28:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.963+0000 7f18bffff640 1 -- 172.21.15.16:0/422786622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0031b30 con 0x7f18c813fcb0 2023-11-24T06:28:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.963+0000 7f18ce221640 1 -- 172.21.15.16:0/422786622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813fcb0 msgr2=0x7f18c80b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.963+0000 7f18ce221640 1 --2- 172.21.15.16:0/422786622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813fcb0 0x7f18c80b2040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009870 tx=0x7f18c002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.964+0000 7f18ce221640 1 -- 172.21.15.16:0/422786622 shutdown_connections 2023-11-24T06:28:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.964+0000 7f18ce221640 1 --2- 172.21.15.16:0/422786622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813fcb0 0x7f18c80b2040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.964+0000 7f18ce221640 1 --2- 172.21.15.16:0/422786622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813cfa0 0x7f18c815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.964+0000 7f18ce221640 1 -- 172.21.15.16:0/422786622 >> 172.21.15.16:0/422786622 conn(0x7f18c806a800 msgr2=0x7f18c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:05.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.964+0000 7f18ce221640 1 -- 172.21.15.16:0/422786622 shutdown_connections 2023-11-24T06:28:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.964+0000 7f18ce221640 1 -- 172.21.15.16:0/422786622 wait complete. 2023-11-24T06:28:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.965+0000 7f18ce221640 1 Processor -- start 2023-11-24T06:28:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.966+0000 7f18ce221640 1 -- start start 2023-11-24T06:28:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18ce221640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c80cd530 0x7f18c81f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18ce221640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813cfa0 0x7f18c81f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18ce221640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813fcb0 0x7f18c80ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18ce221640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80ae1b0 con 0x7f18c813cfa0 2023-11-24T06:28:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18ce221640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8155cb0 con 0x7f18c80cd530 2023-11-24T06:28:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18ce221640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8155f90 con 0x7f18c813fcb0 2023-11-24T06:28:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cca1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813cfa0 0x7f18c81f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cca1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813cfa0 0x7f18c81f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39422/0 (socket says 172.21.15.16:39422) 2023-11-24T06:28:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cca1e640 1 -- 172.21.15.16:0/4204844101 learned_addr learned my addr 172.21.15.16:0/4204844101 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cd21f640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c80cd530 0x7f18c81f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cda20640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813fcb0 0x7f18c80ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cca1e640 1 -- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813fcb0 msgr2=0x7f18c80ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cca1e640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813fcb0 0x7f18c80ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cca1e640 1 -- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c80cd530 msgr2=0x7f18c81f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cca1e640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c80cd530 0x7f18c81f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cca1e640 1 -- 172.21.15.16:0/4204844101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c00094d0 con 0x7f18c813cfa0 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cd21f640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c80cd530 0x7f18c81f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.967+0000 7f18cda20640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813fcb0 0x7f18c80ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.968+0000 7f18cca1e640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813cfa0 0x7f18c81f3f40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c400 tx=0x7f18b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.968+0000 7f18bdffb640 1 -- 172.21.15.16:0/4204844101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b80052d0 con 0x7f18c813cfa0 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.968+0000 7f18bdffb640 1 -- 172.21.15.16:0/4204844101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8005450 con 0x7f18c813cfa0 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.968+0000 7f18bdffb640 1 -- 172.21.15.16:0/4204844101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8007260 con 0x7f18c813cfa0 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.968+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c8156270 con 0x7f18c813cfa0 2023-11-24T06:28:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.968+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c81f4660 con 0x7f18c813cfa0 2023-11-24T06:28:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.969+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c8155b10 con 0x7f18c813cfa0 2023-11-24T06:28:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.971+0000 7f18bdffb640 1 -- 172.21.15.16:0/4204844101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18b80073c0 con 0x7f18c813cfa0 2023-11-24T06:28:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.971+0000 7f18bdffb640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f18ac06ae40 0x7f18ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.972+0000 7f18bdffb640 1 -- 172.21.15.16:0/4204844101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f18b8088b30 con 0x7f18c813cfa0 2023-11-24T06:28:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.972+0000 7f18cd21f640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f18ac06ae40 0x7f18ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:05.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.973+0000 7f18cd21f640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f18ac06ae40 0x7f18ac06d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f18c4007330 tx=0x7f18c400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:05.976+0000 7f18bdffb640 1 -- 172.21.15.16:0/4204844101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18c8155b10 con 0x7f18c813cfa0 2023-11-24T06:28:06.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.164+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18c8155b10 con 0x7f18c813cfa0 2023-11-24T06:28:06.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.167+0000 7f18bdffb640 1 -- 172.21.15.16:0/4204844101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7f18b8080e60 con 0x7f18c813cfa0 2023-11-24T06:28:06.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:06.169 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f18ac06ae40 msgr2=0x7f18ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f18ac06ae40 0x7f18ac06d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f18c4007330 tx=0x7f18c400b040 comp rx=0 tx=0).stop 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813cfa0 msgr2=0x7f18c81f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813cfa0 0x7f18c81f3f40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c400 tx=0x7f18b800c8c0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 shutdown_connections 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f18ac06ae40 0x7f18ac06d300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18c813fcb0 0x7f18c80ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18c813cfa0 0x7f18c81f3f40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 --2- 172.21.15.16:0/4204844101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18c80cd530 0x7f18c81f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.169+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 >> 172.21.15.16:0/4204844101 conn(0x7f18c806a800 msgr2=0x7f18c813f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.170+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 shutdown_connections 2023-11-24T06:28:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.170+0000 7f18ce221640 1 -- 172.21.15.16:0/4204844101 wait complete. 2023-11-24T06:28:06.190 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:28:06.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:06.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.402+0000 7ffa25d43640 1 Processor -- start 2023-11-24T06:28:06.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.402+0000 7ffa25d43640 1 -- start start 2023-11-24T06:28:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.404+0000 7ffa25d43640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144410 0x7ffa2006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.404+0000 7ffa25d43640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ffa200b2600 0x7ffa2006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.404+0000 7ffa25d43640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa200acda0 con 0x7ffa20144870 2023-11-24T06:28:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.404+0000 7ffa25d43640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa200af740 con 0x7ffa200b2600 2023-11-24T06:28:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.404+0000 7ffa25d43640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200ad5b0 con 0x7ffa20144410 2023-11-24T06:28:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.404+0000 7ffa24d41640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144410 0x7ffa2006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.405+0000 7ffa1ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ffa200b2600 0x7ffa2006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.405+0000 7ffa24d41640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144410 0x7ffa2006a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:35908/0 (socket says 172.21.15.16:35908) 2023-11-24T06:28:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.405+0000 7ffa24d41640 1 -- 172.21.15.16:0/2330294202 learned_addr learned my addr 172.21.15.16:0/2330294202 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.405+0000 7ffa1ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ffa200b2600 0x7ffa2006a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:39080/0 (socket says 172.21.15.16:39080) 2023-11-24T06:28:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.405+0000 7ffa1ffff640 1 -- 172.21.15.16:0/2330294202 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144410 msgr2=0x7ffa2006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.405+0000 7ffa1ffff640 1 --2- 172.21.15.16:0/2330294202 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144410 0x7ffa2006a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.405+0000 7ffa1ffff640 1 -- 172.21.15.16:0/2330294202 >> v1:172.21.15.173:6789/0 conn(0x7ffa20144870 legacy=0x7ffa20071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.405+0000 7ffa1ffff640 1 -- 172.21.15.16:0/2330294202 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa200b3240 con 0x7ffa200b2600 2023-11-24T06:28:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.407+0000 7ffa1ffff640 1 --2- 172.21.15.16:0/2330294202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200b2600 0x7ffa2006a920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa140097a0 tx=0x7ffa1402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e48ae595276012 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.408+0000 7ffa1f7fe640 1 -- 172.21.15.16:0/2330294202 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14031830 con 0x7ffa200b2600 2023-11-24T06:28:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.408+0000 7ffa1f7fe640 1 -- 172.21.15.16:0/2330294202 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa14031990 con 0x7ffa200b2600 2023-11-24T06:28:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.408+0000 7ffa1f7fe640 1 -- 172.21.15.16:0/2330294202 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14031d20 con 0x7ffa200b2600 2023-11-24T06:28:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.409+0000 7ffa25d43640 1 -- 172.21.15.16:0/2330294202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200b2600 msgr2=0x7ffa2006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.409+0000 7ffa25d43640 1 --2- 172.21.15.16:0/2330294202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200b2600 0x7ffa2006a920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa140097a0 tx=0x7ffa1402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.409+0000 7ffa25d43640 1 -- 172.21.15.16:0/2330294202 shutdown_connections 2023-11-24T06:28:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.409+0000 7ffa25d43640 1 --2- 172.21.15.16:0/2330294202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200b2600 0x7ffa2006a920 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.409+0000 7ffa25d43640 1 --2- 172.21.15.16:0/2330294202 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144410 0x7ffa2006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.409+0000 7ffa25d43640 1 -- 172.21.15.16:0/2330294202 >> 172.21.15.16:0/2330294202 conn(0x7ffa200b19c0 msgr2=0x7ffa200b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.409+0000 7ffa25d43640 1 -- 172.21.15.16:0/2330294202 shutdown_connections 2023-11-24T06:28:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.409+0000 7ffa25d43640 1 -- 172.21.15.16:0/2330294202 wait complete. 2023-11-24T06:28:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.412+0000 7ffa25d43640 1 Processor -- start 2023-11-24T06:28:06.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.412+0000 7ffa25d43640 1 -- start start 2023-11-24T06:28:06.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.413+0000 7ffa25d43640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200b2600 0x7ffa201fa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.413+0000 7ffa25d43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20144410 0x7ffa2013a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.413+0000 7ffa25d43640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144870 0x7ffa2006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.413+0000 7ffa25d43640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa201fa950 con 0x7ffa20144410 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.413+0000 7ffa25d43640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa201fac30 con 0x7ffa200b2600 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.413+0000 7ffa25d43640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa201faf10 con 0x7ffa20144870 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20144410 0x7ffa2013a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20144410 0x7ffa2013a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39428/0 (socket says 172.21.15.16:39428) 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa1ffff640 1 -- 172.21.15.16:0/2098838870 learned_addr learned my addr 172.21.15.16:0/2098838870 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa24d41640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200b2600 0x7ffa201fa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa25542640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144870 0x7ffa2006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa1ffff640 1 -- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144870 msgr2=0x7ffa2006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa1ffff640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144870 0x7ffa2006a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa1ffff640 1 -- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200b2600 msgr2=0x7ffa201fa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa1ffff640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200b2600 0x7ffa201fa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa1ffff640 1 -- 172.21.15.16:0/2098838870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa14009400 con 0x7ffa20144410 2023-11-24T06:28:06.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.414+0000 7ffa25542640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144870 0x7ffa2006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.415+0000 7ffa1ffff640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20144410 0x7ffa2013a3e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa1402bee0 tx=0x7ffa140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.415+0000 7ffa1d7fa640 1 -- 172.21.15.16:0/2098838870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1403b9f0 con 0x7ffa20144410 2023-11-24T06:28:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.415+0000 7ffa1d7fa640 1 -- 172.21.15.16:0/2098838870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1403bb70 con 0x7ffa20144410 2023-11-24T06:28:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.415+0000 7ffa1d7fa640 1 -- 172.21.15.16:0/2098838870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa140453f0 con 0x7ffa20144410 2023-11-24T06:28:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.415+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa201fb1f0 con 0x7ffa20144410 2023-11-24T06:28:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.415+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2013a920 con 0x7ffa20144410 2023-11-24T06:28:06.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.417+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa201fb710 con 0x7ffa20144410 2023-11-24T06:28:06.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.417+0000 7ffa1d7fa640 1 -- 172.21.15.16:0/2098838870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa1404e450 con 0x7ffa20144410 2023-11-24T06:28:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.419+0000 7ffa1d7fa640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0c06ae40 0x7ffa0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.419+0000 7ffa1d7fa640 1 -- 172.21.15.16:0/2098838870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7ffa140ab7a0 con 0x7ffa20144410 2023-11-24T06:28:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.419+0000 7ffa24d41640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0c06ae40 0x7ffa0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.421+0000 7ffa24d41640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0c06ae40 0x7ffa0c06d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffa100113e0 tx=0x7ffa1000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.423+0000 7ffa1d7fa640 1 -- 172.21.15.16:0/2098838870 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa201fb710 con 0x7ffa20144410 2023-11-24T06:28:06.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.653+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa201fb710 con 0x7ffa20144410 2023-11-24T06:28:06.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.653+0000 7ffa1d7fa640 1 -- 172.21.15.16:0/2098838870 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7ffa201fb710 con 0x7ffa20144410 2023-11-24T06:28:06.657 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:06.658 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3019663874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3019663874},{"type":"v1","addr":"172.21.15.173:6837","nonce":3019663874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2239024365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2239024365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2239024365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:26:56.460479+0000","modified":"2023-11-24T06:26:59.516602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/915142029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":915142029},{"type":"v1","addr":"172.21.15.173:6835","nonce":915142029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:28:06.658 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-11-24T06:28:06.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0c06ae40 msgr2=0x7ffa0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0c06ae40 0x7ffa0c06d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffa100113e0 tx=0x7ffa1000b040 comp rx=0 tx=0).stop 2023-11-24T06:28:06.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20144410 msgr2=0x7ffa2013a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20144410 0x7ffa2013a3e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffa1402bee0 tx=0x7ffa140046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 shutdown_connections 2023-11-24T06:28:06.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0c06ae40 0x7ffa0c06d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20144870 0x7ffa2006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20144410 0x7ffa2013a3e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 --2- 172.21.15.16:0/2098838870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200b2600 0x7ffa201fa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.656+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 >> 172.21.15.16:0/2098838870 conn(0x7ffa200b19c0 msgr2=0x7ffa200af930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.657+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 shutdown_connections 2023-11-24T06:28:06.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.657+0000 7ffa25d43640 1 -- 172.21.15.16:0/2098838870 wait complete. 2023-11-24T06:28:06.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:06.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.900+0000 7faf0697a640 1 Processor -- start 2023-11-24T06:28:06.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.901+0000 7faf0697a640 1 -- start start 2023-11-24T06:28:06.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.902+0000 7faf0697a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7faf0006aaf0 0x7faf0006ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.902+0000 7faf0697a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf000cc8b0 con 0x7faf0006ef90 2023-11-24T06:28:06.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.902+0000 7faf0697a640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf000ccb90 con 0x7faf00144150 2023-11-24T06:28:06.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.902+0000 7faf0697a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf000cce70 con 0x7faf0006aaf0 2023-11-24T06:28:06.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.902+0000 7faf06179640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faf0006ef90 0x7faf00164da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51312/0 (socket says 172.21.15.16:51312) 2023-11-24T06:28:06.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.902+0000 7faf06179640 1 -- 172.21.15.16:0/1099970257 learned_addr learned my addr 172.21.15.16:0/1099970257 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:06.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf05177640 1 --2- 172.21.15.16:0/1099970257 >> v2:172.21.15.173:3300/0 conn(0x7faf0006aaf0 0x7faf0006ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510325992 0 0) 0x7faf000cc8b0 con 0x7faf0006ef90 2023-11-24T06:28:06.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee8003520 con 0x7faf0006ef90 2023-11-24T06:28:06.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2182872132 0 0) 0x7faee8003520 con 0x7faf0006ef90 2023-11-24T06:28:06.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 >> v2:172.21.15.173:3300/0 conn(0x7faf0006aaf0 msgr2=0x7faf0006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 --2- 172.21.15.16:0/1099970257 >> v2:172.21.15.173:3300/0 conn(0x7faf0006aaf0 0x7faf0006ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 >> v1:172.21.15.173:6789/0 conn(0x7faf00144150 legacy=0x7faf0006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000cd150 con 0x7faf0006ef90 2023-11-24T06:28:06.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7faef8001670 con 0x7faf0006ef90 2023-11-24T06:28:06.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 >> v1:172.21.15.16:6789/0 conn(0x7faf0006ef90 legacy=0x7faf00164da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faee8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee80047c0 0x7faee8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faee8005180 0x7faee800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf000cce70 con 0x7faee8003970 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee800c020 con 0x7faee80047c0 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.903+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee800c1e0 con 0x7faee8005180 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.904+0000 7faf05978640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faee8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.904+0000 7faf05177640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee80047c0 0x7faee8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.904+0000 7faf05978640 1 -- 172.21.15.16:0/1099970257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faee8005180 msgr2=0x7faee800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.904+0000 7faf05978640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faee8005180 0x7faee800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.904+0000 7faf05978640 1 -- 172.21.15.16:0/1099970257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee80047c0 msgr2=0x7faee8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:06.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.904+0000 7faf05978640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee80047c0 0x7faee8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.904+0000 7faf05978640 1 -- 172.21.15.16:0/1099970257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8003520 con 0x7faee8003970 2023-11-24T06:28:06.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.904+0000 7faf05177640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faee80047c0 0x7faee8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:06.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.904+0000 7faf06179640 1 -- 172.21.15.16:0/1099970257 reap_dead start 2023-11-24T06:28:06.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.905+0000 7faf05978640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faee8003d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faef0001e50 tx=0x7faee80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06b06386321ca0c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.905+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0003b70 con 0x7faee8003970 2023-11-24T06:28:06.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.905+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef0003cd0 con 0x7faee8003970 2023-11-24T06:28:06.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.905+0000 7faf04976640 1 -- 172.21.15.16:0/1099970257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0005310 con 0x7faee8003970 2023-11-24T06:28:06.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.906+0000 7faf0697a640 1 -- 172.21.15.16:0/1099970257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 msgr2=0x7faee8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.906+0000 7faf0697a640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faee8003d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faef0001e50 tx=0x7faee80057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.906+0000 7faf0697a640 1 -- 172.21.15.16:0/1099970257 shutdown_connections 2023-11-24T06:28:06.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.906+0000 7faf0697a640 1 --2- 172.21.15.16:0/1099970257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faee8003d90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.906+0000 7faf0697a640 1 -- 172.21.15.16:0/1099970257 >> 172.21.15.16:0/1099970257 conn(0x7faf000b1d40 msgr2=0x7faf000b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:06.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.906+0000 7faf0697a640 1 -- 172.21.15.16:0/1099970257 shutdown_connections 2023-11-24T06:28:06.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.906+0000 7faf0697a640 1 -- 172.21.15.16:0/1099970257 wait complete. 2023-11-24T06:28:06.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.908+0000 7faf0697a640 1 Processor -- start 2023-11-24T06:28:06.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.908+0000 7faf0697a640 1 -- start start 2023-11-24T06:28:06.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.909+0000 7faf0697a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faf00159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.909+0000 7faf0697a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf0013c920 0x7faf000b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.909+0000 7faf0697a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf000ad030 0x7faf000ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.909+0000 7faf0697a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf000b4730 con 0x7faee8003970 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.909+0000 7faf0697a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf000b3ab0 con 0x7faf0013c920 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.909+0000 7faf0697a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf000b3d90 con 0x7faf000ad030 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.909+0000 7faf05978640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faf00159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.909+0000 7faf05978640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faf00159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39448/0 (socket says 172.21.15.16:39448) 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.909+0000 7faf05978640 1 -- 172.21.15.16:0/1958089699 learned_addr learned my addr 172.21.15.16:0/1958089699 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faf05177640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf0013c920 0x7faf000b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faf05978640 1 -- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf000ad030 msgr2=0x7faf000ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faf05978640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf000ad030 0x7faf000ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faf05978640 1 -- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf0013c920 msgr2=0x7faf000b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faf05978640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf0013c920 0x7faf000b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faf05978640 1 -- 172.21.15.16:0/1958089699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000b4070 con 0x7faee8003970 2023-11-24T06:28:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faf05978640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faf00159f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faef002bf70 tx=0x7faef00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faefe7fc640 1 -- 172.21.15.16:0/1958089699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef003e070 con 0x7faee8003970 2023-11-24T06:28:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faefe7fc640 1 -- 172.21.15.16:0/1958089699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef0006b50 con 0x7faee8003970 2023-11-24T06:28:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faefe7fc640 1 -- 172.21.15.16:0/1958089699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0007020 con 0x7faee8003970 2023-11-24T06:28:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.910+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf000b4350 con 0x7faee8003970 2023-11-24T06:28:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.911+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0015a460 con 0x7faee8003970 2023-11-24T06:28:06.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.912+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf00067b60 con 0x7faee8003970 2023-11-24T06:28:06.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.913+0000 7faefe7fc640 1 -- 172.21.15.16:0/1958089699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faef0007180 con 0x7faee8003970 2023-11-24T06:28:06.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.913+0000 7faefe7fc640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faeec06ae40 0x7faeec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:06.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.914+0000 7faefe7fc640 1 -- 172.21.15.16:0/1958089699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7faef00aad00 con 0x7faee8003970 2023-11-24T06:28:06.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.914+0000 7faf05177640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faeec06ae40 0x7faeec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:06.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.915+0000 7faf05177640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faeec06ae40 0x7faeec06d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faf00143890 tx=0x7faef4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:06.918+0000 7faefe7fc640 1 -- 172.21.15.16:0/1958089699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faef0075d20 con 0x7faee8003970 2023-11-24T06:28:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.154+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf0006a630 con 0x7faee8003970 2023-11-24T06:28:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.155+0000 7faefe7fc640 1 -- 172.21.15.16:0/1958089699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7faef0079340 con 0x7faee8003970 2023-11-24T06:28:07.158 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:07.158 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3019663874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3019663874},{"type":"v1","addr":"172.21.15.173:6837","nonce":3019663874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4227,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2036678386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2036678386},{"type":"v1","addr":"172.21.15.16:6834","nonce":2036678386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4230,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2239024365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2239024365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2239024365}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:26:56.460479+0000","modified":"2023-11-24T06:26:59.516602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/915142029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":915142029},{"type":"v1","addr":"172.21.15.173:6835","nonce":915142029}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:28:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-11-24T06:28:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faeec06ae40 msgr2=0x7faeec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faeec06ae40 0x7faeec06d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faf00143890 tx=0x7faef4008040 comp rx=0 tx=0).stop 2023-11-24T06:28:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 msgr2=0x7faf00159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faf00159f20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faef002bf70 tx=0x7faef00014a0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 shutdown_connections 2023-11-24T06:28:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faeec06ae40 0x7faeec06d300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf000ad030 0x7faf000ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf0013c920 0x7faf000b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 --2- 172.21.15.16:0/1958089699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee8003970 0x7faf00159f20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 >> 172.21.15.16:0/1958089699 conn(0x7faf000b1d40 msgr2=0x7faf0015dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.157+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 shutdown_connections 2023-11-24T06:28:07.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.158+0000 7faf0697a640 1 -- 172.21.15.16:0/1958089699 wait complete. 2023-11-24T06:28:07.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:07.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.395+0000 7f9bdf94c640 1 Processor -- start 2023-11-24T06:28:07.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.395+0000 7f9bdf94c640 1 -- start start 2023-11-24T06:28:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.397+0000 7f9bdf94c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9bd813df80 0x7f9bd815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.397+0000 7f9bdf94c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9bd80cf9a0 0x7f9bd80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.397+0000 7f9bdf94c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd806c470 con 0x7f9bd813e360 2023-11-24T06:28:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.397+0000 7f9bdf94c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd806c750 con 0x7f9bd813df80 2023-11-24T06:28:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.397+0000 7f9bdf94c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd806ca30 con 0x7f9bd80cf9a0 2023-11-24T06:28:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.397+0000 7f9bdd6e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9bd813df80 0x7f9bd815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.397+0000 7f9bdd6e8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9bd813df80 0x7f9bd815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39458/0 (socket says 172.21.15.16:39458) 2023-11-24T06:28:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.397+0000 7f9bdd6e8640 1 -- 172.21.15.16:0/117237852 learned_addr learned my addr 172.21.15.16:0/117237852 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:07.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.398+0000 7f9bddee9640 1 --2- 172.21.15.16:0/117237852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9bd80cf9a0 0x7f9bd80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.398+0000 7f9bdd6e8640 1 -- 172.21.15.16:0/117237852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9bd80cf9a0 msgr2=0x7f9bd80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.398+0000 7f9bdd6e8640 1 --2- 172.21.15.16:0/117237852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9bd80cf9a0 0x7f9bd80cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.398+0000 7f9bdd6e8640 1 -- 172.21.15.16:0/117237852 >> v1:172.21.15.173:6789/0 conn(0x7f9bd813e360 legacy=0x7f9bd80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.398+0000 7f9bdd6e8640 1 -- 172.21.15.16:0/117237852 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd806cd10 con 0x7f9bd813df80 2023-11-24T06:28:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.399+0000 7f9bdd6e8640 1 --2- 172.21.15.16:0/117237852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813df80 0x7f9bd815bee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009870 tx=0x7f9bd402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d934da6cd53440a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.399+0000 7f9bcffff640 1 -- 172.21.15.16:0/117237852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4031640 con 0x7f9bd813df80 2023-11-24T06:28:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bcffff640 1 -- 172.21.15.16:0/117237852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd40317a0 con 0x7f9bd813df80 2023-11-24T06:28:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bcffff640 1 -- 172.21.15.16:0/117237852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4031b30 con 0x7f9bd813df80 2023-11-24T06:28:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bdf94c640 1 -- 172.21.15.16:0/117237852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813df80 msgr2=0x7f9bd815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bdf94c640 1 --2- 172.21.15.16:0/117237852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813df80 0x7f9bd815bee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009870 tx=0x7f9bd402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bdf94c640 1 -- 172.21.15.16:0/117237852 shutdown_connections 2023-11-24T06:28:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bdf94c640 1 --2- 172.21.15.16:0/117237852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9bd80cf9a0 0x7f9bd80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bdf94c640 1 --2- 172.21.15.16:0/117237852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813df80 0x7f9bd815bee0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bdf94c640 1 -- 172.21.15.16:0/117237852 >> 172.21.15.16:0/117237852 conn(0x7f9bd806a800 msgr2=0x7f9bd806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bdf94c640 1 -- 172.21.15.16:0/117237852 shutdown_connections 2023-11-24T06:28:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.400+0000 7f9bdf94c640 1 -- 172.21.15.16:0/117237852 wait complete. 2023-11-24T06:28:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.402+0000 7f9bdf94c640 1 Processor -- start 2023-11-24T06:28:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.402+0000 7f9bdf94c640 1 -- start start 2023-11-24T06:28:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.403+0000 7f9bdf94c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd80cf9a0 0x7f9bd81391a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.403+0000 7f9bdf94c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9bd813df80 0x7f9bd813ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bdf94c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813e360 0x7f9bd80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bdf94c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81e4bf0 con 0x7f9bd813e360 2023-11-24T06:28:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bdf94c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd8153880 con 0x7f9bd80cf9a0 2023-11-24T06:28:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bdf94c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd813c4b0 con 0x7f9bd813df80 2023-11-24T06:28:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bddee9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813e360 0x7f9bd80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bddee9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813e360 0x7f9bd80cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39468/0 (socket says 172.21.15.16:39468) 2023-11-24T06:28:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bddee9640 1 -- 172.21.15.16:0/334764813 learned_addr learned my addr 172.21.15.16:0/334764813 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bdd6e8640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd80cf9a0 0x7f9bd81391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bddee9640 1 -- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9bd813df80 msgr2=0x7f9bd813ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bddee9640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9bd813df80 0x7f9bd813ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bddee9640 1 -- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd80cf9a0 msgr2=0x7f9bd81391a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bddee9640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd80cf9a0 0x7f9bd81391a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.404+0000 7f9bddee9640 1 -- 172.21.15.16:0/334764813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd40094d0 con 0x7f9bd813e360 2023-11-24T06:28:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.405+0000 7f9bddee9640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813e360 0x7f9bd80cd840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009fd0 tx=0x7f9bd000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.405+0000 7f9bcdffb640 1 -- 172.21.15.16:0/334764813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0014420 con 0x7f9bd813e360 2023-11-24T06:28:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.405+0000 7f9bcdffb640 1 -- 172.21.15.16:0/334764813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd0014580 con 0x7f9bd813e360 2023-11-24T06:28:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.405+0000 7f9bcdffb640 1 -- 172.21.15.16:0/334764813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0014930 con 0x7f9bd813e360 2023-11-24T06:28:07.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.405+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd813bb80 con 0x7f9bd813e360 2023-11-24T06:28:07.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.405+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd813c140 con 0x7f9bd813e360 2023-11-24T06:28:07.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.406+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd813c660 con 0x7f9bd813e360 2023-11-24T06:28:07.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.407+0000 7f9bcdffb640 1 -- 172.21.15.16:0/334764813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9bd0014a90 con 0x7f9bd813e360 2023-11-24T06:28:07.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.408+0000 7f9bcdffb640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9bbc06ae40 0x7f9bbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.408+0000 7f9bcdffb640 1 -- 172.21.15.16:0/334764813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7f9bd0089fb0 con 0x7f9bd813e360 2023-11-24T06:28:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.409+0000 7f9bdd6e8640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9bbc06ae40 0x7f9bbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.410+0000 7f9bdd6e8640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9bbc06ae40 0x7f9bbc06d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4036000 tx=0x7f9bd403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.413+0000 7f9bcdffb640 1 -- 172.21.15.16:0/334764813 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bd00bce90 con 0x7f9bd813e360 2023-11-24T06:28:07.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.607+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bd80675d0 con 0x7f9bd813e360 2023-11-24T06:28:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.608+0000 7f9bcdffb640 1 -- 172.21.15.16:0/334764813 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16773 (secure 0 0 0) 0x7f9bd813c660 con 0x7f9bd813e360 2023-11-24T06:28:07.612 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:07.612 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:27:44.028259+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:26:54.688269+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:26:55.486606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2},{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:07.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9bbc06ae40 msgr2=0x7f9bbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9bbc06ae40 0x7f9bbc06d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4036000 tx=0x7f9bd403c040 comp rx=0 tx=0).stop 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813e360 msgr2=0x7f9bd80cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813e360 0x7f9bd80cd840 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009fd0 tx=0x7f9bd000e4b0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 shutdown_connections 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9bbc06ae40 0x7f9bbc06d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd813e360 0x7f9bd80cd840 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9bd813df80 0x7f9bd813ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 --2- 172.21.15.16:0/334764813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bd80cf9a0 0x7f9bd81391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 >> 172.21.15.16:0/334764813 conn(0x7f9bd806a800 msgr2=0x7f9bd8153a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 shutdown_connections 2023-11-24T06:28:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.611+0000 7f9bdf94c640 1 -- 172.21.15.16:0/334764813 wait complete. 2023-11-24T06:28:07.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:28:07.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.859+0000 7fe5749db640 1 Processor -- start 2023-11-24T06:28:07.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.860+0000 7fe5749db640 1 -- start start 2023-11-24T06:28:07.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.861+0000 7fe5749db640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe570071410 0x7fe570146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.861+0000 7fe5749db640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe57006c7c0 0x7fe5700717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.861+0000 7fe5749db640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57006cc20 con 0x7fe570071d30 2023-11-24T06:28:07.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.861+0000 7fe5749db640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57006cec0 con 0x7fe570071410 2023-11-24T06:28:07.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.861+0000 7fe5749db640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57006d1a0 con 0x7fe57006c7c0 2023-11-24T06:28:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.862+0000 7fe56ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe570071d30 0x7fe5700b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51326/0 (socket says 172.21.15.16:51326) 2023-11-24T06:28:07.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.862+0000 7fe56ffff640 1 -- 172.21.15.16:0/3423264764 learned_addr learned my addr 172.21.15.16:0/3423264764 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:07.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.862+0000 7fe567fff640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe57006c7c0 0x7fe5700717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.862+0000 7fe56f7fe640 1 --2- 172.21.15.16:0/3423264764 >> v2:172.21.15.173:3300/0 conn(0x7fe570071410 0x7fe570146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.862+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292089499 0 0) 0x7fe57006cc20 con 0x7fe570071d30 2023-11-24T06:28:07.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.862+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe55c003520 con 0x7fe570071d30 2023-11-24T06:28:07.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 414159602 0 0) 0x7fe55c003520 con 0x7fe570071d30 2023-11-24T06:28:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe57006c7c0 msgr2=0x7fe5700717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe57006c7c0 0x7fe5700717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 >> v2:172.21.15.173:3300/0 conn(0x7fe570071410 msgr2=0x7fe570146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 --2- 172.21.15.16:0/3423264764 >> v2:172.21.15.173:3300/0 conn(0x7fe570071410 0x7fe570146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57006d480 con 0x7fe570071d30 2023-11-24T06:28:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fe5680019f0 con 0x7fe570071d30 2023-11-24T06:28:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 >> v1:172.21.15.16:6789/0 conn(0x7fe570071d30 legacy=0x7fe5700b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c003970 0x7fe55c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe57006c7c0 0x7fe55c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55c004d00 0x7fe55c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57006cec0 con 0x7fe55c004d00 2023-11-24T06:28:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57006d1a0 con 0x7fe55c003970 2023-11-24T06:28:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c00b980 con 0x7fe57006c7c0 2023-11-24T06:28:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.863+0000 7fe56ffff640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55c004d00 0x7fe55c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.864+0000 7fe56f7fe640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c003970 0x7fe55c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.864+0000 7fe567fff640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe57006c7c0 0x7fe55c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.864+0000 7fe56ffff640 1 -- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe57006c7c0 msgr2=0x7fe55c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.864+0000 7fe56ffff640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe57006c7c0 0x7fe55c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.864+0000 7fe56ffff640 1 -- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c003970 msgr2=0x7fe55c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.864+0000 7fe56ffff640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe55c003970 0x7fe55c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.864+0000 7fe56ffff640 1 -- 172.21.15.16:0/3423264764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c003520 con 0x7fe55c004d00 2023-11-24T06:28:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.865+0000 7fe56ffff640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55c004d00 0x7fe55c005180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe56800a860 tx=0x7fe55c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a80b346a512ec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.865+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568034630 con 0x7fe55c004d00 2023-11-24T06:28:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.865+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56802f070 con 0x7fe55c004d00 2023-11-24T06:28:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.865+0000 7fe56effd640 1 -- 172.21.15.16:0/3423264764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680349f0 con 0x7fe55c004d00 2023-11-24T06:28:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.866+0000 7fe5749db640 1 -- 172.21.15.16:0/3423264764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55c004d00 msgr2=0x7fe55c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.866+0000 7fe5749db640 1 --2- 172.21.15.16:0/3423264764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55c004d00 0x7fe55c005180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe56800a860 tx=0x7fe55c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.866+0000 7fe56ffff640 1 -- 172.21.15.16:0/3423264764 reap_dead start 2023-11-24T06:28:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.866+0000 7fe5749db640 1 -- 172.21.15.16:0/3423264764 shutdown_connections 2023-11-24T06:28:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.866+0000 7fe5749db640 1 -- 172.21.15.16:0/3423264764 >> 172.21.15.16:0/3423264764 conn(0x7fe57006a800 msgr2=0x7fe57006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.866+0000 7fe5749db640 1 -- 172.21.15.16:0/3423264764 shutdown_connections 2023-11-24T06:28:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.866+0000 7fe5749db640 1 -- 172.21.15.16:0/3423264764 wait complete. 2023-11-24T06:28:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.868+0000 7fe5749db640 1 Processor -- start 2023-11-24T06:28:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.868+0000 7fe5749db640 1 -- start start 2023-11-24T06:28:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.869+0000 7fe5749db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5700c8960 0x7fe5701307f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.869+0000 7fe5749db640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5700c8d40 0x7fe5700ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.869+0000 7fe5749db640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5700cdd90 0x7fe5700ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.869+0000 7fe5749db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700ce750 con 0x7fe5700c8960 2023-11-24T06:28:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.869+0000 7fe5749db640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5701317a0 con 0x7fe5700c8d40 2023-11-24T06:28:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.869+0000 7fe5749db640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe570130f00 con 0x7fe5700cdd90 2023-11-24T06:28:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.869+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5700c8960 0x7fe5701307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.869+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5700c8960 0x7fe5701307f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39486/0 (socket says 172.21.15.16:39486) 2023-11-24T06:28:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.869+0000 7fe56f7fe640 1 -- 172.21.15.16:0/1491015971 learned_addr learned my addr 172.21.15.16:0/1491015971 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.870+0000 7fe56ffff640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5700cdd90 0x7fe5700ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.870+0000 7fe567fff640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5700c8d40 0x7fe5700ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.870+0000 7fe56f7fe640 1 -- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5700cdd90 msgr2=0x7fe5700ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.870+0000 7fe56f7fe640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5700cdd90 0x7fe5700ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.870+0000 7fe56f7fe640 1 -- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5700c8d40 msgr2=0x7fe5700ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.870+0000 7fe56f7fe640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5700c8d40 0x7fe5700ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.870+0000 7fe56f7fe640 1 -- 172.21.15.16:0/1491015971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5701311e0 con 0x7fe5700c8960 2023-11-24T06:28:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.870+0000 7fe56f7fe640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5700c8960 0x7fe5701307f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe560011600 tx=0x7fe560011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.870+0000 7fe56cff9640 1 -- 172.21.15.16:0/1491015971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56000f920 con 0x7fe5700c8960 2023-11-24T06:28:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.871+0000 7fe56cff9640 1 -- 172.21.15.16:0/1491015971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56000fa80 con 0x7fe5700c8960 2023-11-24T06:28:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.871+0000 7fe56cff9640 1 -- 172.21.15.16:0/1491015971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56000fdf0 con 0x7fe5700c8960 2023-11-24T06:28:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.871+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5701314c0 con 0x7fe5700c8960 2023-11-24T06:28:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.871+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5700c3d70 con 0x7fe5700c8960 2023-11-24T06:28:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.872+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe570067450 con 0x7fe5700c8960 2023-11-24T06:28:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.873+0000 7fe56cff9640 1 -- 172.21.15.16:0/1491015971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe560015460 con 0x7fe5700c8960 2023-11-24T06:28:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.874+0000 7fe56cff9640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe55006ae70 0x7fe55006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.874+0000 7fe56cff9640 1 -- 172.21.15.16:0/1491015971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5833+0+0 (secure 0 0 0) 0x7fe560089b50 con 0x7fe5700c8960 2023-11-24T06:28:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.874+0000 7fe567fff640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe55006ae70 0x7fe55006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.876+0000 7fe567fff640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe55006ae70 0x7fe55006d330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe5700c5330 tx=0x7fe558008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:07.879+0000 7fe56cff9640 1 -- 172.21.15.16:0/1491015971 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe560054b70 con 0x7fe5700c8960 2023-11-24T06:28:08.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.117+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe57015d070 con 0x7fe5700c8960 2023-11-24T06:28:08.121 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-24T06:28:08.118+0000 7f082ae01640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:28:08.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.121+0000 7fe56cff9640 1 -- 172.21.15.16:0/1491015971 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe560022090 con 0x7fe5700c8960 2023-11-24T06:28:08.125 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.124+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe55006ae70 msgr2=0x7fe55006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.124+0000 7fe5749db640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe55006ae70 0x7fe55006d330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe5700c5330 tx=0x7fe558008040 comp rx=0 tx=0).stop 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.124+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5700c8960 msgr2=0x7fe5701307f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.124+0000 7fe5749db640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5700c8960 0x7fe5701307f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe560011600 tx=0x7fe560011ac0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.124+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 shutdown_connections 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.124+0000 7fe5749db640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe55006ae70 0x7fe55006d330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.124+0000 7fe5749db640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5700cdd90 0x7fe5700ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.124+0000 7fe5749db640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5700c8d40 0x7fe5700ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.124+0000 7fe5749db640 1 --2- 172.21.15.16:0/1491015971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5700c8960 0x7fe5701307f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.125+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 >> 172.21.15.16:0/1491015971 conn(0x7fe57006a800 msgr2=0x7fe5700b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:08.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.125+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 shutdown_connections 2023-11-24T06:28:08.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.125+0000 7fe5749db640 1 -- 172.21.15.16:0/1491015971 wait complete. 2023-11-24T06:28:08.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:28:08.156 INFO:tasks.ceph.mds.b.smithi173.stderr: -3834> 2023-11-24T06:27:01.370+0000 7f0fc80d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:28:08.244 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2023-11-24T06:28:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.371+0000 7f3371567640 1 Processor -- start 2023-11-24T06:28:08.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.372+0000 7f3371567640 1 -- start start 2023-11-24T06:28:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.373+0000 7f3371567640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f336c0adf20 0x7f336c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.373+0000 7f3371567640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f336c15bf10 0x7f336c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f3371567640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336c15f4e0 con 0x7f336c0cc980 2023-11-24T06:28:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f3371567640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336c15f970 con 0x7f336c0adf20 2023-11-24T06:28:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f3371567640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336c0ccdd0 con 0x7f336c15bf10 2023-11-24T06:28:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f336c0adf20 0x7f336c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f336c0adf20 0x7f336c070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39494/0 (socket says 172.21.15.16:39494) 2023-11-24T06:28:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336bfff640 1 -- 172.21.15.16:0/996212394 learned_addr learned my addr 172.21.15.16:0/996212394 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:08.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f3370d66640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f336c0cc980 0x7f336c143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51332/0 (socket says 172.21.15.16:51332) 2023-11-24T06:28:08.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336b7fe640 1 --2- 172.21.15.16:0/996212394 >> v2:172.21.15.173:3300/0 conn(0x7f336c15bf10 0x7f336c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336bfff640 1 -- 172.21.15.16:0/996212394 >> v2:172.21.15.173:3300/0 conn(0x7f336c15bf10 msgr2=0x7f336c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336bfff640 1 --2- 172.21.15.16:0/996212394 >> v2:172.21.15.173:3300/0 conn(0x7f336c15bf10 0x7f336c0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336bfff640 1 -- 172.21.15.16:0/996212394 >> v1:172.21.15.16:6789/0 conn(0x7f336c0cc980 legacy=0x7f336c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336bfff640 1 -- 172.21.15.16:0/996212394 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c071130 con 0x7f336c0adf20 2023-11-24T06:28:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336affd640 1 -- 172.21.15.16:0/996212394 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256780486 0 0) 0x7f336c15f4e0 con 0x7f336c0cc980 2023-11-24T06:28:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.374+0000 7f336b7fe640 1 --2- 172.21.15.16:0/996212394 >> v2:172.21.15.173:3300/0 conn(0x7f336c15bf10 0x7f336c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:08.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.376+0000 7f336bfff640 1 --2- 172.21.15.16:0/996212394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c0adf20 0x7f336c070a30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f336c15e9e0 tx=0x7f335c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658240f87ee7a1b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.376+0000 7f336affd640 1 -- 172.21.15.16:0/996212394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c031870 con 0x7f336c0adf20 2023-11-24T06:28:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.376+0000 7f336affd640 1 -- 172.21.15.16:0/996212394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335c0319d0 con 0x7f336c0adf20 2023-11-24T06:28:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.376+0000 7f336affd640 1 -- 172.21.15.16:0/996212394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c031d60 con 0x7f336c0adf20 2023-11-24T06:28:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.377+0000 7f3371567640 1 -- 172.21.15.16:0/996212394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c0adf20 msgr2=0x7f336c070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.377+0000 7f3371567640 1 --2- 172.21.15.16:0/996212394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c0adf20 0x7f336c070a30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f336c15e9e0 tx=0x7f335c02b910 comp rx=0 tx=0).stop 2023-11-24T06:28:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.377+0000 7f3371567640 1 -- 172.21.15.16:0/996212394 shutdown_connections 2023-11-24T06:28:08.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.377+0000 7f3371567640 1 --2- 172.21.15.16:0/996212394 >> v2:172.21.15.173:3300/0 conn(0x7f336c15bf10 0x7f336c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.377+0000 7f3371567640 1 --2- 172.21.15.16:0/996212394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c0adf20 0x7f336c070a30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.377+0000 7f3371567640 1 -- 172.21.15.16:0/996212394 >> 172.21.15.16:0/996212394 conn(0x7f336c06a7d0 msgr2=0x7f336c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.377+0000 7f3371567640 1 -- 172.21.15.16:0/996212394 shutdown_connections 2023-11-24T06:28:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.377+0000 7f3371567640 1 -- 172.21.15.16:0/996212394 wait complete. 2023-11-24T06:28:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.379+0000 7f3371567640 1 Processor -- start 2023-11-24T06:28:08.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.379+0000 7f3371567640 1 -- start start 2023-11-24T06:28:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3371567640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f336c0adf20 0x7f336c0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3371567640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f336c0cc980 0x7f336c15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3371567640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c15bf10 0x7f336c0b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3371567640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c15af20 con 0x7f336c15bf10 2023-11-24T06:28:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3371567640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0c8110 con 0x7f336c0adf20 2023-11-24T06:28:08.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3371567640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0b23d0 con 0x7f336c0cc980 2023-11-24T06:28:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3370d66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c15bf10 0x7f336c0b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3370d66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c15bf10 0x7f336c0b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39510/0 (socket says 172.21.15.16:39510) 2023-11-24T06:28:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3370d66640 1 -- 172.21.15.16:0/306247250 learned_addr learned my addr 172.21.15.16:0/306247250 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f336bfff640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f336c0adf20 0x7f336c0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f336b7fe640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f336c0cc980 0x7f336c15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.380+0000 7f3370d66640 1 -- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f336c0cc980 msgr2=0x7f336c15a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3370d66640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f336c0cc980 0x7f336c15a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3370d66640 1 -- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f336c0adf20 msgr2=0x7f336c0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3370d66640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f336c0adf20 0x7f336c0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3370d66640 1 -- 172.21.15.16:0/306247250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c0094d0 con 0x7f336c15bf10 2023-11-24T06:28:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f336b7fe640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f336c0cc980 0x7f336c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f336bfff640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f336c0adf20 0x7f336c0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:08.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3370d66640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c15bf10 0x7f336c0b1e90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f336400c780 tx=0x7f336400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3368ff9640 1 -- 172.21.15.16:0/306247250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336400ce90 con 0x7f336c15bf10 2023-11-24T06:28:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3368ff9640 1 -- 172.21.15.16:0/306247250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336400a5e0 con 0x7f336c15bf10 2023-11-24T06:28:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3368ff9640 1 -- 172.21.15.16:0/306247250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336400a970 con 0x7f336c15bf10 2023-11-24T06:28:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c0b2670 con 0x7f336c15bf10 2023-11-24T06:28:08.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.381+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c0b2c30 con 0x7f336c15bf10 2023-11-24T06:28:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.382+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336c067450 con 0x7f336c15bf10 2023-11-24T06:28:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.384+0000 7f3368ff9640 1 -- 172.21.15.16:0/306247250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f336400aad0 con 0x7f336c15bf10 2023-11-24T06:28:08.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.385+0000 7f3368ff9640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae40 0x7f335806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.385+0000 7f3368ff9640 1 -- 172.21.15.16:0/306247250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f3364088bb0 con 0x7f336c15bf10 2023-11-24T06:28:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.386+0000 7f336bfff640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae40 0x7f335806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:08.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.387+0000 7f336bfff640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae40 0x7f335806d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f335c036000 tx=0x7f335c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:08.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.389+0000 7f3368ff9640 1 -- 172.21.15.16:0/306247250 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3364053b80 con 0x7f336c15bf10 2023-11-24T06:28:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:08.630+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f336c15e410 con 0x7f336c15bf10 2023-11-24T06:28:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.188+0000 7f3368ff9640 1 -- 172.21.15.16:0/306247250 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3364018020 con 0x7f336c15bf10 2023-11-24T06:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae40 msgr2=0x7f335806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae40 0x7f335806d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f335c036000 tx=0x7f335c03c040 comp rx=0 tx=0).stop 2023-11-24T06:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c15bf10 msgr2=0x7f336c0b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c15bf10 0x7f336c0b1e90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f336400c780 tx=0x7f336400cc40 comp rx=0 tx=0).stop 2023-11-24T06:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 shutdown_connections 2023-11-24T06:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae40 0x7f335806d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336c15bf10 0x7f336c0b1e90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f336c0cc980 0x7f336c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 --2- 172.21.15.16:0/306247250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f336c0adf20 0x7f336c0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.191+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 >> 172.21.15.16:0/306247250 conn(0x7f336c06a7d0 msgr2=0x7f336c0cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.192+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 shutdown_connections 2023-11-24T06:28:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.192+0000 7f3371567640 1 -- 172.21.15.16:0/306247250 wait complete. 2023-11-24T06:28:09.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:28:09.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.438+0000 7f337b14e640 1 Processor -- start 2023-11-24T06:28:09.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.439+0000 7f337b14e640 1 -- start start 2023-11-24T06:28:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.440+0000 7f337b14e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b3f30 0x7f33740d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.440+0000 7f337b14e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33740d0a10 con 0x7f3374160920 2023-11-24T06:28:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.441+0000 7f337b14e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33740cf080 con 0x7f33740d0100 2023-11-24T06:28:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.441+0000 7f337b14e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33740c6be0 con 0x7f33740b3f30 2023-11-24T06:28:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.441+0000 7f337a14c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3374160920 0x7f33740cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51344/0 (socket says 172.21.15.16:51344) 2023-11-24T06:28:09.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.441+0000 7f337a14c640 1 -- 172.21.15.16:0/2348639971 learned_addr learned my addr 172.21.15.16:0/2348639971 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.441+0000 7f337a94d640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b3f30 0x7f33740d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.441+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183920220 0 0) 0x7f33740d0a10 con 0x7f3374160920 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.441+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3364003520 con 0x7f3374160920 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2116504860 0 0) 0x7f3364003520 con 0x7f3374160920 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b3f30 msgr2=0x7f33740d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b3f30 0x7f33740d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 >> v1:172.21.15.173:6789/0 conn(0x7f33740d0100 legacy=0x7f33740d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33740ce7e0 con 0x7f3374160920 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f33680019f0 con 0x7f3374160920 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 >> v1:172.21.15.16:6789/0 conn(0x7f3374160920 legacy=0x7f33740cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b3f30 0x7f33640039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3364004d00 0x7f3364005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33740c6be0 con 0x7f3364003f20 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336400ba60 con 0x7f3364004d00 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336400bc20 con 0x7f33740b3f30 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.442+0000 7f337994b640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.443+0000 7f337994b640 1 -- 172.21.15.16:0/2348639971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b3f30 msgr2=0x7f33640039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.443+0000 7f337994b640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b3f30 0x7f33640039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.443+0000 7f337994b640 1 -- 172.21.15.16:0/2348639971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3364004d00 msgr2=0x7f3364005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.443+0000 7f337994b640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3364004d00 0x7f3364005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.443+0000 7f337994b640 1 -- 172.21.15.16:0/2348639971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3364003520 con 0x7f3364003f20 2023-11-24T06:28:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.444+0000 7f337994b640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33640047c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3360001750 tx=0x7f33640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f0d118da9cea8c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.444+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600052d0 con 0x7f3364003f20 2023-11-24T06:28:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.444+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336002e050 con 0x7f3364003f20 2023-11-24T06:28:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.444+0000 7f337914a640 1 -- 172.21.15.16:0/2348639971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360003830 con 0x7f3364003f20 2023-11-24T06:28:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.444+0000 7f337b14e640 1 -- 172.21.15.16:0/2348639971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 msgr2=0x7f33640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.444+0000 7f337b14e640 1 --2- 172.21.15.16:0/2348639971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33640047c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3360001750 tx=0x7f33640057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.445+0000 7f337a94d640 1 -- 172.21.15.16:0/2348639971 reap_dead start 2023-11-24T06:28:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.445+0000 7f337b14e640 1 -- 172.21.15.16:0/2348639971 shutdown_connections 2023-11-24T06:28:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.445+0000 7f337b14e640 1 -- 172.21.15.16:0/2348639971 >> 172.21.15.16:0/2348639971 conn(0x7f33740d3090 msgr2=0x7f33740d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.445+0000 7f337b14e640 1 -- 172.21.15.16:0/2348639971 shutdown_connections 2023-11-24T06:28:09.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.445+0000 7f337b14e640 1 -- 172.21.15.16:0/2348639971 wait complete. 2023-11-24T06:28:09.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.446+0000 7f337b14e640 1 Processor -- start 2023-11-24T06:28:09.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.447+0000 7f337b14e640 1 -- start start 2023-11-24T06:28:09.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337b14e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33740b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337b14e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b6540 0x7f3374154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337b14e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33740b69a0 0x7f33741546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337b14e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337415a2e0 con 0x7f3364003f20 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337b14e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33740b7020 con 0x7f33740b69a0 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337b14e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337415f7f0 con 0x7f33740b6540 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337a14c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33740b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337a14c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33740b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39524/0 (socket says 172.21.15.16:39524) 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337a14c640 1 -- 172.21.15.16:0/1446348849 learned_addr learned my addr 172.21.15.16:0/1446348849 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337a14c640 1 -- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b6540 msgr2=0x7f3374154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337a14c640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b6540 0x7f3374154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337a14c640 1 -- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33740b69a0 msgr2=0x7f33741546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337a14c640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33740b69a0 0x7f33741546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.448+0000 7f337a14c640 1 -- 172.21.15.16:0/1446348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337415ea30 con 0x7f3364003f20 2023-11-24T06:28:09.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.449+0000 7f337a14c640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33740b85a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3368002230 tx=0x7f336800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:09.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.449+0000 7f3372ffd640 1 -- 172.21.15.16:0/1446348849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336800a740 con 0x7f3364003f20 2023-11-24T06:28:09.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.449+0000 7f3372ffd640 1 -- 172.21.15.16:0/1446348849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336800a8a0 con 0x7f3364003f20 2023-11-24T06:28:09.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.449+0000 7f3372ffd640 1 -- 172.21.15.16:0/1446348849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336800ac70 con 0x7f3364003f20 2023-11-24T06:28:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.449+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337415ed10 con 0x7f3364003f20 2023-11-24T06:28:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.449+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337415f2d0 con 0x7f3364003f20 2023-11-24T06:28:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.450+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3374067450 con 0x7f3364003f20 2023-11-24T06:28:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.451+0000 7f3372ffd640 1 -- 172.21.15.16:0/1446348849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33680070d0 con 0x7f3364003f20 2023-11-24T06:28:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.452+0000 7f3372ffd640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae90 0x7f335806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.452+0000 7f3372ffd640 1 -- 172.21.15.16:0/1446348849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f3368089350 con 0x7f3364003f20 2023-11-24T06:28:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.453+0000 7f337994b640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae90 0x7f335806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:09.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.454+0000 7f337994b640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae90 0x7f335806d350 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3360033000 tx=0x7f336002f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:09.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.456+0000 7f3372ffd640 1 -- 172.21.15.16:0/1446348849 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3368054310 con 0x7f3364003f20 2023-11-24T06:28:09.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:09.649+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f337408c7e0 con 0x7f3364003f20 2023-11-24T06:28:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.207+0000 7f3372ffd640 1 -- 172.21.15.16:0/1446348849 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f336805ce60 con 0x7f3364003f20 2023-11-24T06:28:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:28:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.211+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae90 msgr2=0x7f335806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.211+0000 7f337b14e640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae90 0x7f335806d350 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3360033000 tx=0x7f336002f000 comp rx=0 tx=0).stop 2023-11-24T06:28:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.211+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 msgr2=0x7f33740b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.211+0000 7f337b14e640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33740b85a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3368002230 tx=0x7f336800cf00 comp rx=0 tx=0).stop 2023-11-24T06:28:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.212+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 shutdown_connections 2023-11-24T06:28:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.212+0000 7f337b14e640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335806ae90 0x7f335806d350 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.212+0000 7f337b14e640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33740b69a0 0x7f33741546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.212+0000 7f337b14e640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33740b6540 0x7f3374154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.212+0000 7f337b14e640 1 --2- 172.21.15.16:0/1446348849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3364003f20 0x7f33740b85a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.212+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 >> 172.21.15.16:0/1446348849 conn(0x7f33740d3090 msgr2=0x7f337406aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.213+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 shutdown_connections 2023-11-24T06:28:10.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.213+0000 7f337b14e640 1 -- 172.21.15.16:0/1446348849 wait complete. 2023-11-24T06:28:10.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:28:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.473+0000 7f6249cbb640 1 Processor -- start 2023-11-24T06:28:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.473+0000 7f6249cbb640 1 -- start start 2023-11-24T06:28:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.475+0000 7f6249cbb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f624414b2c0 0x7f62440cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.475+0000 7f6249cbb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f62440cf7d0 0x7f6244070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.475+0000 7f6249cbb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440cfbf0 con 0x7f6244070ea0 2023-11-24T06:28:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.475+0000 7f6249cbb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624414b6a0 con 0x7f624414b2c0 2023-11-24T06:28:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.475+0000 7f6249cbb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6244071690 con 0x7f62440cf7d0 2023-11-24T06:28:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.476+0000 7f62437fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f624414b2c0 0x7f62440cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.476+0000 7f6243fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6244070ea0 0x7f62440b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51346/0 (socket says 172.21.15.16:51346) 2023-11-24T06:28:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.476+0000 7f6243fff640 1 -- 172.21.15.16:0/2466432823 learned_addr learned my addr 172.21.15.16:0/2466432823 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.476+0000 7f623bfff640 1 --2- 172.21.15.16:0/2466432823 >> v2:172.21.15.173:3300/0 conn(0x7f62440cf7d0 0x7f6244070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.476+0000 7f62437fe640 1 -- 172.21.15.16:0/2466432823 >> v2:172.21.15.173:3300/0 conn(0x7f62440cf7d0 msgr2=0x7f6244070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.476+0000 7f62437fe640 1 --2- 172.21.15.16:0/2466432823 >> v2:172.21.15.173:3300/0 conn(0x7f62440cf7d0 0x7f6244070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.476+0000 7f62437fe640 1 -- 172.21.15.16:0/2466432823 >> v1:172.21.15.16:6789/0 conn(0x7f6244070ea0 legacy=0x7f62440b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.476+0000 7f62437fe640 1 -- 172.21.15.16:0/2466432823 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244071970 con 0x7f624414b2c0 2023-11-24T06:28:10.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.476+0000 7f6242ffd640 1 -- 172.21.15.16:0/2466432823 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763880569 0 0) 0x7f62440cfbf0 con 0x7f6244070ea0 2023-11-24T06:28:10.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.477+0000 7f62437fe640 1 --2- 172.21.15.16:0/2466432823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624414b2c0 0x7f62440cf290 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f62440cfea0 tx=0x7f623402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa71537542a0322 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:10.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.478+0000 7f6242ffd640 1 -- 172.21.15.16:0/2466432823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234031830 con 0x7f624414b2c0 2023-11-24T06:28:10.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.478+0000 7f6242ffd640 1 -- 172.21.15.16:0/2466432823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6234031990 con 0x7f624414b2c0 2023-11-24T06:28:10.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.478+0000 7f6242ffd640 1 -- 172.21.15.16:0/2466432823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234031d20 con 0x7f624414b2c0 2023-11-24T06:28:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.478+0000 7f6249cbb640 1 -- 172.21.15.16:0/2466432823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624414b2c0 msgr2=0x7f62440cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.478+0000 7f6249cbb640 1 --2- 172.21.15.16:0/2466432823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624414b2c0 0x7f62440cf290 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f62440cfea0 tx=0x7f623402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.479+0000 7f6249cbb640 1 -- 172.21.15.16:0/2466432823 shutdown_connections 2023-11-24T06:28:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.479+0000 7f6249cbb640 1 --2- 172.21.15.16:0/2466432823 >> v2:172.21.15.173:3300/0 conn(0x7f62440cf7d0 0x7f6244070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.479+0000 7f6249cbb640 1 --2- 172.21.15.16:0/2466432823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624414b2c0 0x7f62440cf290 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.479+0000 7f6249cbb640 1 -- 172.21.15.16:0/2466432823 >> 172.21.15.16:0/2466432823 conn(0x7f624406a800 msgr2=0x7f624406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:10.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.479+0000 7f6249cbb640 1 -- 172.21.15.16:0/2466432823 shutdown_connections 2023-11-24T06:28:10.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.479+0000 7f6249cbb640 1 -- 172.21.15.16:0/2466432823 wait complete. 2023-11-24T06:28:10.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.480+0000 7f6249cbb640 1 Processor -- start 2023-11-24T06:28:10.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.481+0000 7f6249cbb640 1 -- start start 2023-11-24T06:28:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.481+0000 7f6249cbb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6244070ea0 0x7f6244161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.481+0000 7f6249cbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440cf7d0 0x7f6244147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.481+0000 7f6249cbb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f624414b2c0 0x7f62441420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f6249cbb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244163060 con 0x7f62440cf7d0 2023-11-24T06:28:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f6249cbb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244147bd0 con 0x7f6244070ea0 2023-11-24T06:28:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f6249cbb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6244162450 con 0x7f624414b2c0 2023-11-24T06:28:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f623bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440cf7d0 0x7f6244147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f623bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440cf7d0 0x7f6244147690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39542/0 (socket says 172.21.15.16:39542) 2023-11-24T06:28:10.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f623bfff640 1 -- 172.21.15.16:0/824869345 learned_addr learned my addr 172.21.15.16:0/824869345 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:10.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f62437fe640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6244070ea0 0x7f6244161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f6243fff640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f624414b2c0 0x7f62441420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f623bfff640 1 -- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f624414b2c0 msgr2=0x7f62441420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f623bfff640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f624414b2c0 0x7f62441420a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f623bfff640 1 -- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6244070ea0 msgr2=0x7f6244161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f623bfff640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6244070ea0 0x7f6244161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.482+0000 7f623bfff640 1 -- 172.21.15.16:0/824869345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6234009400 con 0x7f62440cf7d0 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.483+0000 7f623bfff640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440cf7d0 0x7f6244147690 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c7f0 tx=0x7f622c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.483+0000 7f6240ff9640 1 -- 172.21.15.16:0/824869345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c004220 con 0x7f62440cf7d0 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.483+0000 7f6240ff9640 1 -- 172.21.15.16:0/824869345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c0043a0 con 0x7f62440cf7d0 2023-11-24T06:28:10.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.483+0000 7f6240ff9640 1 -- 172.21.15.16:0/824869345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c007860 con 0x7f62440cf7d0 2023-11-24T06:28:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.483+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6244162730 con 0x7f62440cf7d0 2023-11-24T06:28:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.483+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6244162cf0 con 0x7f62440cf7d0 2023-11-24T06:28:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.484+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6244163210 con 0x7f62440cf7d0 2023-11-24T06:28:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.485+0000 7f6240ff9640 1 -- 172.21.15.16:0/824869345 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f622c007bd0 con 0x7f62440cf7d0 2023-11-24T06:28:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.486+0000 7f6240ff9640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f622406ae40 0x7f622406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.486+0000 7f6240ff9640 1 -- 172.21.15.16:0/824869345 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5481+0+0 (secure 0 0 0) 0x7f622c0895a0 con 0x7f62440cf7d0 2023-11-24T06:28:10.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.487+0000 7f62437fe640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f622406ae40 0x7f622406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:10.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.488+0000 7f62437fe640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f622406ae40 0x7f622406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6234036000 tx=0x7f623403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.491+0000 7f6240ff9640 1 -- 172.21.15.16:0/824869345 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6244163210 con 0x7f62440cf7d0 2023-11-24T06:28:10.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:10.686+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f62440675d0 con 0x7f62440cf7d0 2023-11-24T06:28:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.219+0000 7f6240ff9640 1 -- 172.21.15.16:0/824869345 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f622c054690 con 0x7f62440cf7d0 2023-11-24T06:28:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-24T06:28:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f622406ae40 msgr2=0x7f622406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f622406ae40 0x7f622406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6234036000 tx=0x7f623403c040 comp rx=0 tx=0).stop 2023-11-24T06:28:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440cf7d0 msgr2=0x7f6244147690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440cf7d0 0x7f6244147690 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c7f0 tx=0x7f622c00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 shutdown_connections 2023-11-24T06:28:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f622406ae40 0x7f622406d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f624414b2c0 0x7f62441420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62440cf7d0 0x7f6244147690 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 --2- 172.21.15.16:0/824869345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6244070ea0 0x7f6244161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.223+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 >> 172.21.15.16:0/824869345 conn(0x7f624406a800 msgr2=0x7f62440b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.224+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 shutdown_connections 2023-11-24T06:28:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.224+0000 7f6249cbb640 1 -- 172.21.15.16:0/824869345 wait complete. 2023-11-24T06:28:11.253 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:28:11.253 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:28:11.254 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:28:11.254 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:11.254 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:28:11.254 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:28:11.254 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:28:11.254 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:11.255 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:28:11.255 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:28:11.255 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:28:11.255 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:11.255 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:28:11.255 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:28:11.255 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:28:11.256 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:11.277 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:28:11.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:28:11.279 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:28:11.279 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:28:11.280 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:28:11.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:28:11.280 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:28:11.281 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:28:11.327 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:28:11.328 DEBUG:teuthology.parallel:result is None 2023-11-24T06:28:11.328 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:28:11.328 DEBUG:teuthology.parallel:result is None 2023-11-24T06:28:11.334 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:28:11.334 DEBUG:teuthology.parallel:result is None 2023-11-24T06:28:11.335 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:28:11.336 DEBUG:teuthology.parallel:result is None 2023-11-24T06:28:11.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:28:11.492 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2023-11-24T06:28:11.493 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2023-11-24T06:28:11.493 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:28:11.494 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-24T06:28:11.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.526+0000 7f662fb1d640 1 Processor -- start 2023-11-24T06:28:11.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.526+0000 7f662fb1d640 1 -- start start 2023-11-24T06:28:11.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.527+0000 7f662fb1d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813df80 0x7f662815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662fb1d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f66280cf9a0 0x7f66280cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662fb1d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662806c470 con 0x7f662813e360 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662fb1d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662806c750 con 0x7f66280cf9a0 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662fb1d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662806ca30 con 0x7f662813df80 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662d8b9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813df80 0x7f662815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662d0b8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f662813e360 0x7f66280cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51368/0 (socket says 172.21.15.16:51368) 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662d0b8640 1 -- 172.21.15.16:0/2265053074 learned_addr learned my addr 172.21.15.16:0/2265053074 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662e0ba640 1 --2- 172.21.15.16:0/2265053074 >> v2:172.21.15.173:3300/0 conn(0x7f66280cf9a0 0x7f66280cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662d8b9640 1 -- 172.21.15.16:0/2265053074 >> v2:172.21.15.173:3300/0 conn(0x7f66280cf9a0 msgr2=0x7f66280cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662d8b9640 1 --2- 172.21.15.16:0/2265053074 >> v2:172.21.15.173:3300/0 conn(0x7f66280cf9a0 0x7f66280cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662d8b9640 1 -- 172.21.15.16:0/2265053074 >> v1:172.21.15.16:6789/0 conn(0x7f662813e360 legacy=0x7f66280cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:11.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.528+0000 7f662d8b9640 1 -- 172.21.15.16:0/2265053074 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662806cd10 con 0x7f662813df80 2023-11-24T06:28:11.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.530+0000 7f662d8b9640 1 --2- 172.21.15.16:0/2265053074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813df80 0x7f662815bee0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66240097a0 tx=0x7f662402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a64fba42df802e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:11.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.530+0000 7f662c8b7640 1 -- 172.21.15.16:0/2265053074 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662402d040 con 0x7f662813df80 2023-11-24T06:28:11.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.530+0000 7f662c8b7640 1 -- 172.21.15.16:0/2265053074 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6624031810 con 0x7f662813df80 2023-11-24T06:28:11.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.530+0000 7f662c8b7640 1 -- 172.21.15.16:0/2265053074 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624031ba0 con 0x7f662813df80 2023-11-24T06:28:11.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.531+0000 7f662fb1d640 1 -- 172.21.15.16:0/2265053074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813df80 msgr2=0x7f662815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:11.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.531+0000 7f662fb1d640 1 --2- 172.21.15.16:0/2265053074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813df80 0x7f662815bee0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66240097a0 tx=0x7f662402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.531+0000 7f662fb1d640 1 -- 172.21.15.16:0/2265053074 shutdown_connections 2023-11-24T06:28:11.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.531+0000 7f662fb1d640 1 --2- 172.21.15.16:0/2265053074 >> v2:172.21.15.173:3300/0 conn(0x7f66280cf9a0 0x7f66280cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.531+0000 7f662fb1d640 1 --2- 172.21.15.16:0/2265053074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813df80 0x7f662815bee0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.531+0000 7f662fb1d640 1 -- 172.21.15.16:0/2265053074 >> 172.21.15.16:0/2265053074 conn(0x7f662806a800 msgr2=0x7f662806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.531+0000 7f662fb1d640 1 -- 172.21.15.16:0/2265053074 shutdown_connections 2023-11-24T06:28:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.531+0000 7f662fb1d640 1 -- 172.21.15.16:0/2265053074 wait complete. 2023-11-24T06:28:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.533+0000 7f662fb1d640 1 Processor -- start 2023-11-24T06:28:11.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.533+0000 7f662fb1d640 1 -- start start 2023-11-24T06:28:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662fb1d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66280cf9a0 0x7f6628156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662fb1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662813df80 0x7f66280b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662fb1d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813e360 0x7f66281516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662fb1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280b3260 con 0x7f662813df80 2023-11-24T06:28:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662fb1d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280b3540 con 0x7f66280cf9a0 2023-11-24T06:28:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662fb1d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66280b3820 con 0x7f662813e360 2023-11-24T06:28:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662d0b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662813df80 0x7f66280b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662d0b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662813df80 0x7f66280b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39580/0 (socket says 172.21.15.16:39580) 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662d0b8640 1 -- 172.21.15.16:0/2603968281 learned_addr learned my addr 172.21.15.16:0/2603968281 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662d8b9640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66280cf9a0 0x7f6628156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.534+0000 7f662e0ba640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813e360 0x7f66281516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f662d8b9640 1 -- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813e360 msgr2=0x7f66281516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f662d8b9640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813e360 0x7f66281516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f662d8b9640 1 -- 172.21.15.16:0/2603968281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662813df80 msgr2=0x7f66280b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f662d8b9640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662813df80 0x7f66280b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f662d8b9640 1 -- 172.21.15.16:0/2603968281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6624009400 con 0x7f66280cf9a0 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f662d0b8640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662813df80 0x7f66280b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f662d8b9640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66280cf9a0 0x7f6628156210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6624033000 tx=0x7f66240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f661e7fc640 1 -- 172.21.15.16:0/2603968281 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624006e10 con 0x7f66280cf9a0 2023-11-24T06:28:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f661e7fc640 1 -- 172.21.15.16:0/2603968281 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662403ad60 con 0x7f66280cf9a0 2023-11-24T06:28:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.535+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66280b3b00 con 0x7f66280cf9a0 2023-11-24T06:28:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.536+0000 7f661e7fc640 1 -- 172.21.15.16:0/2603968281 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624044660 con 0x7f66280cf9a0 2023-11-24T06:28:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.536+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66280b4300 con 0x7f66280cf9a0 2023-11-24T06:28:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.537+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66280675d0 con 0x7f66280cf9a0 2023-11-24T06:28:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.538+0000 7f661e7fc640 1 -- 172.21.15.16:0/2603968281 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f662404d440 con 0x7f66280cf9a0 2023-11-24T06:28:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.539+0000 7f661e7fc640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660c06ae40 0x7f660c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.539+0000 7f661e7fc640 1 -- 172.21.15.16:0/2603968281 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f66240ab900 con 0x7f66280cf9a0 2023-11-24T06:28:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.539+0000 7f662d0b8640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660c06ae40 0x7f660c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:11.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.540+0000 7f662d0b8640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660c06ae40 0x7f660c06d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6628158010 tx=0x7f6618009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.552+0000 7f661e7fc640 1 -- 172.21.15.16:0/2603968281 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6624076dd0 con 0x7f66280cf9a0 2023-11-24T06:28:11.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:11.757+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f662806df00 con 0x7f66280cf9a0 2023-11-24T06:28:13.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.012+0000 7f661e7fc640 1 -- 172.21.15.16:0/2603968281 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f662407a3f0 con 0x7f66280cf9a0 2023-11-24T06:28:13.015 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-11-24T06:28:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660c06ae40 msgr2=0x7f660c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660c06ae40 0x7f660c06d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6628158010 tx=0x7f6618009040 comp rx=0 tx=0).stop 2023-11-24T06:28:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66280cf9a0 msgr2=0x7f6628156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66280cf9a0 0x7f6628156210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6624033000 tx=0x7f66240046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 shutdown_connections 2023-11-24T06:28:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f660c06ae40 0x7f660c06d300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f662813e360 0x7f66281516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662813df80 0x7f66280b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 --2- 172.21.15.16:0/2603968281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66280cf9a0 0x7f6628156210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.015+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 >> 172.21.15.16:0/2603968281 conn(0x7f662806a800 msgr2=0x7f66280702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.016+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 shutdown_connections 2023-11-24T06:28:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.016+0000 7f662fb1d640 1 -- 172.21.15.16:0/2603968281 wait complete. 2023-11-24T06:28:13.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:28:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.266+0000 7ff00759e640 1 Processor -- start 2023-11-24T06:28:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.266+0000 7ff00759e640 1 -- start start 2023-11-24T06:28:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.268+0000 7ff00759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff008146850 0x7ff0080715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.268+0000 7ff00759e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff0080adcd0 0x7ff00806c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.268+0000 7ff00759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff008071f70 con 0x7ff008071b10 2023-11-24T06:28:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.268+0000 7ff00759e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0080ae110 con 0x7ff0080adcd0 2023-11-24T06:28:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.268+0000 7ff00759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080b3490 con 0x7ff008146850 2023-11-24T06:28:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.268+0000 7feffffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff008071b10 0x7ff0080ad5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39188/0 (socket says 172.21.15.16:39188) 2023-11-24T06:28:13.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.268+0000 7feffffff640 1 -- 172.21.15.16:0/412143936 learned_addr learned my addr 172.21.15.16:0/412143936 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff00659c640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff008146850 0x7ff0080715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108459432 0 0) 0x7ff008071f70 con 0x7ff008071b10 2023-11-24T06:28:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff006d9d640 1 --2- 172.21.15.16:0/412143936 >> v2:172.21.15.173:3300/0 conn(0x7ff0080adcd0 0x7ff00806c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff4003520 con 0x7ff008071b10 2023-11-24T06:28:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3279174860 0 0) 0x7feff4003520 con 0x7ff008071b10 2023-11-24T06:28:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff008146850 msgr2=0x7ff0080715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff008146850 0x7ff0080715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 >> v2:172.21.15.173:3300/0 conn(0x7ff0080adcd0 msgr2=0x7ff00806c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 --2- 172.21.15.16:0/412143936 >> v2:172.21.15.173:3300/0 conn(0x7ff0080adcd0 0x7ff00806c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080b3770 con 0x7ff008071b10 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7feff0001670 con 0x7ff008071b10 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 >> v1:172.21.15.16:6789/0 conn(0x7ff008071b10 legacy=0x7ff0080ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4003970 0x7feff4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff40047c0 0x7feff4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff008146850 0x7feff4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080ae110 con 0x7feff4003970 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080b3490 con 0x7feff40047c0 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff400ba30 con 0x7ff008146850 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.269+0000 7ff006d9d640 1 --2- 172.21.15.16:0/412143936 >> v2:172.21.15.173:3300/0 conn(0x7ff0080adcd0 0x7ff00806c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.270+0000 7ff00659c640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4003970 0x7feff4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.270+0000 7ff00659c640 1 -- 172.21.15.16:0/412143936 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff008146850 msgr2=0x7feff4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:13.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.270+0000 7ff00659c640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff008146850 0x7feff4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.270+0000 7ff00659c640 1 -- 172.21.15.16:0/412143936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff40047c0 msgr2=0x7feff4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:13.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.270+0000 7ff00659c640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff40047c0 0x7feff4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.270+0000 7ff00659c640 1 -- 172.21.15.16:0/412143936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4003520 con 0x7feff4003970 2023-11-24T06:28:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.271+0000 7ff00659c640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4003970 0x7feff4003d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feff800ba00 tx=0x7feff40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5199e817d842b93d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.271+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8032630 con 0x7feff4003970 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.271+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff8032790 con 0x7feff4003970 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.271+0000 7ff005d9b640 1 -- 172.21.15.16:0/412143936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8032b60 con 0x7feff4003970 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.272+0000 7ff00759e640 1 -- 172.21.15.16:0/412143936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4003970 msgr2=0x7feff4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.272+0000 7ff00759e640 1 --2- 172.21.15.16:0/412143936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff4003970 0x7feff4003d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feff800ba00 tx=0x7feff40057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.272+0000 7ff006d9d640 1 -- 172.21.15.16:0/412143936 reap_dead start 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.272+0000 7ff00759e640 1 -- 172.21.15.16:0/412143936 shutdown_connections 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.272+0000 7ff00759e640 1 -- 172.21.15.16:0/412143936 >> 172.21.15.16:0/412143936 conn(0x7ff00806a3e0 msgr2=0x7ff00806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.272+0000 7ff00759e640 1 -- 172.21.15.16:0/412143936 shutdown_connections 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.272+0000 7ff00759e640 1 -- 172.21.15.16:0/412143936 wait complete. 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.274+0000 7ff00759e640 1 Processor -- start 2023-11-24T06:28:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.274+0000 7ff00759e640 1 -- start start 2023-11-24T06:28:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.275+0000 7ff00759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff4003970 0x7ff00815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.275+0000 7ff00759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff00815f550 0x7ff0081f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.275+0000 7ff00759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0081f3530 0x7ff0080c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.275+0000 7ff00759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00815f9d0 con 0x7ff0081f3530 2023-11-24T06:28:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.275+0000 7ff00759e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00815b600 con 0x7feff4003970 2023-11-24T06:28:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.275+0000 7ff00759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00815a9f0 con 0x7ff00815f550 2023-11-24T06:28:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.275+0000 7ff006d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0081f3530 0x7ff0080c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.275+0000 7ff006d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0081f3530 0x7ff0080c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47768/0 (socket says 172.21.15.16:47768) 2023-11-24T06:28:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.275+0000 7ff006d9d640 1 -- 172.21.15.16:0/3713299809 learned_addr learned my addr 172.21.15.16:0/3713299809 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7ff006d9d640 1 -- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff00815f550 msgr2=0x7ff0081f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7ff006d9d640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff00815f550 0x7ff0081f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7ff006d9d640 1 -- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff4003970 msgr2=0x7ff00815f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7ff006d9d640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff4003970 0x7ff00815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7ff006d9d640 1 -- 172.21.15.16:0/3713299809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00815acd0 con 0x7ff0081f3530 2023-11-24T06:28:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7ff006d9d640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0081f3530 0x7ff0080c4020 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ae330 tx=0x7ff00000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7feffeffd640 1 -- 172.21.15.16:0/3713299809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000dcd0 con 0x7ff0081f3530 2023-11-24T06:28:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7feffeffd640 1 -- 172.21.15.16:0/3713299809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00000f040 con 0x7ff0081f3530 2023-11-24T06:28:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7feffeffd640 1 -- 172.21.15.16:0/3713299809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000003bd0 con 0x7ff0081f3530 2023-11-24T06:28:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.276+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00815afb0 con 0x7ff0081f3530 2023-11-24T06:28:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.277+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00815a680 con 0x7ff0081f3530 2023-11-24T06:28:13.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.278+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff008067450 con 0x7ff0081f3530 2023-11-24T06:28:13.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.279+0000 7feffeffd640 1 -- 172.21.15.16:0/3713299809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff000004030 con 0x7ff0081f3530 2023-11-24T06:28:13.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.280+0000 7feffeffd640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefe806ae40 0x7fefe806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.280+0000 7feffeffd640 1 -- 172.21.15.16:0/3713299809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff000089390 con 0x7ff0081f3530 2023-11-24T06:28:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.280+0000 7ff00659c640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefe806ae40 0x7fefe806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.282+0000 7ff00659c640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefe806ae40 0x7fefe806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feff40057a0 tx=0x7feff8032000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.284+0000 7feffeffd640 1 -- 172.21.15.16:0/3713299809 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff000054790 con 0x7ff0081f3530 2023-11-24T06:28:13.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.543+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff0080c3de0 con 0x7ff0081f3530 2023-11-24T06:28:13.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.545+0000 7feffeffd640 1 -- 172.21.15.16:0/3713299809 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff000057db0 con 0x7ff0081f3530 2023-11-24T06:28:13.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:13.548 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:28:13.548 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-11-24T06:28:13.548 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.548 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBl2seAMxAAL8jbBlU0yStjuIivg2ehGQ==", 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-11-24T06:28:13.549 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBlbyW+GxAAy15inYIiNwnGVHj7Ghs5hg==", 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl79LhCRAALZnwoYyKju4aeWNS+jF31A==", 2023-11-24T06:28:13.550 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.551 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:28:13.551 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:28:13.551 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:28:13.551 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.551 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.551 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.551 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-11-24T06:28:13.551 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl/NBDLBAAR4qYgD5THE/yNeZT9XgK5g==", 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD4QWBlarihIxAAczpFVQAODZogIKH0TTdMhw==", 2023-11-24T06:28:13.552 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD8QWBlITYFAhAAMlJyCTojY7WZihUegTcBcQ==", 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:28:13.553 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD/QWBlA/N0DRAAThU0hfAL0+MggufKZC4K8g==", 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQACQmBlM7E3JxAAdmfhQHbjK/JGHgeD0urDZw==", 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.555 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAKQmBlrM97ARAApYaqyASXJqB5ITNLeBec/w==", 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQANQmBl3s5dHBAA3spWOZQiqazGS6isl8ft3Q==", 2023-11-24T06:28:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAQQmBlKVmeNhAAuV5j9qZ2NNvEePQx77EM6Q==", 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:28:13.557 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAUQmBlwADrBRAAvS3M6+H3X+OQD1h6xwhnBQ==", 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA==", 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.559 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBlKhfoFxAA8A+MObA/cjawPN7aclqS9Q==", 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-11-24T06:28:13.560 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBliwu9CxAAPcBaB9CWK2QqRD1k2x+4ig==", 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBl8HWRIhAAqzUXni0Uqgbmi/imvnNIZg==", 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDzQWBlo6bBIRAADWG8Peciy2ws4wHagrfFZw==", 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:28:13.562 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAcQmBlK4qBOBAA6QC90xpIwvph0qFB+SCcSg==", 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.563 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAcQmBlP5yBOBAAWn2cMzyGnXCLp7Ck55CrCw==", 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAcQmBlR6qBOBAAw85MClR9mrOO68ESOhr+zA==", 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T06:28:13.564 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAcQmBl1bOBOBAApv3rBEsdos8pvw6TLGEFWA==", 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T06:28:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAcQmBlwLyBOBAAnt9z7+cZE7BBwmNemaZvQA==", 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAcQmBlksWBOBAAjH2pj38UPuUU4tJ3AKaXAA==", 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.566 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlz+KlJhAAXtiyVVA72YrbDKpNSBUnwA==", 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlN4GDGhAAF9YIRqu011Xj6oWUv9LYlw==", 2023-11-24T06:28:13.567 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:28:13.568 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:28:13.568 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:28:13.568 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:28:13.568 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.568 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:13.568 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-11-24T06:28:13.568 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.547+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefe806ae40 msgr2=0x7fefe806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.547+0000 7ff00759e640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefe806ae40 0x7fefe806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feff40057a0 tx=0x7feff8032000 comp rx=0 tx=0).stop 2023-11-24T06:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.547+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0081f3530 msgr2=0x7ff0080c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.547+0000 7ff00759e640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0081f3530 0x7ff0080c4020 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ae330 tx=0x7ff00000d4a0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.548+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 shutdown_connections 2023-11-24T06:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.548+0000 7ff00759e640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefe806ae40 0x7fefe806d300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.548+0000 7ff00759e640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0081f3530 0x7ff0080c4020 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.548+0000 7ff00759e640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff00815f550 0x7ff0081f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.548+0000 7ff00759e640 1 --2- 172.21.15.16:0/3713299809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feff4003970 0x7ff00815f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.548+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 >> 172.21.15.16:0/3713299809 conn(0x7ff00806a3e0 msgr2=0x7ff0080b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.548+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 shutdown_connections 2023-11-24T06:28:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.548+0000 7ff00759e640 1 -- 172.21.15.16:0/3713299809 wait complete. 2023-11-24T06:28:13.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T06:28:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.791+0000 7f03e880d640 1 Processor -- start 2023-11-24T06:28:13.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.791+0000 7f03e880d640 1 -- start start 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.793+0000 7f03e880d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03e41447f0 0x7f03e415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.793+0000 7f03e880d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e415c5c0 con 0x7f03e406c470 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.793+0000 7f03e880d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e406ca10 con 0x7f03e40cccf0 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.793+0000 7f03e880d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e406d690 con 0x7f03e41447f0 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.793+0000 7f03e2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03e41447f0 0x7f03e415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.793+0000 7f03e2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03e41447f0 0x7f03e415c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47772/0 (socket says 172.21.15.16:47772) 2023-11-24T06:28:13.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.793+0000 7f03e2d9d640 1 -- 172.21.15.16:0/1122046954 learned_addr learned my addr 172.21.15.16:0/1122046954 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:13.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.794+0000 7f03e2d9d640 1 -- 172.21.15.16:0/1122046954 >> v1:172.21.15.173:6789/0 conn(0x7f03e40cccf0 legacy=0x7f03e415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:13.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.794+0000 7f03e2d9d640 1 -- 172.21.15.16:0/1122046954 >> v1:172.21.15.16:6789/0 conn(0x7f03e406c470 legacy=0x7f03e40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.794+0000 7f03e2d9d640 1 -- 172.21.15.16:0/1122046954 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e406ccf0 con 0x7f03e41447f0 2023-11-24T06:28:13.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.794+0000 7f03e1d9b640 1 -- 172.21.15.16:0/1122046954 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695811723 0 0) 0x7f03e415c5c0 con 0x7f03e406c470 2023-11-24T06:28:13.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.795+0000 7f03e2d9d640 1 --2- 172.21.15.16:0/1122046954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e41447f0 0x7f03e415c080 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009870 tx=0x7f03d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c52db9316525f08 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.795+0000 7f03e1d9b640 1 -- 172.21.15.16:0/1122046954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4031870 con 0x7f03e41447f0 2023-11-24T06:28:13.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.795+0000 7f03e1d9b640 1 -- 172.21.15.16:0/1122046954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d40319d0 con 0x7f03e41447f0 2023-11-24T06:28:13.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.795+0000 7f03e1d9b640 1 -- 172.21.15.16:0/1122046954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4031d60 con 0x7f03e41447f0 2023-11-24T06:28:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.796+0000 7f03e880d640 1 -- 172.21.15.16:0/1122046954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e41447f0 msgr2=0x7f03e415c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.796+0000 7f03e880d640 1 --2- 172.21.15.16:0/1122046954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e41447f0 0x7f03e415c080 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009870 tx=0x7f03d402b910 comp rx=0 tx=0).stop 2023-11-24T06:28:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.796+0000 7f03e880d640 1 -- 172.21.15.16:0/1122046954 shutdown_connections 2023-11-24T06:28:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.796+0000 7f03e880d640 1 --2- 172.21.15.16:0/1122046954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e41447f0 0x7f03e415c080 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.796+0000 7f03e880d640 1 -- 172.21.15.16:0/1122046954 >> 172.21.15.16:0/1122046954 conn(0x7f03e406a3e0 msgr2=0x7f03e406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.796+0000 7f03e880d640 1 -- 172.21.15.16:0/1122046954 shutdown_connections 2023-11-24T06:28:13.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.796+0000 7f03e880d640 1 -- 172.21.15.16:0/1122046954 wait complete. 2023-11-24T06:28:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.798+0000 7f03e880d640 1 Processor -- start 2023-11-24T06:28:13.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.798+0000 7f03e880d640 1 -- start start 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.799+0000 7f03e880d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e406c470 0x7f03e40cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.799+0000 7f03e880d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e40cccf0 0x7f03e4150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.799+0000 7f03e880d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03e4150860 0x7f03e413d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.799+0000 7f03e880d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4150cf0 con 0x7f03e406c470 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.799+0000 7f03e880d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e413e540 con 0x7f03e40cccf0 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.799+0000 7f03e880d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e413e820 con 0x7f03e4150860 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.799+0000 7f03e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e406c470 0x7f03e40cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.799+0000 7f03e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e406c470 0x7f03e40cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47780/0 (socket says 172.21.15.16:47780) 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.799+0000 7f03e2d9d640 1 -- 172.21.15.16:0/1010558555 learned_addr learned my addr 172.21.15.16:0/1010558555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.800+0000 7f03e2d9d640 1 -- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03e4150860 msgr2=0x7f03e413d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.800+0000 7f03e2d9d640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03e4150860 0x7f03e413d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.800+0000 7f03e2d9d640 1 -- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e40cccf0 msgr2=0x7f03e4150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.800+0000 7f03e259c640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e40cccf0 0x7f03e4150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.800+0000 7f03e2d9d640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e40cccf0 0x7f03e4150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.800+0000 7f03e2d9d640 1 -- 172.21.15.16:0/1010558555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d40094d0 con 0x7f03e406c470 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.800+0000 7f03e259c640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e40cccf0 0x7f03e4150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.800+0000 7f03e2d9d640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e406c470 0x7f03e40cb9b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f03d402bea0 tx=0x7f03d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.801+0000 7f03cb7fe640 1 -- 172.21.15.16:0/1010558555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d403ad30 con 0x7f03e406c470 2023-11-24T06:28:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.801+0000 7f03cb7fe640 1 -- 172.21.15.16:0/1010558555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d403ae90 con 0x7f03e406c470 2023-11-24T06:28:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.801+0000 7f03cb7fe640 1 -- 172.21.15.16:0/1010558555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4044630 con 0x7f03e406c470 2023-11-24T06:28:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.801+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e413eb00 con 0x7f03e406c470 2023-11-24T06:28:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.801+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e40cbef0 con 0x7f03e406c470 2023-11-24T06:28:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.802+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e4067450 con 0x7f03e406c470 2023-11-24T06:28:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.803+0000 7f03cb7fe640 1 -- 172.21.15.16:0/1010558555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f03d404d450 con 0x7f03e406c470 2023-11-24T06:28:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.804+0000 7f03cb7fe640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f03d006ae40 0x7f03d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.804+0000 7f03cb7fe640 1 -- 172.21.15.16:0/1010558555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f03d402d040 con 0x7f03e406c470 2023-11-24T06:28:13.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.804+0000 7f03e259c640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f03d006ae40 0x7f03d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:13.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.806+0000 7f03e259c640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f03d006ae40 0x7f03d006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03e40b1500 tx=0x7f03d8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:13.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:13.809+0000 7f03cb7fe640 1 -- 172.21.15.16:0/1010558555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03d4076e00 con 0x7f03e406c470 2023-11-24T06:28:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.072+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f03e414e2e0 con 0x7f03e406c470 2023-11-24T06:28:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.076+0000 7f03cb7fe640 1 -- 172.21.15.16:0/1010558555 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f03d407a420 con 0x7f03e406c470 2023-11-24T06:28:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f03d006ae40 msgr2=0x7f03d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f03d006ae40 0x7f03d006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f03e40b1500 tx=0x7f03d8009040 comp rx=0 tx=0).stop 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e406c470 msgr2=0x7f03e40cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e406c470 0x7f03e40cb9b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f03d402bea0 tx=0x7f03d40046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 shutdown_connections 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f03d006ae40 0x7f03d006d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03e4150860 0x7f03e413d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e40cccf0 0x7f03e4150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 --2- 172.21.15.16:0/1010558555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03e406c470 0x7f03e40cb9b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 >> 172.21.15.16:0/1010558555 conn(0x7f03e406a3e0 msgr2=0x7f03e40b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.079+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 shutdown_connections 2023-11-24T06:28:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.080+0000 7f03e880d640 1 -- 172.21.15.16:0/1010558555 wait complete. 2023-11-24T06:28:14.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T06:28:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.321+0000 7f2058adf640 1 Processor -- start 2023-11-24T06:28:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.321+0000 7f2058adf640 1 -- start start 2023-11-24T06:28:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.322+0000 7f2058adf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205413cda0 0x7f205415da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.322+0000 7f2058adf640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f20540ce0c0 0x7f2054070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.323+0000 7f2058adf640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20540700f0 con 0x7f205415dfa0 2023-11-24T06:28:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.323+0000 7f2058adf640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205415e3e0 con 0x7f20540ce0c0 2023-11-24T06:28:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.323+0000 7f2058adf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20540ce720 con 0x7f205413cda0 2023-11-24T06:28:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.323+0000 7f20537fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205413cda0 0x7f205415da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.323+0000 7f2053fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f20540ce0c0 0x7f2054070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.324+0000 7f20537fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205413cda0 0x7f205415da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:40964/0 (socket says 172.21.15.16:40964) 2023-11-24T06:28:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.324+0000 7f20537fe640 1 -- 172.21.15.16:0/3701290125 learned_addr learned my addr 172.21.15.16:0/3701290125 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.324+0000 7f2052ffd640 1 --1- 172.21.15.16:0/3701290125 >> v1:172.21.15.173:6789/0 conn(0x7f205415dfa0 0x7f20540cd9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:43686/0 (socket says 172.21.15.16:43686) 2023-11-24T06:28:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.324+0000 7f20527fc640 1 -- 172.21.15.16:0/3701290125 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264841910 0 0) 0x7f20540700f0 con 0x7f205415dfa0 2023-11-24T06:28:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.324+0000 7f20537fe640 1 -- 172.21.15.16:0/3701290125 >> v2:172.21.15.173:3300/0 conn(0x7f20540ce0c0 msgr2=0x7f2054070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.324+0000 7f20537fe640 1 --2- 172.21.15.16:0/3701290125 >> v2:172.21.15.173:3300/0 conn(0x7f20540ce0c0 0x7f2054070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.324+0000 7f20537fe640 1 -- 172.21.15.16:0/3701290125 >> v1:172.21.15.173:6789/0 conn(0x7f205415dfa0 legacy=0x7f20540cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.324+0000 7f20537fe640 1 -- 172.21.15.16:0/3701290125 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20540cea00 con 0x7f205413cda0 2023-11-24T06:28:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.326+0000 7f20537fe640 1 --2- 172.21.15.16:0/3701290125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205413cda0 0x7f205415da60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2044009870 tx=0x7f204402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12a79837275679af server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.327+0000 7f20527fc640 1 -- 172.21.15.16:0/3701290125 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044031640 con 0x7f205413cda0 2023-11-24T06:28:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.327+0000 7f20527fc640 1 -- 172.21.15.16:0/3701290125 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20440317a0 con 0x7f205413cda0 2023-11-24T06:28:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.327+0000 7f20527fc640 1 -- 172.21.15.16:0/3701290125 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044031b30 con 0x7f205413cda0 2023-11-24T06:28:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.327+0000 7f2058adf640 1 -- 172.21.15.16:0/3701290125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205413cda0 msgr2=0x7f205415da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.327+0000 7f2058adf640 1 --2- 172.21.15.16:0/3701290125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205413cda0 0x7f205415da60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2044009870 tx=0x7f204402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.328+0000 7f2058adf640 1 -- 172.21.15.16:0/3701290125 shutdown_connections 2023-11-24T06:28:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.328+0000 7f2058adf640 1 --2- 172.21.15.16:0/3701290125 >> v2:172.21.15.173:3300/0 conn(0x7f20540ce0c0 0x7f2054070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.328+0000 7f2058adf640 1 --2- 172.21.15.16:0/3701290125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205413cda0 0x7f205415da60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.328+0000 7f2058adf640 1 -- 172.21.15.16:0/3701290125 >> 172.21.15.16:0/3701290125 conn(0x7f205406a3e0 msgr2=0x7f205406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.328+0000 7f2058adf640 1 -- 172.21.15.16:0/3701290125 shutdown_connections 2023-11-24T06:28:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.328+0000 7f2058adf640 1 -- 172.21.15.16:0/3701290125 wait complete. 2023-11-24T06:28:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.329+0000 7f2058adf640 1 Processor -- start 2023-11-24T06:28:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.330+0000 7f2058adf640 1 -- start start 2023-11-24T06:28:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.330+0000 7f2058adf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20540ce0c0 0x7f20541563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.330+0000 7f2058adf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205413cda0 0x7f20541396c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.330+0000 7f2058adf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205415dfa0 0x7f20540aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.330+0000 7f2058adf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2054148bc0 con 0x7f205415dfa0 2023-11-24T06:28:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.330+0000 7f2058adf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2054148040 con 0x7f205413cda0 2023-11-24T06:28:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.330+0000 7f2058adf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2054148320 con 0x7f20540ce0c0 2023-11-24T06:28:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.331+0000 7f2053fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205415dfa0 0x7f20540aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.331+0000 7f2053fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205415dfa0 0x7f20540aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47796/0 (socket says 172.21.15.16:47796) 2023-11-24T06:28:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.331+0000 7f2053fff640 1 -- 172.21.15.16:0/2466777007 learned_addr learned my addr 172.21.15.16:0/2466777007 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.331+0000 7f2053fff640 1 -- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20540ce0c0 msgr2=0x7f20541563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.331+0000 7f2053fff640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20540ce0c0 0x7f20541563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.331+0000 7f2053fff640 1 -- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205413cda0 msgr2=0x7f20541396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.331+0000 7f2053fff640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205413cda0 0x7f20541396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.331+0000 7f2053fff640 1 -- 172.21.15.16:0/2466777007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20440094d0 con 0x7f205415dfa0 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.332+0000 7f2053fff640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205415dfa0 0x7f20540aeba0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f204c00efc0 tx=0x7f204c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.332+0000 7f203bfff640 1 -- 172.21.15.16:0/2466777007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c009ba0 con 0x7f205415dfa0 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.332+0000 7f203bfff640 1 -- 172.21.15.16:0/2466777007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204c00f040 con 0x7f205415dfa0 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.332+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2054148600 con 0x7f205415dfa0 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.332+0000 7f203bfff640 1 -- 172.21.15.16:0/2466777007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204c0146c0 con 0x7f205415dfa0 2023-11-24T06:28:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.332+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2054139c00 con 0x7f205415dfa0 2023-11-24T06:28:14.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.333+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20541f1c60 con 0x7f205415dfa0 2023-11-24T06:28:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.334+0000 7f203bfff640 1 -- 172.21.15.16:0/2466777007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f204c014820 con 0x7f205415dfa0 2023-11-24T06:28:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.335+0000 7f203bfff640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204006ae70 0x7f204006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.335+0000 7f203bfff640 1 -- 172.21.15.16:0/2466777007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f204c089e10 con 0x7f205415dfa0 2023-11-24T06:28:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.336+0000 7f20537fe640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204006ae70 0x7f204006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.337+0000 7f20537fe640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204006ae70 0x7f204006d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2044036000 tx=0x7f2044030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.340+0000 7f203bfff640 1 -- 172.21.15.16:0/2466777007 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20541f1c60 con 0x7f205415dfa0 2023-11-24T06:28:14.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.603+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f20541f1c60 con 0x7f205415dfa0 2023-11-24T06:28:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.608+0000 7f203bfff640 1 -- 172.21.15.16:0/2466777007 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f20541f1c60 con 0x7f205415dfa0 2023-11-24T06:28:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204006ae70 msgr2=0x7f204006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204006ae70 0x7f204006d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2044036000 tx=0x7f2044030000 comp rx=0 tx=0).stop 2023-11-24T06:28:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205415dfa0 msgr2=0x7f20540aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205415dfa0 0x7f20540aeba0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f204c00efc0 tx=0x7f204c00c490 comp rx=0 tx=0).stop 2023-11-24T06:28:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 shutdown_connections 2023-11-24T06:28:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f204006ae70 0x7f204006d330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f205415dfa0 0x7f20540aeba0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205413cda0 0x7f20541396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 --2- 172.21.15.16:0/2466777007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20540ce0c0 0x7f20541563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 >> 172.21.15.16:0/2466777007 conn(0x7f205406a3e0 msgr2=0x7f205415d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 shutdown_connections 2023-11-24T06:28:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.611+0000 7f2058adf640 1 -- 172.21.15.16:0/2466777007 wait complete. 2023-11-24T06:28:14.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T06:28:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.854+0000 7fe2749cb640 1 Processor -- start 2023-11-24T06:28:14.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.854+0000 7fe2749cb640 1 -- start start 2023-11-24T06:28:14.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.856+0000 7fe2749cb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe270071ce0 0x7fe2700ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.856+0000 7fe2749cb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2700adb00 0x7fe27006c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.856+0000 7fe2749cb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2700b3490 con 0x7fe270146850 2023-11-24T06:28:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.856+0000 7fe2749cb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2700b3770 con 0x7fe270071ce0 2023-11-24T06:28:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.856+0000 7fe2749cb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2700b3a50 con 0x7fe2700adb00 2023-11-24T06:28:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.856+0000 7fe26effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe270071ce0 0x7fe2700ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.857+0000 7fe26effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe270071ce0 0x7fe2700ad5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47804/0 (socket says 172.21.15.16:47804) 2023-11-24T06:28:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.857+0000 7fe26effd640 1 -- 172.21.15.16:0/2308552695 learned_addr learned my addr 172.21.15.16:0/2308552695 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.857+0000 7fe26f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe270146850 0x7fe2700715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39212/0 (socket says 172.21.15.16:39212) 2023-11-24T06:28:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.857+0000 7fe26ffff640 1 --2- 172.21.15.16:0/2308552695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2700adb00 0x7fe27006c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.857+0000 7fe26effd640 1 -- 172.21.15.16:0/2308552695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2700adb00 msgr2=0x7fe27006c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.857+0000 7fe26effd640 1 --2- 172.21.15.16:0/2308552695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2700adb00 0x7fe27006c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.857+0000 7fe26effd640 1 -- 172.21.15.16:0/2308552695 >> v1:172.21.15.16:6789/0 conn(0x7fe270146850 legacy=0x7fe2700715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.857+0000 7fe26effd640 1 -- 172.21.15.16:0/2308552695 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2700b3d30 con 0x7fe270071ce0 2023-11-24T06:28:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.857+0000 7fe26e7fc640 1 -- 172.21.15.16:0/2308552695 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84887959 0 0) 0x7fe2700b3490 con 0x7fe270146850 2023-11-24T06:28:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.858+0000 7fe26effd640 1 --2- 172.21.15.16:0/2308552695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 0x7fe2700ad5c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe258009870 tx=0x7fe25802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e432b9efebd493e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.859+0000 7fe26e7fc640 1 -- 172.21.15.16:0/2308552695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258030840 con 0x7fe270071ce0 2023-11-24T06:28:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.859+0000 7fe26e7fc640 1 -- 172.21.15.16:0/2308552695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2580309a0 con 0x7fe270071ce0 2023-11-24T06:28:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.859+0000 7fe26e7fc640 1 -- 172.21.15.16:0/2308552695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258030d30 con 0x7fe270071ce0 2023-11-24T06:28:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.859+0000 7fe2749cb640 1 -- 172.21.15.16:0/2308552695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 msgr2=0x7fe2700ad5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.859+0000 7fe2749cb640 1 --2- 172.21.15.16:0/2308552695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 0x7fe2700ad5c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe258009870 tx=0x7fe25802b980 comp rx=0 tx=0).stop 2023-11-24T06:28:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.860+0000 7fe2749cb640 1 -- 172.21.15.16:0/2308552695 shutdown_connections 2023-11-24T06:28:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.860+0000 7fe2749cb640 1 --2- 172.21.15.16:0/2308552695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2700adb00 0x7fe27006c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.860+0000 7fe2749cb640 1 --2- 172.21.15.16:0/2308552695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 0x7fe2700ad5c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.860+0000 7fe2749cb640 1 -- 172.21.15.16:0/2308552695 >> 172.21.15.16:0/2308552695 conn(0x7fe27006a3e0 msgr2=0x7fe27006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.860+0000 7fe2749cb640 1 -- 172.21.15.16:0/2308552695 shutdown_connections 2023-11-24T06:28:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.860+0000 7fe2749cb640 1 -- 172.21.15.16:0/2308552695 wait complete. 2023-11-24T06:28:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.861+0000 7fe2749cb640 1 Processor -- start 2023-11-24T06:28:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.862+0000 7fe2749cb640 1 -- start start 2023-11-24T06:28:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe2749cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 0x7fe27012e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe2749cb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700adb00 0x7fe2701f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe2749cb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe270146850 0x7fe2701f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe2749cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe270140e60 con 0x7fe270071ce0 2023-11-24T06:28:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe2749cb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27006c4e0 con 0x7fe2700adb00 2023-11-24T06:28:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe2749cb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27006c7c0 con 0x7fe270146850 2023-11-24T06:28:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 0x7fe27012e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 0x7fe27012e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47808/0 (socket says 172.21.15.16:47808) 2023-11-24T06:28:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26f7fe640 1 -- 172.21.15.16:0/855722048 learned_addr learned my addr 172.21.15.16:0/855722048 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26effd640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700adb00 0x7fe2701f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26ffff640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe270146850 0x7fe2701f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26f7fe640 1 -- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe270146850 msgr2=0x7fe2701f6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26f7fe640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe270146850 0x7fe2701f6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26f7fe640 1 -- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700adb00 msgr2=0x7fe2701f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26f7fe640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700adb00 0x7fe2701f5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.863+0000 7fe26f7fe640 1 -- 172.21.15.16:0/855722048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2580094d0 con 0x7fe270071ce0 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.864+0000 7fe26ffff640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe270146850 0x7fe2701f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.864+0000 7fe26f7fe640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 0x7fe27012e270 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe26000c890 tx=0x7fe26000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.864+0000 7fe267fff640 1 -- 172.21.15.16:0/855722048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26000a640 con 0x7fe270071ce0 2023-11-24T06:28:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.864+0000 7fe267fff640 1 -- 172.21.15.16:0/855722048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26000a7c0 con 0x7fe270071ce0 2023-11-24T06:28:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.864+0000 7fe267fff640 1 -- 172.21.15.16:0/855722048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26000ab70 con 0x7fe270071ce0 2023-11-24T06:28:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.864+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27006caa0 con 0x7fe270071ce0 2023-11-24T06:28:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.864+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27012e7b0 con 0x7fe270071ce0 2023-11-24T06:28:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.865+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2700b4210 con 0x7fe270071ce0 2023-11-24T06:28:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.866+0000 7fe267fff640 1 -- 172.21.15.16:0/855722048 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe26000acd0 con 0x7fe270071ce0 2023-11-24T06:28:14.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.867+0000 7fe267fff640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe25006ae40 0x7fe25006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:14.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.867+0000 7fe267fff640 1 -- 172.21.15.16:0/855722048 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe2600887f0 con 0x7fe270071ce0 2023-11-24T06:28:14.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.868+0000 7fe26effd640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe25006ae40 0x7fe25006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:14.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.869+0000 7fe26effd640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe25006ae40 0x7fe25006d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe258036000 tx=0x7fe25803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:14.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:14.872+0000 7fe267fff640 1 -- 172.21.15.16:0/855722048 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2700b4210 con 0x7fe270071ce0 2023-11-24T06:28:15.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.136+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fe2700b4210 con 0x7fe270071ce0 2023-11-24T06:28:15.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.140+0000 7fe267fff640 1 -- 172.21.15.16:0/855722048 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe2700b4210 con 0x7fe270071ce0 2023-11-24T06:28:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe25006ae40 msgr2=0x7fe25006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe25006ae40 0x7fe25006d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe258036000 tx=0x7fe25803b040 comp rx=0 tx=0).stop 2023-11-24T06:28:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 msgr2=0x7fe27012e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 0x7fe27012e270 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe26000c890 tx=0x7fe26000cd50 comp rx=0 tx=0).stop 2023-11-24T06:28:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 shutdown_connections 2023-11-24T06:28:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe25006ae40 0x7fe25006d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe270146850 0x7fe2701f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2700adb00 0x7fe2701f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 --2- 172.21.15.16:0/855722048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe270071ce0 0x7fe27012e270 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 >> 172.21.15.16:0/855722048 conn(0x7fe27006a3e0 msgr2=0x7fe2700715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 shutdown_connections 2023-11-24T06:28:15.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.143+0000 7fe2749cb640 1 -- 172.21.15.16:0/855722048 wait complete. 2023-11-24T06:28:15.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T06:28:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.378+0000 7fe785e3d640 1 Processor -- start 2023-11-24T06:28:15.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.378+0000 7fe785e3d640 1 -- start start 2023-11-24T06:28:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.380+0000 7fe785e3d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013df80 0x7fe78015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.380+0000 7fe785e3d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe7800cf9a0 0x7fe7800cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.380+0000 7fe785e3d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78006c470 con 0x7fe78013e360 2023-11-24T06:28:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.380+0000 7fe785e3d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe78006c750 con 0x7fe7800cf9a0 2023-11-24T06:28:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.380+0000 7fe785e3d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78006ca30 con 0x7fe78013df80 2023-11-24T06:28:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.381+0000 7fe77ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe7800cf9a0 0x7fe7800cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.381+0000 7fe77f7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013df80 0x7fe78015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.381+0000 7fe77ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe7800cf9a0 0x7fe7800cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:41202/0 (socket says 172.21.15.16:41202) 2023-11-24T06:28:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.381+0000 7fe77ffff640 1 -- 172.21.15.16:0/363131429 learned_addr learned my addr 172.21.15.16:0/363131429 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:15.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.382+0000 7fe77f7fe640 1 -- 172.21.15.16:0/363131429 >> v2:172.21.15.173:3300/0 conn(0x7fe7800cf9a0 msgr2=0x7fe7800cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:15.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.382+0000 7fe77f7fe640 1 --2- 172.21.15.16:0/363131429 >> v2:172.21.15.173:3300/0 conn(0x7fe7800cf9a0 0x7fe7800cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.382+0000 7fe77f7fe640 1 -- 172.21.15.16:0/363131429 >> v1:172.21.15.173:6789/0 conn(0x7fe78013e360 legacy=0x7fe7800cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.382+0000 7fe77f7fe640 1 -- 172.21.15.16:0/363131429 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78006cd10 con 0x7fe78013df80 2023-11-24T06:28:15.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.383+0000 7fe77f7fe640 1 --2- 172.21.15.16:0/363131429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013df80 0x7fe78015bee0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe7700097a0 tx=0x7fe77002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d03d1f8f78dce73 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.383+0000 7fe77effd640 1 -- 172.21.15.16:0/363131429 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77002d040 con 0x7fe78013df80 2023-11-24T06:28:15.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.383+0000 7fe77effd640 1 -- 172.21.15.16:0/363131429 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe770031810 con 0x7fe78013df80 2023-11-24T06:28:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.384+0000 7fe77effd640 1 -- 172.21.15.16:0/363131429 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770031ba0 con 0x7fe78013df80 2023-11-24T06:28:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.384+0000 7fe785e3d640 1 -- 172.21.15.16:0/363131429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013df80 msgr2=0x7fe78015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.384+0000 7fe785e3d640 1 --2- 172.21.15.16:0/363131429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013df80 0x7fe78015bee0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe7700097a0 tx=0x7fe77002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.384+0000 7fe785e3d640 1 -- 172.21.15.16:0/363131429 shutdown_connections 2023-11-24T06:28:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.384+0000 7fe785e3d640 1 --2- 172.21.15.16:0/363131429 >> v2:172.21.15.173:3300/0 conn(0x7fe7800cf9a0 0x7fe7800cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.384+0000 7fe785e3d640 1 --2- 172.21.15.16:0/363131429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013df80 0x7fe78015bee0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.384+0000 7fe785e3d640 1 -- 172.21.15.16:0/363131429 >> 172.21.15.16:0/363131429 conn(0x7fe78006a800 msgr2=0x7fe78006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.384+0000 7fe785e3d640 1 -- 172.21.15.16:0/363131429 shutdown_connections 2023-11-24T06:28:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.384+0000 7fe785e3d640 1 -- 172.21.15.16:0/363131429 wait complete. 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.386+0000 7fe785e3d640 1 Processor -- start 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.386+0000 7fe785e3d640 1 -- start start 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.387+0000 7fe785e3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7800cf9a0 0x7fe7801f0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.387+0000 7fe785e3d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe78013df80 0x7fe780159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.387+0000 7fe785e3d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013e360 0x7fe780159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.387+0000 7fe785e3d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800702c0 con 0x7fe7800cf9a0 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.387+0000 7fe785e3d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801f0850 con 0x7fe78013df80 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.387+0000 7fe785e3d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801f3390 con 0x7fe78013e360 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7800cf9a0 0x7fe7801f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7800cf9a0 0x7fe7801f0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47822/0 (socket says 172.21.15.16:47822) 2023-11-24T06:28:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77f7fe640 1 -- 172.21.15.16:0/3405519608 learned_addr learned my addr 172.21.15.16:0/3405519608 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77ffff640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013e360 0x7fe780159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77f7fe640 1 -- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013e360 msgr2=0x7fe780159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe776ffd640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe78013df80 0x7fe780159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77f7fe640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013e360 0x7fe780159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77f7fe640 1 -- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe78013df80 msgr2=0x7fe780159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77f7fe640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe78013df80 0x7fe780159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77f7fe640 1 -- 172.21.15.16:0/3405519608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770009400 con 0x7fe7800cf9a0 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.388+0000 7fe77f7fe640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7800cf9a0 0x7fe7801f0310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe7700098c0 tx=0x7fe7700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.389+0000 7fe77cff9640 1 -- 172.21.15.16:0/3405519608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77003ace0 con 0x7fe7800cf9a0 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.389+0000 7fe77cff9640 1 -- 172.21.15.16:0/3405519608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77003ae60 con 0x7fe7800cf9a0 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.389+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7801f3670 con 0x7fe7800cf9a0 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.389+0000 7fe77cff9640 1 -- 172.21.15.16:0/3405519608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770044640 con 0x7fe7800cf9a0 2023-11-24T06:28:15.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.389+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7801f3c30 con 0x7fe7800cf9a0 2023-11-24T06:28:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.390+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7800675d0 con 0x7fe7800cf9a0 2023-11-24T06:28:15.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.391+0000 7fe77cff9640 1 -- 172.21.15.16:0/3405519608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe77004d440 con 0x7fe7800cf9a0 2023-11-24T06:28:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.392+0000 7fe77cff9640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe76006ae70 0x7fe76006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.392+0000 7fe77cff9640 1 -- 172.21.15.16:0/3405519608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe7700ab600 con 0x7fe7800cf9a0 2023-11-24T06:28:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.393+0000 7fe776ffd640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe76006ae70 0x7fe76006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.394+0000 7fe776ffd640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe76006ae70 0x7fe76006d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe78015c150 tx=0x7fe768009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.396+0000 7fe77cff9640 1 -- 172.21.15.16:0/3405519608 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe770076a80 con 0x7fe7800cf9a0 2023-11-24T06:28:15.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.662+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fe78006df00 con 0x7fe7800cf9a0 2023-11-24T06:28:15.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.666+0000 7fe77cff9640 1 -- 172.21.15.16:0/3405519608 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe77007a0a0 con 0x7fe7800cf9a0 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe76006ae70 msgr2=0x7fe76006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe76006ae70 0x7fe76006d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe78015c150 tx=0x7fe768009040 comp rx=0 tx=0).stop 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7800cf9a0 msgr2=0x7fe7801f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7800cf9a0 0x7fe7801f0310 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe7700098c0 tx=0x7fe7700046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 shutdown_connections 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe76006ae70 0x7fe76006d330 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe78013e360 0x7fe780159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe78013df80 0x7fe780159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 --2- 172.21.15.16:0/3405519608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7800cf9a0 0x7fe7801f0310 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.669+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 >> 172.21.15.16:0/3405519608 conn(0x7fe78006a800 msgr2=0x7fe7800ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.670+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 shutdown_connections 2023-11-24T06:28:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.670+0000 7fe785e3d640 1 -- 172.21.15.16:0/3405519608 wait complete. 2023-11-24T06:28:15.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T06:28:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.915+0000 7f8eef260640 1 Processor -- start 2023-11-24T06:28:15.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.915+0000 7f8eef260640 1 -- start start 2023-11-24T06:28:15.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.917+0000 7f8eef260640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ee80cc500 0x7f8ee80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.917+0000 7f8eef260640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee8070e50 con 0x7f8ee8143c40 2023-11-24T06:28:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.917+0000 7f8eef260640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee8144020 con 0x7f8ee806d160 2023-11-24T06:28:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.917+0000 7f8eef260640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee80723c0 con 0x7f8ee80cc500 2023-11-24T06:28:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.917+0000 7f8eee25e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8ee8143c40 0x7f8ee8071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39216/0 (socket says 172.21.15.16:39216) 2023-11-24T06:28:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.917+0000 7f8eee25e640 1 -- 172.21.15.16:0/921555117 learned_addr learned my addr 172.21.15.16:0/921555117 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.917+0000 7f8eeda5d640 1 --2- 172.21.15.16:0/921555117 >> v2:172.21.15.16:3300/0 conn(0x7f8ee80cc500 0x7f8ee80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.918+0000 7f8eed25c640 1 -- 172.21.15.16:0/921555117 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632707273 0 0) 0x7f8ee8070e50 con 0x7f8ee8143c40 2023-11-24T06:28:15.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.918+0000 7f8eed25c640 1 -- 172.21.15.16:0/921555117 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed4003520 con 0x7f8ee8143c40 2023-11-24T06:28:15.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.918+0000 7f8eeda5d640 1 -- 172.21.15.16:0/921555117 >> v1:172.21.15.173:6789/0 conn(0x7f8ee806d160 legacy=0x7f8ee806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.918+0000 7f8eeda5d640 1 -- 172.21.15.16:0/921555117 >> v1:172.21.15.16:6789/0 conn(0x7f8ee8143c40 legacy=0x7f8ee8071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.918+0000 7f8eeda5d640 1 -- 172.21.15.16:0/921555117 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee8071e00 con 0x7f8ee80cc500 2023-11-24T06:28:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.919+0000 7f8eeda5d640 1 --2- 172.21.15.16:0/921555117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 0x7f8ee80cc990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80097a0 tx=0x7f8ed802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8967edc3e7462f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.919+0000 7f8eed25c640 1 -- 172.21.15.16:0/921555117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8031830 con 0x7f8ee80cc500 2023-11-24T06:28:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.920+0000 7f8eed25c640 1 -- 172.21.15.16:0/921555117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed8031990 con 0x7f8ee80cc500 2023-11-24T06:28:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.920+0000 7f8eed25c640 1 -- 172.21.15.16:0/921555117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8031d00 con 0x7f8ee80cc500 2023-11-24T06:28:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.920+0000 7f8eef260640 1 -- 172.21.15.16:0/921555117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 msgr2=0x7f8ee80cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.920+0000 7f8eef260640 1 --2- 172.21.15.16:0/921555117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 0x7f8ee80cc990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80097a0 tx=0x7f8ed802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.920+0000 7f8eef260640 1 -- 172.21.15.16:0/921555117 shutdown_connections 2023-11-24T06:28:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.920+0000 7f8eef260640 1 --2- 172.21.15.16:0/921555117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 0x7f8ee80cc990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.920+0000 7f8eef260640 1 -- 172.21.15.16:0/921555117 >> 172.21.15.16:0/921555117 conn(0x7f8ee806a800 msgr2=0x7f8ee806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.920+0000 7f8eef260640 1 -- 172.21.15.16:0/921555117 shutdown_connections 2023-11-24T06:28:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.920+0000 7f8eef260640 1 -- 172.21.15.16:0/921555117 wait complete. 2023-11-24T06:28:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.922+0000 7f8eef260640 1 Processor -- start 2023-11-24T06:28:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.922+0000 7f8eef260640 1 -- start start 2023-11-24T06:28:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.923+0000 7f8eef260640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee806d160 0x7f8ee815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.923+0000 7f8eef260640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 0x7f8ee812b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.923+0000 7f8eef260640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee8143c40 0x7f8ee80cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.923+0000 7f8eef260640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80ca6f0 con 0x7f8ee80cc500 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.923+0000 7f8eef260640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80c9b20 con 0x7f8ee8143c40 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.923+0000 7f8eef260640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee80c9dc0 con 0x7f8ee806d160 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.923+0000 7f8eeda5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 0x7f8ee812b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.923+0000 7f8eeda5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 0x7f8ee812b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47840/0 (socket says 172.21.15.16:47840) 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.923+0000 7f8eeda5d640 1 -- 172.21.15.16:0/3939452629 learned_addr learned my addr 172.21.15.16:0/3939452629 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.924+0000 7f8eee25e640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee806d160 0x7f8ee815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.924+0000 7f8eeea5f640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee8143c40 0x7f8ee80cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.924+0000 7f8eeda5d640 1 -- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee806d160 msgr2=0x7f8ee815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.924+0000 7f8eeda5d640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee806d160 0x7f8ee815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.924+0000 7f8eeda5d640 1 -- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee8143c40 msgr2=0x7f8ee80cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:15.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.924+0000 7f8eeda5d640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee8143c40 0x7f8ee80cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:15.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.924+0000 7f8eeda5d640 1 -- 172.21.15.16:0/3939452629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8009400 con 0x7f8ee80cc500 2023-11-24T06:28:15.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.924+0000 7f8eeea5f640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee8143c40 0x7f8ee80cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:15.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.924+0000 7f8eeda5d640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 0x7f8ee812b390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8000c00 tx=0x7f8ed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.925+0000 7f8edeffd640 1 -- 172.21.15.16:0/3939452629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8004230 con 0x7f8ee80cc500 2023-11-24T06:28:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.925+0000 7f8edeffd640 1 -- 172.21.15.16:0/3939452629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed8004390 con 0x7f8ee80cc500 2023-11-24T06:28:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.925+0000 7f8edeffd640 1 -- 172.21.15.16:0/3939452629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed803a860 con 0x7f8ee80cc500 2023-11-24T06:28:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.925+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee80ca0a0 con 0x7f8ee80cc500 2023-11-24T06:28:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.925+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee815ae30 con 0x7f8ee80cc500 2023-11-24T06:28:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.926+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee80cb6d0 con 0x7f8ee80cc500 2023-11-24T06:28:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.927+0000 7f8edeffd640 1 -- 172.21.15.16:0/3939452629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ed803a9c0 con 0x7f8ee80cc500 2023-11-24T06:28:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.928+0000 7f8edeffd640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ecc06ae40 0x7f8ecc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.928+0000 7f8edeffd640 1 -- 172.21.15.16:0/3939452629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8ed80abc00 con 0x7f8ee80cc500 2023-11-24T06:28:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.929+0000 7f8eee25e640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ecc06ae40 0x7f8ecc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:15.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.930+0000 7f8eee25e640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ecc06ae40 0x7f8ecc06d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40011e0 tx=0x7f8ee4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:15.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:15.932+0000 7f8edeffd640 1 -- 172.21.15.16:0/3939452629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ed8077080 con 0x7f8ee80cc500 2023-11-24T06:28:16.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.197+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f8ee8067450 con 0x7f8ee80cc500 2023-11-24T06:28:16.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.202+0000 7f8edeffd640 1 -- 172.21.15.16:0/3939452629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8ed807a6a0 con 0x7f8ee80cc500 2023-11-24T06:28:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.205+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ecc06ae40 msgr2=0x7f8ecc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ecc06ae40 0x7f8ecc06d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ee40011e0 tx=0x7f8ee4009040 comp rx=0 tx=0).stop 2023-11-24T06:28:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 msgr2=0x7f8ee812b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 0x7f8ee812b390 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8000c00 tx=0x7f8ed80046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 shutdown_connections 2023-11-24T06:28:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ecc06ae40 0x7f8ecc06d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee8143c40 0x7f8ee80cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee80cc500 0x7f8ee812b390 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 --2- 172.21.15.16:0/3939452629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee806d160 0x7f8ee815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 >> 172.21.15.16:0/3939452629 conn(0x7f8ee806a800 msgr2=0x7f8ee8071290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 shutdown_connections 2023-11-24T06:28:16.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.206+0000 7f8eef260640 1 -- 172.21.15.16:0/3939452629 wait complete. 2023-11-24T06:28:16.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T06:28:16.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.451+0000 7fdd58f0d640 1 Processor -- start 2023-11-24T06:28:16.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.451+0000 7fdd58f0d640 1 -- start start 2023-11-24T06:28:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.453+0000 7fdd58f0d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdd541447f0 0x7fdd5415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.453+0000 7fdd58f0d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406c470 0x7fdd540cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.453+0000 7fdd58f0d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd5406dab0 0x7fdd540ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.453+0000 7fdd58f0d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5406d690 con 0x7fdd5406dab0 2023-11-24T06:28:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.453+0000 7fdd58f0d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd540cebb0 con 0x7fdd541447f0 2023-11-24T06:28:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.453+0000 7fdd58f0d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5415c5c0 con 0x7fdd5406c470 2023-11-24T06:28:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.453+0000 7fdd53fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd5406dab0 0x7fdd540ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.453+0000 7fdd53fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd5406dab0 0x7fdd540ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47846/0 (socket says 172.21.15.16:47846) 2023-11-24T06:28:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.453+0000 7fdd53fff640 1 -- 172.21.15.16:0/3647426990 learned_addr learned my addr 172.21.15.16:0/3647426990 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.454+0000 7fdd537fe640 1 --2- 172.21.15.16:0/3647426990 >> v2:172.21.15.173:3300/0 conn(0x7fdd541447f0 0x7fdd5415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.454+0000 7fdd4affd640 1 --2- 172.21.15.16:0/3647426990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406c470 0x7fdd540cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.454+0000 7fdd53fff640 1 -- 172.21.15.16:0/3647426990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406c470 msgr2=0x7fdd540cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.454+0000 7fdd53fff640 1 --2- 172.21.15.16:0/3647426990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406c470 0x7fdd540cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.454+0000 7fdd53fff640 1 -- 172.21.15.16:0/3647426990 >> v2:172.21.15.173:3300/0 conn(0x7fdd541447f0 msgr2=0x7fdd5415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.454+0000 7fdd53fff640 1 --2- 172.21.15.16:0/3647426990 >> v2:172.21.15.173:3300/0 conn(0x7fdd541447f0 0x7fdd5415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.454+0000 7fdd53fff640 1 -- 172.21.15.16:0/3647426990 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5406ca10 con 0x7fdd5406dab0 2023-11-24T06:28:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.454+0000 7fdd4affd640 1 --2- 172.21.15.16:0/3647426990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406c470 0x7fdd540cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:16.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.455+0000 7fdd53fff640 1 --2- 172.21.15.16:0/3647426990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406dab0 0x7fdd540ccb20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd54072520 tx=0x7fdd4c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc407052a429abf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.455+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3647426990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0317e0 con 0x7fdd5406dab0 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.456+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3647426990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c031940 con 0x7fdd5406dab0 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.456+0000 7fdd52ffd640 1 -- 172.21.15.16:0/3647426990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c031cd0 con 0x7fdd5406dab0 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.456+0000 7fdd58f0d640 1 -- 172.21.15.16:0/3647426990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406dab0 msgr2=0x7fdd540ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.456+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/3647426990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406dab0 0x7fdd540ccb20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd54072520 tx=0x7fdd4c02b880 comp rx=0 tx=0).stop 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.456+0000 7fdd58f0d640 1 -- 172.21.15.16:0/3647426990 shutdown_connections 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.456+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/3647426990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406dab0 0x7fdd540ccb20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.456+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/3647426990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406c470 0x7fdd540cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.456+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/3647426990 >> v2:172.21.15.173:3300/0 conn(0x7fdd541447f0 0x7fdd5415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.456+0000 7fdd58f0d640 1 -- 172.21.15.16:0/3647426990 >> 172.21.15.16:0/3647426990 conn(0x7fdd5406a3e0 msgr2=0x7fdd5406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.457+0000 7fdd58f0d640 1 -- 172.21.15.16:0/3647426990 shutdown_connections 2023-11-24T06:28:16.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.457+0000 7fdd58f0d640 1 -- 172.21.15.16:0/3647426990 wait complete. 2023-11-24T06:28:16.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.458+0000 7fdd58f0d640 1 Processor -- start 2023-11-24T06:28:16.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.459+0000 7fdd58f0d640 1 -- start start 2023-11-24T06:28:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.459+0000 7fdd58f0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406c470 0x7fdd54155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.459+0000 7fdd58f0d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406dab0 0x7fdd5413eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd58f0d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd541447f0 0x7fdd5413f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd58f0d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54156e70 con 0x7fdd5406c470 2023-11-24T06:28:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd58f0d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540aea80 con 0x7fdd541447f0 2023-11-24T06:28:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd58f0d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540af040 con 0x7fdd5406dab0 2023-11-24T06:28:16.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406c470 0x7fdd54155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406c470 0x7fdd54155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47858/0 (socket says 172.21.15.16:47858) 2023-11-24T06:28:16.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd537fe640 1 -- 172.21.15.16:0/1921023709 learned_addr learned my addr 172.21.15.16:0/1921023709 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd537fe640 1 -- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406dab0 msgr2=0x7fdd5413eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd537fe640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406dab0 0x7fdd5413eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd537fe640 1 -- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd541447f0 msgr2=0x7fdd5413f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.460+0000 7fdd537fe640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd541447f0 0x7fdd5413f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.461+0000 7fdd537fe640 1 -- 172.21.15.16:0/1921023709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c0094d0 con 0x7fdd5406c470 2023-11-24T06:28:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.461+0000 7fdd537fe640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406c470 0x7fdd54155e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdd44007ef0 tx=0x7fdd4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.461+0000 7fdd50ff9640 1 -- 172.21.15.16:0/1921023709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44014620 con 0x7fdd5406c470 2023-11-24T06:28:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.461+0000 7fdd50ff9640 1 -- 172.21.15.16:0/1921023709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd44014780 con 0x7fdd5406c470 2023-11-24T06:28:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.461+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd54156380 con 0x7fdd5406c470 2023-11-24T06:28:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.461+0000 7fdd50ff9640 1 -- 172.21.15.16:0/1921023709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44014b30 con 0x7fdd5406c470 2023-11-24T06:28:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.461+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5413ad40 con 0x7fdd5406c470 2023-11-24T06:28:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.463+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd54067450 con 0x7fdd5406c470 2023-11-24T06:28:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.464+0000 7fdd50ff9640 1 -- 172.21.15.16:0/1921023709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd44014c90 con 0x7fdd5406c470 2023-11-24T06:28:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.464+0000 7fdd50ff9640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd3406ae40 0x7fdd3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.465+0000 7fdd50ff9640 1 -- 172.21.15.16:0/1921023709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdd44089de0 con 0x7fdd5406c470 2023-11-24T06:28:16.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.465+0000 7fdd4affd640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd3406ae40 0x7fdd3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.466+0000 7fdd4affd640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd3406ae40 0x7fdd3406d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c006b10 tx=0x7fdd3c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.469+0000 7fdd50ff9640 1 -- 172.21.15.16:0/1921023709 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd44055140 con 0x7fdd5406c470 2023-11-24T06:28:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.733+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fdd5413af00 con 0x7fdd5406c470 2023-11-24T06:28:16.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.737+0000 7fdd50ff9640 1 -- 172.21.15.16:0/1921023709 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdd44058760 con 0x7fdd5406c470 2023-11-24T06:28:16.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd3406ae40 msgr2=0x7fdd3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd3406ae40 0x7fdd3406d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c006b10 tx=0x7fdd3c008040 comp rx=0 tx=0).stop 2023-11-24T06:28:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406c470 msgr2=0x7fdd54155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406c470 0x7fdd54155e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdd44007ef0 tx=0x7fdd4400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 shutdown_connections 2023-11-24T06:28:16.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd3406ae40 0x7fdd3406d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd541447f0 0x7fdd5413f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd5406dab0 0x7fdd5413eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 --2- 172.21.15.16:0/1921023709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd5406c470 0x7fdd54155e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 >> 172.21.15.16:0/1921023709 conn(0x7fdd5406a3e0 msgr2=0x7fdd54071ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 shutdown_connections 2023-11-24T06:28:16.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.740+0000 7fdd58f0d640 1 -- 172.21.15.16:0/1921023709 wait complete. 2023-11-24T06:28:16.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:16.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.984+0000 7f107c589640 1 Processor -- start 2023-11-24T06:28:16.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.984+0000 7f107c589640 1 -- start start 2023-11-24T06:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.986+0000 7f107c589640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1074143ae0 0x7f10740cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.986+0000 7f107c589640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10740cf7d0 0x7f107415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.986+0000 7f107c589640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10740cfc30 con 0x7f107415e1e0 2023-11-24T06:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.986+0000 7f107c589640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10740b1ee0 con 0x7f10740cf7d0 2023-11-24T06:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.986+0000 7f107c589640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10740b2780 con 0x7f1074143ae0 2023-11-24T06:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.987+0000 7f1079b24640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10740cf7d0 0x7f107415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.987+0000 7f1079b24640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10740cf7d0 0x7f107415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47870/0 (socket says 172.21.15.16:47870) 2023-11-24T06:28:16.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.987+0000 7f1079b24640 1 -- 172.21.15.16:0/1077974475 learned_addr learned my addr 172.21.15.16:0/1077974475 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.987+0000 7f107a325640 1 --2- 172.21.15.16:0/1077974475 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1074143ae0 0x7f10740cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.987+0000 7f1079b24640 1 -- 172.21.15.16:0/1077974475 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1074143ae0 msgr2=0x7f10740cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.987+0000 7f1079b24640 1 --2- 172.21.15.16:0/1077974475 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1074143ae0 0x7f10740cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.987+0000 7f1079b24640 1 -- 172.21.15.16:0/1077974475 >> v1:172.21.15.173:6789/0 conn(0x7f107415e1e0 legacy=0x7f107415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.987+0000 7f1079b24640 1 -- 172.21.15.16:0/1077974475 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107415e990 con 0x7f10740cf7d0 2023-11-24T06:28:16.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.988+0000 7f1079b24640 1 --2- 172.21.15.16:0/1077974475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 0x7f107415dca0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1064009870 tx=0x7f106402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ef144d44819928 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.989+0000 7f1079323640 1 -- 172.21.15.16:0/1077974475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064030840 con 0x7f10740cf7d0 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.989+0000 7f1079323640 1 -- 172.21.15.16:0/1077974475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10640309a0 con 0x7f10740cf7d0 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.989+0000 7f1079323640 1 -- 172.21.15.16:0/1077974475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064030d30 con 0x7f10740cf7d0 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.989+0000 7f107c589640 1 -- 172.21.15.16:0/1077974475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 msgr2=0x7f107415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.989+0000 7f107c589640 1 --2- 172.21.15.16:0/1077974475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 0x7f107415dca0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1064009870 tx=0x7f106402b980 comp rx=0 tx=0).stop 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.989+0000 7f107c589640 1 -- 172.21.15.16:0/1077974475 shutdown_connections 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.989+0000 7f107c589640 1 --2- 172.21.15.16:0/1077974475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 0x7f107415dca0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.989+0000 7f107c589640 1 --2- 172.21.15.16:0/1077974475 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1074143ae0 0x7f10740cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.989+0000 7f107c589640 1 -- 172.21.15.16:0/1077974475 >> 172.21.15.16:0/1077974475 conn(0x7f107406a800 msgr2=0x7f107406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:16.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.990+0000 7f107c589640 1 -- 172.21.15.16:0/1077974475 shutdown_connections 2023-11-24T06:28:16.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.990+0000 7f107c589640 1 -- 172.21.15.16:0/1077974475 wait complete. 2023-11-24T06:28:16.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.991+0000 7f107c589640 1 Processor -- start 2023-11-24T06:28:16.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.991+0000 7f107c589640 1 -- start start 2023-11-24T06:28:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.992+0000 7f107c589640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 0x7f10740c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.992+0000 7f107c589640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1074143ae0 0x7f10740afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.992+0000 7f107c589640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f107415e1e0 0x7f10740c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.992+0000 7f107c589640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10740c5c40 con 0x7f10740cf7d0 2023-11-24T06:28:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.992+0000 7f107c589640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10740c4660 con 0x7f107415e1e0 2023-11-24T06:28:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.992+0000 7f107c589640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10740c4940 con 0x7f1074143ae0 2023-11-24T06:28:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.993+0000 7f107a325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 0x7f10740c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.993+0000 7f107a325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 0x7f10740c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47876/0 (socket says 172.21.15.16:47876) 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.993+0000 7f107a325640 1 -- 172.21.15.16:0/540475115 learned_addr learned my addr 172.21.15.16:0/540475115 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.993+0000 7f107a325640 1 -- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1074143ae0 msgr2=0x7f10740afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.993+0000 7f107a325640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1074143ae0 0x7f10740afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.993+0000 7f107a325640 1 -- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f107415e1e0 msgr2=0x7f10740c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.993+0000 7f107a325640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f107415e1e0 0x7f10740c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.993+0000 7f107a325640 1 -- 172.21.15.16:0/540475115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640094d0 con 0x7f10740cf7d0 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.993+0000 7f107a325640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 0x7f10740c5700 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f107000d800 tx=0x7f107000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.994+0000 7f106affd640 1 -- 172.21.15.16:0/540475115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107000b6c0 con 0x7f10740cf7d0 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.994+0000 7f106affd640 1 -- 172.21.15.16:0/540475115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107000b820 con 0x7f10740cf7d0 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.994+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10740c4c20 con 0x7f10740cf7d0 2023-11-24T06:28:16.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.994+0000 7f106affd640 1 -- 172.21.15.16:0/540475115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107000bbb0 con 0x7f10740cf7d0 2023-11-24T06:28:16.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.994+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10740c51e0 con 0x7f10740cf7d0 2023-11-24T06:28:16.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.995+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10740c53a0 con 0x7f10740cf7d0 2023-11-24T06:28:16.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.996+0000 7f106affd640 1 -- 172.21.15.16:0/540475115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f107000bd10 con 0x7f10740cf7d0 2023-11-24T06:28:17.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.997+0000 7f106affd640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f105806ae40 0x7f105806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.997+0000 7f106affd640 1 -- 172.21.15.16:0/540475115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1070089870 con 0x7f10740cf7d0 2023-11-24T06:28:17.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.997+0000 7f1079b24640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f105806ae40 0x7f105806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:16.999+0000 7f1079b24640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f105806ae40 0x7f105806d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1064036000 tx=0x7f106403b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.002+0000 7f106affd640 1 -- 172.21.15.16:0/540475115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1070054c60 con 0x7f10740cf7d0 2023-11-24T06:28:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.196+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10740b4420 con 0x7f10740cf7d0 2023-11-24T06:28:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.198+0000 7f106affd640 1 -- 172.21.15.16:0/540475115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12857 (secure 0 0 0) 0x7f1070058280 con 0x7f10740cf7d0 2023-11-24T06:28:17.201 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:17.201 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":30,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:12.997500+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f105806ae40 msgr2=0x7f105806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f105806ae40 0x7f105806d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1064036000 tx=0x7f106403b040 comp rx=0 tx=0).stop 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 msgr2=0x7f10740c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 0x7f10740c5700 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f107000d800 tx=0x7f107000dcc0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 shutdown_connections 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f105806ae40 0x7f105806d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f107415e1e0 0x7f10740c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1074143ae0 0x7f10740afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 --2- 172.21.15.16:0/540475115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10740cf7d0 0x7f10740c5700 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 >> 172.21.15.16:0/540475115 conn(0x7f107406a800 msgr2=0x7f107415aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 shutdown_connections 2023-11-24T06:28:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.201+0000 7f107c589640 1 -- 172.21.15.16:0/540475115 wait complete. 2023-11-24T06:28:17.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:17.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:17.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.451+0000 7fe523fff640 1 Processor -- start 2023-11-24T06:28:17.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.451+0000 7fe523fff640 1 -- start start 2023-11-24T06:28:17.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.453+0000 7fe523fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe52406c730 0x7fe5240b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.453+0000 7fe523fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52406cb10 0x7fe52406cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.453+0000 7fe523fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5240708d0 con 0x7fe524070160 2023-11-24T06:28:17.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.453+0000 7fe523fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe524070bb0 con 0x7fe52406c730 2023-11-24T06:28:17.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.453+0000 7fe523fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe524070e90 con 0x7fe52406cb10 2023-11-24T06:28:17.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.453+0000 7fe522ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe52406c730 0x7fe5240b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe522ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe52406c730 0x7fe5240b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47884/0 (socket says 172.21.15.16:47884) 2023-11-24T06:28:17.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe522ffd640 1 -- 172.21.15.16:0/876770807 learned_addr learned my addr 172.21.15.16:0/876770807 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:17.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe51bfff640 1 --1- 172.21.15.16:0/876770807 >> v1:172.21.15.16:6789/0 conn(0x7fe524070160 0x7fe524071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39220/0 (socket says 172.21.15.16:39220) 2023-11-24T06:28:17.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe5237fe640 1 --2- 172.21.15.16:0/876770807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52406cb10 0x7fe52406cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe522ffd640 1 -- 172.21.15.16:0/876770807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52406cb10 msgr2=0x7fe52406cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe522ffd640 1 --2- 172.21.15.16:0/876770807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52406cb10 0x7fe52406cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe522ffd640 1 -- 172.21.15.16:0/876770807 >> v1:172.21.15.16:6789/0 conn(0x7fe524070160 legacy=0x7fe524071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe522ffd640 1 -- 172.21.15.16:0/876770807 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524071170 con 0x7fe52406c730 2023-11-24T06:28:17.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe5227fc640 1 -- 172.21.15.16:0/876770807 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076273690 0 0) 0x7fe5240708d0 con 0x7fe524070160 2023-11-24T06:28:17.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.454+0000 7fe5237fe640 1 --2- 172.21.15.16:0/876770807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52406cb10 0x7fe52406cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:17.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.455+0000 7fe522ffd640 1 --2- 172.21.15.16:0/876770807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406c730 0x7fe5240b2e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe514009870 tx=0x7fe51402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6052fb874eb47512 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe5227fc640 1 -- 172.21.15.16:0/876770807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514030840 con 0x7fe52406c730 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe5227fc640 1 -- 172.21.15.16:0/876770807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5140309a0 con 0x7fe52406c730 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe5227fc640 1 -- 172.21.15.16:0/876770807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514030d30 con 0x7fe52406c730 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe523fff640 1 -- 172.21.15.16:0/876770807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406c730 msgr2=0x7fe5240b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe523fff640 1 --2- 172.21.15.16:0/876770807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406c730 0x7fe5240b2e00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe514009870 tx=0x7fe51402b980 comp rx=0 tx=0).stop 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe523fff640 1 -- 172.21.15.16:0/876770807 shutdown_connections 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe523fff640 1 --2- 172.21.15.16:0/876770807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52406cb10 0x7fe52406cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe523fff640 1 --2- 172.21.15.16:0/876770807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406c730 0x7fe5240b2e00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe523fff640 1 -- 172.21.15.16:0/876770807 >> 172.21.15.16:0/876770807 conn(0x7fe52406a7d0 msgr2=0x7fe52406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe523fff640 1 -- 172.21.15.16:0/876770807 shutdown_connections 2023-11-24T06:28:17.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.456+0000 7fe523fff640 1 -- 172.21.15.16:0/876770807 wait complete. 2023-11-24T06:28:17.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.458+0000 7fe523fff640 1 Processor -- start 2023-11-24T06:28:17.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.458+0000 7fe523fff640 1 -- start start 2023-11-24T06:28:17.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.459+0000 7fe523fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe52406c730 0x7fe5240a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.459+0000 7fe523fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406cb10 0x7fe5241f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.459+0000 7fe523fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe524070160 0x7fe52415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.459+0000 7fe523fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240a8a00 con 0x7fe52406cb10 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.459+0000 7fe523fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52415e850 con 0x7fe52406c730 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.459+0000 7fe523fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52415dc40 con 0x7fe524070160 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406cb10 0x7fe5241f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406cb10 0x7fe5241f9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47890/0 (socket says 172.21.15.16:47890) 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe51bfff640 1 -- 172.21.15.16:0/722430004 learned_addr learned my addr 172.21.15.16:0/722430004 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe5237fe640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe524070160 0x7fe52415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe51bfff640 1 -- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe524070160 msgr2=0x7fe52415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe51bfff640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe524070160 0x7fe52415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe51bfff640 1 -- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe52406c730 msgr2=0x7fe5240a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe51bfff640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe52406c730 0x7fe5240a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe51bfff640 1 -- 172.21.15.16:0/722430004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140094d0 con 0x7fe52406cb10 2023-11-24T06:28:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe5237fe640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe524070160 0x7fe52415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.460+0000 7fe51bfff640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406cb10 0x7fe5241f9950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00c410 tx=0x7fe50c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.461+0000 7fe51b7fe640 1 -- 172.21.15.16:0/722430004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0074e0 con 0x7fe52406cb10 2023-11-24T06:28:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.461+0000 7fe51b7fe640 1 -- 172.21.15.16:0/722430004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c007640 con 0x7fe52406cb10 2023-11-24T06:28:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.461+0000 7fe51b7fe640 1 -- 172.21.15.16:0/722430004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c004a20 con 0x7fe52406cb10 2023-11-24T06:28:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.461+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52415df20 con 0x7fe52406cb10 2023-11-24T06:28:17.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.461+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52415e4e0 con 0x7fe52406cb10 2023-11-24T06:28:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.462+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe524067450 con 0x7fe52406cb10 2023-11-24T06:28:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.463+0000 7fe51b7fe640 1 -- 172.21.15.16:0/722430004 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe50c004b80 con 0x7fe52406cb10 2023-11-24T06:28:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.464+0000 7fe51b7fe640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe50406ae70 0x7fe50406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.464+0000 7fe51b7fe640 1 -- 172.21.15.16:0/722430004 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe50c088d80 con 0x7fe52406cb10 2023-11-24T06:28:17.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.465+0000 7fe522ffd640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe50406ae70 0x7fe50406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.466+0000 7fe522ffd640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe50406ae70 0x7fe50406d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe514036000 tx=0x7fe51403b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.469+0000 7fe51b7fe640 1 -- 172.21.15.16:0/722430004 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe50c057b70 con 0x7fe52406cb10 2023-11-24T06:28:17.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.663+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe52415e990 con 0x7fe52406cb10 2023-11-24T06:28:17.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.664+0000 7fe51b7fe640 1 -- 172.21.15.16:0/722430004 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe50c057700 con 0x7fe52406cb10 2023-11-24T06:28:17.668 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:28:17.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe50406ae70 msgr2=0x7fe50406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe50406ae70 0x7fe50406d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe514036000 tx=0x7fe51403b040 comp rx=0 tx=0).stop 2023-11-24T06:28:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406cb10 msgr2=0x7fe5241f9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406cb10 0x7fe5241f9950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe50c00c410 tx=0x7fe50c00c8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 shutdown_connections 2023-11-24T06:28:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe50406ae70 0x7fe50406d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe524070160 0x7fe52415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe52406cb10 0x7fe5241f9950 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 --2- 172.21.15.16:0/722430004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe52406c730 0x7fe5240a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 >> 172.21.15.16:0/722430004 conn(0x7fe52406a7d0 msgr2=0x7fe5240af380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 shutdown_connections 2023-11-24T06:28:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.667+0000 7fe523fff640 1 -- 172.21.15.16:0/722430004 wait complete. 2023-11-24T06:28:17.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:28:17.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.916+0000 7f26a13ec640 1 Processor -- start 2023-11-24T06:28:17.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.916+0000 7f26a13ec640 1 -- start start 2023-11-24T06:28:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f26a13ec640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f269c0cf9a0 0x7f269c166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f26a13ec640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269c15c5f0 con 0x7f269c13df80 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f26a13ec640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269c06c470 con 0x7f269c13e360 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f26a13ec640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269c06c750 con 0x7f269c0cf9a0 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f269b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f269c0cf9a0 0x7f269c166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f269b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f269c0cf9a0 0x7f269c166030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47904/0 (socket says 172.21.15.16:47904) 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f269affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f269c13df80 0x7f269c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39224/0 (socket says 172.21.15.16:39224) 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f269b7fe640 1 -- 172.21.15.16:0/844045883 learned_addr learned my addr 172.21.15.16:0/844045883 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f269b7fe640 1 -- 172.21.15.16:0/844045883 >> v1:172.21.15.173:6789/0 conn(0x7f269c13e360 legacy=0x7f269c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.919+0000 7f269b7fe640 1 -- 172.21.15.16:0/844045883 >> v1:172.21.15.16:6789/0 conn(0x7f269c13df80 legacy=0x7f269c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.920+0000 7f269b7fe640 1 -- 172.21.15.16:0/844045883 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c06ca30 con 0x7f269c0cf9a0 2023-11-24T06:28:17.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.920+0000 7f269a7fc640 1 -- 172.21.15.16:0/844045883 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078458103 0 0) 0x7f269c15c5f0 con 0x7f269c13df80 2023-11-24T06:28:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.921+0000 7f269b7fe640 1 --2- 172.21.15.16:0/844045883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c0cf9a0 0x7f269c166030 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2694009870 tx=0x7f269402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2070519f5a3c9988 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.921+0000 7f269a7fc640 1 -- 172.21.15.16:0/844045883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694030860 con 0x7f269c0cf9a0 2023-11-24T06:28:17.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.921+0000 7f269a7fc640 1 -- 172.21.15.16:0/844045883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26940309c0 con 0x7f269c0cf9a0 2023-11-24T06:28:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.921+0000 7f269a7fc640 1 -- 172.21.15.16:0/844045883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694030d50 con 0x7f269c0cf9a0 2023-11-24T06:28:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.922+0000 7f26a13ec640 1 -- 172.21.15.16:0/844045883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c0cf9a0 msgr2=0x7f269c166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.922+0000 7f26a13ec640 1 --2- 172.21.15.16:0/844045883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c0cf9a0 0x7f269c166030 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2694009870 tx=0x7f269402b9a0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.922+0000 7f26a13ec640 1 -- 172.21.15.16:0/844045883 shutdown_connections 2023-11-24T06:28:17.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.922+0000 7f26a13ec640 1 --2- 172.21.15.16:0/844045883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c0cf9a0 0x7f269c166030 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.922+0000 7f26a13ec640 1 -- 172.21.15.16:0/844045883 >> 172.21.15.16:0/844045883 conn(0x7f269c06a800 msgr2=0x7f269c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:17.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.922+0000 7f26a13ec640 1 -- 172.21.15.16:0/844045883 shutdown_connections 2023-11-24T06:28:17.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.922+0000 7f26a13ec640 1 -- 172.21.15.16:0/844045883 wait complete. 2023-11-24T06:28:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.924+0000 7f26a13ec640 1 Processor -- start 2023-11-24T06:28:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.924+0000 7f26a13ec640 1 -- start start 2023-11-24T06:28:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.925+0000 7f26a13ec640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c0cf9a0 0x7f269c13d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.925+0000 7f26a13ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c13df80 0x7f269c070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.925+0000 7f26a13ec640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c13e360 0x7f269c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.925+0000 7f26a13ec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c13db00 con 0x7f269c13df80 2023-11-24T06:28:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.925+0000 7f26a13ec640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c071d10 con 0x7f269c0cf9a0 2023-11-24T06:28:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.925+0000 7f26a13ec640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c071750 con 0x7f269c13e360 2023-11-24T06:28:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.925+0000 7f26927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c13df80 0x7f269c070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.925+0000 7f26927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c13df80 0x7f269c070cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47910/0 (socket says 172.21.15.16:47910) 2023-11-24T06:28:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.925+0000 7f26927fc640 1 -- 172.21.15.16:0/578726507 learned_addr learned my addr 172.21.15.16:0/578726507 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f269b7fe640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c13e360 0x7f269c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f269affd640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c0cf9a0 0x7f269c13d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f26927fc640 1 -- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c13e360 msgr2=0x7f269c152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f26927fc640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c13e360 0x7f269c152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f26927fc640 1 -- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c0cf9a0 msgr2=0x7f269c13d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f26927fc640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c0cf9a0 0x7f269c13d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f26927fc640 1 -- 172.21.15.16:0/578726507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940094d0 con 0x7f269c13df80 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f269affd640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c0cf9a0 0x7f269c13d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f269b7fe640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c13e360 0x7f269c152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f26927fc640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c13df80 0x7f269c070cd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f268400c780 tx=0x7f268400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f2693fff640 1 -- 172.21.15.16:0/578726507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400aa50 con 0x7f269c13df80 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f2693fff640 1 -- 172.21.15.16:0/578726507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268400ce90 con 0x7f269c13df80 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f2693fff640 1 -- 172.21.15.16:0/578726507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400ae30 con 0x7f269c13df80 2023-11-24T06:28:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.926+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c071a30 con 0x7f269c13df80 2023-11-24T06:28:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.927+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269c14e420 con 0x7f269c13df80 2023-11-24T06:28:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.928+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269c14e640 con 0x7f269c13df80 2023-11-24T06:28:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.929+0000 7f2693fff640 1 -- 172.21.15.16:0/578726507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2684012420 con 0x7f269c13df80 2023-11-24T06:28:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.930+0000 7f2693fff640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06ae40 0x7f267c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.930+0000 7f2693fff640 1 -- 172.21.15.16:0/578726507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2684089530 con 0x7f269c13df80 2023-11-24T06:28:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.931+0000 7f269affd640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06ae40 0x7f267c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.932+0000 7f269affd640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06ae40 0x7f267c06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f268c000f60 tx=0x7f268c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:17.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:17.935+0000 7f2693fff640 1 -- 172.21.15.16:0/578726507 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2684054920 con 0x7f269c13df80 2023-11-24T06:28:18.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.132+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f269c0675d0 con 0x7f269c13df80 2023-11-24T06:28:18.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.133+0000 7f2693fff640 1 -- 172.21.15.16:0/578726507 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2684016090 con 0x7f269c13df80 2023-11-24T06:28:18.136 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:18.137 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:28:18.137 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:18.137 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-24T06:28:18.137 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-24T06:28:18.137 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:18.137 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:28:18.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06ae40 msgr2=0x7f267c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06ae40 0x7f267c06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f268c000f60 tx=0x7f268c009040 comp rx=0 tx=0).stop 2023-11-24T06:28:18.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c13df80 msgr2=0x7f269c070cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c13df80 0x7f269c070cd0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f268400c780 tx=0x7f268400cc40 comp rx=0 tx=0).stop 2023-11-24T06:28:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 shutdown_connections 2023-11-24T06:28:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06ae40 0x7f267c06d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c13e360 0x7f269c152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269c13df80 0x7f269c070cd0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 --2- 172.21.15.16:0/578726507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c0cf9a0 0x7f269c13d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 >> 172.21.15.16:0/578726507 conn(0x7f269c06a800 msgr2=0x7f269c15cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 shutdown_connections 2023-11-24T06:28:18.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.136+0000 7f26a13ec640 1 -- 172.21.15.16:0/578726507 wait complete. 2023-11-24T06:28:18.158 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:28:18.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:28:18.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.377+0000 7f278a547640 1 Processor -- start 2023-11-24T06:28:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.377+0000 7f278a547640 1 -- start start 2023-11-24T06:28:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.379+0000 7f278a547640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 0x7f2784071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.379+0000 7f278a547640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27840ccc10 0x7f278406d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.379+0000 7f278a547640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2784070e50 con 0x7f27840717d0 2023-11-24T06:28:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.379+0000 7f278a547640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840723c0 con 0x7f27840ccc10 2023-11-24T06:28:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.379+0000 7f278a547640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2784144020 con 0x7f2784143c40 2023-11-24T06:28:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.379+0000 7f2789d46640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27840ccc10 0x7f278406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.380+0000 7f2789d46640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27840ccc10 0x7f278406d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47924/0 (socket says 172.21.15.16:47924) 2023-11-24T06:28:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.380+0000 7f2789d46640 1 -- 172.21.15.16:0/3680251687 learned_addr learned my addr 172.21.15.16:0/3680251687 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.380+0000 7f2789545640 1 --2- 172.21.15.16:0/3680251687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 0x7f2784071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.380+0000 7f2789d46640 1 -- 172.21.15.16:0/3680251687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 msgr2=0x7f2784071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.380+0000 7f2789d46640 1 --2- 172.21.15.16:0/3680251687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 0x7f2784071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.380+0000 7f2789d46640 1 -- 172.21.15.16:0/3680251687 >> v1:172.21.15.173:6789/0 conn(0x7f27840717d0 legacy=0x7f27840cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:18.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.380+0000 7f2789d46640 1 -- 172.21.15.16:0/3680251687 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2784071e00 con 0x7f27840ccc10 2023-11-24T06:28:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.381+0000 7f2789d46640 1 --2- 172.21.15.16:0/3680251687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840ccc10 0x7f278406d160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f277c009e50 tx=0x7f277c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874beb78757310a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.382+0000 7f277bfff640 1 -- 172.21.15.16:0/3680251687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c02c9e0 con 0x7f27840ccc10 2023-11-24T06:28:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.382+0000 7f277bfff640 1 -- 172.21.15.16:0/3680251687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c02cb40 con 0x7f27840ccc10 2023-11-24T06:28:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.382+0000 7f277bfff640 1 -- 172.21.15.16:0/3680251687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c0343f0 con 0x7f27840ccc10 2023-11-24T06:28:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.382+0000 7f278a547640 1 -- 172.21.15.16:0/3680251687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840ccc10 msgr2=0x7f278406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.382+0000 7f278a547640 1 --2- 172.21.15.16:0/3680251687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840ccc10 0x7f278406d160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f277c009e50 tx=0x7f277c02be40 comp rx=0 tx=0).stop 2023-11-24T06:28:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.382+0000 7f278a547640 1 -- 172.21.15.16:0/3680251687 shutdown_connections 2023-11-24T06:28:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.382+0000 7f278a547640 1 --2- 172.21.15.16:0/3680251687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840ccc10 0x7f278406d160 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.382+0000 7f278a547640 1 --2- 172.21.15.16:0/3680251687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 0x7f2784071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.382+0000 7f278a547640 1 -- 172.21.15.16:0/3680251687 >> 172.21.15.16:0/3680251687 conn(0x7f278406a800 msgr2=0x7f278406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.383+0000 7f278a547640 1 -- 172.21.15.16:0/3680251687 shutdown_connections 2023-11-24T06:28:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.383+0000 7f278a547640 1 -- 172.21.15.16:0/3680251687 wait complete. 2023-11-24T06:28:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.384+0000 7f278a547640 1 Processor -- start 2023-11-24T06:28:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.384+0000 7f278a547640 1 -- start start 2023-11-24T06:28:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.385+0000 7f278a547640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840717d0 0x7f27840af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.385+0000 7f278a547640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27840ccc10 0x7f278412e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.385+0000 7f278a547640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 0x7f278412ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.385+0000 7f278a547640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278412fd00 con 0x7f27840717d0 2023-11-24T06:28:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.385+0000 7f278a547640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278412f1d0 con 0x7f27840ccc10 2023-11-24T06:28:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.385+0000 7f278a547640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27841f1150 con 0x7f2784143c40 2023-11-24T06:28:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2789545640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840717d0 0x7f27840af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2789545640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840717d0 0x7f27840af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47930/0 (socket says 172.21.15.16:47930) 2023-11-24T06:28:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2789545640 1 -- 172.21.15.16:0/1486119440 learned_addr learned my addr 172.21.15.16:0/1486119440 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2789d46640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 0x7f278412ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2789545640 1 -- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 msgr2=0x7f278412ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2789545640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 0x7f278412ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2789545640 1 -- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27840ccc10 msgr2=0x7f278412e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2788d44640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27840ccc10 0x7f278412e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2789545640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27840ccc10 0x7f278412e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.386+0000 7f2789545640 1 -- 172.21.15.16:0/1486119440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c009b30 con 0x7f27840717d0 2023-11-24T06:28:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.387+0000 7f2789545640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840717d0 0x7f27840af050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f278000d5f0 tx=0x7f278000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.387+0000 7f2779ffb640 1 -- 172.21.15.16:0/1486119440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27800125f0 con 0x7f27840717d0 2023-11-24T06:28:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.387+0000 7f2779ffb640 1 -- 172.21.15.16:0/1486119440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2780012750 con 0x7f27840717d0 2023-11-24T06:28:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.387+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278412f460 con 0x7f27840717d0 2023-11-24T06:28:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.387+0000 7f2779ffb640 1 -- 172.21.15.16:0/1486119440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780012b00 con 0x7f27840717d0 2023-11-24T06:28:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.387+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278412fa20 con 0x7f27840717d0 2023-11-24T06:28:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.388+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27840cdef0 con 0x7f27840717d0 2023-11-24T06:28:18.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.389+0000 7f2779ffb640 1 -- 172.21.15.16:0/1486119440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2780012c60 con 0x7f27840717d0 2023-11-24T06:28:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.390+0000 7f2779ffb640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.390+0000 7f2779ffb640 1 -- 172.21.15.16:0/1486119440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2780089830 con 0x7f27840717d0 2023-11-24T06:28:18.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.391+0000 7f2788d44640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.392+0000 7f2788d44640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f278413c860 tx=0x7f2774009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.395+0000 7f2779ffb640 1 -- 172.21.15.16:0/1486119440 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2780054c20 con 0x7f27840717d0 2023-11-24T06:28:18.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.593+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f27840714e0 con 0x7f27840717d0 2023-11-24T06:28:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.686+0000 7f2779ffb640 1 -- 172.21.15.16:0/1486119440 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2780058240 con 0x7f27840717d0 2023-11-24T06:28:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-24T06:28:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 msgr2=0x7f276806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f278413c860 tx=0x7f2774009040 comp rx=0 tx=0).stop 2023-11-24T06:28:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840717d0 msgr2=0x7f27840af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840717d0 0x7f27840af050 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f278000d5f0 tx=0x7f278000dab0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 shutdown_connections 2023-11-24T06:28:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806ae40 0x7f276806d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784143c40 0x7f278412ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27840ccc10 0x7f278412e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 --2- 172.21.15.16:0/1486119440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840717d0 0x7f27840af050 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.690+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 >> 172.21.15.16:0/1486119440 conn(0x7f278406a800 msgr2=0x7f278413fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.691+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 shutdown_connections 2023-11-24T06:28:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.691+0000 7f278a547640 1 -- 172.21.15.16:0/1486119440 wait complete. 2023-11-24T06:28:18.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:28:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.928+0000 7fbb1bfff640 1 Processor -- start 2023-11-24T06:28:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.928+0000 7fbb1bfff640 1 -- start start 2023-11-24T06:28:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.930+0000 7fbb1bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb1c143c40 0x7fbb1c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.930+0000 7fbb1bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb1c0717d0 0x7fbb1c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.930+0000 7fbb1bfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbb1c0cca40 0x7fbb1c15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.930+0000 7fbb1bfff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c0723c0 con 0x7fbb1c0717d0 2023-11-24T06:28:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.930+0000 7fbb1bfff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c070e50 con 0x7fbb1c0cca40 2023-11-24T06:28:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.930+0000 7fbb1bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cce70 con 0x7fbb1c143c40 2023-11-24T06:28:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb1c0717d0 0x7fbb1c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1a7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb1c0717d0 0x7fbb1c0cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47934/0 (socket says 172.21.15.16:47934) 2023-11-24T06:28:18.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1a7fc640 1 -- 172.21.15.16:0/1629800337 learned_addr learned my addr 172.21.15.16:0/1629800337 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1affd640 1 --2- 172.21.15.16:0/1629800337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb1c143c40 0x7fbb1c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1b7fe640 1 --2- 172.21.15.16:0/1629800337 >> v2:172.21.15.173:3300/0 conn(0x7fbb1c0cca40 0x7fbb1c15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1a7fc640 1 -- 172.21.15.16:0/1629800337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb1c143c40 msgr2=0x7fbb1c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1a7fc640 1 --2- 172.21.15.16:0/1629800337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb1c143c40 0x7fbb1c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1a7fc640 1 -- 172.21.15.16:0/1629800337 >> v2:172.21.15.173:3300/0 conn(0x7fbb1c0cca40 msgr2=0x7fbb1c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1a7fc640 1 --2- 172.21.15.16:0/1629800337 >> v2:172.21.15.173:3300/0 conn(0x7fbb1c0cca40 0x7fbb1c15eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.931+0000 7fbb1a7fc640 1 -- 172.21.15.16:0/1629800337 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c144020 con 0x7fbb1c0717d0 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.933+0000 7fbb1a7fc640 1 --2- 172.21.15.16:0/1629800337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 0x7fbb1c0cc500 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0cc3f0 tx=0x7fbb1002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27dfe14317305c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.933+0000 7fbb19ffb640 1 -- 172.21.15.16:0/1629800337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb100317e0 con 0x7fbb1c0717d0 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.933+0000 7fbb19ffb640 1 -- 172.21.15.16:0/1629800337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb10031940 con 0x7fbb1c0717d0 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.933+0000 7fbb19ffb640 1 -- 172.21.15.16:0/1629800337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10031cd0 con 0x7fbb1c0717d0 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.934+0000 7fbb1bfff640 1 -- 172.21.15.16:0/1629800337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 msgr2=0x7fbb1c0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.934+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/1629800337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 0x7fbb1c0cc500 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0cc3f0 tx=0x7fbb1002b880 comp rx=0 tx=0).stop 2023-11-24T06:28:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.934+0000 7fbb1bfff640 1 -- 172.21.15.16:0/1629800337 shutdown_connections 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.934+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/1629800337 >> v2:172.21.15.173:3300/0 conn(0x7fbb1c0cca40 0x7fbb1c15eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.934+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/1629800337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 0x7fbb1c0cc500 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.934+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/1629800337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb1c143c40 0x7fbb1c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.934+0000 7fbb1bfff640 1 -- 172.21.15.16:0/1629800337 >> 172.21.15.16:0/1629800337 conn(0x7fbb1c06a800 msgr2=0x7fbb1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.934+0000 7fbb1bfff640 1 -- 172.21.15.16:0/1629800337 shutdown_connections 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.934+0000 7fbb1bfff640 1 -- 172.21.15.16:0/1629800337 wait complete. 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.936+0000 7fbb1bfff640 1 Processor -- start 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.936+0000 7fbb1bfff640 1 -- start start 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.937+0000 7fbb1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 0x7fbb1c1e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.937+0000 7fbb1bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb1c0cca40 0x7fbb1c1e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.937+0000 7fbb1bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb1c143c40 0x7fbb1c134470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.937+0000 7fbb1bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1e3210 con 0x7fbb1c0717d0 2023-11-24T06:28:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.937+0000 7fbb1bfff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1349b0 con 0x7fbb1c0cca40 2023-11-24T06:28:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.937+0000 7fbb1bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1e2600 con 0x7fbb1c143c40 2023-11-24T06:28:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.937+0000 7fbb1affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 0x7fbb1c1e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.937+0000 7fbb1affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 0x7fbb1c1e1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47944/0 (socket says 172.21.15.16:47944) 2023-11-24T06:28:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.937+0000 7fbb1affd640 1 -- 172.21.15.16:0/2521111934 learned_addr learned my addr 172.21.15.16:0/2521111934 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.938+0000 7fbb1affd640 1 -- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb1c143c40 msgr2=0x7fbb1c134470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.938+0000 7fbb1affd640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb1c143c40 0x7fbb1c134470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.938+0000 7fbb1affd640 1 -- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb1c0cca40 msgr2=0x7fbb1c1e36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.938+0000 7fbb1affd640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb1c0cca40 0x7fbb1c1e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.938+0000 7fbb1affd640 1 -- 172.21.15.16:0/2521111934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100094d0 con 0x7fbb1c0717d0 2023-11-24T06:28:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.938+0000 7fbb1affd640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 0x7fbb1c1e1f70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00d800 tx=0x7fbb0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.939+0000 7fbb037fe640 1 -- 172.21.15.16:0/2521111934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00b6c0 con 0x7fbb1c0717d0 2023-11-24T06:28:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.939+0000 7fbb037fe640 1 -- 172.21.15.16:0/2521111934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c00b820 con 0x7fbb1c0717d0 2023-11-24T06:28:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.939+0000 7fbb037fe640 1 -- 172.21.15.16:0/2521111934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00bbb0 con 0x7fbb1c0717d0 2023-11-24T06:28:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.939+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c1e28e0 con 0x7fbb1c0717d0 2023-11-24T06:28:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.939+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c1e2ea0 con 0x7fbb1c0717d0 2023-11-24T06:28:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.940+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c067450 con 0x7fbb1c0717d0 2023-11-24T06:28:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.941+0000 7fbb037fe640 1 -- 172.21.15.16:0/2521111934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb0c00bd10 con 0x7fbb1c0717d0 2023-11-24T06:28:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.942+0000 7fbb037fe640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb0806ae40 0x7fbb0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.942+0000 7fbb037fe640 1 -- 172.21.15.16:0/2521111934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbb0c08a4a0 con 0x7fbb1c0717d0 2023-11-24T06:28:18.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.943+0000 7fbb1a7fc640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb0806ae40 0x7fbb0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.944+0000 7fbb1a7fc640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb0806ae40 0x7fbb0806d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb10036000 tx=0x7fbb10030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:18.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:18.946+0000 7fbb037fe640 1 -- 172.21.15.16:0/2521111934 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb0c055710 con 0x7fbb1c0717d0 2023-11-24T06:28:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.143+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbb1c0ceb30 con 0x7fbb1c0717d0 2023-11-24T06:28:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.705+0000 7fbb037fe640 1 -- 172.21.15.16:0/2521111934 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbb0c058d30 con 0x7fbb1c0717d0 2023-11-24T06:28:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-24T06:28:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.709+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb0806ae40 msgr2=0x7fbb0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.709+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb0806ae40 0x7fbb0806d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb10036000 tx=0x7fbb10030000 comp rx=0 tx=0).stop 2023-11-24T06:28:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.709+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 msgr2=0x7fbb1c1e1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.709+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 0x7fbb1c1e1f70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00d800 tx=0x7fbb0c00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.710+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 shutdown_connections 2023-11-24T06:28:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.710+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb0806ae40 0x7fbb0806d300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.710+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb1c143c40 0x7fbb1c134470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.710+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb1c0cca40 0x7fbb1c1e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.710+0000 7fbb1bfff640 1 --2- 172.21.15.16:0/2521111934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1c0717d0 0x7fbb1c1e1f70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.710+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 >> 172.21.15.16:0/2521111934 conn(0x7fbb1c06a800 msgr2=0x7fbb1c06cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.710+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 shutdown_connections 2023-11-24T06:28:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.713+0000 7fbb1bfff640 1 -- 172.21.15.16:0/2521111934 wait complete. 2023-11-24T06:28:19.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:28:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.938+0000 7f4b0d589640 1 Processor -- start 2023-11-24T06:28:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.938+0000 7f4b0d589640 1 -- start start 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.940+0000 7f4b0d589640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b080cf9a0 0x7f4b08070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.940+0000 7f4b0d589640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08070ea0 0x7f4b080cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.940+0000 7f4b0d589640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b08071690 con 0x7f4b0814b2c0 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.940+0000 7f4b0d589640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0814b6a0 con 0x7f4b080cf9a0 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.940+0000 7f4b0d589640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08071970 con 0x7f4b08070ea0 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.940+0000 7f4afe7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b080cf9a0 0x7f4b08070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.941+0000 7f4afe7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b080cf9a0 0x7f4b08070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47958/0 (socket says 172.21.15.16:47958) 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.941+0000 7f4afe7fc640 1 -- 172.21.15.16:0/1712459808 learned_addr learned my addr 172.21.15.16:0/1712459808 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.941+0000 7f4b077fe640 1 --2- 172.21.15.16:0/1712459808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08070ea0 0x7f4b080cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.941+0000 7f4afe7fc640 1 -- 172.21.15.16:0/1712459808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08070ea0 msgr2=0x7f4b080cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.941+0000 7f4afe7fc640 1 --2- 172.21.15.16:0/1712459808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08070ea0 0x7f4b080cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.941+0000 7f4afe7fc640 1 -- 172.21.15.16:0/1712459808 >> v1:172.21.15.173:6789/0 conn(0x7f4b0814b2c0 legacy=0x7f4b080cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.941+0000 7f4afe7fc640 1 -- 172.21.15.16:0/1712459808 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08071c50 con 0x7f4b080cf9a0 2023-11-24T06:28:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.942+0000 7f4afe7fc640 1 --2- 172.21.15.16:0/1712459808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 0x7f4b08070960 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009870 tx=0x7f4af002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d365cda833b68bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.942+0000 7f4b067fc640 1 -- 172.21.15.16:0/1712459808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0030840 con 0x7f4b080cf9a0 2023-11-24T06:28:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b067fc640 1 -- 172.21.15.16:0/1712459808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af00309a0 con 0x7f4b080cf9a0 2023-11-24T06:28:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b067fc640 1 -- 172.21.15.16:0/1712459808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0030d30 con 0x7f4b080cf9a0 2023-11-24T06:28:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b0d589640 1 -- 172.21.15.16:0/1712459808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 msgr2=0x7f4b08070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b0d589640 1 --2- 172.21.15.16:0/1712459808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 0x7f4b08070960 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009870 tx=0x7f4af002b980 comp rx=0 tx=0).stop 2023-11-24T06:28:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b0d589640 1 -- 172.21.15.16:0/1712459808 shutdown_connections 2023-11-24T06:28:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b0d589640 1 --2- 172.21.15.16:0/1712459808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08070ea0 0x7f4b080cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b0d589640 1 --2- 172.21.15.16:0/1712459808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 0x7f4b08070960 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b0d589640 1 -- 172.21.15.16:0/1712459808 >> 172.21.15.16:0/1712459808 conn(0x7f4b0806a800 msgr2=0x7f4b0806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b0d589640 1 -- 172.21.15.16:0/1712459808 shutdown_connections 2023-11-24T06:28:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.943+0000 7f4b0d589640 1 -- 172.21.15.16:0/1712459808 wait complete. 2023-11-24T06:28:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.945+0000 7f4b0d589640 1 Processor -- start 2023-11-24T06:28:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.945+0000 7f4b0d589640 1 -- start start 2023-11-24T06:28:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.946+0000 7f4b0d589640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b08070ea0 0x7f4b081fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.946+0000 7f4b0d589640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 0x7f4b08146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.946+0000 7f4b0d589640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b0814b2c0 0x7f4b081458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.946+0000 7f4b0d589640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081f86e0 con 0x7f4b080cf9a0 2023-11-24T06:28:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.946+0000 7f4b0d589640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08145e10 con 0x7f4b08070ea0 2023-11-24T06:28:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.946+0000 7f4b0d589640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081f8120 con 0x7f4b0814b2c0 2023-11-24T06:28:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4afe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 0x7f4b08146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4afe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 0x7f4b08146340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47972/0 (socket says 172.21.15.16:47972) 2023-11-24T06:28:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4afe7fc640 1 -- 172.21.15.16:0/3609162990 learned_addr learned my addr 172.21.15.16:0/3609162990 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4b06ffd640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b08070ea0 0x7f4b081fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4b077fe640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b0814b2c0 0x7f4b081458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4afe7fc640 1 -- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b0814b2c0 msgr2=0x7f4b081458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4afe7fc640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b0814b2c0 0x7f4b081458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4afe7fc640 1 -- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b08070ea0 msgr2=0x7f4b081fa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4afe7fc640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b08070ea0 0x7f4b081fa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4afe7fc640 1 -- 172.21.15.16:0/3609162990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00094d0 con 0x7f4b080cf9a0 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.947+0000 7f4afe7fc640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 0x7f4b08146340 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4af0000c00 tx=0x7f4af0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.948+0000 7f4afffff640 1 -- 172.21.15.16:0/3609162990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0039c90 con 0x7f4b080cf9a0 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.948+0000 7f4afffff640 1 -- 172.21.15.16:0/3609162990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af0039df0 con 0x7f4b080cf9a0 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.948+0000 7f4afffff640 1 -- 172.21.15.16:0/3609162990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0043640 con 0x7f4b080cf9a0 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.948+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b081f8400 con 0x7f4b080cf9a0 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.948+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b081628b0 con 0x7f4b080cf9a0 2023-11-24T06:28:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.949+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08162430 con 0x7f4b080cf9a0 2023-11-24T06:28:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.951+0000 7f4afffff640 1 -- 172.21.15.16:0/3609162990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4af00437a0 con 0x7f4b080cf9a0 2023-11-24T06:28:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.952+0000 7f4afffff640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ae806ae70 0x7f4ae806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.952+0000 7f4afffff640 1 -- 172.21.15.16:0/3609162990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4af003e070 con 0x7f4b080cf9a0 2023-11-24T06:28:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.952+0000 7f4b06ffd640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ae806ae70 0x7f4ae806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.953+0000 7f4b06ffd640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ae806ae70 0x7f4ae806d330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4af80012e0 tx=0x7f4af8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:19.956+0000 7f4afffff640 1 -- 172.21.15.16:0/3609162990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4af0078090 con 0x7f4b080cf9a0 2023-11-24T06:28:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.186+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4b08162430 con 0x7f4b080cf9a0 2023-11-24T06:28:20.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.730+0000 7f4afffff640 1 -- 172.21.15.16:0/3609162990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4b08162430 con 0x7f4b080cf9a0 2023-11-24T06:28:20.738 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T06:28:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.741+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ae806ae70 msgr2=0x7f4ae806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.741+0000 7f4b0d589640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ae806ae70 0x7f4ae806d330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4af80012e0 tx=0x7f4af8009040 comp rx=0 tx=0).stop 2023-11-24T06:28:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.741+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 msgr2=0x7f4b08146340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.741+0000 7f4b0d589640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 0x7f4b08146340 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4af0000c00 tx=0x7f4af0004730 comp rx=0 tx=0).stop 2023-11-24T06:28:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.742+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 shutdown_connections 2023-11-24T06:28:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.742+0000 7f4b0d589640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ae806ae70 0x7f4ae806d330 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.742+0000 7f4b0d589640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b0814b2c0 0x7f4b081458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.742+0000 7f4b0d589640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cf9a0 0x7f4b08146340 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.742+0000 7f4b0d589640 1 --2- 172.21.15.16:0/3609162990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b08070ea0 0x7f4b081fa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.742+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 >> 172.21.15.16:0/3609162990 conn(0x7f4b0806a800 msgr2=0x7f4b0814a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.745+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 shutdown_connections 2023-11-24T06:28:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.745+0000 7f4b0d589640 1 -- 172.21.15.16:0/3609162990 wait complete. 2023-11-24T06:28:20.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.959+0000 7f0209c83640 1 Processor -- start 2023-11-24T06:28:20.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.959+0000 7f0209c83640 1 -- start start 2023-11-24T06:28:20.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.961+0000 7f0209c83640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 0x7f020415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.961+0000 7f0209c83640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f020413e360 0x7f02040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:20.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.961+0000 7f0209c83640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020406c470 con 0x7f020415f9c0 2023-11-24T06:28:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.961+0000 7f0209c83640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020406c750 con 0x7f020413e360 2023-11-24T06:28:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.961+0000 7f0209c83640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020406ca30 con 0x7f020413df80 2023-11-24T06:28:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.961+0000 7f01fbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f020413e360 0x7f02040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.961+0000 7f01fbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f020413e360 0x7f02040cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47976/0 (socket says 172.21.15.16:47976) 2023-11-24T06:28:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.961+0000 7f01fbfff640 1 -- 172.21.15.16:0/352006924 learned_addr learned my addr 172.21.15.16:0/352006924 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.962+0000 7f01fbfff640 1 -- 172.21.15.16:0/352006924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 msgr2=0x7f020415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.962+0000 7f01fbfff640 1 --2- 172.21.15.16:0/352006924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 0x7f020415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.962+0000 7f01fbfff640 1 -- 172.21.15.16:0/352006924 >> v1:172.21.15.173:6789/0 conn(0x7f020415f9c0 legacy=0x7f02041408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.962+0000 7f01fbfff640 1 -- 172.21.15.16:0/352006924 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020406cd10 con 0x7f020413e360 2023-11-24T06:28:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.963+0000 7f01fbfff640 1 --2- 172.21.15.16:0/352006924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020413e360 0x7f02040cf290 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009870 tx=0x7f01ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3955dcc38303c8c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.963+0000 7f0202ffd640 1 -- 172.21.15.16:0/352006924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec031640 con 0x7f020413e360 2023-11-24T06:28:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.963+0000 7f0202ffd640 1 -- 172.21.15.16:0/352006924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec0317a0 con 0x7f020413e360 2023-11-24T06:28:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.963+0000 7f0202ffd640 1 -- 172.21.15.16:0/352006924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec031b30 con 0x7f020413e360 2023-11-24T06:28:21.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.964+0000 7f0209c83640 1 -- 172.21.15.16:0/352006924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020413e360 msgr2=0x7f02040cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.964+0000 7f0209c83640 1 --2- 172.21.15.16:0/352006924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020413e360 0x7f02040cf290 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009870 tx=0x7f01ec02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.964+0000 7f0209c83640 1 -- 172.21.15.16:0/352006924 shutdown_connections 2023-11-24T06:28:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.964+0000 7f0209c83640 1 --2- 172.21.15.16:0/352006924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020413e360 0x7f02040cf290 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.964+0000 7f0209c83640 1 --2- 172.21.15.16:0/352006924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 0x7f020415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.964+0000 7f0209c83640 1 -- 172.21.15.16:0/352006924 >> 172.21.15.16:0/352006924 conn(0x7f020406a800 msgr2=0x7f020406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:21.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.964+0000 7f0209c83640 1 -- 172.21.15.16:0/352006924 shutdown_connections 2023-11-24T06:28:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.964+0000 7f0209c83640 1 -- 172.21.15.16:0/352006924 wait complete. 2023-11-24T06:28:21.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.966+0000 7f0209c83640 1 Processor -- start 2023-11-24T06:28:21.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.966+0000 7f0209c83640 1 -- start start 2023-11-24T06:28:21.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.967+0000 7f0209c83640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 0x7f020413d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.967+0000 7f0209c83640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f020413e360 0x7f020414ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.967+0000 7f0209c83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020415f9c0 0x7f0204153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.967+0000 7f0209c83640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02041516c0 con 0x7f020415f9c0 2023-11-24T06:28:21.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.967+0000 7f0209c83640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020414d1f0 con 0x7f020413e360 2023-11-24T06:28:21.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.967+0000 7f0209c83640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020415a3b0 con 0x7f020413df80 2023-11-24T06:28:21.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.967+0000 7f0203fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020415f9c0 0x7f0204153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.967+0000 7f0203fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020415f9c0 0x7f0204153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47978/0 (socket says 172.21.15.16:47978) 2023-11-24T06:28:21.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.967+0000 7f0203fff640 1 -- 172.21.15.16:0/1282857712 learned_addr learned my addr 172.21.15.16:0/1282857712 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:21.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f01fbfff640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f020413e360 0x7f020414ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f02037fe640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 0x7f020413d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0203fff640 1 -- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 msgr2=0x7f020413d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0203fff640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 0x7f020413d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0203fff640 1 -- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f020413e360 msgr2=0x7f020414ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0203fff640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f020413e360 0x7f020414ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0203fff640 1 -- 172.21.15.16:0/1282857712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0094d0 con 0x7f020415f9c0 2023-11-24T06:28:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f01fbfff640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f020413e360 0x7f020414ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f02037fe640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 0x7f020413d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:21.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0203fff640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020415f9c0 0x7f0204153ca0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00c780 tx=0x7f01fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0200ff9640 1 -- 172.21.15.16:0/1282857712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc003d80 con 0x7f020415f9c0 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0200ff9640 1 -- 172.21.15.16:0/1282857712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc00ce90 con 0x7f020415f9c0 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0200ff9640 1 -- 172.21.15.16:0/1282857712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0052b0 con 0x7f020415f9c0 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.968+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020415a690 con 0x7f020415f9c0 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.969+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020415ac50 con 0x7f020415f9c0 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.970+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02040675d0 con 0x7f020415f9c0 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.971+0000 7f0200ff9640 1 -- 172.21.15.16:0/1282857712 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01fc005410 con 0x7f020415f9c0 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.971+0000 7f0200ff9640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01e406ae40 0x7f01e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.972+0000 7f0200ff9640 1 -- 172.21.15.16:0/1282857712 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01fc088b80 con 0x7f020415f9c0 2023-11-24T06:28:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.972+0000 7f02037fe640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01e406ae40 0x7f01e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.973+0000 7f02037fe640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01e406ae40 0x7f01e406d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f020406d920 tx=0x7f01f4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:20.976+0000 7f0200ff9640 1 -- 172.21.15.16:0/1282857712 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01fc053b90 con 0x7f020415f9c0 2023-11-24T06:28:21.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.166+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f020414d420 con 0x7f020415f9c0 2023-11-24T06:28:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.169+0000 7f0200ff9640 1 -- 172.21.15.16:0/1282857712 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f01fc088e60 con 0x7f020415f9c0 2023-11-24T06:28:21.174 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:21.174 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:20.189038+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:28:18.648898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:28:19.200832+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.172+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01e406ae40 msgr2=0x7f01e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.172+0000 7f0209c83640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01e406ae40 0x7f01e406d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f020406d920 tx=0x7f01f4008040 comp rx=0 tx=0).stop 2023-11-24T06:28:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.172+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020415f9c0 msgr2=0x7f0204153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.172+0000 7f0209c83640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020415f9c0 0x7f0204153ca0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00c780 tx=0x7f01fc00cc40 comp rx=0 tx=0).stop 2023-11-24T06:28:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.173+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 shutdown_connections 2023-11-24T06:28:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.173+0000 7f0209c83640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01e406ae40 0x7f01e406d300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.173+0000 7f0209c83640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f020415f9c0 0x7f0204153ca0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.173+0000 7f0209c83640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f020413e360 0x7f020414ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.173+0000 7f0209c83640 1 --2- 172.21.15.16:0/1282857712 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f020413df80 0x7f020413d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.173+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 >> 172.21.15.16:0/1282857712 conn(0x7f020406a800 msgr2=0x7f02040b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.173+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 shutdown_connections 2023-11-24T06:28:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.173+0000 7f0209c83640 1 -- 172.21.15.16:0/1282857712 wait complete. 2023-11-24T06:28:21.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:21.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.408+0000 7f3f1addb640 1 Processor -- start 2023-11-24T06:28:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.408+0000 7f3f1addb640 1 -- start start 2023-11-24T06:28:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.410+0000 7f3f1addb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f14162430 0x7f3f140b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.410+0000 7f3f1addb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f140b3650 0x7f3f1406c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.410+0000 7f3f1addb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f140702e0 con 0x7f3f14162810 2023-11-24T06:28:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.410+0000 7f3f1addb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f140705c0 con 0x7f3f14162430 2023-11-24T06:28:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.410+0000 7f3f1addb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140708a0 con 0x7f3f140b3650 2023-11-24T06:28:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.410+0000 7f3f19dd9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f14162430 0x7f3f140b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.411+0000 7f3f19dd9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f14162430 0x7f3f140b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47980/0 (socket says 172.21.15.16:47980) 2023-11-24T06:28:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.411+0000 7f3f19dd9640 1 -- 172.21.15.16:0/2847717381 learned_addr learned my addr 172.21.15.16:0/2847717381 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.411+0000 7f3f1a5da640 1 --2- 172.21.15.16:0/2847717381 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f140b3650 0x7f3f1406c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.411+0000 7f3f19dd9640 1 -- 172.21.15.16:0/2847717381 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f140b3650 msgr2=0x7f3f1406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.411+0000 7f3f19dd9640 1 --2- 172.21.15.16:0/2847717381 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f140b3650 0x7f3f1406c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.411+0000 7f3f19dd9640 1 -- 172.21.15.16:0/2847717381 >> v1:172.21.15.16:6789/0 conn(0x7f3f14162810 legacy=0x7f3f14071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.411+0000 7f3f19dd9640 1 -- 172.21.15.16:0/2847717381 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14070b80 con 0x7f3f14162430 2023-11-24T06:28:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.412+0000 7f3f19dd9640 1 --2- 172.21.15.16:0/2847717381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f14162430 0x7f3f140b3110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009870 tx=0x7f3f1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a77102ff3c8d32 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.412+0000 7f3f18dd7640 1 -- 172.21.15.16:0/2847717381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10030840 con 0x7f3f14162430 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f18dd7640 1 -- 172.21.15.16:0/2847717381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f100309a0 con 0x7f3f14162430 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f18dd7640 1 -- 172.21.15.16:0/2847717381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10030d30 con 0x7f3f14162430 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f1addb640 1 -- 172.21.15.16:0/2847717381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f14162430 msgr2=0x7f3f140b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f1addb640 1 --2- 172.21.15.16:0/2847717381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f14162430 0x7f3f140b3110 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009870 tx=0x7f3f1002b980 comp rx=0 tx=0).stop 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f1addb640 1 -- 172.21.15.16:0/2847717381 shutdown_connections 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f1addb640 1 --2- 172.21.15.16:0/2847717381 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f140b3650 0x7f3f1406c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f1addb640 1 --2- 172.21.15.16:0/2847717381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f14162430 0x7f3f140b3110 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f1addb640 1 -- 172.21.15.16:0/2847717381 >> 172.21.15.16:0/2847717381 conn(0x7f3f1406a800 msgr2=0x7f3f1406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f1addb640 1 -- 172.21.15.16:0/2847717381 shutdown_connections 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.413+0000 7f3f1addb640 1 -- 172.21.15.16:0/2847717381 wait complete. 2023-11-24T06:28:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.415+0000 7f3f1addb640 1 Processor -- start 2023-11-24T06:28:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.415+0000 7f3f1addb640 1 -- start start 2023-11-24T06:28:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.416+0000 7f3f1addb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f140b3650 0x7f3f141408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.416+0000 7f3f1addb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f14162430 0x7f3f14153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.416+0000 7f3f1addb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f1415b310 0x7f3f140cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.416+0000 7f3f1addb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14140df0 con 0x7f3f140b3650 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.416+0000 7f3f1addb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f141405f0 con 0x7f3f14162430 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.416+0000 7f3f1addb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1415b7d0 con 0x7f3f1415b310 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.416+0000 7f3f19dd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f140b3650 0x7f3f141408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.416+0000 7f3f19dd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f140b3650 0x7f3f141408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47986/0 (socket says 172.21.15.16:47986) 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.416+0000 7f3f19dd9640 1 -- 172.21.15.16:0/4035642865 learned_addr learned my addr 172.21.15.16:0/4035642865 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.417+0000 7f3f1a5da640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f1415b310 0x7f3f140cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.417+0000 7f3f19dd9640 1 -- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f1415b310 msgr2=0x7f3f140cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.417+0000 7f3f19dd9640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f1415b310 0x7f3f140cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.417+0000 7f3f195d8640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f14162430 0x7f3f14153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.417+0000 7f3f19dd9640 1 -- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f14162430 msgr2=0x7f3f14153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.417+0000 7f3f19dd9640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f14162430 0x7f3f14153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.417+0000 7f3f19dd9640 1 -- 172.21.15.16:0/4035642865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f100094d0 con 0x7f3f140b3650 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.417+0000 7f3f195d8640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f14162430 0x7f3f14153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.417+0000 7f3f19dd9640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f140b3650 0x7f3f141408b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3f10000c00 tx=0x7f3f10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.418+0000 7f3f0a7fc640 1 -- 172.21.15.16:0/4035642865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10039c90 con 0x7f3f140b3650 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.418+0000 7f3f0a7fc640 1 -- 172.21.15.16:0/4035642865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f10039df0 con 0x7f3f140b3650 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.418+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f14165720 con 0x7f3f140b3650 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.418+0000 7f3f0a7fc640 1 -- 172.21.15.16:0/4035642865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10043640 con 0x7f3f140b3650 2023-11-24T06:28:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.418+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f14153ca0 con 0x7f3f140b3650 2023-11-24T06:28:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.419+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f14165c40 con 0x7f3f140b3650 2023-11-24T06:28:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.420+0000 7f3f0a7fc640 1 -- 172.21.15.16:0/4035642865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f100437a0 con 0x7f3f140b3650 2023-11-24T06:28:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.421+0000 7f3f0a7fc640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ef806ae40 0x7f3ef806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.421+0000 7f3f0a7fc640 1 -- 172.21.15.16:0/4035642865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f1003e070 con 0x7f3f140b3650 2023-11-24T06:28:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.422+0000 7f3f195d8640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ef806ae40 0x7f3ef806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.423+0000 7f3f195d8640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ef806ae40 0x7f3ef806d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3f140ce610 tx=0x7f3f04009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.425+0000 7f3f0a7fc640 1 -- 172.21.15.16:0/4035642865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f100777b0 con 0x7f3f140b3650 2023-11-24T06:28:21.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.619+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f14067450 con 0x7f3f140b3650 2023-11-24T06:28:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.621+0000 7f3f0a7fc640 1 -- 172.21.15.16:0/4035642865 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f3f1007add0 con 0x7f3f140b3650 2023-11-24T06:28:21.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:21.624 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:20.189038+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:28:18.648898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:28:19.200832+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:21.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ef806ae40 msgr2=0x7f3ef806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ef806ae40 0x7f3ef806d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3f140ce610 tx=0x7f3f04009040 comp rx=0 tx=0).stop 2023-11-24T06:28:21.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f140b3650 msgr2=0x7f3f141408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f140b3650 0x7f3f141408b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3f10000c00 tx=0x7f3f10004730 comp rx=0 tx=0).stop 2023-11-24T06:28:21.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 shutdown_connections 2023-11-24T06:28:21.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3ef806ae40 0x7f3ef806d300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f1415b310 0x7f3f140cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f14162430 0x7f3f14153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 --2- 172.21.15.16:0/4035642865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f140b3650 0x7f3f141408b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.623+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 >> 172.21.15.16:0/4035642865 conn(0x7f3f1406a800 msgr2=0x7f3f1406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:21.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.624+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 shutdown_connections 2023-11-24T06:28:21.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.624+0000 7f3f1addb640 1 -- 172.21.15.16:0/4035642865 wait complete. 2023-11-24T06:28:21.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:28:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.848+0000 7fc06dfde640 1 Processor -- start 2023-11-24T06:28:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.849+0000 7fc06dfde640 1 -- start start 2023-11-24T06:28:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.850+0000 7fc06dfde640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 0x7fc0680b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.850+0000 7fc06dfde640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc06806c6b0 0x7fc0680cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.850+0000 7fc06dfde640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06806cea0 con 0x7fc0680cd240 2023-11-24T06:28:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.850+0000 7fc06dfde640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06806cbc0 con 0x7fc06806c6b0 2023-11-24T06:28:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.850+0000 7fc06dfde640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06806f350 con 0x7fc06809a170 2023-11-24T06:28:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.851+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 0x7fc0680b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.851+0000 7fc067fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc06806c6b0 0x7fc0680cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.851+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 0x7fc0680b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:41184/0 (socket says 172.21.15.16:41184) 2023-11-24T06:28:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.851+0000 7fc0677fe640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fc0680cd240 0x7fc068099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:43764/0 (socket says 172.21.15.16:43764) 2023-11-24T06:28:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.852+0000 7fc05ffff640 1 -- 172.21.15.16:0/745319064 learned_addr learned my addr 172.21.15.16:0/745319064 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.852+0000 7fc067fff640 1 -- 172.21.15.16:0/745319064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 msgr2=0x7fc0680b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.852+0000 7fc067fff640 1 --2- 172.21.15.16:0/745319064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 0x7fc0680b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.852+0000 7fc067fff640 1 -- 172.21.15.16:0/745319064 >> v1:172.21.15.173:6789/0 conn(0x7fc0680cd240 legacy=0x7fc068099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.852+0000 7fc067fff640 1 -- 172.21.15.16:0/745319064 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06806f630 con 0x7fc06806c6b0 2023-11-24T06:28:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.854+0000 7fc067fff640 1 --2- 172.21.15.16:0/745319064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc06806c6b0 0x7fc0680cf640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc060009d90 tx=0x7fc06002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdf4d740d2efdf8a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.854+0000 7fc066ffd640 1 -- 172.21.15.16:0/745319064 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06002cc50 con 0x7fc06806c6b0 2023-11-24T06:28:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.854+0000 7fc066ffd640 1 -- 172.21.15.16:0/745319064 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06002cdb0 con 0x7fc06806c6b0 2023-11-24T06:28:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.854+0000 7fc066ffd640 1 -- 172.21.15.16:0/745319064 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060034660 con 0x7fc06806c6b0 2023-11-24T06:28:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.855+0000 7fc06dfde640 1 -- 172.21.15.16:0/745319064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc06806c6b0 msgr2=0x7fc0680cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.855+0000 7fc06dfde640 1 --2- 172.21.15.16:0/745319064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc06806c6b0 0x7fc0680cf640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc060009d90 tx=0x7fc06002bea0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.855+0000 7fc06dfde640 1 -- 172.21.15.16:0/745319064 shutdown_connections 2023-11-24T06:28:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.855+0000 7fc06dfde640 1 --2- 172.21.15.16:0/745319064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc06806c6b0 0x7fc0680cf640 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.855+0000 7fc06dfde640 1 --2- 172.21.15.16:0/745319064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 0x7fc0680b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.855+0000 7fc06dfde640 1 -- 172.21.15.16:0/745319064 >> 172.21.15.16:0/745319064 conn(0x7fc0680b2d00 msgr2=0x7fc0680b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.855+0000 7fc06dfde640 1 -- 172.21.15.16:0/745319064 shutdown_connections 2023-11-24T06:28:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.855+0000 7fc06dfde640 1 -- 172.21.15.16:0/745319064 wait complete. 2023-11-24T06:28:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.857+0000 7fc06dfde640 1 Processor -- start 2023-11-24T06:28:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.857+0000 7fc06dfde640 1 -- start start 2023-11-24T06:28:21.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc06dfde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06806c6b0 0x7fc0681f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc06dfde640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 0x7fc06814fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc06dfde640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0680cd240 0x7fc068151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc06dfde640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680b5ed0 con 0x7fc06806c6b0 2023-11-24T06:28:21.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc06dfde640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068152430 con 0x7fc0680cd240 2023-11-24T06:28:21.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc06dfde640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681526d0 con 0x7fc06809a170 2023-11-24T06:28:21.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc0677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06806c6b0 0x7fc0681f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc0677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06806c6b0 0x7fc0681f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48002/0 (socket says 172.21.15.16:48002) 2023-11-24T06:28:21.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc0677fe640 1 -- 172.21.15.16:0/3584045857 learned_addr learned my addr 172.21.15.16:0/3584045857 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc067fff640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0680cd240 0x7fc068151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.858+0000 7fc05ffff640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 0x7fc06814fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc0677fe640 1 -- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 msgr2=0x7fc06814fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc0677fe640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 0x7fc06814fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc0677fe640 1 -- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0680cd240 msgr2=0x7fc068151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc0677fe640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0680cd240 0x7fc068151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc0677fe640 1 -- 172.21.15.16:0/3584045857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009a70 con 0x7fc06806c6b0 2023-11-24T06:28:21.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc0677fe640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06806c6b0 0x7fc0681f6e20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c4d0 tx=0x7fc05800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc064ff9640 1 -- 172.21.15.16:0/3584045857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058007480 con 0x7fc06806c6b0 2023-11-24T06:28:21.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc064ff9640 1 -- 172.21.15.16:0/3584045857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0580075e0 con 0x7fc06806c6b0 2023-11-24T06:28:21.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc064ff9640 1 -- 172.21.15.16:0/3584045857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0580049e0 con 0x7fc06806c6b0 2023-11-24T06:28:21.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0681529b0 con 0x7fc06806c6b0 2023-11-24T06:28:21.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.859+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0680b5b50 con 0x7fc06806c6b0 2023-11-24T06:28:21.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.861+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0680675d0 con 0x7fc06806c6b0 2023-11-24T06:28:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.862+0000 7fc064ff9640 1 -- 172.21.15.16:0/3584045857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc058004b40 con 0x7fc06806c6b0 2023-11-24T06:28:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.862+0000 7fc064ff9640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc04806ae40 0x7fc04806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.862+0000 7fc064ff9640 1 -- 172.21.15.16:0/3584045857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0580886f0 con 0x7fc06806c6b0 2023-11-24T06:28:21.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.863+0000 7fc05ffff640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc04806ae40 0x7fc04806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:21.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.864+0000 7fc05ffff640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc04806ae40 0x7fc04806d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0680b2070 tx=0x7fc050008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:21.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:21.867+0000 7fc064ff9640 1 -- 172.21.15.16:0/3584045857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc058053850 con 0x7fc06806c6b0 2023-11-24T06:28:22.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.099+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc0680b35d0 con 0x7fc06806c6b0 2023-11-24T06:28:22.247 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-11-24T06:28:22.244+0000 7fc38754f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:28:22.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.968+0000 7fc064ff9640 1 -- 172.21.15.16:0/3584045857 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc0580180b0 con 0x7fc06806c6b0 2023-11-24T06:28:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.971+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc04806ae40 msgr2=0x7fc04806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.971+0000 7fc06dfde640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc04806ae40 0x7fc04806d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0680b2070 tx=0x7fc050008040 comp rx=0 tx=0).stop 2023-11-24T06:28:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.971+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06806c6b0 msgr2=0x7fc0681f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.971+0000 7fc06dfde640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06806c6b0 0x7fc0681f6e20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c4d0 tx=0x7fc05800c990 comp rx=0 tx=0).stop 2023-11-24T06:28:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.972+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 shutdown_connections 2023-11-24T06:28:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.972+0000 7fc06dfde640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc04806ae40 0x7fc04806d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.972+0000 7fc06dfde640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0680cd240 0x7fc068151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.972+0000 7fc06dfde640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc06809a170 0x7fc06814fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.972+0000 7fc06dfde640 1 --2- 172.21.15.16:0/3584045857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc06806c6b0 0x7fc0681f6e20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.972+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 >> 172.21.15.16:0/3584045857 conn(0x7fc0680b2d00 msgr2=0x7fc068160850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.972+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 shutdown_connections 2023-11-24T06:28:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:22.972+0000 7fc06dfde640 1 -- 172.21.15.16:0/3584045857 wait complete. 2023-11-24T06:28:22.995 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:28:22.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:28:23.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:23.060 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:23.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.216+0000 7f4966708640 1 Processor -- start 2023-11-24T06:28:23.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.217+0000 7f4966708640 1 -- start start 2023-11-24T06:28:23.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.218+0000 7f4966708640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f496013e360 0x7f49600cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.218+0000 7f4966708640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f496015f9c0 0x7f49600cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.218+0000 7f4966708640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496015c5f0 con 0x7f496013df80 2023-11-24T06:28:23.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.218+0000 7f4966708640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496006c470 con 0x7f496013e360 2023-11-24T06:28:23.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.218+0000 7f4966708640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496006c750 con 0x7f496015f9c0 2023-11-24T06:28:23.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.219+0000 7f4957fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f496013e360 0x7f49600cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.219+0000 7f4957fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f496013e360 0x7f49600cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50222/0 (socket says 172.21.15.16:50222) 2023-11-24T06:28:23.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.219+0000 7f4957fff640 1 -- 172.21.15.16:0/2421911293 learned_addr learned my addr 172.21.15.16:0/2421911293 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:23.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.219+0000 7f4964ca5640 1 --2- 172.21.15.16:0/2421911293 >> v2:172.21.15.173:3300/0 conn(0x7f496015f9c0 0x7f49600cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.219+0000 7f4957fff640 1 -- 172.21.15.16:0/2421911293 >> v2:172.21.15.173:3300/0 conn(0x7f496015f9c0 msgr2=0x7f49600cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:23.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.219+0000 7f4957fff640 1 --2- 172.21.15.16:0/2421911293 >> v2:172.21.15.173:3300/0 conn(0x7f496015f9c0 0x7f49600cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.219+0000 7f4957fff640 1 -- 172.21.15.16:0/2421911293 >> v1:172.21.15.173:6789/0 conn(0x7f496013df80 legacy=0x7f496015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:23.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.219+0000 7f4957fff640 1 -- 172.21.15.16:0/2421911293 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496006ca30 con 0x7f496013e360 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.220+0000 7f4957fff640 1 --2- 172.21.15.16:0/2421911293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 0x7f49600cf290 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f496006ded0 tx=0x7f494802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6dd0597f5feae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.221+0000 7f495f7fe640 1 -- 172.21.15.16:0/2421911293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948031640 con 0x7f496013e360 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.221+0000 7f495f7fe640 1 -- 172.21.15.16:0/2421911293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49480317a0 con 0x7f496013e360 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.221+0000 7f495f7fe640 1 -- 172.21.15.16:0/2421911293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948031b30 con 0x7f496013e360 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.221+0000 7f4966708640 1 -- 172.21.15.16:0/2421911293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 msgr2=0x7f49600cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.221+0000 7f4966708640 1 --2- 172.21.15.16:0/2421911293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 0x7f49600cf290 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f496006ded0 tx=0x7f494802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.222+0000 7f4966708640 1 -- 172.21.15.16:0/2421911293 shutdown_connections 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.222+0000 7f4966708640 1 --2- 172.21.15.16:0/2421911293 >> v2:172.21.15.173:3300/0 conn(0x7f496015f9c0 0x7f49600cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.222+0000 7f4966708640 1 --2- 172.21.15.16:0/2421911293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 0x7f49600cf290 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.222+0000 7f4966708640 1 -- 172.21.15.16:0/2421911293 >> 172.21.15.16:0/2421911293 conn(0x7f496006a800 msgr2=0x7f496006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.222+0000 7f4966708640 1 -- 172.21.15.16:0/2421911293 shutdown_connections 2023-11-24T06:28:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.222+0000 7f4966708640 1 -- 172.21.15.16:0/2421911293 wait complete. 2023-11-24T06:28:23.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.223+0000 7f4966708640 1 Processor -- start 2023-11-24T06:28:23.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.224+0000 7f4966708640 1 -- start start 2023-11-24T06:28:23.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.224+0000 7f4966708640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f496013df80 0x7f49601304f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.224+0000 7f4966708640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 0x7f49601f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.224+0000 7f4966708640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f496015f9c0 0x7f496014c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.224+0000 7f4966708640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496014da80 con 0x7f496013e360 2023-11-24T06:28:23.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4966708640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496014cbc0 con 0x7f496013df80 2023-11-24T06:28:23.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4966708640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f496014ce70 con 0x7f496015f9c0 2023-11-24T06:28:23.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4957fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 0x7f49601f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4957fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 0x7f49601f4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50236/0 (socket says 172.21.15.16:50236) 2023-11-24T06:28:23.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4957fff640 1 -- 172.21.15.16:0/2832533729 learned_addr learned my addr 172.21.15.16:0/2832533729 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:23.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f495ffff640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f496013df80 0x7f49601304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4957fff640 1 -- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f496015f9c0 msgr2=0x7f496014c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4957fff640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f496015f9c0 0x7f496014c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4957fff640 1 -- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f496013df80 msgr2=0x7f49601304f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4957fff640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f496013df80 0x7f49601304f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.225+0000 7f4957fff640 1 -- 172.21.15.16:0/2832533729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480094d0 con 0x7f496013e360 2023-11-24T06:28:23.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.226+0000 7f4957fff640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 0x7f49601f4240 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4948000c00 tx=0x7f49480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:23.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.226+0000 7f495d7fa640 1 -- 172.21.15.16:0/2832533729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494803ac60 con 0x7f496013e360 2023-11-24T06:28:23.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.226+0000 7f495d7fa640 1 -- 172.21.15.16:0/2832533729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494803adc0 con 0x7f496013e360 2023-11-24T06:28:23.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.226+0000 7f495d7fa640 1 -- 172.21.15.16:0/2832533729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948044610 con 0x7f496013e360 2023-11-24T06:28:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.226+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496014d150 con 0x7f496013e360 2023-11-24T06:28:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.226+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496014d710 con 0x7f496013e360 2023-11-24T06:28:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.227+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49600675d0 con 0x7f496013e360 2023-11-24T06:28:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.228+0000 7f495d7fa640 1 -- 172.21.15.16:0/2832533729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f494802f050 con 0x7f496013e360 2023-11-24T06:28:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.229+0000 7f495d7fa640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f494006ae40 0x7f494006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.229+0000 7f495d7fa640 1 -- 172.21.15.16:0/2832533729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49480ab530 con 0x7f496013e360 2023-11-24T06:28:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.230+0000 7f495ffff640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f494006ae40 0x7f494006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.231+0000 7f495ffff640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f494006ae40 0x7f494006d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4950006fd0 tx=0x7f4950008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.234+0000 7f495d7fa640 1 -- 172.21.15.16:0/2832533729 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49480765e0 con 0x7f496013e360 2023-11-24T06:28:23.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.466+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f496014dbc0 con 0x7f496013e360 2023-11-24T06:28:23.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.978+0000 7f495d7fa640 1 -- 172.21.15.16:0/2832533729 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4948031e60 con 0x7f496013e360 2023-11-24T06:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.981+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f494006ae40 msgr2=0x7f494006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.981+0000 7f4966708640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f494006ae40 0x7f494006d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4950006fd0 tx=0x7f4950008040 comp rx=0 tx=0).stop 2023-11-24T06:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 msgr2=0x7f49601f4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 0x7f49601f4240 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4948000c00 tx=0x7f49480046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 shutdown_connections 2023-11-24T06:28:23.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f494006ae40 0x7f494006d300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f496015f9c0 0x7f496014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496013e360 0x7f49601f4240 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 --2- 172.21.15.16:0/2832533729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f496013df80 0x7f49601304f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 >> 172.21.15.16:0/2832533729 conn(0x7f496006a800 msgr2=0x7f496015de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 shutdown_connections 2023-11-24T06:28:23.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:23.982+0000 7f4966708640 1 -- 172.21.15.16:0/2832533729 wait complete. 2023-11-24T06:28:24.004 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:28:24.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.274+0000 7f205234e640 1 Processor -- start 2023-11-24T06:28:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.274+0000 7f205234e640 1 -- start start 2023-11-24T06:28:24.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.275+0000 7f205234e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f204c0cc990 0x7f204c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.275+0000 7f205234e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c070e50 con 0x7f204c143c40 2023-11-24T06:28:24.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.275+0000 7f205234e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c144020 con 0x7f204c0cc500 2023-11-24T06:28:24.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.275+0000 7f205234e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c0723c0 con 0x7f204c0cc990 2023-11-24T06:28:24.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.276+0000 7f205134c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f204c143c40 0x7f204c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60724/0 (socket says 172.21.15.16:60724) 2023-11-24T06:28:24.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.276+0000 7f205134c640 1 -- 172.21.15.16:0/173154642 learned_addr learned my addr 172.21.15.16:0/173154642 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.276+0000 7f2051b4d640 1 --2- 172.21.15.16:0/173154642 >> v2:172.21.15.173:3300/0 conn(0x7f204c0cc990 0x7f204c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.276+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212602268 0 0) 0x7f204c070e50 con 0x7f204c143c40 2023-11-24T06:28:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.276+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2038003520 con 0x7f204c143c40 2023-11-24T06:28:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2752806251 0 0) 0x7f2038003520 con 0x7f204c143c40 2023-11-24T06:28:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 >> v2:172.21.15.173:3300/0 conn(0x7f204c0cc990 msgr2=0x7f204c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 --2- 172.21.15.16:0/173154642 >> v2:172.21.15.173:3300/0 conn(0x7f204c0cc990 0x7f204c06d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 >> v1:172.21.15.173:6789/0 conn(0x7f204c0cc500 legacy=0x7f204c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c071e00 con 0x7f204c143c40 2023-11-24T06:28:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f2048001670 con 0x7f204c143c40 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 >> v1:172.21.15.16:6789/0 conn(0x7f204c143c40 legacy=0x7f204c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20380039c0 0x7f2038003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f2038004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20380051d0 0x7f203800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0723c0 con 0x7f20380047c0 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203800c120 con 0x7f20380051d0 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203800c2e0 con 0x7f20380039c0 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2050b4b640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f2038004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.277+0000 7f2051b4d640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20380051d0 0x7f203800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.278+0000 7f205134c640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20380039c0 0x7f2038003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.278+0000 7f2050b4b640 1 -- 172.21.15.16:0/173154642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20380039c0 msgr2=0x7f2038003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.278+0000 7f2050b4b640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20380039c0 0x7f2038003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.278+0000 7f2050b4b640 1 -- 172.21.15.16:0/173154642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20380051d0 msgr2=0x7f203800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:24.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.278+0000 7f2050b4b640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20380051d0 0x7f203800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.278+0000 7f2050b4b640 1 -- 172.21.15.16:0/173154642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2038003520 con 0x7f20380047c0 2023-11-24T06:28:24.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.278+0000 7f2051b4d640 1 -- 172.21.15.16:0/173154642 reap_dead start 2023-11-24T06:28:24.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.279+0000 7f2050b4b640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f2038004c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f203c001720 tx=0x7f20380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a7668d7fe07481 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.279+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c0052d0 con 0x7f20380047c0 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.279+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c02e050 con 0x7f20380047c0 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.279+0000 7f2043fff640 1 -- 172.21.15.16:0/173154642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c003830 con 0x7f20380047c0 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.280+0000 7f205234e640 1 -- 172.21.15.16:0/173154642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 msgr2=0x7f2038004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.280+0000 7f205234e640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f2038004c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f203c001720 tx=0x7f20380057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.280+0000 7f205234e640 1 -- 172.21.15.16:0/173154642 shutdown_connections 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.280+0000 7f205234e640 1 --2- 172.21.15.16:0/173154642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f2038004c40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.280+0000 7f205234e640 1 -- 172.21.15.16:0/173154642 >> 172.21.15.16:0/173154642 conn(0x7f204c06a800 msgr2=0x7f204c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.280+0000 7f205234e640 1 -- 172.21.15.16:0/173154642 shutdown_connections 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.280+0000 7f205234e640 1 -- 172.21.15.16:0/173154642 wait complete. 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.282+0000 7f205234e640 1 Processor -- start 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.282+0000 7f205234e640 1 -- start start 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.283+0000 7f205234e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f204c13cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.283+0000 7f205234e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f204c1e57a0 0x7f204c1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.283+0000 7f205234e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f204c13ddc0 0x7f204c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.283+0000 7f205234e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c1e5380 con 0x7f20380047c0 2023-11-24T06:28:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.283+0000 7f205234e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c159ee0 con 0x7f204c1e57a0 2023-11-24T06:28:24.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.283+0000 7f205234e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c13d390 con 0x7f204c13ddc0 2023-11-24T06:28:24.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.283+0000 7f205134c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f204c13cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.283+0000 7f205134c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f204c13cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50254/0 (socket says 172.21.15.16:50254) 2023-11-24T06:28:24.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.283+0000 7f205134c640 1 -- 172.21.15.16:0/3225353826 learned_addr learned my addr 172.21.15.16:0/3225353826 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:24.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f2050b4b640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f204c1e57a0 0x7f204c1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f205134c640 1 -- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f204c13ddc0 msgr2=0x7f204c0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f2051b4d640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f204c13ddc0 0x7f204c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f205134c640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f204c13ddc0 0x7f204c0b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f205134c640 1 -- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f204c1e57a0 msgr2=0x7f204c1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f205134c640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f204c1e57a0 0x7f204c1599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f205134c640 1 -- 172.21.15.16:0/3225353826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c15a3a0 con 0x7f20380047c0 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f205134c640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f204c13cc90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f204800e760 tx=0x7f204800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f2041ffb640 1 -- 172.21.15.16:0/3225353826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800c660 con 0x7f20380047c0 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f2041ffb640 1 -- 172.21.15.16:0/3225353826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204800c7c0 con 0x7f20380047c0 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f2041ffb640 1 -- 172.21.15.16:0/3225353826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800cb70 con 0x7f20380047c0 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.284+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c1e4a20 con 0x7f20380047c0 2023-11-24T06:28:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.285+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c1e4fa0 con 0x7f20380047c0 2023-11-24T06:28:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.286+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c070290 con 0x7f20380047c0 2023-11-24T06:28:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.287+0000 7f2041ffb640 1 -- 172.21.15.16:0/3225353826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f204800ccd0 con 0x7f20380047c0 2023-11-24T06:28:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.288+0000 7f2041ffb640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f203006ae40 0x7f203006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.289+0000 7f2041ffb640 1 -- 172.21.15.16:0/3225353826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2048089410 con 0x7f20380047c0 2023-11-24T06:28:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.289+0000 7f2050b4b640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f203006ae40 0x7f203006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.290+0000 7f2050b4b640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f203006ae40 0x7f203006d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f203c033000 tx=0x7f203c009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.293+0000 7f2041ffb640 1 -- 172.21.15.16:0/3225353826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20480544b0 con 0x7f20380047c0 2023-11-24T06:28:24.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.525+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f204c067450 con 0x7f20380047c0 2023-11-24T06:28:24.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.526+0000 7f2041ffb640 1 -- 172.21.15.16:0/3225353826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f2048057ad0 con 0x7f20380047c0 2023-11-24T06:28:24.530 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:24.530 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/147489979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":147489979},{"type":"v1","addr":"172.21.15.173:6836","nonce":147489979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1330874839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1330874839},{"type":"v1","addr":"172.21.15.173:6837","nonce":1330874839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2341098684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2341098684},{"type":"v1","addr":"172.21.15.16:6835","nonce":2341098684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:28:20.189904+0000","modified":"2023-11-24T06:28:23.468425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6834/4049479925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4049479925},{"type":"v1","addr":"172.21.15.16:6834","nonce":4049479925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:28:24.530 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2023-11-24T06:28:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f203006ae40 msgr2=0x7f203006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f203006ae40 0x7f203006d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f203c033000 tx=0x7f203c009030 comp rx=0 tx=0).stop 2023-11-24T06:28:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 msgr2=0x7f204c13cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f204c13cc90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f204800e760 tx=0x7f204800ec20 comp rx=0 tx=0).stop 2023-11-24T06:28:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 shutdown_connections 2023-11-24T06:28:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f203006ae40 0x7f203006d300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f204c13ddc0 0x7f204c0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f204c1e57a0 0x7f204c1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 --2- 172.21.15.16:0/3225353826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380047c0 0x7f204c13cc90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 >> 172.21.15.16:0/3225353826 conn(0x7f204c06a800 msgr2=0x7f204c0af400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 shutdown_connections 2023-11-24T06:28:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.529+0000 7f205234e640 1 -- 172.21.15.16:0/3225353826 wait complete. 2023-11-24T06:28:24.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.776+0000 7f2436859640 1 Processor -- start 2023-11-24T06:28:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.776+0000 7f2436859640 1 -- start start 2023-11-24T06:28:24.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2436859640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2430143c40 0x7f2430071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2436859640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24300ccc10 0x7f243006d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2436859640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2430070e50 con 0x7f24300717d0 2023-11-24T06:28:24.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2436859640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24300723c0 con 0x7f24300ccc10 2023-11-24T06:28:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2436859640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2430144020 con 0x7f2430143c40 2023-11-24T06:28:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2436058640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24300ccc10 0x7f243006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2436058640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24300ccc10 0x7f243006d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50258/0 (socket says 172.21.15.16:50258) 2023-11-24T06:28:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2436058640 1 -- 172.21.15.16:0/3637796953 learned_addr learned my addr 172.21.15.16:0/3637796953 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2435056640 1 --1- 172.21.15.16:0/3637796953 >> v1:172.21.15.16:6789/0 conn(0x7f24300717d0 0x7f24300cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60728/0 (socket says 172.21.15.16:60728) 2023-11-24T06:28:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.778+0000 7f2435857640 1 --2- 172.21.15.16:0/3637796953 >> v2:172.21.15.173:3300/0 conn(0x7f2430143c40 0x7f2430071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.779+0000 7f2436058640 1 -- 172.21.15.16:0/3637796953 >> v2:172.21.15.173:3300/0 conn(0x7f2430143c40 msgr2=0x7f2430071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.779+0000 7f2436058640 1 --2- 172.21.15.16:0/3637796953 >> v2:172.21.15.173:3300/0 conn(0x7f2430143c40 0x7f2430071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.779+0000 7f2436058640 1 -- 172.21.15.16:0/3637796953 >> v1:172.21.15.16:6789/0 conn(0x7f24300717d0 legacy=0x7f24300cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.779+0000 7f2436058640 1 -- 172.21.15.16:0/3637796953 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430071e00 con 0x7f24300ccc10 2023-11-24T06:28:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.780+0000 7f2436058640 1 --2- 172.21.15.16:0/3637796953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 0x7f243006d160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f243006e410 tx=0x7f242802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a5b2bad85e7295 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.780+0000 7f2434855640 1 -- 172.21.15.16:0/3637796953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242802ca80 con 0x7f24300ccc10 2023-11-24T06:28:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.780+0000 7f2434855640 1 -- 172.21.15.16:0/3637796953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242802cbe0 con 0x7f24300ccc10 2023-11-24T06:28:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.780+0000 7f2434855640 1 -- 172.21.15.16:0/3637796953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24280343f0 con 0x7f24300ccc10 2023-11-24T06:28:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.781+0000 7f2436859640 1 -- 172.21.15.16:0/3637796953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 msgr2=0x7f243006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.781+0000 7f2436859640 1 --2- 172.21.15.16:0/3637796953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 0x7f243006d160 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f243006e410 tx=0x7f242802bee0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.781+0000 7f2436859640 1 -- 172.21.15.16:0/3637796953 shutdown_connections 2023-11-24T06:28:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.781+0000 7f2436859640 1 --2- 172.21.15.16:0/3637796953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 0x7f243006d160 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.781+0000 7f2436859640 1 --2- 172.21.15.16:0/3637796953 >> v2:172.21.15.173:3300/0 conn(0x7f2430143c40 0x7f2430071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.781+0000 7f2436859640 1 -- 172.21.15.16:0/3637796953 >> 172.21.15.16:0/3637796953 conn(0x7f243006a800 msgr2=0x7f243006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.781+0000 7f2436859640 1 -- 172.21.15.16:0/3637796953 shutdown_connections 2023-11-24T06:28:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.781+0000 7f2436859640 1 -- 172.21.15.16:0/3637796953 wait complete. 2023-11-24T06:28:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.783+0000 7f2436859640 1 Processor -- start 2023-11-24T06:28:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.783+0000 7f2436859640 1 -- start start 2023-11-24T06:28:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.784+0000 7f2436859640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24300717d0 0x7f243015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.784+0000 7f2436859640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 0x7f24300c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.784+0000 7f2436859640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2430143c40 0x7f24300c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.784+0000 7f2436859640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243012d670 con 0x7f24300ccc10 2023-11-24T06:28:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.784+0000 7f2436859640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24301f6e50 con 0x7f24300717d0 2023-11-24T06:28:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.784+0000 7f2436859640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24300c4d60 con 0x7f2430143c40 2023-11-24T06:28:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.784+0000 7f2435056640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 0x7f24300c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.784+0000 7f2435056640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 0x7f24300c4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50268/0 (socket says 172.21.15.16:50268) 2023-11-24T06:28:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.784+0000 7f2435056640 1 -- 172.21.15.16:0/3617245692 learned_addr learned my addr 172.21.15.16:0/3617245692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f2435857640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24300717d0 0x7f243015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f2436058640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2430143c40 0x7f24300c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f2435056640 1 -- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2430143c40 msgr2=0x7f24300c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f2435056640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2430143c40 0x7f24300c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f2435056640 1 -- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24300717d0 msgr2=0x7f243015a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f2435056640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24300717d0 0x7f243015a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f2435056640 1 -- 172.21.15.16:0/3617245692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428009b30 con 0x7f24300ccc10 2023-11-24T06:28:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f2435056640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 0x7f24300c4820 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f242000c3e0 tx=0x7f242000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f24267fc640 1 -- 172.21.15.16:0/3617245692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200074e0 con 0x7f24300ccc10 2023-11-24T06:28:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f24267fc640 1 -- 172.21.15.16:0/3617245692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2420007640 con 0x7f24300ccc10 2023-11-24T06:28:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f24267fc640 1 -- 172.21.15.16:0/3617245692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200049e0 con 0x7f24300ccc10 2023-11-24T06:28:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.785+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24301f65b0 con 0x7f24300ccc10 2023-11-24T06:28:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.786+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24301f6b70 con 0x7f24300ccc10 2023-11-24T06:28:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.787+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2430067450 con 0x7f24300ccc10 2023-11-24T06:28:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.788+0000 7f24267fc640 1 -- 172.21.15.16:0/3617245692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2420004b40 con 0x7f24300ccc10 2023-11-24T06:28:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.789+0000 7f24267fc640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241406ae40 0x7f241406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.789+0000 7f24267fc640 1 -- 172.21.15.16:0/3617245692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2420089080 con 0x7f24300ccc10 2023-11-24T06:28:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.790+0000 7f2435857640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241406ae40 0x7f241406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.791+0000 7f2435857640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241406ae40 0x7f241406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f242c006fd0 tx=0x7f242c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.794+0000 7f24267fc640 1 -- 172.21.15.16:0/3617245692 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24200bbe90 con 0x7f24300ccc10 2023-11-24T06:28:24.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.989+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24300ceb30 con 0x7f24300ccc10 2023-11-24T06:28:24.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.991+0000 7f24267fc640 1 -- 172.21.15.16:0/3617245692 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f2420054090 con 0x7f24300ccc10 2023-11-24T06:28:24.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:24.994 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:21.731503+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:28:18.648898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:28:19.200832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:24.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.994+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241406ae40 msgr2=0x7f241406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.994+0000 7f2436859640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241406ae40 0x7f241406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f242c006fd0 tx=0x7f242c008040 comp rx=0 tx=0).stop 2023-11-24T06:28:24.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.994+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 msgr2=0x7f24300c4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:24.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.994+0000 7f2436859640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 0x7f24300c4820 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f242000c3e0 tx=0x7f242000c8a0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.995+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 shutdown_connections 2023-11-24T06:28:24.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.995+0000 7f2436859640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241406ae40 0x7f241406d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.995+0000 7f2436859640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2430143c40 0x7f24300c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.995+0000 7f2436859640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24300ccc10 0x7f24300c4820 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.995+0000 7f2436859640 1 --2- 172.21.15.16:0/3617245692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24300717d0 0x7f243015a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:24.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.995+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 >> 172.21.15.16:0/3617245692 conn(0x7f243006a800 msgr2=0x7f24300cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:24.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.995+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 shutdown_connections 2023-11-24T06:28:24.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:24.995+0000 7f2436859640 1 -- 172.21.15.16:0/3617245692 wait complete. 2023-11-24T06:28:25.016 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:28:25.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.237+0000 7ff1e0fac640 1 Processor -- start 2023-11-24T06:28:25.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.238+0000 7ff1e0fac640 1 -- start start 2023-11-24T06:28:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.239+0000 7ff1e0fac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1dc160920 0x7ff1dc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.239+0000 7ff1e0fac640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff1dc0cf9f0 0x7ff1dc0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.239+0000 7ff1e0fac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc0d0a10 con 0x7ff1dc161010 2023-11-24T06:28:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.239+0000 7ff1e0fac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1dc0d03e0 con 0x7ff1dc160920 2023-11-24T06:28:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.239+0000 7ff1e0fac640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1dc0d1f50 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.240+0000 7ff1db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1dc160920 0x7ff1dc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.240+0000 7ff1db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1dc160920 0x7ff1dc0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50276/0 (socket says 172.21.15.16:50276) 2023-11-24T06:28:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.240+0000 7ff1db7fe640 1 -- 172.21.15.16:0/711921594 learned_addr learned my addr 172.21.15.16:0/711921594 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.240+0000 7ff1dbfff640 1 --1- 172.21.15.16:0/711921594 >> v1:172.21.15.16:6789/0 conn(0x7ff1dc161010 0x7ff1dc161480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60734/0 (socket says 172.21.15.16:60734) 2023-11-24T06:28:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.240+0000 7ff1db7fe640 1 -- 172.21.15.16:0/711921594 >> v2:172.21.15.173:3300/0 conn(0x7ff1dc0cf9f0 msgr2=0x7ff1dc0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.240+0000 7ff1db7fe640 1 --2- 172.21.15.16:0/711921594 >> v2:172.21.15.173:3300/0 conn(0x7ff1dc0cf9f0 0x7ff1dc0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.240+0000 7ff1db7fe640 1 -- 172.21.15.16:0/711921594 >> v1:172.21.15.16:6789/0 conn(0x7ff1dc161010 legacy=0x7ff1dc161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.240+0000 7ff1db7fe640 1 -- 172.21.15.16:0/711921594 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc0cf080 con 0x7ff1dc160920 2023-11-24T06:28:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.241+0000 7ff1db7fe640 1 --2- 172.21.15.16:0/711921594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc160920 0x7ff1dc0b3f30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc08c1f0 tx=0x7ff1cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b286d0ea17e2e5f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1daffd640 1 -- 172.21.15.16:0/711921594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc030860 con 0x7ff1dc160920 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1daffd640 1 -- 172.21.15.16:0/711921594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc0309c0 con 0x7ff1dc160920 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1daffd640 1 -- 172.21.15.16:0/711921594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc030d50 con 0x7ff1dc160920 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1e0fac640 1 -- 172.21.15.16:0/711921594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc160920 msgr2=0x7ff1dc0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1e0fac640 1 --2- 172.21.15.16:0/711921594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc160920 0x7ff1dc0b3f30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc08c1f0 tx=0x7ff1cc02b9a0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1e0fac640 1 -- 172.21.15.16:0/711921594 shutdown_connections 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1e0fac640 1 --2- 172.21.15.16:0/711921594 >> v2:172.21.15.173:3300/0 conn(0x7ff1dc0cf9f0 0x7ff1dc0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1e0fac640 1 --2- 172.21.15.16:0/711921594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc160920 0x7ff1dc0b3f30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1e0fac640 1 -- 172.21.15.16:0/711921594 >> 172.21.15.16:0/711921594 conn(0x7ff1dc0d3090 msgr2=0x7ff1dc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.242+0000 7ff1e0fac640 1 -- 172.21.15.16:0/711921594 shutdown_connections 2023-11-24T06:28:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.243+0000 7ff1e0fac640 1 -- 172.21.15.16:0/711921594 wait complete. 2023-11-24T06:28:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.244+0000 7ff1e0fac640 1 Processor -- start 2023-11-24T06:28:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.245+0000 7ff1e0fac640 1 -- start start 2023-11-24T06:28:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1e0fac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc0cf9f0 0x7ff1dc154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1e0fac640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1dc160920 0x7ff1dc0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1e0fac640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1dc161010 0x7ff1dc1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1e0fac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0b6620 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1e0fac640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0b5cf0 con 0x7ff1dc160920 2023-11-24T06:28:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1e0fac640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0b5fd0 con 0x7ff1dc161010 2023-11-24T06:28:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc0cf9f0 0x7ff1dc154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc0cf9f0 0x7ff1dc154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50290/0 (socket says 172.21.15.16:50290) 2023-11-24T06:28:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1db7fe640 1 -- 172.21.15.16:0/1313237222 learned_addr learned my addr 172.21.15.16:0/1313237222 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1db7fe640 1 -- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1dc161010 msgr2=0x7ff1dc1572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1db7fe640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1dc161010 0x7ff1dc1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1db7fe640 1 -- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1dc160920 msgr2=0x7ff1dc0b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1db7fe640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1dc160920 0x7ff1dc0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.246+0000 7ff1db7fe640 1 -- 172.21.15.16:0/1313237222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0094d0 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.247+0000 7ff1db7fe640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc0cf9f0 0x7ff1dc154db0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc02b9e0 tx=0x7ff1cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.247+0000 7ff1d8ff9640 1 -- 172.21.15.16:0/1313237222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc030850 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.247+0000 7ff1d8ff9640 1 -- 172.21.15.16:0/1313237222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc043400 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.247+0000 7ff1d8ff9640 1 -- 172.21.15.16:0/1313237222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0437b0 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.247+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc0b62b0 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.247+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc0b83f0 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.248+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc067450 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.250+0000 7ff1d8ff9640 1 -- 172.21.15.16:0/1313237222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff1cc043910 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.250+0000 7ff1d8ff9640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1bc06aec0 0x7ff1bc06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.251+0000 7ff1d8ff9640 1 -- 172.21.15.16:0/1313237222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1cc03e070 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.251+0000 7ff1d3fff640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1bc06aec0 0x7ff1bc06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.253+0000 7ff1d3fff640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1bc06aec0 0x7ff1bc06d380 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc158ce0 tx=0x7ff1c4006bf0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.258+0000 7ff1d8ff9640 1 -- 172.21.15.16:0/1313237222 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1cc077d20 con 0x7ff1dc0cf9f0 2023-11-24T06:28:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.455+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1dc0ceac0 con 0x7ff1bc06aec0 2023-11-24T06:28:25.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.483+0000 7ff1d8ff9640 1 -- 172.21.15.16:0/1313237222 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274714 (secure 0 0 0) 0x7ff1c4012390 con 0x7ff1bc06aec0 2023-11-24T06:28:25.490 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:25.492 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:28:25.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1bc06aec0 msgr2=0x7ff1bc06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1bc06aec0 0x7ff1bc06d380 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc158ce0 tx=0x7ff1c4006bf0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc0cf9f0 msgr2=0x7ff1dc154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc0cf9f0 0x7ff1dc154db0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc02b9e0 tx=0x7ff1cc004730 comp rx=0 tx=0).stop 2023-11-24T06:28:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 shutdown_connections 2023-11-24T06:28:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1bc06aec0 0x7ff1bc06d380 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1dc161010 0x7ff1dc1572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1dc160920 0x7ff1dc0b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 --2- 172.21.15.16:0/1313237222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1dc0cf9f0 0x7ff1dc154db0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 >> 172.21.15.16:0/1313237222 conn(0x7ff1dc0d3090 msgr2=0x7ff1dc161480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 shutdown_connections 2023-11-24T06:28:25.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.488+0000 7ff1e0fac640 1 -- 172.21.15.16:0/1313237222 wait complete. 2023-11-24T06:28:25.514 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-11-24T06:28:24.132156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236676,"kb_used_data":3868,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738044,"statfs":{"total":773094113280,"available":772851757056,"internally_reserved":0,"allocated":3960832,"data_stored":2235695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.440385"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.705175+0000","last_change":"2023-11-24T06:28:18.705175+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.705175+0000","last_undegraded":"2023-11-24T06:28:18.705175+0000","last_fullsized":"2023-11-24T06:28:18.705175+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695945+0000","last_change":"2023-11-24T06:28:18.695945+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695945+0000","last_undegraded":"2023-11-24T06:28:18.695945+0000","last_fullsized":"2023-11-24T06:28:18.695945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.697812+0000","last_change":"2023-11-24T06:28:18.697812+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.697812+0000","last_undegraded":"2023-11-24T06:28:18.697812+0000","last_fullsized":"2023-11-24T06:28:18.697812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.706454+0000","last_change":"2023-11-24T06:28:18.706454+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.706454+0000","last_undegraded":"2023-11-24T06:28:18.706454+0000","last_fullsized":"2023-11-24T06:28:18.706454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.713422+0000","last_change":"2023-11-24T06:28:18.713422+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.713422+0000","last_undegraded":"2023-11-24T06:28:18.713422+0000","last_fullsized":"2023-11-24T06:28:18.713422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.711845+0000","last_change":"2023-11-24T06:28:18.711845+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.711845+0000","last_undegraded":"2023-11-24T06:28:18.711845+0000","last_fullsized":"2023-11-24T06:28:18.711845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695981+0000","last_change":"2023-11-24T06:28:18.695981+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695981+0000","last_undegraded":"2023-11-24T06:28:18.695981+0000","last_fullsized":"2023-11-24T06:28:18.695981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.711915+0000","last_change":"2023-11-24T06:28:18.711915+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.711915+0000","last_undegraded":"2023-11-24T06:28:18.711915+0000","last_fullsized":"2023-11-24T06:28:18.711915+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695471+0000","last_change":"2023-11-24T06:28:18.695471+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695471+0000","last_undegraded":"2023-11-24T06:28:18.695471+0000","last_fullsized":"2023-11-24T06:28:18.695471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.694233+0000","last_change":"2023-11-24T06:28:18.694233+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.694233+0000","last_undegraded":"2023-11-24T06:28:18.694233+0000","last_fullsized":"2023-11-24T06:28:18.694233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.700202+0000","last_change":"2023-11-24T06:28:18.700202+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.700202+0000","last_undegraded":"2023-11-24T06:28:18.700202+0000","last_fullsized":"2023-11-24T06:28:18.700202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.697763+0000","last_change":"2023-11-24T06:28:18.697763+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.697763+0000","last_undegraded":"2023-11-24T06:28:18.697763+0000","last_fullsized":"2023-11-24T06:28:18.697763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.712103+0000","last_change":"2023-11-24T06:28:18.712103+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.712103+0000","last_undegraded":"2023-11-24T06:28:18.712103+0000","last_fullsized":"2023-11-24T06:28:18.712103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.699816+0000","last_change":"2023-11-24T06:28:18.699816+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.699816+0000","last_undegraded":"2023-11-24T06:28:18.699816+0000","last_fullsized":"2023-11-24T06:28:18.699816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.696609+0000","last_change":"2023-11-24T06:28:18.696609+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.696609+0000","last_undegraded":"2023-11-24T06:28:18.696609+0000","last_fullsized":"2023-11-24T06:28:18.696609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695808+0000","last_change":"2023-11-24T06:28:18.695808+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695808+0000","last_undegraded":"2023-11-24T06:28:18.695808+0000","last_fullsized":"2023-11-24T06:28:18.695808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.699979+0000","last_change":"2023-11-24T06:28:18.699979+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.699979+0000","last_undegraded":"2023-11-24T06:28:18.699979+0000","last_fullsized":"2023-11-24T06:28:18.699979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.712310+0000","last_change":"2023-11-24T06:28:18.712310+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.712310+0000","last_undegraded":"2023-11-24T06:28:18.712310+0000","last_fullsized":"2023-11-24T06:28:18.712310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.712100+0000","last_change":"2023-11-24T06:28:18.712100+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.712100+0000","last_undegraded":"2023-11-24T06:28:18.712100+0000","last_fullsized":"2023-11-24T06:28:18.712100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.712121+0000","last_change":"2023-11-24T06:28:18.712121+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.712121+0000","last_undegraded":"2023-11-24T06:28:18.712121+0000","last_fullsized":"2023-11-24T06:28:18.712121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695108+0000","last_change":"2023-11-24T06:28:18.695108+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695108+0000","last_undegraded":"2023-11-24T06:28:18.695108+0000","last_fullsized":"2023-11-24T06:28:18.695108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.705565+0000","last_change":"2023-11-24T06:28:18.705565+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.705565+0000","last_undegraded":"2023-11-24T06:28:18.705565+0000","last_fullsized":"2023-11-24T06:28:18.705565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.713277+0000","last_change":"2023-11-24T06:28:18.713277+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.713277+0000","last_undegraded":"2023-11-24T06:28:18.713277+0000","last_fullsized":"2023-11-24T06:28:18.713277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.699954+0000","last_change":"2023-11-24T06:28:18.699954+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.699954+0000","last_undegraded":"2023-11-24T06:28:18.699954+0000","last_fullsized":"2023-11-24T06:28:18.699954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.699494+0000","last_change":"2023-11-24T06:28:18.699494+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.699494+0000","last_undegraded":"2023-11-24T06:28:18.699494+0000","last_fullsized":"2023-11-24T06:28:18.699494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.694974+0000","last_change":"2023-11-24T06:28:18.694974+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.694974+0000","last_undegraded":"2023-11-24T06:28:18.694974+0000","last_fullsized":"2023-11-24T06:28:18.694974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.698502+0000","last_change":"2023-11-24T06:28:18.698502+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.698502+0000","last_undegraded":"2023-11-24T06:28:18.698502+0000","last_fullsized":"2023-11-24T06:28:18.698502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.694291+0000","last_change":"2023-11-24T06:28:18.694291+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.694291+0000","last_undegraded":"2023-11-24T06:28:18.694291+0000","last_fullsized":"2023-11-24T06:28:18.694291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.697229+0000","last_change":"2023-11-24T06:28:18.697229+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.697229+0000","last_undegraded":"2023-11-24T06:28:18.697229+0000","last_fullsized":"2023-11-24T06:28:18.697229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.711854+0000","last_change":"2023-11-24T06:28:18.711854+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.711854+0000","last_undegraded":"2023-11-24T06:28:18.711854+0000","last_fullsized":"2023-11-24T06:28:18.711854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695239+0000","last_change":"2023-11-24T06:28:18.695239+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695239+0000","last_undegraded":"2023-11-24T06:28:18.695239+0000","last_fullsized":"2023-11-24T06:28:18.695239+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.705158+0000","last_change":"2023-11-24T06:28:18.705158+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.705158+0000","last_undegraded":"2023-11-24T06:28:18.705158+0000","last_fullsized":"2023-11-24T06:28:18.705158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-24T06:28:13.012624+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:28:13.012624+0000","last_peered":"2023-11-24T06:28:13.012624+0000","last_clean":"2023-11-24T06:28:13.012624+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:28:13.012624+0000","last_undegraded":"2023-11-24T06:28:13.012624+0000","last_fullsized":"2023-11-24T06:28:13.012624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.216},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.76}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":6.3319999999999999,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.585}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.3380000000000001,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.056}]},{"osd":3,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.637},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.2909999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":1.798}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.417},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":6.7359999999999998,"5min":6.7359999999999998,"15min":6.7359999999999998},"last":1.845}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.905,"15min":1.905},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":7.6449999999999996,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.907},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":1.7230000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.351}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.999},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.88},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.77500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.167},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.24}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.113},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":1.478}]},{"osd":5,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.774,"5min":3.774,"15min":3.774},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":1.548},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":3.649,"5min":3.649,"15min":3.649},"last":1.72}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.998}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.0489999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.085},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.2010000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.554,"15min":1.554},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.085,"5min":3.085,"15min":3.085},"last":1.2809999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.431},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.238}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.389},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.333}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.92700000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.359}]},{"osd":2,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.252}]},{"osd":3,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.4970000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5950000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.581,"15min":1.581},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.52,"5min":3.52,"15min":3.52},"last":1.649}]},{"osd":6,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.403},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":1.5509999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.796},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":1.6060000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.0569999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.252}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.397},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.5910000000000002,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.2030000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.081,"5min":3.081,"15min":3.081},"last":1.619},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.677}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.6960000000000002,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.357},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":8.8879999999999999,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.526}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":2.1030000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":2.1829999999999998},{"interface":"front","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":1.5760000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":335872,"data_stored":126780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.91400000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.157},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.476},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":3.7410000000000001,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.4039999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":4.1210000000000004,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.895}]},{"osd":7,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.8119999999999998,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.381},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.996}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.84299999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.72499999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.012}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.49},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.1659999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.581},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":1.5940000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.234},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.52}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":722965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.878},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.499}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.999}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.0680000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.091},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1259999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.91300000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.151},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:25.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.779+0000 7f50381fb640 1 Processor -- start 2023-11-24T06:28:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.779+0000 7f50381fb640 1 -- start start 2023-11-24T06:28:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.781+0000 7f50381fb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f503013e360 0x7f50300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.781+0000 7f50381fb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503015f9c0 0x7f50300cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.781+0000 7f50381fb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503015c5f0 con 0x7f503013df80 2023-11-24T06:28:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.781+0000 7f50381fb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503006c470 con 0x7f503013e360 2023-11-24T06:28:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.781+0000 7f50381fb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503006c750 con 0x7f503015f9c0 2023-11-24T06:28:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.781+0000 7f5035f97640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f503013df80 0x7f503015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60740/0 (socket says 172.21.15.16:60740) 2023-11-24T06:28:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.781+0000 7f5035f97640 1 -- 172.21.15.16:0/2050622729 learned_addr learned my addr 172.21.15.16:0/2050622729 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.781+0000 7f5035796640 1 --2- 172.21.15.16:0/2050622729 >> v2:172.21.15.16:3300/0 conn(0x7f503013e360 0x7f50300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.782+0000 7f5036798640 1 --2- 172.21.15.16:0/2050622729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503015f9c0 0x7f50300cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.782+0000 7f5035796640 1 -- 172.21.15.16:0/2050622729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503015f9c0 msgr2=0x7f50300cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.782+0000 7f5034f95640 1 -- 172.21.15.16:0/2050622729 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101194582 0 0) 0x7f503015c5f0 con 0x7f503013df80 2023-11-24T06:28:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.782+0000 7f5035796640 1 --2- 172.21.15.16:0/2050622729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503015f9c0 0x7f50300cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.782+0000 7f5035796640 1 -- 172.21.15.16:0/2050622729 >> v1:172.21.15.16:6789/0 conn(0x7f503013df80 legacy=0x7f503015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.782+0000 7f5035796640 1 -- 172.21.15.16:0/2050622729 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503006ca30 con 0x7f503013e360 2023-11-24T06:28:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.783+0000 7f5035796640 1 --2- 172.21.15.16:0/2050622729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013e360 0x7f50300cf290 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f50200097a0 tx=0x7f502002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d99099c3e28f149 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.783+0000 7f5034f95640 1 -- 172.21.15.16:0/2050622729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020031830 con 0x7f503013e360 2023-11-24T06:28:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.783+0000 7f5034f95640 1 -- 172.21.15.16:0/2050622729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020031990 con 0x7f503013e360 2023-11-24T06:28:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.783+0000 7f5034f95640 1 -- 172.21.15.16:0/2050622729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020031d20 con 0x7f503013e360 2023-11-24T06:28:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.784+0000 7f50381fb640 1 -- 172.21.15.16:0/2050622729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013e360 msgr2=0x7f50300cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.784+0000 7f50381fb640 1 --2- 172.21.15.16:0/2050622729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013e360 0x7f50300cf290 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f50200097a0 tx=0x7f502002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.784+0000 7f50381fb640 1 -- 172.21.15.16:0/2050622729 shutdown_connections 2023-11-24T06:28:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.784+0000 7f50381fb640 1 --2- 172.21.15.16:0/2050622729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503015f9c0 0x7f50300cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.784+0000 7f50381fb640 1 --2- 172.21.15.16:0/2050622729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013e360 0x7f50300cf290 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.784+0000 7f50381fb640 1 -- 172.21.15.16:0/2050622729 >> 172.21.15.16:0/2050622729 conn(0x7f503006a800 msgr2=0x7f503006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.784+0000 7f50381fb640 1 -- 172.21.15.16:0/2050622729 shutdown_connections 2023-11-24T06:28:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.784+0000 7f50381fb640 1 -- 172.21.15.16:0/2050622729 wait complete. 2023-11-24T06:28:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.786+0000 7f50381fb640 1 Processor -- start 2023-11-24T06:28:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.786+0000 7f50381fb640 1 -- start start 2023-11-24T06:28:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.787+0000 7f50381fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013df80 0x7f503014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.787+0000 7f50381fb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503013e360 0x7f503013d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.787+0000 7f50381fb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f503015f9c0 0x7f5030070930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.787+0000 7f50381fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5030071d20 con 0x7f503013df80 2023-11-24T06:28:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.787+0000 7f50381fb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5030070e70 con 0x7f503015f9c0 2023-11-24T06:28:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.787+0000 7f50381fb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5030071110 con 0x7f503013e360 2023-11-24T06:28:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.787+0000 7f5035f97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013df80 0x7f503014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.787+0000 7f5035f97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013df80 0x7f503014bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50312/0 (socket says 172.21.15.16:50312) 2023-11-24T06:28:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.787+0000 7f5035f97640 1 -- 172.21.15.16:0/3962519698 learned_addr learned my addr 172.21.15.16:0/3962519698 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f5035796640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503013e360 0x7f503013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f5036798640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f503015f9c0 0x7f5030070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f5035f97640 1 -- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503013e360 msgr2=0x7f503013d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f5035f97640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503013e360 0x7f503013d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f5035f97640 1 -- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f503015f9c0 msgr2=0x7f5030070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f5035f97640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f503015f9c0 0x7f5030070930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f5035f97640 1 -- 172.21.15.16:0/3962519698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5020009400 con 0x7f503013df80 2023-11-24T06:28:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f5035f97640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013df80 0x7f503014bf00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c400 tx=0x7f502c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f50267fc640 1 -- 172.21.15.16:0/3962519698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c005ce0 con 0x7f503013df80 2023-11-24T06:28:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.788+0000 7f50267fc640 1 -- 172.21.15.16:0/3962519698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c005e40 con 0x7f503013df80 2023-11-24T06:28:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.789+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50300713f0 con 0x7f503013df80 2023-11-24T06:28:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.789+0000 7f50267fc640 1 -- 172.21.15.16:0/3962519698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c007310 con 0x7f503013df80 2023-11-24T06:28:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.789+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50300719b0 con 0x7f503013df80 2023-11-24T06:28:25.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.790+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50300675d0 con 0x7f503013df80 2023-11-24T06:28:25.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.791+0000 7f50267fc640 1 -- 172.21.15.16:0/3962519698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f502c007530 con 0x7f503013df80 2023-11-24T06:28:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.792+0000 7f50267fc640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f501406ae40 0x7f501406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.792+0000 7f50267fc640 1 -- 172.21.15.16:0/3962519698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f502c088c40 con 0x7f503013df80 2023-11-24T06:28:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.792+0000 7f5035796640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f501406ae40 0x7f501406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:25.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.794+0000 7f5035796640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f501406ae40 0x7f501406d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5020036000 tx=0x7f5020008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.796+0000 7f50267fc640 1 -- 172.21.15.16:0/3962519698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f502c053ce0 con 0x7f503013df80 2023-11-24T06:28:25.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:25.988+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f503006cff0 con 0x7f501406ae40 2023-11-24T06:28:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.012+0000 7f50267fc640 1 -- 172.21.15.16:0/3962519698 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274714 (secure 0 0 0) 0x7f5020031b00 con 0x7f501406ae40 2023-11-24T06:28:26.016 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:26.021 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:28:26.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f501406ae40 msgr2=0x7f501406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f501406ae40 0x7f501406d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5020036000 tx=0x7f5020008030 comp rx=0 tx=0).stop 2023-11-24T06:28:26.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013df80 msgr2=0x7f503014bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013df80 0x7f503014bf00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c400 tx=0x7f502c00c8c0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 shutdown_connections 2023-11-24T06:28:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f501406ae40 0x7f501406d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f503015f9c0 0x7f5030070930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f503013e360 0x7f503013d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 --2- 172.21.15.16:0/3962519698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f503013df80 0x7f503014bf00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.018+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 >> 172.21.15.16:0/3962519698 conn(0x7f503006a800 msgr2=0x7f5030153a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.019+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 shutdown_connections 2023-11-24T06:28:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.019+0000 7f50381fb640 1 -- 172.21.15.16:0/3962519698 wait complete. 2023-11-24T06:28:26.044 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-11-24T06:28:24.132156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236676,"kb_used_data":3868,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738044,"statfs":{"total":773094113280,"available":772851757056,"internally_reserved":0,"allocated":3960832,"data_stored":2235695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.440385"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.705175+0000","last_change":"2023-11-24T06:28:18.705175+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.705175+0000","last_undegraded":"2023-11-24T06:28:18.705175+0000","last_fullsized":"2023-11-24T06:28:18.705175+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695945+0000","last_change":"2023-11-24T06:28:18.695945+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695945+0000","last_undegraded":"2023-11-24T06:28:18.695945+0000","last_fullsized":"2023-11-24T06:28:18.695945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.697812+0000","last_change":"2023-11-24T06:28:18.697812+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.697812+0000","last_undegraded":"2023-11-24T06:28:18.697812+0000","last_fullsized":"2023-11-24T06:28:18.697812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.706454+0000","last_change":"2023-11-24T06:28:18.706454+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.706454+0000","last_undegraded":"2023-11-24T06:28:18.706454+0000","last_fullsized":"2023-11-24T06:28:18.706454+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.713422+0000","last_change":"2023-11-24T06:28:18.713422+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.713422+0000","last_undegraded":"2023-11-24T06:28:18.713422+0000","last_fullsized":"2023-11-24T06:28:18.713422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.711845+0000","last_change":"2023-11-24T06:28:18.711845+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.711845+0000","last_undegraded":"2023-11-24T06:28:18.711845+0000","last_fullsized":"2023-11-24T06:28:18.711845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695981+0000","last_change":"2023-11-24T06:28:18.695981+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695981+0000","last_undegraded":"2023-11-24T06:28:18.695981+0000","last_fullsized":"2023-11-24T06:28:18.695981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.711915+0000","last_change":"2023-11-24T06:28:18.711915+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.711915+0000","last_undegraded":"2023-11-24T06:28:18.711915+0000","last_fullsized":"2023-11-24T06:28:18.711915+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695471+0000","last_change":"2023-11-24T06:28:18.695471+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695471+0000","last_undegraded":"2023-11-24T06:28:18.695471+0000","last_fullsized":"2023-11-24T06:28:18.695471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.694233+0000","last_change":"2023-11-24T06:28:18.694233+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.694233+0000","last_undegraded":"2023-11-24T06:28:18.694233+0000","last_fullsized":"2023-11-24T06:28:18.694233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.700202+0000","last_change":"2023-11-24T06:28:18.700202+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.700202+0000","last_undegraded":"2023-11-24T06:28:18.700202+0000","last_fullsized":"2023-11-24T06:28:18.700202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.697763+0000","last_change":"2023-11-24T06:28:18.697763+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.697763+0000","last_undegraded":"2023-11-24T06:28:18.697763+0000","last_fullsized":"2023-11-24T06:28:18.697763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.712103+0000","last_change":"2023-11-24T06:28:18.712103+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.712103+0000","last_undegraded":"2023-11-24T06:28:18.712103+0000","last_fullsized":"2023-11-24T06:28:18.712103+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.699816+0000","last_change":"2023-11-24T06:28:18.699816+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.699816+0000","last_undegraded":"2023-11-24T06:28:18.699816+0000","last_fullsized":"2023-11-24T06:28:18.699816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.696609+0000","last_change":"2023-11-24T06:28:18.696609+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.696609+0000","last_undegraded":"2023-11-24T06:28:18.696609+0000","last_fullsized":"2023-11-24T06:28:18.696609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695808+0000","last_change":"2023-11-24T06:28:18.695808+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695808+0000","last_undegraded":"2023-11-24T06:28:18.695808+0000","last_fullsized":"2023-11-24T06:28:18.695808+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.699979+0000","last_change":"2023-11-24T06:28:18.699979+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.699979+0000","last_undegraded":"2023-11-24T06:28:18.699979+0000","last_fullsized":"2023-11-24T06:28:18.699979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.712310+0000","last_change":"2023-11-24T06:28:18.712310+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.712310+0000","last_undegraded":"2023-11-24T06:28:18.712310+0000","last_fullsized":"2023-11-24T06:28:18.712310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.712100+0000","last_change":"2023-11-24T06:28:18.712100+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.712100+0000","last_undegraded":"2023-11-24T06:28:18.712100+0000","last_fullsized":"2023-11-24T06:28:18.712100+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.712121+0000","last_change":"2023-11-24T06:28:18.712121+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.712121+0000","last_undegraded":"2023-11-24T06:28:18.712121+0000","last_fullsized":"2023-11-24T06:28:18.712121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695108+0000","last_change":"2023-11-24T06:28:18.695108+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695108+0000","last_undegraded":"2023-11-24T06:28:18.695108+0000","last_fullsized":"2023-11-24T06:28:18.695108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.705565+0000","last_change":"2023-11-24T06:28:18.705565+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.705565+0000","last_undegraded":"2023-11-24T06:28:18.705565+0000","last_fullsized":"2023-11-24T06:28:18.705565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.713277+0000","last_change":"2023-11-24T06:28:18.713277+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.713277+0000","last_undegraded":"2023-11-24T06:28:18.713277+0000","last_fullsized":"2023-11-24T06:28:18.713277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.699954+0000","last_change":"2023-11-24T06:28:18.699954+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.699954+0000","last_undegraded":"2023-11-24T06:28:18.699954+0000","last_fullsized":"2023-11-24T06:28:18.699954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.699494+0000","last_change":"2023-11-24T06:28:18.699494+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.699494+0000","last_undegraded":"2023-11-24T06:28:18.699494+0000","last_fullsized":"2023-11-24T06:28:18.699494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.694974+0000","last_change":"2023-11-24T06:28:18.694974+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.694974+0000","last_undegraded":"2023-11-24T06:28:18.694974+0000","last_fullsized":"2023-11-24T06:28:18.694974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.698502+0000","last_change":"2023-11-24T06:28:18.698502+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.698502+0000","last_undegraded":"2023-11-24T06:28:18.698502+0000","last_fullsized":"2023-11-24T06:28:18.698502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.694291+0000","last_change":"2023-11-24T06:28:18.694291+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.694291+0000","last_undegraded":"2023-11-24T06:28:18.694291+0000","last_fullsized":"2023-11-24T06:28:18.694291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.697229+0000","last_change":"2023-11-24T06:28:18.697229+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.697229+0000","last_undegraded":"2023-11-24T06:28:18.697229+0000","last_fullsized":"2023-11-24T06:28:18.697229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.711854+0000","last_change":"2023-11-24T06:28:18.711854+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.711854+0000","last_undegraded":"2023-11-24T06:28:18.711854+0000","last_fullsized":"2023-11-24T06:28:18.711854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.695239+0000","last_change":"2023-11-24T06:28:18.695239+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.695239+0000","last_undegraded":"2023-11-24T06:28:18.695239+0000","last_fullsized":"2023-11-24T06:28:18.695239+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:28:19.688317+0000","last_change":"2023-11-24T06:28:19.688317+0000","last_active":"2023-11-24T06:28:19.688317+0000","last_peered":"2023-11-24T06:28:19.688317+0000","last_clean":"2023-11-24T06:28:19.688317+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:19.688317+0000","last_undegraded":"2023-11-24T06:28:19.688317+0000","last_fullsized":"2023-11-24T06:28:19.688317+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-24T06:28:18.705158+0000","last_change":"2023-11-24T06:28:18.705158+0000","last_active":"2023-11-24T06:28:18.674747+0000","last_peered":"2023-11-24T06:28:18.674747+0000","last_clean":"2023-11-24T06:28:18.674747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:28:18.705158+0000","last_undegraded":"2023-11-24T06:28:18.705158+0000","last_fullsized":"2023-11-24T06:28:18.705158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":176,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-24T06:28:13.012624+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:28:13.012624+0000","last_peered":"2023-11-24T06:28:13.012624+0000","last_clean":"2023-11-24T06:28:13.012624+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:28:13.012624+0000","last_undegraded":"2023-11-24T06:28:13.012624+0000","last_fullsized":"2023-11-24T06:28:13.012624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.216},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.76}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":6.3319999999999999,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.585}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.3380000000000001,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.056}]},{"osd":3,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.637},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.2909999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":1.798}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.417},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":6.7359999999999998,"5min":6.7359999999999998,"15min":6.7359999999999998},"last":1.845}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.905,"15min":1.905},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":7.6449999999999996,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.907},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":1.7230000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.351}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.999},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.88},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.77500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.167},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.24}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.113},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":1.478}]},{"osd":5,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.774,"5min":3.774,"15min":3.774},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":1.548},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":3.649,"5min":3.649,"15min":3.649},"last":1.72}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.998}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.0489999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.085},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.2010000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.554,"15min":1.554},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.085,"5min":3.085,"15min":3.085},"last":1.2809999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.431},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.238}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.389},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.333}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.92700000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.359}]},{"osd":2,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.252}]},{"osd":3,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.4970000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5950000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.581,"15min":1.581},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.52,"5min":3.52,"15min":3.52},"last":1.649}]},{"osd":6,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.403},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":1.5509999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.796},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":1.6060000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.0569999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.252}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.397},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.5910000000000002,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.2030000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.081,"5min":3.081,"15min":3.081},"last":1.619},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.677}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.6960000000000002,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.357},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":8.8879999999999999,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.526}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":2.1030000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":2.1829999999999998},{"interface":"front","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":1.5760000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":335872,"data_stored":126780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.91400000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.157},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.476},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":3.7410000000000001,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.4039999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":4.1210000000000004,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.895}]},{"osd":7,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.8119999999999998,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.381},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.996}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":132597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.84299999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.72499999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.012}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.49},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.1659999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.581},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":1.5940000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.234},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.52}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":722965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.878},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.499}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.999}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.0680000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.091},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1259999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.91300000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.151},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:26.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.313+0000 7f70f759e640 1 Processor -- start 2023-11-24T06:28:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.314+0000 7f70f759e640 1 -- start start 2023-11-24T06:28:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.315+0000 7f70f759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8071f00 0x7f70f8146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.316+0000 7f70f759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f80b2e10 con 0x7f70f8071410 2023-11-24T06:28:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.316+0000 7f70f759e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f806cec0 con 0x7f70f806c7c0 2023-11-24T06:28:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.316+0000 7f70f759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f806d1a0 con 0x7f70f8071f00 2023-11-24T06:28:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.316+0000 7f70f659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f70f8071410 0x7f70f80717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60746/0 (socket says 172.21.15.16:60746) 2023-11-24T06:28:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.316+0000 7f70f659c640 1 -- 172.21.15.16:0/4291016532 learned_addr learned my addr 172.21.15.16:0/4291016532 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.316+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783624943 0 0) 0x7f70f80b2e10 con 0x7f70f8071410 2023-11-24T06:28:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.316+0000 7f70effff640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8071f00 0x7f70f8146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.316+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e4003520 con 0x7f70f8071410 2023-11-24T06:28:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 114551786 0 0) 0x7f70e4003520 con 0x7f70f8071410 2023-11-24T06:28:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8071f00 msgr2=0x7f70f8146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8071f00 0x7f70f8146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 >> v1:172.21.15.173:6789/0 conn(0x7f70f806c7c0 legacy=0x7f70f8166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f806d480 con 0x7f70f8071410 2023-11-24T06:28:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f70e8001670 con 0x7f70f8071410 2023-11-24T06:28:26.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 >> v1:172.21.15.16:6789/0 conn(0x7f70f8071410 legacy=0x7f70f80717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70e40039a0 0x7f70e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8071f00 0x7f70e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f806d1a0 con 0x7f70e4004d00 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e400ba60 con 0x7f70e40039a0 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e400bc20 con 0x7f70f8071f00 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.317+0000 7f70f6d9d640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.318+0000 7f70f659c640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70e40039a0 0x7f70e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.318+0000 7f70effff640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8071f00 0x7f70e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.318+0000 7f70f6d9d640 1 -- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8071f00 msgr2=0x7f70e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:26.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.318+0000 7f70f6d9d640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8071f00 0x7f70e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.318+0000 7f70f6d9d640 1 -- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70e40039a0 msgr2=0x7f70e4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.318+0000 7f70f6d9d640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70e40039a0 0x7f70e4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.318+0000 7f70f6d9d640 1 -- 172.21.15.16:0/4291016532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4003520 con 0x7f70e4004d00 2023-11-24T06:28:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.318+0000 7f70effff640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8071f00 0x7f70e40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:26.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.318+0000 7f70f659c640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70e40039a0 0x7f70e4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.319+0000 7f70f6d9d640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70e4005180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f70f0001e20 tx=0x7f70e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ef3833ace8ef7c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.319+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0003b70 con 0x7f70e4004d00 2023-11-24T06:28:26.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.319+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0003cd0 con 0x7f70e4004d00 2023-11-24T06:28:26.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.319+0000 7f70f5d9b640 1 -- 172.21.15.16:0/4291016532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0005310 con 0x7f70e4004d00 2023-11-24T06:28:26.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.320+0000 7f70f759e640 1 -- 172.21.15.16:0/4291016532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 msgr2=0x7f70e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.320+0000 7f70f759e640 1 --2- 172.21.15.16:0/4291016532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70e4005180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f70f0001e20 tx=0x7f70e40057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.320+0000 7f70f6d9d640 1 -- 172.21.15.16:0/4291016532 reap_dead start 2023-11-24T06:28:26.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.320+0000 7f70f759e640 1 -- 172.21.15.16:0/4291016532 shutdown_connections 2023-11-24T06:28:26.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.320+0000 7f70f759e640 1 -- 172.21.15.16:0/4291016532 >> 172.21.15.16:0/4291016532 conn(0x7f70f806a800 msgr2=0x7f70f806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:26.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.320+0000 7f70f759e640 1 -- 172.21.15.16:0/4291016532 shutdown_connections 2023-11-24T06:28:26.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.320+0000 7f70f759e640 1 -- 172.21.15.16:0/4291016532 wait complete. 2023-11-24T06:28:26.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.322+0000 7f70f759e640 1 Processor -- start 2023-11-24T06:28:26.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.322+0000 7f70f759e640 1 -- start start 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.323+0000 7f70f759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70f813be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.323+0000 7f70f759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70f815f330 0x7f70f813b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.323+0000 7f70f759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8141e80 0x7f70f8142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.323+0000 7f70f759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f8142840 con 0x7f70e4004d00 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.323+0000 7f70f759e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f813cf80 con 0x7f70f815f330 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.323+0000 7f70f759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f813c6e0 con 0x7f70f8141e80 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.323+0000 7f70f659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70f813be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.323+0000 7f70f659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70f813be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50320/0 (socket says 172.21.15.16:50320) 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.323+0000 7f70f659c640 1 -- 172.21.15.16:0/394391423 learned_addr learned my addr 172.21.15.16:0/394391423 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70f659c640 1 -- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8141e80 msgr2=0x7f70f8142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70f659c640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8141e80 0x7f70f8142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70effff640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70f815f330 0x7f70f813b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70f659c640 1 -- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70f815f330 msgr2=0x7f70f813b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70f659c640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70f815f330 0x7f70f813b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70f659c640 1 -- 172.21.15.16:0/394391423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f813c9c0 con 0x7f70e4004d00 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70f659c640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70f813be40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e750 tx=0x7f70e800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70eeffd640 1 -- 172.21.15.16:0/394391423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e800ee60 con 0x7f70e4004d00 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70eeffd640 1 -- 172.21.15.16:0/394391423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e800c870 con 0x7f70e4004d00 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70eeffd640 1 -- 172.21.15.16:0/394391423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e800cbe0 con 0x7f70e4004d00 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.324+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f813cca0 con 0x7f70e4004d00 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.325+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f813c380 con 0x7f70e4004d00 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.326+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f80cbf40 con 0x7f70e4004d00 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.327+0000 7f70eeffd640 1 -- 172.21.15.16:0/394391423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70e800cd40 con 0x7f70e4004d00 2023-11-24T06:28:26.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.328+0000 7f70eeffd640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f70d806ae40 0x7f70d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.328+0000 7f70eeffd640 1 -- 172.21.15.16:0/394391423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70e8089570 con 0x7f70e4004d00 2023-11-24T06:28:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.328+0000 7f70effff640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f70d806ae40 0x7f70d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.329+0000 7f70effff640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f70d806ae40 0x7f70d806d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f70f80b1880 tx=0x7f70e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.332+0000 7f70eeffd640 1 -- 172.21.15.16:0/394391423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70f80cbf40 con 0x7f70e4004d00 2023-11-24T06:28:26.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.533+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70f806cbe0 con 0x7f70d806ae40 2023-11-24T06:28:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.556+0000 7f70eeffd640 1 -- 172.21.15.16:0/394391423 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286889 (secure 0 0 0) 0x7f70e0012950 con 0x7f70d806ae40 2023-11-24T06:28:26.564 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:26.566 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f70d806ae40 msgr2=0x7f70d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f70d806ae40 0x7f70d806d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f70f80b1880 tx=0x7f70e0008040 comp rx=0 tx=0).stop 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 msgr2=0x7f70f813be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70f813be40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e750 tx=0x7f70e800ec10 comp rx=0 tx=0).stop 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 shutdown_connections 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f70d806ae40 0x7f70d806d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70f8141e80 0x7f70f8142300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70f815f330 0x7f70f813b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 --2- 172.21.15.16:0/394391423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70e4004d00 0x7f70f813be40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 >> 172.21.15.16:0/394391423 conn(0x7f70f806a800 msgr2=0x7f70f8145230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 shutdown_connections 2023-11-24T06:28:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.562+0000 7f70f759e640 1 -- 172.21.15.16:0/394391423 wait complete. 2023-11-24T06:28:26.587 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2023-11-24T06:28:26.132716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237064,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737656,"statfs":{"total":773094113280,"available":772851359744,"internally_reserved":0,"allocated":4333568,"data_stored":2401068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.421970"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362259+0000","last_change":"2023-11-24T06:28:20.744890+0000","last_active":"2023-11-24T06:28:22.362259+0000","last_peered":"2023-11-24T06:28:22.362259+0000","last_clean":"2023-11-24T06:28:22.362259+0000","last_became_active":"2023-11-24T06:28:20.744570+0000","last_became_peered":"2023-11-24T06:28:20.744570+0000","last_unstale":"2023-11-24T06:28:22.362259+0000","last_undegraded":"2023-11-24T06:28:22.362259+0000","last_fullsized":"2023-11-24T06:28:22.362259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:34.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364147+0000","last_change":"2023-11-24T06:28:20.740082+0000","last_active":"2023-11-24T06:28:22.364147+0000","last_peered":"2023-11-24T06:28:22.364147+0000","last_clean":"2023-11-24T06:28:22.364147+0000","last_became_active":"2023-11-24T06:28:20.739812+0000","last_became_peered":"2023-11-24T06:28:20.739812+0000","last_unstale":"2023-11-24T06:28:22.364147+0000","last_undegraded":"2023-11-24T06:28:22.364147+0000","last_fullsized":"2023-11-24T06:28:22.364147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:36.198976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.750217+0000","last_change":"2023-11-24T06:28:20.745391+0000","last_active":"2023-11-24T06:28:21.750217+0000","last_peered":"2023-11-24T06:28:21.750217+0000","last_clean":"2023-11-24T06:28:21.750217+0000","last_became_active":"2023-11-24T06:28:20.744851+0000","last_became_peered":"2023-11-24T06:28:20.744851+0000","last_unstale":"2023-11-24T06:28:21.750217+0000","last_undegraded":"2023-11-24T06:28:21.750217+0000","last_fullsized":"2023-11-24T06:28:21.750217+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:14.252801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741692+0000","last_change":"2023-11-24T06:28:20.758056+0000","last_active":"2023-11-24T06:28:21.741692+0000","last_peered":"2023-11-24T06:28:21.741692+0000","last_clean":"2023-11-24T06:28:21.741692+0000","last_became_active":"2023-11-24T06:28:20.757439+0000","last_became_peered":"2023-11-24T06:28:20.757439+0000","last_unstale":"2023-11-24T06:28:21.741692+0000","last_undegraded":"2023-11-24T06:28:21.741692+0000","last_fullsized":"2023-11-24T06:28:21.741692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:44:14.949226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.360988+0000","last_change":"2023-11-24T06:28:20.738890+0000","last_active":"2023-11-24T06:28:22.360988+0000","last_peered":"2023-11-24T06:28:22.360988+0000","last_clean":"2023-11-24T06:28:22.360988+0000","last_became_active":"2023-11-24T06:28:20.738441+0000","last_became_peered":"2023-11-24T06:28:20.738441+0000","last_unstale":"2023-11-24T06:28:22.360988+0000","last_undegraded":"2023-11-24T06:28:22.360988+0000","last_fullsized":"2023-11-24T06:28:22.360988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:26.380103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363652+0000","last_change":"2023-11-24T06:28:20.744786+0000","last_active":"2023-11-24T06:28:22.363652+0000","last_peered":"2023-11-24T06:28:22.363652+0000","last_clean":"2023-11-24T06:28:22.363652+0000","last_became_active":"2023-11-24T06:28:20.744487+0000","last_became_peered":"2023-11-24T06:28:20.744487+0000","last_unstale":"2023-11-24T06:28:22.363652+0000","last_undegraded":"2023-11-24T06:28:22.363652+0000","last_fullsized":"2023-11-24T06:28:22.363652+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:14.252801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748125+0000","last_change":"2023-11-24T06:28:20.753428+0000","last_active":"2023-11-24T06:28:21.748125+0000","last_peered":"2023-11-24T06:28:21.748125+0000","last_clean":"2023-11-24T06:28:21.748125+0000","last_became_active":"2023-11-24T06:28:20.750476+0000","last_became_peered":"2023-11-24T06:28:20.750476+0000","last_unstale":"2023-11-24T06:28:21.748125+0000","last_undegraded":"2023-11-24T06:28:21.748125+0000","last_fullsized":"2023-11-24T06:28:21.748125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:18.305749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747710+0000","last_change":"2023-11-24T06:28:20.750080+0000","last_active":"2023-11-24T06:28:21.747710+0000","last_peered":"2023-11-24T06:28:21.747710+0000","last_clean":"2023-11-24T06:28:21.747710+0000","last_became_active":"2023-11-24T06:28:20.749694+0000","last_became_peered":"2023-11-24T06:28:20.749694+0000","last_unstale":"2023-11-24T06:28:21.747710+0000","last_undegraded":"2023-11-24T06:28:21.747710+0000","last_fullsized":"2023-11-24T06:28:21.747710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:41.832504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363811+0000","last_change":"2023-11-24T06:28:20.760441+0000","last_active":"2023-11-24T06:28:22.363811+0000","last_peered":"2023-11-24T06:28:22.363811+0000","last_clean":"2023-11-24T06:28:22.363811+0000","last_became_active":"2023-11-24T06:28:20.758860+0000","last_became_peered":"2023-11-24T06:28:20.758860+0000","last_unstale":"2023-11-24T06:28:22.363811+0000","last_undegraded":"2023-11-24T06:28:22.363811+0000","last_fullsized":"2023-11-24T06:28:22.363811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:21.582740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742175+0000","last_change":"2023-11-24T06:28:20.744185+0000","last_active":"2023-11-24T06:28:21.742175+0000","last_peered":"2023-11-24T06:28:21.742175+0000","last_clean":"2023-11-24T06:28:21.742175+0000","last_became_active":"2023-11-24T06:28:20.743866+0000","last_became_peered":"2023-11-24T06:28:20.743866+0000","last_unstale":"2023-11-24T06:28:21.742175+0000","last_undegraded":"2023-11-24T06:28:21.742175+0000","last_fullsized":"2023-11-24T06:28:21.742175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:41.487786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748764+0000","last_change":"2023-11-24T06:28:20.748422+0000","last_active":"2023-11-24T06:28:21.748764+0000","last_peered":"2023-11-24T06:28:21.748764+0000","last_clean":"2023-11-24T06:28:21.748764+0000","last_became_active":"2023-11-24T06:28:20.747375+0000","last_became_peered":"2023-11-24T06:28:20.747375+0000","last_unstale":"2023-11-24T06:28:21.748764+0000","last_undegraded":"2023-11-24T06:28:21.748764+0000","last_fullsized":"2023-11-24T06:28:21.748764+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:21.555443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749684+0000","last_change":"2023-11-24T06:28:20.741727+0000","last_active":"2023-11-24T06:28:21.749684+0000","last_peered":"2023-11-24T06:28:21.749684+0000","last_clean":"2023-11-24T06:28:21.749684+0000","last_became_active":"2023-11-24T06:28:20.741428+0000","last_became_peered":"2023-11-24T06:28:20.741428+0000","last_unstale":"2023-11-24T06:28:21.749684+0000","last_undegraded":"2023-11-24T06:28:21.749684+0000","last_fullsized":"2023-11-24T06:28:21.749684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:18.305749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361169+0000","last_change":"2023-11-24T06:28:20.740157+0000","last_active":"2023-11-24T06:28:22.361169+0000","last_peered":"2023-11-24T06:28:22.361169+0000","last_clean":"2023-11-24T06:28:22.361169+0000","last_became_active":"2023-11-24T06:28:20.739896+0000","last_became_peered":"2023-11-24T06:28:20.739896+0000","last_unstale":"2023-11-24T06:28:22.361169+0000","last_undegraded":"2023-11-24T06:28:22.361169+0000","last_fullsized":"2023-11-24T06:28:22.361169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:41.168494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363756+0000","last_change":"2023-11-24T06:28:20.744520+0000","last_active":"2023-11-24T06:28:22.363756+0000","last_peered":"2023-11-24T06:28:22.363756+0000","last_clean":"2023-11-24T06:28:22.363756+0000","last_became_active":"2023-11-24T06:28:20.744131+0000","last_became_peered":"2023-11-24T06:28:20.744131+0000","last_unstale":"2023-11-24T06:28:22.363756+0000","last_undegraded":"2023-11-24T06:28:22.363756+0000","last_fullsized":"2023-11-24T06:28:22.363756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:52.718291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749170+0000","last_change":"2023-11-24T06:28:20.749484+0000","last_active":"2023-11-24T06:28:21.749170+0000","last_peered":"2023-11-24T06:28:21.749170+0000","last_clean":"2023-11-24T06:28:21.749170+0000","last_became_active":"2023-11-24T06:28:20.749186+0000","last_became_peered":"2023-11-24T06:28:20.749186+0000","last_unstale":"2023-11-24T06:28:21.749170+0000","last_undegraded":"2023-11-24T06:28:21.749170+0000","last_fullsized":"2023-11-24T06:28:21.749170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:48.588430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748803+0000","last_change":"2023-11-24T06:28:20.755151+0000","last_active":"2023-11-24T06:28:21.748803+0000","last_peered":"2023-11-24T06:28:21.748803+0000","last_clean":"2023-11-24T06:28:21.748803+0000","last_became_active":"2023-11-24T06:28:20.754783+0000","last_became_peered":"2023-11-24T06:28:20.754783+0000","last_unstale":"2023-11-24T06:28:21.748803+0000","last_undegraded":"2023-11-24T06:28:21.748803+0000","last_fullsized":"2023-11-24T06:28:21.748803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:52.033759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748672+0000","last_change":"2023-11-24T06:28:20.751616+0000","last_active":"2023-11-24T06:28:21.748672+0000","last_peered":"2023-11-24T06:28:21.748672+0000","last_clean":"2023-11-24T06:28:21.748672+0000","last_became_active":"2023-11-24T06:28:20.751347+0000","last_became_peered":"2023-11-24T06:28:20.751347+0000","last_unstale":"2023-11-24T06:28:21.748672+0000","last_undegraded":"2023-11-24T06:28:21.748672+0000","last_fullsized":"2023-11-24T06:28:21.748672+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:52.718291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364306+0000","last_change":"2023-11-24T06:28:20.752060+0000","last_active":"2023-11-24T06:28:22.364306+0000","last_peered":"2023-11-24T06:28:22.364306+0000","last_clean":"2023-11-24T06:28:22.364306+0000","last_became_active":"2023-11-24T06:28:20.751597+0000","last_became_peered":"2023-11-24T06:28:20.751597+0000","last_unstale":"2023-11-24T06:28:22.364306+0000","last_undegraded":"2023-11-24T06:28:22.364306+0000","last_fullsized":"2023-11-24T06:28:22.364306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:51.206064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363421+0000","last_change":"2023-11-24T06:28:20.751828+0000","last_active":"2023-11-24T06:28:22.363421+0000","last_peered":"2023-11-24T06:28:22.363421+0000","last_clean":"2023-11-24T06:28:22.363421+0000","last_became_active":"2023-11-24T06:28:20.751496+0000","last_became_peered":"2023-11-24T06:28:20.751496+0000","last_unstale":"2023-11-24T06:28:22.363421+0000","last_undegraded":"2023-11-24T06:28:22.363421+0000","last_fullsized":"2023-11-24T06:28:22.363421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:00.075286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361993+0000","last_change":"2023-11-24T06:28:20.742826+0000","last_active":"2023-11-24T06:28:22.361993+0000","last_peered":"2023-11-24T06:28:22.361993+0000","last_clean":"2023-11-24T06:28:22.361993+0000","last_became_active":"2023-11-24T06:28:20.742479+0000","last_became_peered":"2023-11-24T06:28:20.742479+0000","last_unstale":"2023-11-24T06:28:22.361993+0000","last_undegraded":"2023-11-24T06:28:22.361993+0000","last_fullsized":"2023-11-24T06:28:22.361993+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:06.929632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363090+0000","last_change":"2023-11-24T06:28:20.752827+0000","last_active":"2023-11-24T06:28:22.363090+0000","last_peered":"2023-11-24T06:28:22.363090+0000","last_clean":"2023-11-24T06:28:22.363090+0000","last_became_active":"2023-11-24T06:28:20.752553+0000","last_became_peered":"2023-11-24T06:28:20.752553+0000","last_unstale":"2023-11-24T06:28:22.363090+0000","last_undegraded":"2023-11-24T06:28:22.363090+0000","last_fullsized":"2023-11-24T06:28:22.363090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:34.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362788+0000","last_change":"2023-11-24T06:28:20.747287+0000","last_active":"2023-11-24T06:28:22.362788+0000","last_peered":"2023-11-24T06:28:22.362788+0000","last_clean":"2023-11-24T06:28:22.362788+0000","last_became_active":"2023-11-24T06:28:20.746913+0000","last_became_peered":"2023-11-24T06:28:20.746913+0000","last_unstale":"2023-11-24T06:28:22.362788+0000","last_undegraded":"2023-11-24T06:28:22.362788+0000","last_fullsized":"2023-11-24T06:28:22.362788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:08.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361398+0000","last_change":"2023-11-24T06:28:20.755636+0000","last_active":"2023-11-24T06:28:22.361398+0000","last_peered":"2023-11-24T06:28:22.361398+0000","last_clean":"2023-11-24T06:28:22.361398+0000","last_became_active":"2023-11-24T06:28:20.755151+0000","last_became_peered":"2023-11-24T06:28:20.755151+0000","last_unstale":"2023-11-24T06:28:22.361398+0000","last_undegraded":"2023-11-24T06:28:22.361398+0000","last_fullsized":"2023-11-24T06:28:22.361398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:34.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747983+0000","last_change":"2023-11-24T06:28:20.738640+0000","last_active":"2023-11-24T06:28:21.747983+0000","last_peered":"2023-11-24T06:28:21.747983+0000","last_clean":"2023-11-24T06:28:21.747983+0000","last_became_active":"2023-11-24T06:28:20.737115+0000","last_became_peered":"2023-11-24T06:28:20.737115+0000","last_unstale":"2023-11-24T06:28:21.747983+0000","last_undegraded":"2023-11-24T06:28:21.747983+0000","last_fullsized":"2023-11-24T06:28:21.747983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:41.832504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742269+0000","last_change":"2023-11-24T06:28:20.754501+0000","last_active":"2023-11-24T06:28:21.742269+0000","last_peered":"2023-11-24T06:28:21.742269+0000","last_clean":"2023-11-24T06:28:21.742269+0000","last_became_active":"2023-11-24T06:28:20.749001+0000","last_became_peered":"2023-11-24T06:28:20.749001+0000","last_unstale":"2023-11-24T06:28:21.742269+0000","last_undegraded":"2023-11-24T06:28:21.742269+0000","last_fullsized":"2023-11-24T06:28:21.742269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:35.498445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364527+0000","last_change":"2023-11-24T06:28:20.749900+0000","last_active":"2023-11-24T06:28:22.364527+0000","last_peered":"2023-11-24T06:28:22.364527+0000","last_clean":"2023-11-24T06:28:22.364527+0000","last_became_active":"2023-11-24T06:28:20.749612+0000","last_became_peered":"2023-11-24T06:28:20.749612+0000","last_unstale":"2023-11-24T06:28:22.364527+0000","last_undegraded":"2023-11-24T06:28:22.364527+0000","last_fullsized":"2023-11-24T06:28:22.364527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:02.550632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742049+0000","last_change":"2023-11-24T06:28:20.757641+0000","last_active":"2023-11-24T06:28:21.742049+0000","last_peered":"2023-11-24T06:28:21.742049+0000","last_clean":"2023-11-24T06:28:21.742049+0000","last_became_active":"2023-11-24T06:28:20.757307+0000","last_became_peered":"2023-11-24T06:28:20.757307+0000","last_unstale":"2023-11-24T06:28:21.742049+0000","last_undegraded":"2023-11-24T06:28:21.742049+0000","last_fullsized":"2023-11-24T06:28:21.742049+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:36.858522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749834+0000","last_change":"2023-11-24T06:28:20.738500+0000","last_active":"2023-11-24T06:28:21.749834+0000","last_peered":"2023-11-24T06:28:21.749834+0000","last_clean":"2023-11-24T06:28:21.749834+0000","last_became_active":"2023-11-24T06:28:20.738051+0000","last_became_peered":"2023-11-24T06:28:20.738051+0000","last_unstale":"2023-11-24T06:28:21.749834+0000","last_undegraded":"2023-11-24T06:28:21.749834+0000","last_fullsized":"2023-11-24T06:28:21.749834+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:46.026504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363095+0000","last_change":"2023-11-24T06:28:20.743275+0000","last_active":"2023-11-24T06:28:22.363095+0000","last_peered":"2023-11-24T06:28:22.363095+0000","last_clean":"2023-11-24T06:28:22.363095+0000","last_became_active":"2023-11-24T06:28:20.742978+0000","last_became_peered":"2023-11-24T06:28:20.742978+0000","last_unstale":"2023-11-24T06:28:22.363095+0000","last_undegraded":"2023-11-24T06:28:22.363095+0000","last_fullsized":"2023-11-24T06:28:22.363095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:12.693448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747735+0000","last_change":"2023-11-24T06:28:20.751681+0000","last_active":"2023-11-24T06:28:21.747735+0000","last_peered":"2023-11-24T06:28:21.747735+0000","last_clean":"2023-11-24T06:28:21.747735+0000","last_became_active":"2023-11-24T06:28:20.751402+0000","last_became_peered":"2023-11-24T06:28:20.751402+0000","last_unstale":"2023-11-24T06:28:21.747735+0000","last_undegraded":"2023-11-24T06:28:21.747735+0000","last_fullsized":"2023-11-24T06:28:21.747735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:36.384991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361179+0000","last_change":"2023-11-24T06:28:20.754007+0000","last_active":"2023-11-24T06:28:22.361179+0000","last_peered":"2023-11-24T06:28:22.361179+0000","last_clean":"2023-11-24T06:28:22.361179+0000","last_became_active":"2023-11-24T06:28:20.752557+0000","last_became_peered":"2023-11-24T06:28:20.752557+0000","last_unstale":"2023-11-24T06:28:22.361179+0000","last_undegraded":"2023-11-24T06:28:22.361179+0000","last_fullsized":"2023-11-24T06:28:22.361179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:51.904162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748122+0000","last_change":"2023-11-24T06:28:20.757581+0000","last_active":"2023-11-24T06:28:21.748122+0000","last_peered":"2023-11-24T06:28:21.748122+0000","last_clean":"2023-11-24T06:28:21.748122+0000","last_became_active":"2023-11-24T06:28:20.757233+0000","last_became_peered":"2023-11-24T06:28:20.757233+0000","last_unstale":"2023-11-24T06:28:21.748122+0000","last_undegraded":"2023-11-24T06:28:21.748122+0000","last_fullsized":"2023-11-24T06:28:21.748122+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:23.123298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361983+0000","last_change":"2023-11-24T06:28:20.747847+0000","last_active":"2023-11-24T06:28:22.361983+0000","last_peered":"2023-11-24T06:28:22.361983+0000","last_clean":"2023-11-24T06:28:22.361983+0000","last_became_active":"2023-11-24T06:28:20.747551+0000","last_became_peered":"2023-11-24T06:28:20.747551+0000","last_unstale":"2023-11-24T06:28:22.361983+0000","last_undegraded":"2023-11-24T06:28:22.361983+0000","last_fullsized":"2023-11-24T06:28:22.361983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:39.039302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.770328+0000","last_change":"2023-11-24T06:28:19.748835+0000","last_active":"2023-11-24T06:28:20.770328+0000","last_peered":"2023-11-24T06:28:20.770328+0000","last_clean":"2023-11-24T06:28:20.770328+0000","last_became_active":"2023-11-24T06:28:19.748412+0000","last_became_peered":"2023-11-24T06:28:19.748412+0000","last_unstale":"2023-11-24T06:28:20.770328+0000","last_undegraded":"2023-11-24T06:28:20.770328+0000","last_fullsized":"2023-11-24T06:28:20.770328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:48.325368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362430+0000","last_change":"2023-11-24T06:28:20.748498+0000","last_active":"2023-11-24T06:28:22.362430+0000","last_peered":"2023-11-24T06:28:22.362430+0000","last_clean":"2023-11-24T06:28:22.362430+0000","last_became_active":"2023-11-24T06:28:20.748101+0000","last_became_peered":"2023-11-24T06:28:20.748101+0000","last_unstale":"2023-11-24T06:28:22.362430+0000","last_undegraded":"2023-11-24T06:28:22.362430+0000","last_fullsized":"2023-11-24T06:28:22.362430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:12.693448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.771276+0000","last_change":"2023-11-24T06:28:19.742903+0000","last_active":"2023-11-24T06:28:20.771276+0000","last_peered":"2023-11-24T06:28:20.771276+0000","last_clean":"2023-11-24T06:28:20.771276+0000","last_became_active":"2023-11-24T06:28:19.742598+0000","last_became_peered":"2023-11-24T06:28:19.742598+0000","last_unstale":"2023-11-24T06:28:20.771276+0000","last_undegraded":"2023-11-24T06:28:20.771276+0000","last_fullsized":"2023-11-24T06:28:20.771276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:29.191620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363955+0000","last_change":"2023-11-24T06:28:20.751153+0000","last_active":"2023-11-24T06:28:22.363955+0000","last_peered":"2023-11-24T06:28:22.363955+0000","last_clean":"2023-11-24T06:28:22.363955+0000","last_became_active":"2023-11-24T06:28:20.749651+0000","last_became_peered":"2023-11-24T06:28:20.749651+0000","last_unstale":"2023-11-24T06:28:22.363955+0000","last_undegraded":"2023-11-24T06:28:22.363955+0000","last_fullsized":"2023-11-24T06:28:22.363955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:23.869661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.773975+0000","last_change":"2023-11-24T06:28:19.733747+0000","last_active":"2023-11-24T06:28:20.773975+0000","last_peered":"2023-11-24T06:28:20.773975+0000","last_clean":"2023-11-24T06:28:20.773975+0000","last_became_active":"2023-11-24T06:28:19.733405+0000","last_became_peered":"2023-11-24T06:28:19.733405+0000","last_unstale":"2023-11-24T06:28:20.773975+0000","last_undegraded":"2023-11-24T06:28:20.773975+0000","last_fullsized":"2023-11-24T06:28:20.773975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:22.109728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742130+0000","last_change":"2023-11-24T06:28:20.757546+0000","last_active":"2023-11-24T06:28:21.742130+0000","last_peered":"2023-11-24T06:28:21.742130+0000","last_clean":"2023-11-24T06:28:21.742130+0000","last_became_active":"2023-11-24T06:28:20.757229+0000","last_became_peered":"2023-11-24T06:28:20.757229+0000","last_unstale":"2023-11-24T06:28:21.742130+0000","last_undegraded":"2023-11-24T06:28:21.742130+0000","last_fullsized":"2023-11-24T06:28:21.742130+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:16.647539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.772837+0000","last_change":"2023-11-24T06:28:19.733959+0000","last_active":"2023-11-24T06:28:20.772837+0000","last_peered":"2023-11-24T06:28:20.772837+0000","last_clean":"2023-11-24T06:28:20.772837+0000","last_became_active":"2023-11-24T06:28:19.733633+0000","last_became_peered":"2023-11-24T06:28:19.733633+0000","last_unstale":"2023-11-24T06:28:20.772837+0000","last_undegraded":"2023-11-24T06:28:20.772837+0000","last_fullsized":"2023-11-24T06:28:20.772837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:07:48.392066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748219+0000","last_change":"2023-11-24T06:28:20.759715+0000","last_active":"2023-11-24T06:28:21.748219+0000","last_peered":"2023-11-24T06:28:21.748219+0000","last_clean":"2023-11-24T06:28:21.748219+0000","last_became_active":"2023-11-24T06:28:20.757687+0000","last_became_peered":"2023-11-24T06:28:20.757687+0000","last_unstale":"2023-11-24T06:28:21.748219+0000","last_undegraded":"2023-11-24T06:28:21.748219+0000","last_fullsized":"2023-11-24T06:28:21.748219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:46.026504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.768459+0000","last_change":"2023-11-24T06:28:19.740944+0000","last_active":"2023-11-24T06:28:20.768459+0000","last_peered":"2023-11-24T06:28:20.768459+0000","last_clean":"2023-11-24T06:28:20.768459+0000","last_became_active":"2023-11-24T06:28:19.740201+0000","last_became_peered":"2023-11-24T06:28:19.740201+0000","last_unstale":"2023-11-24T06:28:20.768459+0000","last_undegraded":"2023-11-24T06:28:20.768459+0000","last_fullsized":"2023-11-24T06:28:20.768459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:07:48.392066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749401+0000","last_change":"2023-11-24T06:28:20.747089+0000","last_active":"2023-11-24T06:28:21.749401+0000","last_peered":"2023-11-24T06:28:21.749401+0000","last_clean":"2023-11-24T06:28:21.749401+0000","last_became_active":"2023-11-24T06:28:20.742220+0000","last_became_peered":"2023-11-24T06:28:20.742220+0000","last_unstale":"2023-11-24T06:28:21.749401+0000","last_undegraded":"2023-11-24T06:28:21.749401+0000","last_fullsized":"2023-11-24T06:28:21.749401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:36.384991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.767921+0000","last_change":"2023-11-24T06:28:19.737597+0000","last_active":"2023-11-24T06:28:20.767921+0000","last_peered":"2023-11-24T06:28:20.767921+0000","last_clean":"2023-11-24T06:28:20.767921+0000","last_became_active":"2023-11-24T06:28:19.737301+0000","last_became_peered":"2023-11-24T06:28:19.737301+0000","last_unstale":"2023-11-24T06:28:20.767921+0000","last_undegraded":"2023-11-24T06:28:20.767921+0000","last_fullsized":"2023-11-24T06:28:20.767921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:07.528212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361176+0000","last_change":"2023-11-24T06:28:20.752076+0000","last_active":"2023-11-24T06:28:22.361176+0000","last_peered":"2023-11-24T06:28:22.361176+0000","last_clean":"2023-11-24T06:28:22.361176+0000","last_became_active":"2023-11-24T06:28:20.751527+0000","last_became_peered":"2023-11-24T06:28:20.751527+0000","last_unstale":"2023-11-24T06:28:22.361176+0000","last_undegraded":"2023-11-24T06:28:22.361176+0000","last_fullsized":"2023-11-24T06:28:22.361176+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:51.904162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.728185+0000","last_change":"2023-11-24T06:28:19.732965+0000","last_active":"2023-11-24T06:28:20.728185+0000","last_peered":"2023-11-24T06:28:20.728185+0000","last_clean":"2023-11-24T06:28:20.728185+0000","last_became_active":"2023-11-24T06:28:19.731006+0000","last_became_peered":"2023-11-24T06:28:19.731006+0000","last_unstale":"2023-11-24T06:28:20.728185+0000","last_undegraded":"2023-11-24T06:28:20.728185+0000","last_fullsized":"2023-11-24T06:28:20.728185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:40.818934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741644+0000","last_change":"2023-11-24T06:28:20.755312+0000","last_active":"2023-11-24T06:28:21.741644+0000","last_peered":"2023-11-24T06:28:21.741644+0000","last_clean":"2023-11-24T06:28:21.741644+0000","last_became_active":"2023-11-24T06:28:20.753599+0000","last_became_peered":"2023-11-24T06:28:20.753599+0000","last_unstale":"2023-11-24T06:28:21.741644+0000","last_undegraded":"2023-11-24T06:28:21.741644+0000","last_fullsized":"2023-11-24T06:28:21.741644+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:41.967768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.734014+0000","last_change":"2023-11-24T06:28:19.740715+0000","last_active":"2023-11-24T06:28:20.734014+0000","last_peered":"2023-11-24T06:28:20.734014+0000","last_clean":"2023-11-24T06:28:20.734014+0000","last_became_active":"2023-11-24T06:28:19.738599+0000","last_became_peered":"2023-11-24T06:28:19.738599+0000","last_unstale":"2023-11-24T06:28:20.734014+0000","last_undegraded":"2023-11-24T06:28:20.734014+0000","last_fullsized":"2023-11-24T06:28:20.734014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:02.155451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749992+0000","last_change":"2023-11-24T06:28:20.741845+0000","last_active":"2023-11-24T06:28:21.749992+0000","last_peered":"2023-11-24T06:28:21.749992+0000","last_clean":"2023-11-24T06:28:21.749992+0000","last_became_active":"2023-11-24T06:28:20.741520+0000","last_became_peered":"2023-11-24T06:28:20.741520+0000","last_unstale":"2023-11-24T06:28:21.749992+0000","last_undegraded":"2023-11-24T06:28:21.749992+0000","last_fullsized":"2023-11-24T06:28:21.749992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:00.075286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.729679+0000","last_change":"2023-11-24T06:28:19.742211+0000","last_active":"2023-11-24T06:28:20.729679+0000","last_peered":"2023-11-24T06:28:20.729679+0000","last_clean":"2023-11-24T06:28:20.729679+0000","last_became_active":"2023-11-24T06:28:19.741525+0000","last_became_peered":"2023-11-24T06:28:19.741525+0000","last_unstale":"2023-11-24T06:28:20.729679+0000","last_undegraded":"2023-11-24T06:28:20.729679+0000","last_fullsized":"2023-11-24T06:28:20.729679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:17.292179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363780+0000","last_change":"2023-11-24T06:28:20.752528+0000","last_active":"2023-11-24T06:28:22.363780+0000","last_peered":"2023-11-24T06:28:22.363780+0000","last_clean":"2023-11-24T06:28:22.363780+0000","last_became_active":"2023-11-24T06:28:20.752084+0000","last_became_peered":"2023-11-24T06:28:20.752084+0000","last_unstale":"2023-11-24T06:28:22.363780+0000","last_undegraded":"2023-11-24T06:28:22.363780+0000","last_fullsized":"2023-11-24T06:28:22.363780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:21.555443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.723909+0000","last_change":"2023-11-24T06:28:19.735685+0000","last_active":"2023-11-24T06:28:20.723909+0000","last_peered":"2023-11-24T06:28:20.723909+0000","last_clean":"2023-11-24T06:28:20.723909+0000","last_became_active":"2023-11-24T06:28:19.735419+0000","last_became_peered":"2023-11-24T06:28:19.735419+0000","last_unstale":"2023-11-24T06:28:20.723909+0000","last_undegraded":"2023-11-24T06:28:20.723909+0000","last_fullsized":"2023-11-24T06:28:20.723909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:07.528212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364602+0000","last_change":"2023-11-24T06:28:20.748284+0000","last_active":"2023-11-24T06:28:22.364602+0000","last_peered":"2023-11-24T06:28:22.364602+0000","last_clean":"2023-11-24T06:28:22.364602+0000","last_became_active":"2023-11-24T06:28:20.748000+0000","last_became_peered":"2023-11-24T06:28:20.748000+0000","last_unstale":"2023-11-24T06:28:22.364602+0000","last_undegraded":"2023-11-24T06:28:22.364602+0000","last_fullsized":"2023-11-24T06:28:22.364602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:23.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.770416+0000","last_change":"2023-11-24T06:28:19.742398+0000","last_active":"2023-11-24T06:28:20.770416+0000","last_peered":"2023-11-24T06:28:20.770416+0000","last_clean":"2023-11-24T06:28:20.770416+0000","last_became_active":"2023-11-24T06:28:19.739648+0000","last_became_peered":"2023-11-24T06:28:19.739648+0000","last_unstale":"2023-11-24T06:28:20.770416+0000","last_undegraded":"2023-11-24T06:28:20.770416+0000","last_fullsized":"2023-11-24T06:28:20.770416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:40.474217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749587+0000","last_change":"2023-11-24T06:28:20.745174+0000","last_active":"2023-11-24T06:28:21.749587+0000","last_peered":"2023-11-24T06:28:21.749587+0000","last_clean":"2023-11-24T06:28:21.749587+0000","last_became_active":"2023-11-24T06:28:20.744317+0000","last_became_peered":"2023-11-24T06:28:20.744317+0000","last_unstale":"2023-11-24T06:28:21.749587+0000","last_undegraded":"2023-11-24T06:28:21.749587+0000","last_fullsized":"2023-11-24T06:28:21.749587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:23.123298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.776450+0000","last_change":"2023-11-24T06:28:19.732266+0000","last_active":"2023-11-24T06:28:20.776450+0000","last_peered":"2023-11-24T06:28:20.776450+0000","last_clean":"2023-11-24T06:28:20.776450+0000","last_became_active":"2023-11-24T06:28:19.731925+0000","last_became_peered":"2023-11-24T06:28:19.731925+0000","last_unstale":"2023-11-24T06:28:20.776450+0000","last_undegraded":"2023-11-24T06:28:20.776450+0000","last_fullsized":"2023-11-24T06:28:20.776450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:47.574860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741618+0000","last_change":"2023-11-24T06:28:20.749699+0000","last_active":"2023-11-24T06:28:21.741618+0000","last_peered":"2023-11-24T06:28:21.741618+0000","last_clean":"2023-11-24T06:28:21.741618+0000","last_became_active":"2023-11-24T06:28:20.749273+0000","last_became_peered":"2023-11-24T06:28:20.749273+0000","last_unstale":"2023-11-24T06:28:21.741618+0000","last_undegraded":"2023-11-24T06:28:21.741618+0000","last_fullsized":"2023-11-24T06:28:21.741618+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:23.869661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.769745+0000","last_change":"2023-11-24T06:28:19.740983+0000","last_active":"2023-11-24T06:28:20.769745+0000","last_peered":"2023-11-24T06:28:20.769745+0000","last_clean":"2023-11-24T06:28:20.769745+0000","last_became_active":"2023-11-24T06:28:19.740647+0000","last_became_peered":"2023-11-24T06:28:19.740647+0000","last_unstale":"2023-11-24T06:28:20.769745+0000","last_undegraded":"2023-11-24T06:28:20.769745+0000","last_fullsized":"2023-11-24T06:28:20.769745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:11:28.316281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363921+0000","last_change":"2023-11-24T06:28:20.758815+0000","last_active":"2023-11-24T06:28:22.363921+0000","last_peered":"2023-11-24T06:28:22.363921+0000","last_clean":"2023-11-24T06:28:22.363921+0000","last_became_active":"2023-11-24T06:28:20.757556+0000","last_became_peered":"2023-11-24T06:28:20.757556+0000","last_unstale":"2023-11-24T06:28:22.363921+0000","last_undegraded":"2023-11-24T06:28:22.363921+0000","last_fullsized":"2023-11-24T06:28:22.363921+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:52.033759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.739778+0000","last_change":"2023-11-24T06:28:19.741817+0000","last_active":"2023-11-24T06:28:20.739778+0000","last_peered":"2023-11-24T06:28:20.739778+0000","last_clean":"2023-11-24T06:28:20.739778+0000","last_became_active":"2023-11-24T06:28:19.741553+0000","last_became_peered":"2023-11-24T06:28:19.741553+0000","last_unstale":"2023-11-24T06:28:20.739778+0000","last_undegraded":"2023-11-24T06:28:20.739778+0000","last_fullsized":"2023-11-24T06:28:20.739778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:07.764462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748482+0000","last_change":"2023-11-24T06:28:20.757476+0000","last_active":"2023-11-24T06:28:21.748482+0000","last_peered":"2023-11-24T06:28:21.748482+0000","last_clean":"2023-11-24T06:28:21.748482+0000","last_became_active":"2023-11-24T06:28:20.757215+0000","last_became_peered":"2023-11-24T06:28:20.757215+0000","last_unstale":"2023-11-24T06:28:21.748482+0000","last_undegraded":"2023-11-24T06:28:21.748482+0000","last_fullsized":"2023-11-24T06:28:21.748482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:00.075286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.727760+0000","last_change":"2023-11-24T06:28:19.737019+0000","last_active":"2023-11-24T06:28:20.727760+0000","last_peered":"2023-11-24T06:28:20.727760+0000","last_clean":"2023-11-24T06:28:20.727760+0000","last_became_active":"2023-11-24T06:28:19.736587+0000","last_became_peered":"2023-11-24T06:28:19.736587+0000","last_unstale":"2023-11-24T06:28:20.727760+0000","last_undegraded":"2023-11-24T06:28:20.727760+0000","last_fullsized":"2023-11-24T06:28:20.727760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:34.484876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361269+0000","last_change":"2023-11-24T06:28:20.756359+0000","last_active":"2023-11-24T06:28:22.361269+0000","last_peered":"2023-11-24T06:28:22.361269+0000","last_clean":"2023-11-24T06:28:22.361269+0000","last_became_active":"2023-11-24T06:28:20.756086+0000","last_became_peered":"2023-11-24T06:28:20.756086+0000","last_unstale":"2023-11-24T06:28:22.361269+0000","last_undegraded":"2023-11-24T06:28:22.361269+0000","last_fullsized":"2023-11-24T06:28:22.361269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:19.935685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.774855+0000","last_change":"2023-11-24T06:28:19.728784+0000","last_active":"2023-11-24T06:28:20.774855+0000","last_peered":"2023-11-24T06:28:20.774855+0000","last_clean":"2023-11-24T06:28:20.774855+0000","last_became_active":"2023-11-24T06:28:19.728379+0000","last_became_peered":"2023-11-24T06:28:19.728379+0000","last_unstale":"2023-11-24T06:28:20.774855+0000","last_undegraded":"2023-11-24T06:28:20.774855+0000","last_fullsized":"2023-11-24T06:28:20.774855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:35.371421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363620+0000","last_change":"2023-11-24T06:28:20.748159+0000","last_active":"2023-11-24T06:28:22.363620+0000","last_peered":"2023-11-24T06:28:22.363620+0000","last_clean":"2023-11-24T06:28:22.363620+0000","last_became_active":"2023-11-24T06:28:20.747300+0000","last_became_peered":"2023-11-24T06:28:20.747300+0000","last_unstale":"2023-11-24T06:28:22.363620+0000","last_undegraded":"2023-11-24T06:28:22.363620+0000","last_fullsized":"2023-11-24T06:28:22.363620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:39.039302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.773830+0000","last_change":"2023-11-24T06:28:19.738232+0000","last_active":"2023-11-24T06:28:20.773830+0000","last_peered":"2023-11-24T06:28:20.773830+0000","last_clean":"2023-11-24T06:28:20.773830+0000","last_became_active":"2023-11-24T06:28:19.737493+0000","last_became_peered":"2023-11-24T06:28:19.737493+0000","last_unstale":"2023-11-24T06:28:20.773830+0000","last_undegraded":"2023-11-24T06:28:20.773830+0000","last_fullsized":"2023-11-24T06:28:20.773830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:38.025733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.750413+0000","last_change":"2023-11-24T06:28:20.752165+0000","last_active":"2023-11-24T06:28:21.750413+0000","last_peered":"2023-11-24T06:28:21.750413+0000","last_clean":"2023-11-24T06:28:21.750413+0000","last_became_active":"2023-11-24T06:28:20.751192+0000","last_became_peered":"2023-11-24T06:28:20.751192+0000","last_unstale":"2023-11-24T06:28:21.750413+0000","last_undegraded":"2023-11-24T06:28:21.750413+0000","last_fullsized":"2023-11-24T06:28:21.750413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:52.718291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.769292+0000","last_change":"2023-11-24T06:28:19.740458+0000","last_active":"2023-11-24T06:28:20.769292+0000","last_peered":"2023-11-24T06:28:20.769292+0000","last_clean":"2023-11-24T06:28:20.769292+0000","last_became_active":"2023-11-24T06:28:19.740072+0000","last_became_peered":"2023-11-24T06:28:19.740072+0000","last_unstale":"2023-11-24T06:28:20.769292+0000","last_undegraded":"2023-11-24T06:28:20.769292+0000","last_fullsized":"2023-11-24T06:28:20.769292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:14.621798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741778+0000","last_change":"2023-11-24T06:28:20.754060+0000","last_active":"2023-11-24T06:28:21.741778+0000","last_peered":"2023-11-24T06:28:21.741778+0000","last_clean":"2023-11-24T06:28:21.741778+0000","last_became_active":"2023-11-24T06:28:20.753777+0000","last_became_peered":"2023-11-24T06:28:20.753777+0000","last_unstale":"2023-11-24T06:28:21.741778+0000","last_undegraded":"2023-11-24T06:28:21.741778+0000","last_fullsized":"2023-11-24T06:28:21.741778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:29.453645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.767249+0000","last_change":"2023-11-24T06:28:19.742059+0000","last_active":"2023-11-24T06:28:20.767249+0000","last_peered":"2023-11-24T06:28:20.767249+0000","last_clean":"2023-11-24T06:28:20.767249+0000","last_became_active":"2023-11-24T06:28:19.741784+0000","last_became_peered":"2023-11-24T06:28:19.741784+0000","last_unstale":"2023-11-24T06:28:20.767249+0000","last_undegraded":"2023-11-24T06:28:20.767249+0000","last_fullsized":"2023-11-24T06:28:20.767249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:40.154924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361968+0000","last_change":"2023-11-24T06:28:20.749842+0000","last_active":"2023-11-24T06:28:22.361968+0000","last_peered":"2023-11-24T06:28:22.361968+0000","last_clean":"2023-11-24T06:28:22.361968+0000","last_became_active":"2023-11-24T06:28:20.749578+0000","last_became_peered":"2023-11-24T06:28:20.749578+0000","last_unstale":"2023-11-24T06:28:22.361968+0000","last_undegraded":"2023-11-24T06:28:22.361968+0000","last_fullsized":"2023-11-24T06:28:22.361968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:30.205190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.735299+0000","last_change":"2023-11-24T06:28:19.736327+0000","last_active":"2023-11-24T06:28:20.735299+0000","last_peered":"2023-11-24T06:28:20.735299+0000","last_clean":"2023-11-24T06:28:20.735299+0000","last_became_active":"2023-11-24T06:28:19.736017+0000","last_became_peered":"2023-11-24T06:28:19.736017+0000","last_unstale":"2023-11-24T06:28:20.735299+0000","last_undegraded":"2023-11-24T06:28:20.735299+0000","last_fullsized":"2023-11-24T06:28:20.735299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:05.916062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.360985+0000","last_change":"2023-11-24T06:28:20.749162+0000","last_active":"2023-11-24T06:28:22.360985+0000","last_peered":"2023-11-24T06:28:22.360985+0000","last_clean":"2023-11-24T06:28:22.360985+0000","last_became_active":"2023-11-24T06:28:20.748814+0000","last_became_peered":"2023-11-24T06:28:20.748814+0000","last_unstale":"2023-11-24T06:28:22.360985+0000","last_undegraded":"2023-11-24T06:28:22.360985+0000","last_fullsized":"2023-11-24T06:28:22.360985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:15.635368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.771526+0000","last_change":"2023-11-24T06:28:19.732595+0000","last_active":"2023-11-24T06:28:20.771526+0000","last_peered":"2023-11-24T06:28:20.771526+0000","last_clean":"2023-11-24T06:28:20.771526+0000","last_became_active":"2023-11-24T06:28:19.732320+0000","last_became_peered":"2023-11-24T06:28:19.732320+0000","last_unstale":"2023-11-24T06:28:20.771526+0000","last_undegraded":"2023-11-24T06:28:20.771526+0000","last_fullsized":"2023-11-24T06:28:20.771526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:50.890592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361338+0000","last_change":"2023-11-24T06:28:20.742637+0000","last_active":"2023-11-24T06:28:22.361338+0000","last_peered":"2023-11-24T06:28:22.361338+0000","last_clean":"2023-11-24T06:28:22.361338+0000","last_became_active":"2023-11-24T06:28:20.742277+0000","last_became_peered":"2023-11-24T06:28:20.742277+0000","last_unstale":"2023-11-24T06:28:22.361338+0000","last_undegraded":"2023-11-24T06:28:22.361338+0000","last_fullsized":"2023-11-24T06:28:22.361338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:57.469275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.736299+0000","last_change":"2023-11-24T06:28:19.745300+0000","last_active":"2023-11-24T06:28:20.736299+0000","last_peered":"2023-11-24T06:28:20.736299+0000","last_clean":"2023-11-24T06:28:20.736299+0000","last_became_active":"2023-11-24T06:28:19.744997+0000","last_became_peered":"2023-11-24T06:28:19.744997+0000","last_unstale":"2023-11-24T06:28:20.736299+0000","last_undegraded":"2023-11-24T06:28:20.736299+0000","last_fullsized":"2023-11-24T06:28:20.736299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:05.916062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361396+0000","last_change":"2023-11-24T06:28:20.746645+0000","last_active":"2023-11-24T06:28:22.361396+0000","last_peered":"2023-11-24T06:28:22.361396+0000","last_clean":"2023-11-24T06:28:22.361396+0000","last_became_active":"2023-11-24T06:28:20.742228+0000","last_became_peered":"2023-11-24T06:28:20.742228+0000","last_unstale":"2023-11-24T06:28:22.361396+0000","last_undegraded":"2023-11-24T06:28:22.361396+0000","last_fullsized":"2023-11-24T06:28:22.361396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:03.169021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.728527+0000","last_change":"2023-11-24T06:28:19.741555+0000","last_active":"2023-11-24T06:28:20.728527+0000","last_peered":"2023-11-24T06:28:20.728527+0000","last_clean":"2023-11-24T06:28:20.728527+0000","last_became_active":"2023-11-24T06:28:19.739895+0000","last_became_peered":"2023-11-24T06:28:19.739895+0000","last_unstale":"2023-11-24T06:28:20.728527+0000","last_undegraded":"2023-11-24T06:28:20.728527+0000","last_fullsized":"2023-11-24T06:28:20.728527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:55:19.561292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747734+0000","last_change":"2023-11-24T06:28:20.750118+0000","last_active":"2023-11-24T06:28:21.747734+0000","last_peered":"2023-11-24T06:28:21.747734+0000","last_clean":"2023-11-24T06:28:21.747734+0000","last_became_active":"2023-11-24T06:28:20.749804+0000","last_became_peered":"2023-11-24T06:28:20.749804+0000","last_unstale":"2023-11-24T06:28:21.747734+0000","last_undegraded":"2023-11-24T06:28:21.747734+0000","last_fullsized":"2023-11-24T06:28:21.747734+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:48.588430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.726227+0000","last_change":"2023-11-24T06:28:19.739558+0000","last_active":"2023-11-24T06:28:20.726227+0000","last_peered":"2023-11-24T06:28:20.726227+0000","last_clean":"2023-11-24T06:28:20.726227+0000","last_became_active":"2023-11-24T06:28:19.736344+0000","last_became_peered":"2023-11-24T06:28:19.736344+0000","last_unstale":"2023-11-24T06:28:20.726227+0000","last_undegraded":"2023-11-24T06:28:20.726227+0000","last_fullsized":"2023-11-24T06:28:20.726227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:40.954198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361860+0000","last_change":"2023-11-24T06:28:20.745248+0000","last_active":"2023-11-24T06:28:22.361860+0000","last_peered":"2023-11-24T06:28:22.361860+0000","last_clean":"2023-11-24T06:28:22.361860+0000","last_became_active":"2023-11-24T06:28:20.744949+0000","last_became_peered":"2023-11-24T06:28:20.744949+0000","last_unstale":"2023-11-24T06:28:22.361860+0000","last_undegraded":"2023-11-24T06:28:22.361860+0000","last_fullsized":"2023-11-24T06:28:22.361860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:49.338938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.769325+0000","last_change":"2023-11-24T06:28:19.739351+0000","last_active":"2023-11-24T06:28:20.769325+0000","last_peered":"2023-11-24T06:28:20.769325+0000","last_clean":"2023-11-24T06:28:20.769325+0000","last_became_active":"2023-11-24T06:28:19.737557+0000","last_became_peered":"2023-11-24T06:28:19.737557+0000","last_unstale":"2023-11-24T06:28:20.769325+0000","last_undegraded":"2023-11-24T06:28:20.769325+0000","last_fullsized":"2023-11-24T06:28:20.769325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:20.569170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.750531+0000","last_change":"2023-11-24T06:28:20.756413+0000","last_active":"2023-11-24T06:28:21.750531+0000","last_peered":"2023-11-24T06:28:21.750531+0000","last_clean":"2023-11-24T06:28:21.750531+0000","last_became_active":"2023-11-24T06:28:20.756144+0000","last_became_peered":"2023-11-24T06:28:20.756144+0000","last_unstale":"2023-11-24T06:28:21.750531+0000","last_undegraded":"2023-11-24T06:28:21.750531+0000","last_fullsized":"2023-11-24T06:28:21.750531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:21.555443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.723507+0000","last_change":"2023-11-24T06:28:19.733326+0000","last_active":"2023-11-24T06:28:20.723507+0000","last_peered":"2023-11-24T06:28:20.723507+0000","last_clean":"2023-11-24T06:28:20.723507+0000","last_became_active":"2023-11-24T06:28:19.733070+0000","last_became_peered":"2023-11-24T06:28:19.733070+0000","last_unstale":"2023-11-24T06:28:20.723507+0000","last_undegraded":"2023-11-24T06:28:20.723507+0000","last_fullsized":"2023-11-24T06:28:20.723507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:05.916062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361892+0000","last_change":"2023-11-24T06:28:20.741782+0000","last_active":"2023-11-24T06:28:22.361892+0000","last_peered":"2023-11-24T06:28:22.361892+0000","last_clean":"2023-11-24T06:28:22.361892+0000","last_became_active":"2023-11-24T06:28:20.741508+0000","last_became_peered":"2023-11-24T06:28:20.741508+0000","last_unstale":"2023-11-24T06:28:22.361892+0000","last_undegraded":"2023-11-24T06:28:22.361892+0000","last_fullsized":"2023-11-24T06:28:22.361892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:12.693448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.740853+0000","last_change":"2023-11-24T06:28:19.741328+0000","last_active":"2023-11-24T06:28:20.740853+0000","last_peered":"2023-11-24T06:28:20.740853+0000","last_clean":"2023-11-24T06:28:20.740853+0000","last_became_active":"2023-11-24T06:28:19.740211+0000","last_became_peered":"2023-11-24T06:28:19.740211+0000","last_unstale":"2023-11-24T06:28:20.740853+0000","last_undegraded":"2023-11-24T06:28:20.740853+0000","last_fullsized":"2023-11-24T06:28:20.740853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:11.679878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361842+0000","last_change":"2023-11-24T06:28:20.746488+0000","last_active":"2023-11-24T06:28:22.361842+0000","last_peered":"2023-11-24T06:28:22.361842+0000","last_clean":"2023-11-24T06:28:22.361842+0000","last_became_active":"2023-11-24T06:28:20.746196+0000","last_became_peered":"2023-11-24T06:28:20.746196+0000","last_unstale":"2023-11-24T06:28:22.361842+0000","last_undegraded":"2023-11-24T06:28:22.361842+0000","last_fullsized":"2023-11-24T06:28:22.361842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:08.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.724893+0000","last_change":"2023-11-24T06:28:19.738778+0000","last_active":"2023-11-24T06:28:20.724893+0000","last_peered":"2023-11-24T06:28:20.724893+0000","last_clean":"2023-11-24T06:28:20.724893+0000","last_became_active":"2023-11-24T06:28:19.738482+0000","last_became_peered":"2023-11-24T06:28:19.738482+0000","last_unstale":"2023-11-24T06:28:20.724893+0000","last_undegraded":"2023-11-24T06:28:20.724893+0000","last_fullsized":"2023-11-24T06:28:20.724893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:18.922115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363974+0000","last_change":"2023-11-24T06:28:20.746401+0000","last_active":"2023-11-24T06:28:22.363974+0000","last_peered":"2023-11-24T06:28:22.363974+0000","last_clean":"2023-11-24T06:28:22.363974+0000","last_became_active":"2023-11-24T06:28:20.745992+0000","last_became_peered":"2023-11-24T06:28:20.745992+0000","last_unstale":"2023-11-24T06:28:22.363974+0000","last_undegraded":"2023-11-24T06:28:22.363974+0000","last_fullsized":"2023-11-24T06:28:22.363974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:21.582740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.739931+0000","last_change":"2023-11-24T06:28:19.737632+0000","last_active":"2023-11-24T06:28:20.739931+0000","last_peered":"2023-11-24T06:28:20.739931+0000","last_clean":"2023-11-24T06:28:20.739931+0000","last_became_active":"2023-11-24T06:28:19.736632+0000","last_became_peered":"2023-11-24T06:28:19.736632+0000","last_unstale":"2023-11-24T06:28:20.739931+0000","last_undegraded":"2023-11-24T06:28:20.739931+0000","last_fullsized":"2023-11-24T06:28:20.739931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:33.275487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362995+0000","last_change":"2023-11-24T06:28:20.753196+0000","last_active":"2023-11-24T06:28:22.362995+0000","last_peered":"2023-11-24T06:28:22.362995+0000","last_clean":"2023-11-24T06:28:22.362995+0000","last_became_active":"2023-11-24T06:28:20.752928+0000","last_became_peered":"2023-11-24T06:28:20.752928+0000","last_unstale":"2023-11-24T06:28:22.362995+0000","last_undegraded":"2023-11-24T06:28:22.362995+0000","last_fullsized":"2023-11-24T06:28:22.362995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:46.026504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.728819+0000","last_change":"2023-11-24T06:28:19.744231+0000","last_active":"2023-11-24T06:28:20.728819+0000","last_peered":"2023-11-24T06:28:20.728819+0000","last_clean":"2023-11-24T06:28:20.728819+0000","last_became_active":"2023-11-24T06:28:19.743719+0000","last_became_peered":"2023-11-24T06:28:19.743719+0000","last_unstale":"2023-11-24T06:28:20.728819+0000","last_undegraded":"2023-11-24T06:28:20.728819+0000","last_fullsized":"2023-11-24T06:28:20.728819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:44:59.965561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361501+0000","last_change":"2023-11-24T06:28:20.742551+0000","last_active":"2023-11-24T06:28:22.361501+0000","last_peered":"2023-11-24T06:28:22.361501+0000","last_clean":"2023-11-24T06:28:22.361501+0000","last_became_active":"2023-11-24T06:28:20.742211+0000","last_became_peered":"2023-11-24T06:28:20.742211+0000","last_unstale":"2023-11-24T06:28:22.361501+0000","last_undegraded":"2023-11-24T06:28:22.361501+0000","last_fullsized":"2023-11-24T06:28:22.361501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:19.935685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.726724+0000","last_change":"2023-11-24T06:28:19.737801+0000","last_active":"2023-11-24T06:28:20.726724+0000","last_peered":"2023-11-24T06:28:20.726724+0000","last_clean":"2023-11-24T06:28:20.726724+0000","last_became_active":"2023-11-24T06:28:19.736632+0000","last_became_peered":"2023-11-24T06:28:19.736632+0000","last_unstale":"2023-11-24T06:28:20.726724+0000","last_undegraded":"2023-11-24T06:28:20.726724+0000","last_fullsized":"2023-11-24T06:28:20.726724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:15.633969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748625+0000","last_change":"2023-11-24T06:28:20.748776+0000","last_active":"2023-11-24T06:28:21.748625+0000","last_peered":"2023-11-24T06:28:21.748625+0000","last_clean":"2023-11-24T06:28:21.748625+0000","last_became_active":"2023-11-24T06:28:20.745731+0000","last_became_peered":"2023-11-24T06:28:20.745731+0000","last_unstale":"2023-11-24T06:28:21.748625+0000","last_undegraded":"2023-11-24T06:28:21.748625+0000","last_fullsized":"2023-11-24T06:28:21.748625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:14.252801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.736666+0000","last_change":"2023-11-24T06:28:19.745039+0000","last_active":"2023-11-24T06:28:20.736666+0000","last_peered":"2023-11-24T06:28:20.736666+0000","last_clean":"2023-11-24T06:28:20.736666+0000","last_became_active":"2023-11-24T06:28:19.744548+0000","last_became_peered":"2023-11-24T06:28:19.744548+0000","last_unstale":"2023-11-24T06:28:20.736666+0000","last_undegraded":"2023-11-24T06:28:20.736666+0000","last_fullsized":"2023-11-24T06:28:20.736666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:07.528212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.742551+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:28:20.742551+0000","last_peered":"2023-11-24T06:28:20.742551+0000","last_clean":"2023-11-24T06:28:20.742551+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:28:20.742551+0000","last_undegraded":"2023-11-24T06:28:20.742551+0000","last_fullsized":"2023-11-24T06:28:20.742551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":742397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.6819999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":6.3319999999999999,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":0.86699999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.3380000000000001,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.613}]},{"osd":3,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.474},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.3640000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":1.286}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.522},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":6.7359999999999998,"5min":6.7359999999999998,"15min":6.7359999999999998},"last":1.762}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.905,"15min":1.905},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":7.6449999999999996,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.423},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":1.1359999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.85699999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.472},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.5649999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.3859999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.5},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.6259999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.109},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":0.98899999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.044},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.774,"5min":3.774,"15min":3.774},"last":1.5960000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":3.649,"5min":3.649,"15min":3.649},"last":1.177}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.885},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":2.4039999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.6819999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.992},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.776}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.518},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.617}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.554,"15min":1.554},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.085,"5min":3.085,"15min":3.085},"last":1.1299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":2.0470000000000002},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.96099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.724},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":2.157}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.526},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.0489999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.4610000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.194},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.27},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.3680000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5950000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.581,"15min":1.581},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.52,"5min":3.52,"15min":3.52},"last":1.49}]},{"osd":6,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.95299999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":1.3049999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":153573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.89100000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.752},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.141}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.25},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.5910000000000002,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.089}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.081,"5min":3.081,"15min":3.081},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.6960000000000002,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":8.8879999999999999,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.202},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.641}]},{"osd":7,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":1.714},{"interface":"front","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":1.3}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":152009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.161}]},{"osd":1,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.278}]},{"osd":3,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.597}]},{"osd":4,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.446},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.6899999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":3.7410000000000001,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.367},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.327}]},{"osd":6,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.49},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":4.1210000000000004,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.546}]},{"osd":7,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.8119999999999998,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.6559999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":153573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.702}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.97599999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.212},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.5149999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.042}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.823},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":1.464}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.27},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.9279999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":742307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.52800000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.97099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.93700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:26.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.855+0000 7effb010f640 1 Processor -- start 2023-11-24T06:28:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.855+0000 7effb010f640 1 -- start start 2023-11-24T06:28:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.857+0000 7effb010f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7effa80ae5a0 0x7effa815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.857+0000 7effb010f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80cf290 0x7effa80cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.857+0000 7effb010f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa8070c40 con 0x7effa80b3ae0 2023-11-24T06:28:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.857+0000 7effb010f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa8071450 con 0x7effa80ae5a0 2023-11-24T06:28:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.857+0000 7effb010f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa80cfc60 con 0x7effa80cf290 2023-11-24T06:28:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.857+0000 7effad6aa640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7effa80b3ae0 0x7effa806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60760/0 (socket says 172.21.15.16:60760) 2023-11-24T06:28:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.857+0000 7effad6aa640 1 -- 172.21.15.16:0/592959757 learned_addr learned my addr 172.21.15.16:0/592959757 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.858+0000 7effae6ac640 1 --2- 172.21.15.16:0/592959757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80cf290 0x7effa80cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.858+0000 7effadeab640 1 --2- 172.21.15.16:0/592959757 >> v2:172.21.15.173:3300/0 conn(0x7effa80ae5a0 0x7effa815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.858+0000 7effacea9640 1 -- 172.21.15.16:0/592959757 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547722681 0 0) 0x7effa8070c40 con 0x7effa80b3ae0 2023-11-24T06:28:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.858+0000 7effacea9640 1 -- 172.21.15.16:0/592959757 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff94003520 con 0x7effa80b3ae0 2023-11-24T06:28:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.858+0000 7effae6ac640 1 -- 172.21.15.16:0/592959757 >> v2:172.21.15.173:3300/0 conn(0x7effa80ae5a0 msgr2=0x7effa815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.858+0000 7effae6ac640 1 --2- 172.21.15.16:0/592959757 >> v2:172.21.15.173:3300/0 conn(0x7effa80ae5a0 0x7effa815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.858+0000 7effae6ac640 1 -- 172.21.15.16:0/592959757 >> v1:172.21.15.16:6789/0 conn(0x7effa80b3ae0 legacy=0x7effa806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.858+0000 7effae6ac640 1 -- 172.21.15.16:0/592959757 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa806c470 con 0x7effa80cf290 2023-11-24T06:28:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.860+0000 7effae6ac640 1 --2- 172.21.15.16:0/592959757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80cf290 0x7effa80cf720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7effa00097a0 tx=0x7effa002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b70360a5a1716941 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.860+0000 7effacea9640 1 -- 172.21.15.16:0/592959757 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0030800 con 0x7effa80cf290 2023-11-24T06:28:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.860+0000 7effacea9640 1 -- 172.21.15.16:0/592959757 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa0030960 con 0x7effa80cf290 2023-11-24T06:28:26.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.860+0000 7effacea9640 1 -- 172.21.15.16:0/592959757 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0030cf0 con 0x7effa80cf290 2023-11-24T06:28:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.861+0000 7effb010f640 1 -- 172.21.15.16:0/592959757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80cf290 msgr2=0x7effa80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.861+0000 7effb010f640 1 --2- 172.21.15.16:0/592959757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80cf290 0x7effa80cf720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7effa00097a0 tx=0x7effa002b940 comp rx=0 tx=0).stop 2023-11-24T06:28:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.861+0000 7effb010f640 1 -- 172.21.15.16:0/592959757 shutdown_connections 2023-11-24T06:28:26.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.861+0000 7effb010f640 1 --2- 172.21.15.16:0/592959757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80cf290 0x7effa80cf720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.861+0000 7effb010f640 1 --2- 172.21.15.16:0/592959757 >> v2:172.21.15.173:3300/0 conn(0x7effa80ae5a0 0x7effa815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.861+0000 7effb010f640 1 -- 172.21.15.16:0/592959757 >> 172.21.15.16:0/592959757 conn(0x7effa806a800 msgr2=0x7effa806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.861+0000 7effb010f640 1 -- 172.21.15.16:0/592959757 shutdown_connections 2023-11-24T06:28:26.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.861+0000 7effb010f640 1 -- 172.21.15.16:0/592959757 wait complete. 2023-11-24T06:28:26.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.863+0000 7effb010f640 1 Processor -- start 2023-11-24T06:28:26.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.864+0000 7effb010f640 1 -- start start 2023-11-24T06:28:26.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.865+0000 7effb010f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa80ae5a0 0x7effa815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.865+0000 7effb010f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80b3ae0 0x7effa81f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.865+0000 7effb010f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effa80cf290 0x7effa80b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.865+0000 7effb010f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa81506d0 con 0x7effa80cf290 2023-11-24T06:28:26.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.865+0000 7effb010f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa81510b0 con 0x7effa80ae5a0 2023-11-24T06:28:26.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.865+0000 7effb010f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa8151390 con 0x7effa80b3ae0 2023-11-24T06:28:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.865+0000 7effae6ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effa80cf290 0x7effa80b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.865+0000 7effae6ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effa80cf290 0x7effa80b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50336/0 (socket says 172.21.15.16:50336) 2023-11-24T06:28:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.865+0000 7effae6ac640 1 -- 172.21.15.16:0/888638240 learned_addr learned my addr 172.21.15.16:0/888638240 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.866+0000 7effad6aa640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80b3ae0 0x7effa81f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.866+0000 7effadeab640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa80ae5a0 0x7effa815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.866+0000 7effae6ac640 1 -- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80b3ae0 msgr2=0x7effa81f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.866+0000 7effae6ac640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80b3ae0 0x7effa81f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.866+0000 7effae6ac640 1 -- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa80ae5a0 msgr2=0x7effa815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.866+0000 7effae6ac640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa80ae5a0 0x7effa815ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.866+0000 7effae6ac640 1 -- 172.21.15.16:0/888638240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa0009400 con 0x7effa80cf290 2023-11-24T06:28:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.866+0000 7effae6ac640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effa80cf290 0x7effa80b2aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7effa00098c0 tx=0x7effa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.867+0000 7eff9e7fc640 1 -- 172.21.15.16:0/888638240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa003e070 con 0x7effa80cf290 2023-11-24T06:28:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.867+0000 7eff9e7fc640 1 -- 172.21.15.16:0/888638240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa0039910 con 0x7effa80cf290 2023-11-24T06:28:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.867+0000 7eff9e7fc640 1 -- 172.21.15.16:0/888638240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0039cc0 con 0x7effa80cf290 2023-11-24T06:28:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.867+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa81f2dc0 con 0x7effa80cf290 2023-11-24T06:28:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.867+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa815f1b0 con 0x7effa80cf290 2023-11-24T06:28:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.868+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa815f340 con 0x7effa80cf290 2023-11-24T06:28:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.869+0000 7eff9e7fc640 1 -- 172.21.15.16:0/888638240 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effa00423f0 con 0x7effa80cf290 2023-11-24T06:28:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.870+0000 7eff9e7fc640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff8c06ae90 0x7eff8c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:26.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.870+0000 7eff9e7fc640 1 -- 172.21.15.16:0/888638240 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effa00ab250 con 0x7effa80cf290 2023-11-24T06:28:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.871+0000 7effadeab640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff8c06ae90 0x7eff8c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:26.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.872+0000 7effadeab640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff8c06ae90 0x7eff8c06d350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effa8150a50 tx=0x7effa400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:26.876+0000 7eff9e7fc640 1 -- 172.21.15.16:0/888638240 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7effa815f340 con 0x7effa80cf290 2023-11-24T06:28:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.149+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7effa81f32e0 con 0x7effa80cf290 2023-11-24T06:28:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.150+0000 7eff9e7fc640 1 -- 172.21.15.16:0/888638240 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1012 (secure 0 0 0) 0x7effa81f32e0 con 0x7effa80cf290 2023-11-24T06:28:27.154 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:27.154 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":110,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1700807208,"num_in_osds":8,"osd_in_since":1700807200,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":242753536,"bytes_avail":772851359744,"bytes_total":773094113280,"write_bytes_sec":2232,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5211}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-11-24T06:28:02.124517+0000","services":{}},"progress_events":{}} 2023-11-24T06:28:27.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff8c06ae90 msgr2=0x7eff8c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff8c06ae90 0x7eff8c06d350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effa8150a50 tx=0x7effa400b040 comp rx=0 tx=0).stop 2023-11-24T06:28:27.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effa80cf290 msgr2=0x7effa80b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effa80cf290 0x7effa80b2aa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7effa00098c0 tx=0x7effa00046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 shutdown_connections 2023-11-24T06:28:27.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff8c06ae90 0x7eff8c06d350 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effa80cf290 0x7effa80b2aa0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa80b3ae0 0x7effa81f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 --2- 172.21.15.16:0/888638240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa80ae5a0 0x7effa815ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 >> 172.21.15.16:0/888638240 conn(0x7effa806a800 msgr2=0x7effa815d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:27.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.153+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 shutdown_connections 2023-11-24T06:28:27.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.154+0000 7effb010f640 1 -- 172.21.15.16:0/888638240 wait complete. 2023-11-24T06:28:27.177 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T06:28:27.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:27.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.397+0000 7f3a7cc59640 1 Processor -- start 2023-11-24T06:28:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.397+0000 7f3a7cc59640 1 -- start start 2023-11-24T06:28:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.399+0000 7f3a7cc59640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a78071f00 0x7f3a78146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.399+0000 7f3a7cc59640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 0x7f3a78166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.399+0000 7f3a7cc59640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a780b2e10 con 0x7f3a78071410 2023-11-24T06:28:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.399+0000 7f3a7cc59640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7806cc10 con 0x7f3a78071f00 2023-11-24T06:28:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.399+0000 7f3a7cc59640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7806cec0 con 0x7f3a7806c7c0 2023-11-24T06:28:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.399+0000 7f3a6ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a78071f00 0x7f3a78146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.400+0000 7f3a6ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a78071f00 0x7f3a78146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50350/0 (socket says 172.21.15.16:50350) 2023-11-24T06:28:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.400+0000 7f3a6ffff640 1 -- 172.21.15.16:0/1583909923 learned_addr learned my addr 172.21.15.16:0/1583909923 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.400+0000 7f3a77fff640 1 --2- 172.21.15.16:0/1583909923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 0x7f3a78166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.400+0000 7f3a6ffff640 1 -- 172.21.15.16:0/1583909923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 msgr2=0x7f3a78166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.400+0000 7f3a6ffff640 1 --2- 172.21.15.16:0/1583909923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 0x7f3a78166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.400+0000 7f3a6ffff640 1 -- 172.21.15.16:0/1583909923 >> v1:172.21.15.173:6789/0 conn(0x7f3a78071410 legacy=0x7f3a780717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.400+0000 7f3a6ffff640 1 -- 172.21.15.16:0/1583909923 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7806d1a0 con 0x7f3a78071f00 2023-11-24T06:28:27.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.401+0000 7f3a6ffff640 1 --2- 172.21.15.16:0/1583909923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071f00 0x7f3a78146770 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009870 tx=0x7f3a6002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6173544831feda server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:27.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a76ffd640 1 -- 172.21.15.16:0/1583909923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600317c0 con 0x7f3a78071f00 2023-11-24T06:28:27.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a76ffd640 1 -- 172.21.15.16:0/1583909923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60031920 con 0x7f3a78071f00 2023-11-24T06:28:27.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a76ffd640 1 -- 172.21.15.16:0/1583909923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60031cb0 con 0x7f3a78071f00 2023-11-24T06:28:27.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a7cc59640 1 -- 172.21.15.16:0/1583909923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071f00 msgr2=0x7f3a78146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a7cc59640 1 --2- 172.21.15.16:0/1583909923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071f00 0x7f3a78146770 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009870 tx=0x7f3a6002b860 comp rx=0 tx=0).stop 2023-11-24T06:28:27.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a7cc59640 1 -- 172.21.15.16:0/1583909923 shutdown_connections 2023-11-24T06:28:27.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a7cc59640 1 --2- 172.21.15.16:0/1583909923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 0x7f3a78166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a7cc59640 1 --2- 172.21.15.16:0/1583909923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071f00 0x7f3a78146770 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a7cc59640 1 -- 172.21.15.16:0/1583909923 >> 172.21.15.16:0/1583909923 conn(0x7f3a7806a800 msgr2=0x7f3a7806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:27.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a7cc59640 1 -- 172.21.15.16:0/1583909923 shutdown_connections 2023-11-24T06:28:27.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.402+0000 7f3a7cc59640 1 -- 172.21.15.16:0/1583909923 wait complete. 2023-11-24T06:28:27.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.404+0000 7f3a7cc59640 1 Processor -- start 2023-11-24T06:28:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.405+0000 7f3a7cc59640 1 -- start start 2023-11-24T06:28:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.406+0000 7f3a7cc59640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 0x7f3a7813d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.406+0000 7f3a7cc59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071410 0x7f3a780b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.406+0000 7f3a7cc59640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a78071f00 0x7f3a780b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.406+0000 7f3a7cc59640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7813dd80 con 0x7f3a78071410 2023-11-24T06:28:27.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.406+0000 7f3a7cc59640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780c4e70 con 0x7f3a78071f00 2023-11-24T06:28:27.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.406+0000 7f3a7cc59640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780c4b90 con 0x7f3a7806c7c0 2023-11-24T06:28:27.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.406+0000 7f3a6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071410 0x7f3a780b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.406+0000 7f3a6ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071410 0x7f3a780b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50366/0 (socket says 172.21.15.16:50366) 2023-11-24T06:28:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.406+0000 7f3a6ffff640 1 -- 172.21.15.16:0/826065518 learned_addr learned my addr 172.21.15.16:0/826065518 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a777fe640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 0x7f3a7813d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a77fff640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a78071f00 0x7f3a780b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a6ffff640 1 -- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 msgr2=0x7f3a7813d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a6ffff640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 0x7f3a7813d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a6ffff640 1 -- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a78071f00 msgr2=0x7f3a780b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a6ffff640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a78071f00 0x7f3a780b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a6ffff640 1 -- 172.21.15.16:0/826065518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a600094d0 con 0x7f3a78071410 2023-11-24T06:28:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a777fe640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 0x7f3a7813d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a77fff640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a78071f00 0x7f3a780b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a6ffff640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071410 0x7f3a780b1b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009870 tx=0x7f3a600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.407+0000 7f3a74ff9640 1 -- 172.21.15.16:0/826065518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60032860 con 0x7f3a78071410 2023-11-24T06:28:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.408+0000 7f3a74ff9640 1 -- 172.21.15.16:0/826065518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a600329c0 con 0x7f3a78071410 2023-11-24T06:28:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.408+0000 7f3a74ff9640 1 -- 172.21.15.16:0/826065518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60032d70 con 0x7f3a78071410 2023-11-24T06:28:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.408+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7815d620 con 0x7f3a78071410 2023-11-24T06:28:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.408+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7815dbe0 con 0x7f3a78071410 2023-11-24T06:28:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.409+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a78067450 con 0x7f3a78071410 2023-11-24T06:28:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.410+0000 7f3a74ff9640 1 -- 172.21.15.16:0/826065518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a6004e450 con 0x7f3a78071410 2023-11-24T06:28:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.411+0000 7f3a74ff9640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3a5806ae40 0x7f3a5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.411+0000 7f3a74ff9640 1 -- 172.21.15.16:0/826065518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a600ab970 con 0x7f3a78071410 2023-11-24T06:28:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.411+0000 7f3a777fe640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3a5806ae40 0x7f3a5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.412+0000 7f3a777fe640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3a5806ae40 0x7f3a5806d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a78070230 tx=0x7f3a68009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.416+0000 7f3a74ff9640 1 -- 172.21.15.16:0/826065518 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a60076980 con 0x7f3a78071410 2023-11-24T06:28:27.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.607+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a7815d4a0 con 0x7f3a5806ae40 2023-11-24T06:28:27.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.628+0000 7f3a74ff9640 1 -- 172.21.15.16:0/826065518 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286889 (secure 0 0 0) 0x7f3a68005050 con 0x7f3a5806ae40 2023-11-24T06:28:27.640 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:28:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3a5806ae40 msgr2=0x7f3a5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3a5806ae40 0x7f3a5806d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a78070230 tx=0x7f3a68009040 comp rx=0 tx=0).stop 2023-11-24T06:28:27.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071410 msgr2=0x7f3a780b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071410 0x7f3a780b1b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009870 tx=0x7f3a600046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 shutdown_connections 2023-11-24T06:28:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3a5806ae40 0x7f3a5806d300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a78071f00 0x7f3a780b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a78071410 0x7f3a780b1b90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 --2- 172.21.15.16:0/826065518 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a7806c7c0 0x7f3a7813d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:27.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 >> 172.21.15.16:0/826065518 conn(0x7f3a7806a800 msgr2=0x7f3a7806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 shutdown_connections 2023-11-24T06:28:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:27.634+0000 7f3a7cc59640 1 -- 172.21.15.16:0/826065518 wait complete. 2023-11-24T06:28:27.660 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2023-11-24T06:28:26.132716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237064,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737656,"statfs":{"total":773094113280,"available":772851359744,"internally_reserved":0,"allocated":4333568,"data_stored":2401068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.421970"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362259+0000","last_change":"2023-11-24T06:28:20.744890+0000","last_active":"2023-11-24T06:28:22.362259+0000","last_peered":"2023-11-24T06:28:22.362259+0000","last_clean":"2023-11-24T06:28:22.362259+0000","last_became_active":"2023-11-24T06:28:20.744570+0000","last_became_peered":"2023-11-24T06:28:20.744570+0000","last_unstale":"2023-11-24T06:28:22.362259+0000","last_undegraded":"2023-11-24T06:28:22.362259+0000","last_fullsized":"2023-11-24T06:28:22.362259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:34.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364147+0000","last_change":"2023-11-24T06:28:20.740082+0000","last_active":"2023-11-24T06:28:22.364147+0000","last_peered":"2023-11-24T06:28:22.364147+0000","last_clean":"2023-11-24T06:28:22.364147+0000","last_became_active":"2023-11-24T06:28:20.739812+0000","last_became_peered":"2023-11-24T06:28:20.739812+0000","last_unstale":"2023-11-24T06:28:22.364147+0000","last_undegraded":"2023-11-24T06:28:22.364147+0000","last_fullsized":"2023-11-24T06:28:22.364147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:36.198976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.750217+0000","last_change":"2023-11-24T06:28:20.745391+0000","last_active":"2023-11-24T06:28:21.750217+0000","last_peered":"2023-11-24T06:28:21.750217+0000","last_clean":"2023-11-24T06:28:21.750217+0000","last_became_active":"2023-11-24T06:28:20.744851+0000","last_became_peered":"2023-11-24T06:28:20.744851+0000","last_unstale":"2023-11-24T06:28:21.750217+0000","last_undegraded":"2023-11-24T06:28:21.750217+0000","last_fullsized":"2023-11-24T06:28:21.750217+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:14.252801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741692+0000","last_change":"2023-11-24T06:28:20.758056+0000","last_active":"2023-11-24T06:28:21.741692+0000","last_peered":"2023-11-24T06:28:21.741692+0000","last_clean":"2023-11-24T06:28:21.741692+0000","last_became_active":"2023-11-24T06:28:20.757439+0000","last_became_peered":"2023-11-24T06:28:20.757439+0000","last_unstale":"2023-11-24T06:28:21.741692+0000","last_undegraded":"2023-11-24T06:28:21.741692+0000","last_fullsized":"2023-11-24T06:28:21.741692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:44:14.949226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.360988+0000","last_change":"2023-11-24T06:28:20.738890+0000","last_active":"2023-11-24T06:28:22.360988+0000","last_peered":"2023-11-24T06:28:22.360988+0000","last_clean":"2023-11-24T06:28:22.360988+0000","last_became_active":"2023-11-24T06:28:20.738441+0000","last_became_peered":"2023-11-24T06:28:20.738441+0000","last_unstale":"2023-11-24T06:28:22.360988+0000","last_undegraded":"2023-11-24T06:28:22.360988+0000","last_fullsized":"2023-11-24T06:28:22.360988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:26.380103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363652+0000","last_change":"2023-11-24T06:28:20.744786+0000","last_active":"2023-11-24T06:28:22.363652+0000","last_peered":"2023-11-24T06:28:22.363652+0000","last_clean":"2023-11-24T06:28:22.363652+0000","last_became_active":"2023-11-24T06:28:20.744487+0000","last_became_peered":"2023-11-24T06:28:20.744487+0000","last_unstale":"2023-11-24T06:28:22.363652+0000","last_undegraded":"2023-11-24T06:28:22.363652+0000","last_fullsized":"2023-11-24T06:28:22.363652+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:14.252801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748125+0000","last_change":"2023-11-24T06:28:20.753428+0000","last_active":"2023-11-24T06:28:21.748125+0000","last_peered":"2023-11-24T06:28:21.748125+0000","last_clean":"2023-11-24T06:28:21.748125+0000","last_became_active":"2023-11-24T06:28:20.750476+0000","last_became_peered":"2023-11-24T06:28:20.750476+0000","last_unstale":"2023-11-24T06:28:21.748125+0000","last_undegraded":"2023-11-24T06:28:21.748125+0000","last_fullsized":"2023-11-24T06:28:21.748125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:18.305749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747710+0000","last_change":"2023-11-24T06:28:20.750080+0000","last_active":"2023-11-24T06:28:21.747710+0000","last_peered":"2023-11-24T06:28:21.747710+0000","last_clean":"2023-11-24T06:28:21.747710+0000","last_became_active":"2023-11-24T06:28:20.749694+0000","last_became_peered":"2023-11-24T06:28:20.749694+0000","last_unstale":"2023-11-24T06:28:21.747710+0000","last_undegraded":"2023-11-24T06:28:21.747710+0000","last_fullsized":"2023-11-24T06:28:21.747710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:41.832504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363811+0000","last_change":"2023-11-24T06:28:20.760441+0000","last_active":"2023-11-24T06:28:22.363811+0000","last_peered":"2023-11-24T06:28:22.363811+0000","last_clean":"2023-11-24T06:28:22.363811+0000","last_became_active":"2023-11-24T06:28:20.758860+0000","last_became_peered":"2023-11-24T06:28:20.758860+0000","last_unstale":"2023-11-24T06:28:22.363811+0000","last_undegraded":"2023-11-24T06:28:22.363811+0000","last_fullsized":"2023-11-24T06:28:22.363811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:21.582740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742175+0000","last_change":"2023-11-24T06:28:20.744185+0000","last_active":"2023-11-24T06:28:21.742175+0000","last_peered":"2023-11-24T06:28:21.742175+0000","last_clean":"2023-11-24T06:28:21.742175+0000","last_became_active":"2023-11-24T06:28:20.743866+0000","last_became_peered":"2023-11-24T06:28:20.743866+0000","last_unstale":"2023-11-24T06:28:21.742175+0000","last_undegraded":"2023-11-24T06:28:21.742175+0000","last_fullsized":"2023-11-24T06:28:21.742175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:41.487786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748764+0000","last_change":"2023-11-24T06:28:20.748422+0000","last_active":"2023-11-24T06:28:21.748764+0000","last_peered":"2023-11-24T06:28:21.748764+0000","last_clean":"2023-11-24T06:28:21.748764+0000","last_became_active":"2023-11-24T06:28:20.747375+0000","last_became_peered":"2023-11-24T06:28:20.747375+0000","last_unstale":"2023-11-24T06:28:21.748764+0000","last_undegraded":"2023-11-24T06:28:21.748764+0000","last_fullsized":"2023-11-24T06:28:21.748764+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:21.555443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749684+0000","last_change":"2023-11-24T06:28:20.741727+0000","last_active":"2023-11-24T06:28:21.749684+0000","last_peered":"2023-11-24T06:28:21.749684+0000","last_clean":"2023-11-24T06:28:21.749684+0000","last_became_active":"2023-11-24T06:28:20.741428+0000","last_became_peered":"2023-11-24T06:28:20.741428+0000","last_unstale":"2023-11-24T06:28:21.749684+0000","last_undegraded":"2023-11-24T06:28:21.749684+0000","last_fullsized":"2023-11-24T06:28:21.749684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:18.305749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361169+0000","last_change":"2023-11-24T06:28:20.740157+0000","last_active":"2023-11-24T06:28:22.361169+0000","last_peered":"2023-11-24T06:28:22.361169+0000","last_clean":"2023-11-24T06:28:22.361169+0000","last_became_active":"2023-11-24T06:28:20.739896+0000","last_became_peered":"2023-11-24T06:28:20.739896+0000","last_unstale":"2023-11-24T06:28:22.361169+0000","last_undegraded":"2023-11-24T06:28:22.361169+0000","last_fullsized":"2023-11-24T06:28:22.361169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:41.168494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363756+0000","last_change":"2023-11-24T06:28:20.744520+0000","last_active":"2023-11-24T06:28:22.363756+0000","last_peered":"2023-11-24T06:28:22.363756+0000","last_clean":"2023-11-24T06:28:22.363756+0000","last_became_active":"2023-11-24T06:28:20.744131+0000","last_became_peered":"2023-11-24T06:28:20.744131+0000","last_unstale":"2023-11-24T06:28:22.363756+0000","last_undegraded":"2023-11-24T06:28:22.363756+0000","last_fullsized":"2023-11-24T06:28:22.363756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:52.718291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749170+0000","last_change":"2023-11-24T06:28:20.749484+0000","last_active":"2023-11-24T06:28:21.749170+0000","last_peered":"2023-11-24T06:28:21.749170+0000","last_clean":"2023-11-24T06:28:21.749170+0000","last_became_active":"2023-11-24T06:28:20.749186+0000","last_became_peered":"2023-11-24T06:28:20.749186+0000","last_unstale":"2023-11-24T06:28:21.749170+0000","last_undegraded":"2023-11-24T06:28:21.749170+0000","last_fullsized":"2023-11-24T06:28:21.749170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:48.588430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748803+0000","last_change":"2023-11-24T06:28:20.755151+0000","last_active":"2023-11-24T06:28:21.748803+0000","last_peered":"2023-11-24T06:28:21.748803+0000","last_clean":"2023-11-24T06:28:21.748803+0000","last_became_active":"2023-11-24T06:28:20.754783+0000","last_became_peered":"2023-11-24T06:28:20.754783+0000","last_unstale":"2023-11-24T06:28:21.748803+0000","last_undegraded":"2023-11-24T06:28:21.748803+0000","last_fullsized":"2023-11-24T06:28:21.748803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:52.033759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748672+0000","last_change":"2023-11-24T06:28:20.751616+0000","last_active":"2023-11-24T06:28:21.748672+0000","last_peered":"2023-11-24T06:28:21.748672+0000","last_clean":"2023-11-24T06:28:21.748672+0000","last_became_active":"2023-11-24T06:28:20.751347+0000","last_became_peered":"2023-11-24T06:28:20.751347+0000","last_unstale":"2023-11-24T06:28:21.748672+0000","last_undegraded":"2023-11-24T06:28:21.748672+0000","last_fullsized":"2023-11-24T06:28:21.748672+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:52.718291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364306+0000","last_change":"2023-11-24T06:28:20.752060+0000","last_active":"2023-11-24T06:28:22.364306+0000","last_peered":"2023-11-24T06:28:22.364306+0000","last_clean":"2023-11-24T06:28:22.364306+0000","last_became_active":"2023-11-24T06:28:20.751597+0000","last_became_peered":"2023-11-24T06:28:20.751597+0000","last_unstale":"2023-11-24T06:28:22.364306+0000","last_undegraded":"2023-11-24T06:28:22.364306+0000","last_fullsized":"2023-11-24T06:28:22.364306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:51.206064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363421+0000","last_change":"2023-11-24T06:28:20.751828+0000","last_active":"2023-11-24T06:28:22.363421+0000","last_peered":"2023-11-24T06:28:22.363421+0000","last_clean":"2023-11-24T06:28:22.363421+0000","last_became_active":"2023-11-24T06:28:20.751496+0000","last_became_peered":"2023-11-24T06:28:20.751496+0000","last_unstale":"2023-11-24T06:28:22.363421+0000","last_undegraded":"2023-11-24T06:28:22.363421+0000","last_fullsized":"2023-11-24T06:28:22.363421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:00.075286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361993+0000","last_change":"2023-11-24T06:28:20.742826+0000","last_active":"2023-11-24T06:28:22.361993+0000","last_peered":"2023-11-24T06:28:22.361993+0000","last_clean":"2023-11-24T06:28:22.361993+0000","last_became_active":"2023-11-24T06:28:20.742479+0000","last_became_peered":"2023-11-24T06:28:20.742479+0000","last_unstale":"2023-11-24T06:28:22.361993+0000","last_undegraded":"2023-11-24T06:28:22.361993+0000","last_fullsized":"2023-11-24T06:28:22.361993+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:06.929632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363090+0000","last_change":"2023-11-24T06:28:20.752827+0000","last_active":"2023-11-24T06:28:22.363090+0000","last_peered":"2023-11-24T06:28:22.363090+0000","last_clean":"2023-11-24T06:28:22.363090+0000","last_became_active":"2023-11-24T06:28:20.752553+0000","last_became_peered":"2023-11-24T06:28:20.752553+0000","last_unstale":"2023-11-24T06:28:22.363090+0000","last_undegraded":"2023-11-24T06:28:22.363090+0000","last_fullsized":"2023-11-24T06:28:22.363090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:34.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362788+0000","last_change":"2023-11-24T06:28:20.747287+0000","last_active":"2023-11-24T06:28:22.362788+0000","last_peered":"2023-11-24T06:28:22.362788+0000","last_clean":"2023-11-24T06:28:22.362788+0000","last_became_active":"2023-11-24T06:28:20.746913+0000","last_became_peered":"2023-11-24T06:28:20.746913+0000","last_unstale":"2023-11-24T06:28:22.362788+0000","last_undegraded":"2023-11-24T06:28:22.362788+0000","last_fullsized":"2023-11-24T06:28:22.362788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:08.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361398+0000","last_change":"2023-11-24T06:28:20.755636+0000","last_active":"2023-11-24T06:28:22.361398+0000","last_peered":"2023-11-24T06:28:22.361398+0000","last_clean":"2023-11-24T06:28:22.361398+0000","last_became_active":"2023-11-24T06:28:20.755151+0000","last_became_peered":"2023-11-24T06:28:20.755151+0000","last_unstale":"2023-11-24T06:28:22.361398+0000","last_undegraded":"2023-11-24T06:28:22.361398+0000","last_fullsized":"2023-11-24T06:28:22.361398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:34.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747983+0000","last_change":"2023-11-24T06:28:20.738640+0000","last_active":"2023-11-24T06:28:21.747983+0000","last_peered":"2023-11-24T06:28:21.747983+0000","last_clean":"2023-11-24T06:28:21.747983+0000","last_became_active":"2023-11-24T06:28:20.737115+0000","last_became_peered":"2023-11-24T06:28:20.737115+0000","last_unstale":"2023-11-24T06:28:21.747983+0000","last_undegraded":"2023-11-24T06:28:21.747983+0000","last_fullsized":"2023-11-24T06:28:21.747983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:41.832504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742269+0000","last_change":"2023-11-24T06:28:20.754501+0000","last_active":"2023-11-24T06:28:21.742269+0000","last_peered":"2023-11-24T06:28:21.742269+0000","last_clean":"2023-11-24T06:28:21.742269+0000","last_became_active":"2023-11-24T06:28:20.749001+0000","last_became_peered":"2023-11-24T06:28:20.749001+0000","last_unstale":"2023-11-24T06:28:21.742269+0000","last_undegraded":"2023-11-24T06:28:21.742269+0000","last_fullsized":"2023-11-24T06:28:21.742269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:35.498445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364527+0000","last_change":"2023-11-24T06:28:20.749900+0000","last_active":"2023-11-24T06:28:22.364527+0000","last_peered":"2023-11-24T06:28:22.364527+0000","last_clean":"2023-11-24T06:28:22.364527+0000","last_became_active":"2023-11-24T06:28:20.749612+0000","last_became_peered":"2023-11-24T06:28:20.749612+0000","last_unstale":"2023-11-24T06:28:22.364527+0000","last_undegraded":"2023-11-24T06:28:22.364527+0000","last_fullsized":"2023-11-24T06:28:22.364527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:02.550632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742049+0000","last_change":"2023-11-24T06:28:20.757641+0000","last_active":"2023-11-24T06:28:21.742049+0000","last_peered":"2023-11-24T06:28:21.742049+0000","last_clean":"2023-11-24T06:28:21.742049+0000","last_became_active":"2023-11-24T06:28:20.757307+0000","last_became_peered":"2023-11-24T06:28:20.757307+0000","last_unstale":"2023-11-24T06:28:21.742049+0000","last_undegraded":"2023-11-24T06:28:21.742049+0000","last_fullsized":"2023-11-24T06:28:21.742049+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:36.858522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749834+0000","last_change":"2023-11-24T06:28:20.738500+0000","last_active":"2023-11-24T06:28:21.749834+0000","last_peered":"2023-11-24T06:28:21.749834+0000","last_clean":"2023-11-24T06:28:21.749834+0000","last_became_active":"2023-11-24T06:28:20.738051+0000","last_became_peered":"2023-11-24T06:28:20.738051+0000","last_unstale":"2023-11-24T06:28:21.749834+0000","last_undegraded":"2023-11-24T06:28:21.749834+0000","last_fullsized":"2023-11-24T06:28:21.749834+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:46.026504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363095+0000","last_change":"2023-11-24T06:28:20.743275+0000","last_active":"2023-11-24T06:28:22.363095+0000","last_peered":"2023-11-24T06:28:22.363095+0000","last_clean":"2023-11-24T06:28:22.363095+0000","last_became_active":"2023-11-24T06:28:20.742978+0000","last_became_peered":"2023-11-24T06:28:20.742978+0000","last_unstale":"2023-11-24T06:28:22.363095+0000","last_undegraded":"2023-11-24T06:28:22.363095+0000","last_fullsized":"2023-11-24T06:28:22.363095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:12.693448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747735+0000","last_change":"2023-11-24T06:28:20.751681+0000","last_active":"2023-11-24T06:28:21.747735+0000","last_peered":"2023-11-24T06:28:21.747735+0000","last_clean":"2023-11-24T06:28:21.747735+0000","last_became_active":"2023-11-24T06:28:20.751402+0000","last_became_peered":"2023-11-24T06:28:20.751402+0000","last_unstale":"2023-11-24T06:28:21.747735+0000","last_undegraded":"2023-11-24T06:28:21.747735+0000","last_fullsized":"2023-11-24T06:28:21.747735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:36.384991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361179+0000","last_change":"2023-11-24T06:28:20.754007+0000","last_active":"2023-11-24T06:28:22.361179+0000","last_peered":"2023-11-24T06:28:22.361179+0000","last_clean":"2023-11-24T06:28:22.361179+0000","last_became_active":"2023-11-24T06:28:20.752557+0000","last_became_peered":"2023-11-24T06:28:20.752557+0000","last_unstale":"2023-11-24T06:28:22.361179+0000","last_undegraded":"2023-11-24T06:28:22.361179+0000","last_fullsized":"2023-11-24T06:28:22.361179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:51.904162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748122+0000","last_change":"2023-11-24T06:28:20.757581+0000","last_active":"2023-11-24T06:28:21.748122+0000","last_peered":"2023-11-24T06:28:21.748122+0000","last_clean":"2023-11-24T06:28:21.748122+0000","last_became_active":"2023-11-24T06:28:20.757233+0000","last_became_peered":"2023-11-24T06:28:20.757233+0000","last_unstale":"2023-11-24T06:28:21.748122+0000","last_undegraded":"2023-11-24T06:28:21.748122+0000","last_fullsized":"2023-11-24T06:28:21.748122+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:23.123298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361983+0000","last_change":"2023-11-24T06:28:20.747847+0000","last_active":"2023-11-24T06:28:22.361983+0000","last_peered":"2023-11-24T06:28:22.361983+0000","last_clean":"2023-11-24T06:28:22.361983+0000","last_became_active":"2023-11-24T06:28:20.747551+0000","last_became_peered":"2023-11-24T06:28:20.747551+0000","last_unstale":"2023-11-24T06:28:22.361983+0000","last_undegraded":"2023-11-24T06:28:22.361983+0000","last_fullsized":"2023-11-24T06:28:22.361983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:39.039302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.770328+0000","last_change":"2023-11-24T06:28:19.748835+0000","last_active":"2023-11-24T06:28:20.770328+0000","last_peered":"2023-11-24T06:28:20.770328+0000","last_clean":"2023-11-24T06:28:20.770328+0000","last_became_active":"2023-11-24T06:28:19.748412+0000","last_became_peered":"2023-11-24T06:28:19.748412+0000","last_unstale":"2023-11-24T06:28:20.770328+0000","last_undegraded":"2023-11-24T06:28:20.770328+0000","last_fullsized":"2023-11-24T06:28:20.770328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:48.325368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362430+0000","last_change":"2023-11-24T06:28:20.748498+0000","last_active":"2023-11-24T06:28:22.362430+0000","last_peered":"2023-11-24T06:28:22.362430+0000","last_clean":"2023-11-24T06:28:22.362430+0000","last_became_active":"2023-11-24T06:28:20.748101+0000","last_became_peered":"2023-11-24T06:28:20.748101+0000","last_unstale":"2023-11-24T06:28:22.362430+0000","last_undegraded":"2023-11-24T06:28:22.362430+0000","last_fullsized":"2023-11-24T06:28:22.362430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:12.693448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.771276+0000","last_change":"2023-11-24T06:28:19.742903+0000","last_active":"2023-11-24T06:28:20.771276+0000","last_peered":"2023-11-24T06:28:20.771276+0000","last_clean":"2023-11-24T06:28:20.771276+0000","last_became_active":"2023-11-24T06:28:19.742598+0000","last_became_peered":"2023-11-24T06:28:19.742598+0000","last_unstale":"2023-11-24T06:28:20.771276+0000","last_undegraded":"2023-11-24T06:28:20.771276+0000","last_fullsized":"2023-11-24T06:28:20.771276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:29.191620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363955+0000","last_change":"2023-11-24T06:28:20.751153+0000","last_active":"2023-11-24T06:28:22.363955+0000","last_peered":"2023-11-24T06:28:22.363955+0000","last_clean":"2023-11-24T06:28:22.363955+0000","last_became_active":"2023-11-24T06:28:20.749651+0000","last_became_peered":"2023-11-24T06:28:20.749651+0000","last_unstale":"2023-11-24T06:28:22.363955+0000","last_undegraded":"2023-11-24T06:28:22.363955+0000","last_fullsized":"2023-11-24T06:28:22.363955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:23.869661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.773975+0000","last_change":"2023-11-24T06:28:19.733747+0000","last_active":"2023-11-24T06:28:20.773975+0000","last_peered":"2023-11-24T06:28:20.773975+0000","last_clean":"2023-11-24T06:28:20.773975+0000","last_became_active":"2023-11-24T06:28:19.733405+0000","last_became_peered":"2023-11-24T06:28:19.733405+0000","last_unstale":"2023-11-24T06:28:20.773975+0000","last_undegraded":"2023-11-24T06:28:20.773975+0000","last_fullsized":"2023-11-24T06:28:20.773975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:22.109728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742130+0000","last_change":"2023-11-24T06:28:20.757546+0000","last_active":"2023-11-24T06:28:21.742130+0000","last_peered":"2023-11-24T06:28:21.742130+0000","last_clean":"2023-11-24T06:28:21.742130+0000","last_became_active":"2023-11-24T06:28:20.757229+0000","last_became_peered":"2023-11-24T06:28:20.757229+0000","last_unstale":"2023-11-24T06:28:21.742130+0000","last_undegraded":"2023-11-24T06:28:21.742130+0000","last_fullsized":"2023-11-24T06:28:21.742130+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:16.647539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.772837+0000","last_change":"2023-11-24T06:28:19.733959+0000","last_active":"2023-11-24T06:28:20.772837+0000","last_peered":"2023-11-24T06:28:20.772837+0000","last_clean":"2023-11-24T06:28:20.772837+0000","last_became_active":"2023-11-24T06:28:19.733633+0000","last_became_peered":"2023-11-24T06:28:19.733633+0000","last_unstale":"2023-11-24T06:28:20.772837+0000","last_undegraded":"2023-11-24T06:28:20.772837+0000","last_fullsized":"2023-11-24T06:28:20.772837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:07:48.392066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748219+0000","last_change":"2023-11-24T06:28:20.759715+0000","last_active":"2023-11-24T06:28:21.748219+0000","last_peered":"2023-11-24T06:28:21.748219+0000","last_clean":"2023-11-24T06:28:21.748219+0000","last_became_active":"2023-11-24T06:28:20.757687+0000","last_became_peered":"2023-11-24T06:28:20.757687+0000","last_unstale":"2023-11-24T06:28:21.748219+0000","last_undegraded":"2023-11-24T06:28:21.748219+0000","last_fullsized":"2023-11-24T06:28:21.748219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:46.026504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.768459+0000","last_change":"2023-11-24T06:28:19.740944+0000","last_active":"2023-11-24T06:28:20.768459+0000","last_peered":"2023-11-24T06:28:20.768459+0000","last_clean":"2023-11-24T06:28:20.768459+0000","last_became_active":"2023-11-24T06:28:19.740201+0000","last_became_peered":"2023-11-24T06:28:19.740201+0000","last_unstale":"2023-11-24T06:28:20.768459+0000","last_undegraded":"2023-11-24T06:28:20.768459+0000","last_fullsized":"2023-11-24T06:28:20.768459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:07:48.392066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749401+0000","last_change":"2023-11-24T06:28:20.747089+0000","last_active":"2023-11-24T06:28:21.749401+0000","last_peered":"2023-11-24T06:28:21.749401+0000","last_clean":"2023-11-24T06:28:21.749401+0000","last_became_active":"2023-11-24T06:28:20.742220+0000","last_became_peered":"2023-11-24T06:28:20.742220+0000","last_unstale":"2023-11-24T06:28:21.749401+0000","last_undegraded":"2023-11-24T06:28:21.749401+0000","last_fullsized":"2023-11-24T06:28:21.749401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:36.384991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.767921+0000","last_change":"2023-11-24T06:28:19.737597+0000","last_active":"2023-11-24T06:28:20.767921+0000","last_peered":"2023-11-24T06:28:20.767921+0000","last_clean":"2023-11-24T06:28:20.767921+0000","last_became_active":"2023-11-24T06:28:19.737301+0000","last_became_peered":"2023-11-24T06:28:19.737301+0000","last_unstale":"2023-11-24T06:28:20.767921+0000","last_undegraded":"2023-11-24T06:28:20.767921+0000","last_fullsized":"2023-11-24T06:28:20.767921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:07.528212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361176+0000","last_change":"2023-11-24T06:28:20.752076+0000","last_active":"2023-11-24T06:28:22.361176+0000","last_peered":"2023-11-24T06:28:22.361176+0000","last_clean":"2023-11-24T06:28:22.361176+0000","last_became_active":"2023-11-24T06:28:20.751527+0000","last_became_peered":"2023-11-24T06:28:20.751527+0000","last_unstale":"2023-11-24T06:28:22.361176+0000","last_undegraded":"2023-11-24T06:28:22.361176+0000","last_fullsized":"2023-11-24T06:28:22.361176+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:51.904162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.728185+0000","last_change":"2023-11-24T06:28:19.732965+0000","last_active":"2023-11-24T06:28:20.728185+0000","last_peered":"2023-11-24T06:28:20.728185+0000","last_clean":"2023-11-24T06:28:20.728185+0000","last_became_active":"2023-11-24T06:28:19.731006+0000","last_became_peered":"2023-11-24T06:28:19.731006+0000","last_unstale":"2023-11-24T06:28:20.728185+0000","last_undegraded":"2023-11-24T06:28:20.728185+0000","last_fullsized":"2023-11-24T06:28:20.728185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:40.818934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741644+0000","last_change":"2023-11-24T06:28:20.755312+0000","last_active":"2023-11-24T06:28:21.741644+0000","last_peered":"2023-11-24T06:28:21.741644+0000","last_clean":"2023-11-24T06:28:21.741644+0000","last_became_active":"2023-11-24T06:28:20.753599+0000","last_became_peered":"2023-11-24T06:28:20.753599+0000","last_unstale":"2023-11-24T06:28:21.741644+0000","last_undegraded":"2023-11-24T06:28:21.741644+0000","last_fullsized":"2023-11-24T06:28:21.741644+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:41.967768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.734014+0000","last_change":"2023-11-24T06:28:19.740715+0000","last_active":"2023-11-24T06:28:20.734014+0000","last_peered":"2023-11-24T06:28:20.734014+0000","last_clean":"2023-11-24T06:28:20.734014+0000","last_became_active":"2023-11-24T06:28:19.738599+0000","last_became_peered":"2023-11-24T06:28:19.738599+0000","last_unstale":"2023-11-24T06:28:20.734014+0000","last_undegraded":"2023-11-24T06:28:20.734014+0000","last_fullsized":"2023-11-24T06:28:20.734014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:02.155451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749992+0000","last_change":"2023-11-24T06:28:20.741845+0000","last_active":"2023-11-24T06:28:21.749992+0000","last_peered":"2023-11-24T06:28:21.749992+0000","last_clean":"2023-11-24T06:28:21.749992+0000","last_became_active":"2023-11-24T06:28:20.741520+0000","last_became_peered":"2023-11-24T06:28:20.741520+0000","last_unstale":"2023-11-24T06:28:21.749992+0000","last_undegraded":"2023-11-24T06:28:21.749992+0000","last_fullsized":"2023-11-24T06:28:21.749992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:00.075286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.729679+0000","last_change":"2023-11-24T06:28:19.742211+0000","last_active":"2023-11-24T06:28:20.729679+0000","last_peered":"2023-11-24T06:28:20.729679+0000","last_clean":"2023-11-24T06:28:20.729679+0000","last_became_active":"2023-11-24T06:28:19.741525+0000","last_became_peered":"2023-11-24T06:28:19.741525+0000","last_unstale":"2023-11-24T06:28:20.729679+0000","last_undegraded":"2023-11-24T06:28:20.729679+0000","last_fullsized":"2023-11-24T06:28:20.729679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:17.292179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363780+0000","last_change":"2023-11-24T06:28:20.752528+0000","last_active":"2023-11-24T06:28:22.363780+0000","last_peered":"2023-11-24T06:28:22.363780+0000","last_clean":"2023-11-24T06:28:22.363780+0000","last_became_active":"2023-11-24T06:28:20.752084+0000","last_became_peered":"2023-11-24T06:28:20.752084+0000","last_unstale":"2023-11-24T06:28:22.363780+0000","last_undegraded":"2023-11-24T06:28:22.363780+0000","last_fullsized":"2023-11-24T06:28:22.363780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:21.555443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.723909+0000","last_change":"2023-11-24T06:28:19.735685+0000","last_active":"2023-11-24T06:28:20.723909+0000","last_peered":"2023-11-24T06:28:20.723909+0000","last_clean":"2023-11-24T06:28:20.723909+0000","last_became_active":"2023-11-24T06:28:19.735419+0000","last_became_peered":"2023-11-24T06:28:19.735419+0000","last_unstale":"2023-11-24T06:28:20.723909+0000","last_undegraded":"2023-11-24T06:28:20.723909+0000","last_fullsized":"2023-11-24T06:28:20.723909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:07.528212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364602+0000","last_change":"2023-11-24T06:28:20.748284+0000","last_active":"2023-11-24T06:28:22.364602+0000","last_peered":"2023-11-24T06:28:22.364602+0000","last_clean":"2023-11-24T06:28:22.364602+0000","last_became_active":"2023-11-24T06:28:20.748000+0000","last_became_peered":"2023-11-24T06:28:20.748000+0000","last_unstale":"2023-11-24T06:28:22.364602+0000","last_undegraded":"2023-11-24T06:28:22.364602+0000","last_fullsized":"2023-11-24T06:28:22.364602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:23.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.770416+0000","last_change":"2023-11-24T06:28:19.742398+0000","last_active":"2023-11-24T06:28:20.770416+0000","last_peered":"2023-11-24T06:28:20.770416+0000","last_clean":"2023-11-24T06:28:20.770416+0000","last_became_active":"2023-11-24T06:28:19.739648+0000","last_became_peered":"2023-11-24T06:28:19.739648+0000","last_unstale":"2023-11-24T06:28:20.770416+0000","last_undegraded":"2023-11-24T06:28:20.770416+0000","last_fullsized":"2023-11-24T06:28:20.770416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:40.474217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749587+0000","last_change":"2023-11-24T06:28:20.745174+0000","last_active":"2023-11-24T06:28:21.749587+0000","last_peered":"2023-11-24T06:28:21.749587+0000","last_clean":"2023-11-24T06:28:21.749587+0000","last_became_active":"2023-11-24T06:28:20.744317+0000","last_became_peered":"2023-11-24T06:28:20.744317+0000","last_unstale":"2023-11-24T06:28:21.749587+0000","last_undegraded":"2023-11-24T06:28:21.749587+0000","last_fullsized":"2023-11-24T06:28:21.749587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:23.123298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.776450+0000","last_change":"2023-11-24T06:28:19.732266+0000","last_active":"2023-11-24T06:28:20.776450+0000","last_peered":"2023-11-24T06:28:20.776450+0000","last_clean":"2023-11-24T06:28:20.776450+0000","last_became_active":"2023-11-24T06:28:19.731925+0000","last_became_peered":"2023-11-24T06:28:19.731925+0000","last_unstale":"2023-11-24T06:28:20.776450+0000","last_undegraded":"2023-11-24T06:28:20.776450+0000","last_fullsized":"2023-11-24T06:28:20.776450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:47.574860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741618+0000","last_change":"2023-11-24T06:28:20.749699+0000","last_active":"2023-11-24T06:28:21.741618+0000","last_peered":"2023-11-24T06:28:21.741618+0000","last_clean":"2023-11-24T06:28:21.741618+0000","last_became_active":"2023-11-24T06:28:20.749273+0000","last_became_peered":"2023-11-24T06:28:20.749273+0000","last_unstale":"2023-11-24T06:28:21.741618+0000","last_undegraded":"2023-11-24T06:28:21.741618+0000","last_fullsized":"2023-11-24T06:28:21.741618+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:23.869661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.769745+0000","last_change":"2023-11-24T06:28:19.740983+0000","last_active":"2023-11-24T06:28:20.769745+0000","last_peered":"2023-11-24T06:28:20.769745+0000","last_clean":"2023-11-24T06:28:20.769745+0000","last_became_active":"2023-11-24T06:28:19.740647+0000","last_became_peered":"2023-11-24T06:28:19.740647+0000","last_unstale":"2023-11-24T06:28:20.769745+0000","last_undegraded":"2023-11-24T06:28:20.769745+0000","last_fullsized":"2023-11-24T06:28:20.769745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:11:28.316281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363921+0000","last_change":"2023-11-24T06:28:20.758815+0000","last_active":"2023-11-24T06:28:22.363921+0000","last_peered":"2023-11-24T06:28:22.363921+0000","last_clean":"2023-11-24T06:28:22.363921+0000","last_became_active":"2023-11-24T06:28:20.757556+0000","last_became_peered":"2023-11-24T06:28:20.757556+0000","last_unstale":"2023-11-24T06:28:22.363921+0000","last_undegraded":"2023-11-24T06:28:22.363921+0000","last_fullsized":"2023-11-24T06:28:22.363921+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:52.033759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.739778+0000","last_change":"2023-11-24T06:28:19.741817+0000","last_active":"2023-11-24T06:28:20.739778+0000","last_peered":"2023-11-24T06:28:20.739778+0000","last_clean":"2023-11-24T06:28:20.739778+0000","last_became_active":"2023-11-24T06:28:19.741553+0000","last_became_peered":"2023-11-24T06:28:19.741553+0000","last_unstale":"2023-11-24T06:28:20.739778+0000","last_undegraded":"2023-11-24T06:28:20.739778+0000","last_fullsized":"2023-11-24T06:28:20.739778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:07.764462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748482+0000","last_change":"2023-11-24T06:28:20.757476+0000","last_active":"2023-11-24T06:28:21.748482+0000","last_peered":"2023-11-24T06:28:21.748482+0000","last_clean":"2023-11-24T06:28:21.748482+0000","last_became_active":"2023-11-24T06:28:20.757215+0000","last_became_peered":"2023-11-24T06:28:20.757215+0000","last_unstale":"2023-11-24T06:28:21.748482+0000","last_undegraded":"2023-11-24T06:28:21.748482+0000","last_fullsized":"2023-11-24T06:28:21.748482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:00.075286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.727760+0000","last_change":"2023-11-24T06:28:19.737019+0000","last_active":"2023-11-24T06:28:20.727760+0000","last_peered":"2023-11-24T06:28:20.727760+0000","last_clean":"2023-11-24T06:28:20.727760+0000","last_became_active":"2023-11-24T06:28:19.736587+0000","last_became_peered":"2023-11-24T06:28:19.736587+0000","last_unstale":"2023-11-24T06:28:20.727760+0000","last_undegraded":"2023-11-24T06:28:20.727760+0000","last_fullsized":"2023-11-24T06:28:20.727760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:34.484876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361269+0000","last_change":"2023-11-24T06:28:20.756359+0000","last_active":"2023-11-24T06:28:22.361269+0000","last_peered":"2023-11-24T06:28:22.361269+0000","last_clean":"2023-11-24T06:28:22.361269+0000","last_became_active":"2023-11-24T06:28:20.756086+0000","last_became_peered":"2023-11-24T06:28:20.756086+0000","last_unstale":"2023-11-24T06:28:22.361269+0000","last_undegraded":"2023-11-24T06:28:22.361269+0000","last_fullsized":"2023-11-24T06:28:22.361269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:19.935685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.774855+0000","last_change":"2023-11-24T06:28:19.728784+0000","last_active":"2023-11-24T06:28:20.774855+0000","last_peered":"2023-11-24T06:28:20.774855+0000","last_clean":"2023-11-24T06:28:20.774855+0000","last_became_active":"2023-11-24T06:28:19.728379+0000","last_became_peered":"2023-11-24T06:28:19.728379+0000","last_unstale":"2023-11-24T06:28:20.774855+0000","last_undegraded":"2023-11-24T06:28:20.774855+0000","last_fullsized":"2023-11-24T06:28:20.774855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:35.371421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363620+0000","last_change":"2023-11-24T06:28:20.748159+0000","last_active":"2023-11-24T06:28:22.363620+0000","last_peered":"2023-11-24T06:28:22.363620+0000","last_clean":"2023-11-24T06:28:22.363620+0000","last_became_active":"2023-11-24T06:28:20.747300+0000","last_became_peered":"2023-11-24T06:28:20.747300+0000","last_unstale":"2023-11-24T06:28:22.363620+0000","last_undegraded":"2023-11-24T06:28:22.363620+0000","last_fullsized":"2023-11-24T06:28:22.363620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:39.039302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.773830+0000","last_change":"2023-11-24T06:28:19.738232+0000","last_active":"2023-11-24T06:28:20.773830+0000","last_peered":"2023-11-24T06:28:20.773830+0000","last_clean":"2023-11-24T06:28:20.773830+0000","last_became_active":"2023-11-24T06:28:19.737493+0000","last_became_peered":"2023-11-24T06:28:19.737493+0000","last_unstale":"2023-11-24T06:28:20.773830+0000","last_undegraded":"2023-11-24T06:28:20.773830+0000","last_fullsized":"2023-11-24T06:28:20.773830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:38.025733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.750413+0000","last_change":"2023-11-24T06:28:20.752165+0000","last_active":"2023-11-24T06:28:21.750413+0000","last_peered":"2023-11-24T06:28:21.750413+0000","last_clean":"2023-11-24T06:28:21.750413+0000","last_became_active":"2023-11-24T06:28:20.751192+0000","last_became_peered":"2023-11-24T06:28:20.751192+0000","last_unstale":"2023-11-24T06:28:21.750413+0000","last_undegraded":"2023-11-24T06:28:21.750413+0000","last_fullsized":"2023-11-24T06:28:21.750413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:52.718291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.769292+0000","last_change":"2023-11-24T06:28:19.740458+0000","last_active":"2023-11-24T06:28:20.769292+0000","last_peered":"2023-11-24T06:28:20.769292+0000","last_clean":"2023-11-24T06:28:20.769292+0000","last_became_active":"2023-11-24T06:28:19.740072+0000","last_became_peered":"2023-11-24T06:28:19.740072+0000","last_unstale":"2023-11-24T06:28:20.769292+0000","last_undegraded":"2023-11-24T06:28:20.769292+0000","last_fullsized":"2023-11-24T06:28:20.769292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:14.621798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741778+0000","last_change":"2023-11-24T06:28:20.754060+0000","last_active":"2023-11-24T06:28:21.741778+0000","last_peered":"2023-11-24T06:28:21.741778+0000","last_clean":"2023-11-24T06:28:21.741778+0000","last_became_active":"2023-11-24T06:28:20.753777+0000","last_became_peered":"2023-11-24T06:28:20.753777+0000","last_unstale":"2023-11-24T06:28:21.741778+0000","last_undegraded":"2023-11-24T06:28:21.741778+0000","last_fullsized":"2023-11-24T06:28:21.741778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:29.453645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.767249+0000","last_change":"2023-11-24T06:28:19.742059+0000","last_active":"2023-11-24T06:28:20.767249+0000","last_peered":"2023-11-24T06:28:20.767249+0000","last_clean":"2023-11-24T06:28:20.767249+0000","last_became_active":"2023-11-24T06:28:19.741784+0000","last_became_peered":"2023-11-24T06:28:19.741784+0000","last_unstale":"2023-11-24T06:28:20.767249+0000","last_undegraded":"2023-11-24T06:28:20.767249+0000","last_fullsized":"2023-11-24T06:28:20.767249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:40.154924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361968+0000","last_change":"2023-11-24T06:28:20.749842+0000","last_active":"2023-11-24T06:28:22.361968+0000","last_peered":"2023-11-24T06:28:22.361968+0000","last_clean":"2023-11-24T06:28:22.361968+0000","last_became_active":"2023-11-24T06:28:20.749578+0000","last_became_peered":"2023-11-24T06:28:20.749578+0000","last_unstale":"2023-11-24T06:28:22.361968+0000","last_undegraded":"2023-11-24T06:28:22.361968+0000","last_fullsized":"2023-11-24T06:28:22.361968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:30.205190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.735299+0000","last_change":"2023-11-24T06:28:19.736327+0000","last_active":"2023-11-24T06:28:20.735299+0000","last_peered":"2023-11-24T06:28:20.735299+0000","last_clean":"2023-11-24T06:28:20.735299+0000","last_became_active":"2023-11-24T06:28:19.736017+0000","last_became_peered":"2023-11-24T06:28:19.736017+0000","last_unstale":"2023-11-24T06:28:20.735299+0000","last_undegraded":"2023-11-24T06:28:20.735299+0000","last_fullsized":"2023-11-24T06:28:20.735299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:05.916062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.360985+0000","last_change":"2023-11-24T06:28:20.749162+0000","last_active":"2023-11-24T06:28:22.360985+0000","last_peered":"2023-11-24T06:28:22.360985+0000","last_clean":"2023-11-24T06:28:22.360985+0000","last_became_active":"2023-11-24T06:28:20.748814+0000","last_became_peered":"2023-11-24T06:28:20.748814+0000","last_unstale":"2023-11-24T06:28:22.360985+0000","last_undegraded":"2023-11-24T06:28:22.360985+0000","last_fullsized":"2023-11-24T06:28:22.360985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:15.635368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.771526+0000","last_change":"2023-11-24T06:28:19.732595+0000","last_active":"2023-11-24T06:28:20.771526+0000","last_peered":"2023-11-24T06:28:20.771526+0000","last_clean":"2023-11-24T06:28:20.771526+0000","last_became_active":"2023-11-24T06:28:19.732320+0000","last_became_peered":"2023-11-24T06:28:19.732320+0000","last_unstale":"2023-11-24T06:28:20.771526+0000","last_undegraded":"2023-11-24T06:28:20.771526+0000","last_fullsized":"2023-11-24T06:28:20.771526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:50.890592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361338+0000","last_change":"2023-11-24T06:28:20.742637+0000","last_active":"2023-11-24T06:28:22.361338+0000","last_peered":"2023-11-24T06:28:22.361338+0000","last_clean":"2023-11-24T06:28:22.361338+0000","last_became_active":"2023-11-24T06:28:20.742277+0000","last_became_peered":"2023-11-24T06:28:20.742277+0000","last_unstale":"2023-11-24T06:28:22.361338+0000","last_undegraded":"2023-11-24T06:28:22.361338+0000","last_fullsized":"2023-11-24T06:28:22.361338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:57.469275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.736299+0000","last_change":"2023-11-24T06:28:19.745300+0000","last_active":"2023-11-24T06:28:20.736299+0000","last_peered":"2023-11-24T06:28:20.736299+0000","last_clean":"2023-11-24T06:28:20.736299+0000","last_became_active":"2023-11-24T06:28:19.744997+0000","last_became_peered":"2023-11-24T06:28:19.744997+0000","last_unstale":"2023-11-24T06:28:20.736299+0000","last_undegraded":"2023-11-24T06:28:20.736299+0000","last_fullsized":"2023-11-24T06:28:20.736299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:05.916062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361396+0000","last_change":"2023-11-24T06:28:20.746645+0000","last_active":"2023-11-24T06:28:22.361396+0000","last_peered":"2023-11-24T06:28:22.361396+0000","last_clean":"2023-11-24T06:28:22.361396+0000","last_became_active":"2023-11-24T06:28:20.742228+0000","last_became_peered":"2023-11-24T06:28:20.742228+0000","last_unstale":"2023-11-24T06:28:22.361396+0000","last_undegraded":"2023-11-24T06:28:22.361396+0000","last_fullsized":"2023-11-24T06:28:22.361396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:03.169021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.728527+0000","last_change":"2023-11-24T06:28:19.741555+0000","last_active":"2023-11-24T06:28:20.728527+0000","last_peered":"2023-11-24T06:28:20.728527+0000","last_clean":"2023-11-24T06:28:20.728527+0000","last_became_active":"2023-11-24T06:28:19.739895+0000","last_became_peered":"2023-11-24T06:28:19.739895+0000","last_unstale":"2023-11-24T06:28:20.728527+0000","last_undegraded":"2023-11-24T06:28:20.728527+0000","last_fullsized":"2023-11-24T06:28:20.728527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:55:19.561292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747734+0000","last_change":"2023-11-24T06:28:20.750118+0000","last_active":"2023-11-24T06:28:21.747734+0000","last_peered":"2023-11-24T06:28:21.747734+0000","last_clean":"2023-11-24T06:28:21.747734+0000","last_became_active":"2023-11-24T06:28:20.749804+0000","last_became_peered":"2023-11-24T06:28:20.749804+0000","last_unstale":"2023-11-24T06:28:21.747734+0000","last_undegraded":"2023-11-24T06:28:21.747734+0000","last_fullsized":"2023-11-24T06:28:21.747734+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:48.588430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.726227+0000","last_change":"2023-11-24T06:28:19.739558+0000","last_active":"2023-11-24T06:28:20.726227+0000","last_peered":"2023-11-24T06:28:20.726227+0000","last_clean":"2023-11-24T06:28:20.726227+0000","last_became_active":"2023-11-24T06:28:19.736344+0000","last_became_peered":"2023-11-24T06:28:19.736344+0000","last_unstale":"2023-11-24T06:28:20.726227+0000","last_undegraded":"2023-11-24T06:28:20.726227+0000","last_fullsized":"2023-11-24T06:28:20.726227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:40.954198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361860+0000","last_change":"2023-11-24T06:28:20.745248+0000","last_active":"2023-11-24T06:28:22.361860+0000","last_peered":"2023-11-24T06:28:22.361860+0000","last_clean":"2023-11-24T06:28:22.361860+0000","last_became_active":"2023-11-24T06:28:20.744949+0000","last_became_peered":"2023-11-24T06:28:20.744949+0000","last_unstale":"2023-11-24T06:28:22.361860+0000","last_undegraded":"2023-11-24T06:28:22.361860+0000","last_fullsized":"2023-11-24T06:28:22.361860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:49.338938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.769325+0000","last_change":"2023-11-24T06:28:19.739351+0000","last_active":"2023-11-24T06:28:20.769325+0000","last_peered":"2023-11-24T06:28:20.769325+0000","last_clean":"2023-11-24T06:28:20.769325+0000","last_became_active":"2023-11-24T06:28:19.737557+0000","last_became_peered":"2023-11-24T06:28:19.737557+0000","last_unstale":"2023-11-24T06:28:20.769325+0000","last_undegraded":"2023-11-24T06:28:20.769325+0000","last_fullsized":"2023-11-24T06:28:20.769325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:20.569170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.750531+0000","last_change":"2023-11-24T06:28:20.756413+0000","last_active":"2023-11-24T06:28:21.750531+0000","last_peered":"2023-11-24T06:28:21.750531+0000","last_clean":"2023-11-24T06:28:21.750531+0000","last_became_active":"2023-11-24T06:28:20.756144+0000","last_became_peered":"2023-11-24T06:28:20.756144+0000","last_unstale":"2023-11-24T06:28:21.750531+0000","last_undegraded":"2023-11-24T06:28:21.750531+0000","last_fullsized":"2023-11-24T06:28:21.750531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:21.555443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.723507+0000","last_change":"2023-11-24T06:28:19.733326+0000","last_active":"2023-11-24T06:28:20.723507+0000","last_peered":"2023-11-24T06:28:20.723507+0000","last_clean":"2023-11-24T06:28:20.723507+0000","last_became_active":"2023-11-24T06:28:19.733070+0000","last_became_peered":"2023-11-24T06:28:19.733070+0000","last_unstale":"2023-11-24T06:28:20.723507+0000","last_undegraded":"2023-11-24T06:28:20.723507+0000","last_fullsized":"2023-11-24T06:28:20.723507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:05.916062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361892+0000","last_change":"2023-11-24T06:28:20.741782+0000","last_active":"2023-11-24T06:28:22.361892+0000","last_peered":"2023-11-24T06:28:22.361892+0000","last_clean":"2023-11-24T06:28:22.361892+0000","last_became_active":"2023-11-24T06:28:20.741508+0000","last_became_peered":"2023-11-24T06:28:20.741508+0000","last_unstale":"2023-11-24T06:28:22.361892+0000","last_undegraded":"2023-11-24T06:28:22.361892+0000","last_fullsized":"2023-11-24T06:28:22.361892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:12.693448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.740853+0000","last_change":"2023-11-24T06:28:19.741328+0000","last_active":"2023-11-24T06:28:20.740853+0000","last_peered":"2023-11-24T06:28:20.740853+0000","last_clean":"2023-11-24T06:28:20.740853+0000","last_became_active":"2023-11-24T06:28:19.740211+0000","last_became_peered":"2023-11-24T06:28:19.740211+0000","last_unstale":"2023-11-24T06:28:20.740853+0000","last_undegraded":"2023-11-24T06:28:20.740853+0000","last_fullsized":"2023-11-24T06:28:20.740853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:11.679878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361842+0000","last_change":"2023-11-24T06:28:20.746488+0000","last_active":"2023-11-24T06:28:22.361842+0000","last_peered":"2023-11-24T06:28:22.361842+0000","last_clean":"2023-11-24T06:28:22.361842+0000","last_became_active":"2023-11-24T06:28:20.746196+0000","last_became_peered":"2023-11-24T06:28:20.746196+0000","last_unstale":"2023-11-24T06:28:22.361842+0000","last_undegraded":"2023-11-24T06:28:22.361842+0000","last_fullsized":"2023-11-24T06:28:22.361842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:08.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.724893+0000","last_change":"2023-11-24T06:28:19.738778+0000","last_active":"2023-11-24T06:28:20.724893+0000","last_peered":"2023-11-24T06:28:20.724893+0000","last_clean":"2023-11-24T06:28:20.724893+0000","last_became_active":"2023-11-24T06:28:19.738482+0000","last_became_peered":"2023-11-24T06:28:19.738482+0000","last_unstale":"2023-11-24T06:28:20.724893+0000","last_undegraded":"2023-11-24T06:28:20.724893+0000","last_fullsized":"2023-11-24T06:28:20.724893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:18.922115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363974+0000","last_change":"2023-11-24T06:28:20.746401+0000","last_active":"2023-11-24T06:28:22.363974+0000","last_peered":"2023-11-24T06:28:22.363974+0000","last_clean":"2023-11-24T06:28:22.363974+0000","last_became_active":"2023-11-24T06:28:20.745992+0000","last_became_peered":"2023-11-24T06:28:20.745992+0000","last_unstale":"2023-11-24T06:28:22.363974+0000","last_undegraded":"2023-11-24T06:28:22.363974+0000","last_fullsized":"2023-11-24T06:28:22.363974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:21.582740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.739931+0000","last_change":"2023-11-24T06:28:19.737632+0000","last_active":"2023-11-24T06:28:20.739931+0000","last_peered":"2023-11-24T06:28:20.739931+0000","last_clean":"2023-11-24T06:28:20.739931+0000","last_became_active":"2023-11-24T06:28:19.736632+0000","last_became_peered":"2023-11-24T06:28:19.736632+0000","last_unstale":"2023-11-24T06:28:20.739931+0000","last_undegraded":"2023-11-24T06:28:20.739931+0000","last_fullsized":"2023-11-24T06:28:20.739931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:33.275487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362995+0000","last_change":"2023-11-24T06:28:20.753196+0000","last_active":"2023-11-24T06:28:22.362995+0000","last_peered":"2023-11-24T06:28:22.362995+0000","last_clean":"2023-11-24T06:28:22.362995+0000","last_became_active":"2023-11-24T06:28:20.752928+0000","last_became_peered":"2023-11-24T06:28:20.752928+0000","last_unstale":"2023-11-24T06:28:22.362995+0000","last_undegraded":"2023-11-24T06:28:22.362995+0000","last_fullsized":"2023-11-24T06:28:22.362995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:46.026504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.728819+0000","last_change":"2023-11-24T06:28:19.744231+0000","last_active":"2023-11-24T06:28:20.728819+0000","last_peered":"2023-11-24T06:28:20.728819+0000","last_clean":"2023-11-24T06:28:20.728819+0000","last_became_active":"2023-11-24T06:28:19.743719+0000","last_became_peered":"2023-11-24T06:28:19.743719+0000","last_unstale":"2023-11-24T06:28:20.728819+0000","last_undegraded":"2023-11-24T06:28:20.728819+0000","last_fullsized":"2023-11-24T06:28:20.728819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:44:59.965561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361501+0000","last_change":"2023-11-24T06:28:20.742551+0000","last_active":"2023-11-24T06:28:22.361501+0000","last_peered":"2023-11-24T06:28:22.361501+0000","last_clean":"2023-11-24T06:28:22.361501+0000","last_became_active":"2023-11-24T06:28:20.742211+0000","last_became_peered":"2023-11-24T06:28:20.742211+0000","last_unstale":"2023-11-24T06:28:22.361501+0000","last_undegraded":"2023-11-24T06:28:22.361501+0000","last_fullsized":"2023-11-24T06:28:22.361501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:19.935685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.726724+0000","last_change":"2023-11-24T06:28:19.737801+0000","last_active":"2023-11-24T06:28:20.726724+0000","last_peered":"2023-11-24T06:28:20.726724+0000","last_clean":"2023-11-24T06:28:20.726724+0000","last_became_active":"2023-11-24T06:28:19.736632+0000","last_became_peered":"2023-11-24T06:28:19.736632+0000","last_unstale":"2023-11-24T06:28:20.726724+0000","last_undegraded":"2023-11-24T06:28:20.726724+0000","last_fullsized":"2023-11-24T06:28:20.726724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:15.633969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748625+0000","last_change":"2023-11-24T06:28:20.748776+0000","last_active":"2023-11-24T06:28:21.748625+0000","last_peered":"2023-11-24T06:28:21.748625+0000","last_clean":"2023-11-24T06:28:21.748625+0000","last_became_active":"2023-11-24T06:28:20.745731+0000","last_became_peered":"2023-11-24T06:28:20.745731+0000","last_unstale":"2023-11-24T06:28:21.748625+0000","last_undegraded":"2023-11-24T06:28:21.748625+0000","last_fullsized":"2023-11-24T06:28:21.748625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:14.252801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.736666+0000","last_change":"2023-11-24T06:28:19.745039+0000","last_active":"2023-11-24T06:28:20.736666+0000","last_peered":"2023-11-24T06:28:20.736666+0000","last_clean":"2023-11-24T06:28:20.736666+0000","last_became_active":"2023-11-24T06:28:19.744548+0000","last_became_peered":"2023-11-24T06:28:19.744548+0000","last_unstale":"2023-11-24T06:28:20.736666+0000","last_undegraded":"2023-11-24T06:28:20.736666+0000","last_fullsized":"2023-11-24T06:28:20.736666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:07.528212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.742551+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:28:20.742551+0000","last_peered":"2023-11-24T06:28:20.742551+0000","last_clean":"2023-11-24T06:28:20.742551+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:28:20.742551+0000","last_undegraded":"2023-11-24T06:28:20.742551+0000","last_fullsized":"2023-11-24T06:28:20.742551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":742397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.6819999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":6.3319999999999999,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":0.86699999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.3380000000000001,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.613}]},{"osd":3,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.474},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.3640000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":1.286}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.522},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":6.7359999999999998,"5min":6.7359999999999998,"15min":6.7359999999999998},"last":1.762}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.905,"15min":1.905},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":7.6449999999999996,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.423},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":1.1359999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.85699999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.472},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.5649999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.3859999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.5},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.6259999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.109},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":0.98899999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.044},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.774,"5min":3.774,"15min":3.774},"last":1.5960000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":3.649,"5min":3.649,"15min":3.649},"last":1.177}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":1.885},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":2.4039999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.6819999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.992},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.776}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.518},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.617}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.554,"15min":1.554},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.085,"5min":3.085,"15min":3.085},"last":1.1299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":2.0470000000000002},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":0.96099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.724},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":2.157}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.526},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.0489999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.4610000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.194},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.27},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.3680000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5950000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.581,"15min":1.581},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.52,"5min":3.52,"15min":3.52},"last":1.49}]},{"osd":6,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.95299999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":1.3049999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":153573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.89100000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.752},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.141}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.25},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.5910000000000002,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.089}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.081,"5min":3.081,"15min":3.081},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.6960000000000002,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":8.8879999999999999,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.202},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.641}]},{"osd":7,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":1.714},{"interface":"front","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":1.3}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":152009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.161}]},{"osd":1,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.278}]},{"osd":3,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.597}]},{"osd":4,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.446},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.6899999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":3.7410000000000001,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.367},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.327}]},{"osd":6,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.49},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":4.1210000000000004,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.546}]},{"osd":7,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.8119999999999998,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.6559999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":153573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.702}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.97599999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.212},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.5149999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.042}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.823},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":1.464}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.27},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.9279999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":742307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.52800000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":0.97099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.93700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:30.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:28:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.901+0000 7f06aa4b4640 1 Processor -- start 2023-11-24T06:28:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.902+0000 7f06aa4b4640 1 -- start start 2023-11-24T06:28:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.903+0000 7f06aa4b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06a40cf9a0 0x7f06a415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.903+0000 7f06aa4b4640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f06a415e1e0 0x7f06a40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.903+0000 7f06aa4b4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a40b1ee0 con 0x7f06a4143ae0 2023-11-24T06:28:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.903+0000 7f06aa4b4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40b2780 con 0x7f06a40cf9a0 2023-11-24T06:28:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.903+0000 7f06aa4b4640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a415e990 con 0x7f06a415e1e0 2023-11-24T06:28:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.903+0000 7f069b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06a40cf9a0 0x7f06a415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.903+0000 7f06a3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f06a4143ae0 0x7f06a40cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60766/0 (socket says 172.21.15.16:60766) 2023-11-24T06:28:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.903+0000 7f06a3fff640 1 -- 172.21.15.16:0/3990262915 learned_addr learned my addr 172.21.15.16:0/3990262915 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.904+0000 7f06a8a51640 1 --2- 172.21.15.16:0/3990262915 >> v2:172.21.15.173:3300/0 conn(0x7f06a415e1e0 0x7f06a40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.904+0000 7f069b7fe640 1 -- 172.21.15.16:0/3990262915 >> v2:172.21.15.173:3300/0 conn(0x7f06a415e1e0 msgr2=0x7f06a40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.904+0000 7f069b7fe640 1 --2- 172.21.15.16:0/3990262915 >> v2:172.21.15.173:3300/0 conn(0x7f06a415e1e0 0x7f06a40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.904+0000 7f069b7fe640 1 -- 172.21.15.16:0/3990262915 >> v1:172.21.15.16:6789/0 conn(0x7f06a4143ae0 legacy=0x7f06a40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.904+0000 7f069b7fe640 1 -- 172.21.15.16:0/3990262915 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a40b2130 con 0x7f06a40cf9a0 2023-11-24T06:28:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.905+0000 7f069b7fe640 1 --2- 172.21.15.16:0/3990262915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a40cf9a0 0x7f06a415dca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f06a40ae9d0 tx=0x7f068c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fda0e7b2a85317 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.906+0000 7f06a37fe640 1 -- 172.21.15.16:0/3990262915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c02cc70 con 0x7f06a40cf9a0 2023-11-24T06:28:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.906+0000 7f06a37fe640 1 -- 172.21.15.16:0/3990262915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c02cdd0 con 0x7f06a40cf9a0 2023-11-24T06:28:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.906+0000 7f06a37fe640 1 -- 172.21.15.16:0/3990262915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c034660 con 0x7f06a40cf9a0 2023-11-24T06:28:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.906+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3990262915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a40cf9a0 msgr2=0x7f06a415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.906+0000 7f06aa4b4640 1 --2- 172.21.15.16:0/3990262915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a40cf9a0 0x7f06a415dca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f06a40ae9d0 tx=0x7f068c02bec0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.906+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3990262915 shutdown_connections 2023-11-24T06:28:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.906+0000 7f06aa4b4640 1 --2- 172.21.15.16:0/3990262915 >> v2:172.21.15.173:3300/0 conn(0x7f06a415e1e0 0x7f06a40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.906+0000 7f06aa4b4640 1 --2- 172.21.15.16:0/3990262915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a40cf9a0 0x7f06a415dca0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.906+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3990262915 >> 172.21.15.16:0/3990262915 conn(0x7f06a406a800 msgr2=0x7f06a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.907+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3990262915 shutdown_connections 2023-11-24T06:28:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.907+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3990262915 wait complete. 2023-11-24T06:28:30.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.908+0000 7f06aa4b4640 1 Processor -- start 2023-11-24T06:28:30.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.908+0000 7f06aa4b4640 1 -- start start 2023-11-24T06:28:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.909+0000 7f06aa4b4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06a40cf9a0 0x7f06a40ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.909+0000 7f06aa4b4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06a4143ae0 0x7f06a40cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.909+0000 7f06aa4b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a415e1e0 0x7f06a4159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.909+0000 7f06aa4b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a4141a60 con 0x7f06a415e1e0 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.909+0000 7f06aa4b4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a4141780 con 0x7f06a40cf9a0 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.909+0000 7f06aa4b4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a4140220 con 0x7f06a4143ae0 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a8a51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a415e1e0 0x7f06a4159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a8a51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a415e1e0 0x7f06a4159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50380/0 (socket says 172.21.15.16:50380) 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a8a51640 1 -- 172.21.15.16:0/3192502687 learned_addr learned my addr 172.21.15.16:0/3192502687 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a3fff640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06a40cf9a0 0x7f06a40ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f069b7fe640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06a4143ae0 0x7f06a40cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a8a51640 1 -- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06a4143ae0 msgr2=0x7f06a40cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a8a51640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06a4143ae0 0x7f06a40cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a8a51640 1 -- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06a40cf9a0 msgr2=0x7f06a40ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a8a51640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06a40cf9a0 0x7f06a40ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a8a51640 1 -- 172.21.15.16:0/3192502687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c009a70 con 0x7f06a415e1e0 2023-11-24T06:28:30.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.910+0000 7f06a8a51640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a415e1e0 0x7f06a4159690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f069c00d800 tx=0x7f069c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:30.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.911+0000 7f06a17fa640 1 -- 172.21.15.16:0/3192502687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c00b6c0 con 0x7f06a415e1e0 2023-11-24T06:28:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.911+0000 7f06a17fa640 1 -- 172.21.15.16:0/3192502687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c00b840 con 0x7f06a415e1e0 2023-11-24T06:28:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.911+0000 7f06a17fa640 1 -- 172.21.15.16:0/3192502687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c00bbf0 con 0x7f06a415e1e0 2023-11-24T06:28:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.911+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a4140500 con 0x7f06a415e1e0 2023-11-24T06:28:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.911+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a40ae630 con 0x7f06a415e1e0 2023-11-24T06:28:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.912+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a4140a20 con 0x7f06a415e1e0 2023-11-24T06:28:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.913+0000 7f06a17fa640 1 -- 172.21.15.16:0/3192502687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f069c00bd50 con 0x7f06a415e1e0 2023-11-24T06:28:30.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.914+0000 7f06a17fa640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f068406ae40 0x7f068406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:30.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.914+0000 7f06a17fa640 1 -- 172.21.15.16:0/3192502687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f069c089c40 con 0x7f06a415e1e0 2023-11-24T06:28:30.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.915+0000 7f06a3fff640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f068406ae40 0x7f068406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:30.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.916+0000 7f06a3fff640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f068406ae40 0x7f068406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0694005880 tx=0x7f0694009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:30.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:30.918+0000 7f06a17fa640 1 -- 172.21.15.16:0/3192502687 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06a4140a20 con 0x7f06a415e1e0 2023-11-24T06:28:31.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.110+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06a415e670 con 0x7f068406ae40 2023-11-24T06:28:31.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.133+0000 7f06a17fa640 1 -- 172.21.15.16:0/3192502687 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286882 (secure 0 0 0) 0x7f0694001c50 con 0x7f068406ae40 2023-11-24T06:28:31.138 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:28:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.140+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f068406ae40 msgr2=0x7f068406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.140+0000 7f06aa4b4640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f068406ae40 0x7f068406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0694005880 tx=0x7f0694009040 comp rx=0 tx=0).stop 2023-11-24T06:28:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a415e1e0 msgr2=0x7f06a4159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a415e1e0 0x7f06a4159690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f069c00d800 tx=0x7f069c00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 shutdown_connections 2023-11-24T06:28:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f068406ae40 0x7f068406d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06a415e1e0 0x7f06a4159690 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06a4143ae0 0x7f06a40cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 --2- 172.21.15.16:0/3192502687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06a40cf9a0 0x7f06a40ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 >> 172.21.15.16:0/3192502687 conn(0x7f06a406a800 msgr2=0x7f06a4070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 shutdown_connections 2023-11-24T06:28:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.141+0000 7f06aa4b4640 1 -- 172.21.15.16:0/3192502687 wait complete. 2023-11-24T06:28:31.167 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2023-11-24T06:28:30.134222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237064,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737656,"statfs":{"total":773094113280,"available":772851359744,"internally_reserved":0,"allocated":4333568,"data_stored":2401068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.420899"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362259+0000","last_change":"2023-11-24T06:28:20.744890+0000","last_active":"2023-11-24T06:28:22.362259+0000","last_peered":"2023-11-24T06:28:22.362259+0000","last_clean":"2023-11-24T06:28:22.362259+0000","last_became_active":"2023-11-24T06:28:20.744570+0000","last_became_peered":"2023-11-24T06:28:20.744570+0000","last_unstale":"2023-11-24T06:28:22.362259+0000","last_undegraded":"2023-11-24T06:28:22.362259+0000","last_fullsized":"2023-11-24T06:28:22.362259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:34.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364147+0000","last_change":"2023-11-24T06:28:20.740082+0000","last_active":"2023-11-24T06:28:22.364147+0000","last_peered":"2023-11-24T06:28:22.364147+0000","last_clean":"2023-11-24T06:28:22.364147+0000","last_became_active":"2023-11-24T06:28:20.739812+0000","last_became_peered":"2023-11-24T06:28:20.739812+0000","last_unstale":"2023-11-24T06:28:22.364147+0000","last_undegraded":"2023-11-24T06:28:22.364147+0000","last_fullsized":"2023-11-24T06:28:22.364147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:36.198976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.750217+0000","last_change":"2023-11-24T06:28:20.745391+0000","last_active":"2023-11-24T06:28:21.750217+0000","last_peered":"2023-11-24T06:28:21.750217+0000","last_clean":"2023-11-24T06:28:21.750217+0000","last_became_active":"2023-11-24T06:28:20.744851+0000","last_became_peered":"2023-11-24T06:28:20.744851+0000","last_unstale":"2023-11-24T06:28:21.750217+0000","last_undegraded":"2023-11-24T06:28:21.750217+0000","last_fullsized":"2023-11-24T06:28:21.750217+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:14.252801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741692+0000","last_change":"2023-11-24T06:28:20.758056+0000","last_active":"2023-11-24T06:28:21.741692+0000","last_peered":"2023-11-24T06:28:21.741692+0000","last_clean":"2023-11-24T06:28:21.741692+0000","last_became_active":"2023-11-24T06:28:20.757439+0000","last_became_peered":"2023-11-24T06:28:20.757439+0000","last_unstale":"2023-11-24T06:28:21.741692+0000","last_undegraded":"2023-11-24T06:28:21.741692+0000","last_fullsized":"2023-11-24T06:28:21.741692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:44:14.949226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.360988+0000","last_change":"2023-11-24T06:28:20.738890+0000","last_active":"2023-11-24T06:28:22.360988+0000","last_peered":"2023-11-24T06:28:22.360988+0000","last_clean":"2023-11-24T06:28:22.360988+0000","last_became_active":"2023-11-24T06:28:20.738441+0000","last_became_peered":"2023-11-24T06:28:20.738441+0000","last_unstale":"2023-11-24T06:28:22.360988+0000","last_undegraded":"2023-11-24T06:28:22.360988+0000","last_fullsized":"2023-11-24T06:28:22.360988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:26.380103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363652+0000","last_change":"2023-11-24T06:28:20.744786+0000","last_active":"2023-11-24T06:28:22.363652+0000","last_peered":"2023-11-24T06:28:22.363652+0000","last_clean":"2023-11-24T06:28:22.363652+0000","last_became_active":"2023-11-24T06:28:20.744487+0000","last_became_peered":"2023-11-24T06:28:20.744487+0000","last_unstale":"2023-11-24T06:28:22.363652+0000","last_undegraded":"2023-11-24T06:28:22.363652+0000","last_fullsized":"2023-11-24T06:28:22.363652+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:14.252801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748125+0000","last_change":"2023-11-24T06:28:20.753428+0000","last_active":"2023-11-24T06:28:21.748125+0000","last_peered":"2023-11-24T06:28:21.748125+0000","last_clean":"2023-11-24T06:28:21.748125+0000","last_became_active":"2023-11-24T06:28:20.750476+0000","last_became_peered":"2023-11-24T06:28:20.750476+0000","last_unstale":"2023-11-24T06:28:21.748125+0000","last_undegraded":"2023-11-24T06:28:21.748125+0000","last_fullsized":"2023-11-24T06:28:21.748125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:18.305749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747710+0000","last_change":"2023-11-24T06:28:20.750080+0000","last_active":"2023-11-24T06:28:21.747710+0000","last_peered":"2023-11-24T06:28:21.747710+0000","last_clean":"2023-11-24T06:28:21.747710+0000","last_became_active":"2023-11-24T06:28:20.749694+0000","last_became_peered":"2023-11-24T06:28:20.749694+0000","last_unstale":"2023-11-24T06:28:21.747710+0000","last_undegraded":"2023-11-24T06:28:21.747710+0000","last_fullsized":"2023-11-24T06:28:21.747710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:41.832504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363811+0000","last_change":"2023-11-24T06:28:20.760441+0000","last_active":"2023-11-24T06:28:22.363811+0000","last_peered":"2023-11-24T06:28:22.363811+0000","last_clean":"2023-11-24T06:28:22.363811+0000","last_became_active":"2023-11-24T06:28:20.758860+0000","last_became_peered":"2023-11-24T06:28:20.758860+0000","last_unstale":"2023-11-24T06:28:22.363811+0000","last_undegraded":"2023-11-24T06:28:22.363811+0000","last_fullsized":"2023-11-24T06:28:22.363811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:21.582740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742175+0000","last_change":"2023-11-24T06:28:20.744185+0000","last_active":"2023-11-24T06:28:21.742175+0000","last_peered":"2023-11-24T06:28:21.742175+0000","last_clean":"2023-11-24T06:28:21.742175+0000","last_became_active":"2023-11-24T06:28:20.743866+0000","last_became_peered":"2023-11-24T06:28:20.743866+0000","last_unstale":"2023-11-24T06:28:21.742175+0000","last_undegraded":"2023-11-24T06:28:21.742175+0000","last_fullsized":"2023-11-24T06:28:21.742175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:41.487786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748764+0000","last_change":"2023-11-24T06:28:20.748422+0000","last_active":"2023-11-24T06:28:21.748764+0000","last_peered":"2023-11-24T06:28:21.748764+0000","last_clean":"2023-11-24T06:28:21.748764+0000","last_became_active":"2023-11-24T06:28:20.747375+0000","last_became_peered":"2023-11-24T06:28:20.747375+0000","last_unstale":"2023-11-24T06:28:21.748764+0000","last_undegraded":"2023-11-24T06:28:21.748764+0000","last_fullsized":"2023-11-24T06:28:21.748764+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:21.555443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749684+0000","last_change":"2023-11-24T06:28:20.741727+0000","last_active":"2023-11-24T06:28:21.749684+0000","last_peered":"2023-11-24T06:28:21.749684+0000","last_clean":"2023-11-24T06:28:21.749684+0000","last_became_active":"2023-11-24T06:28:20.741428+0000","last_became_peered":"2023-11-24T06:28:20.741428+0000","last_unstale":"2023-11-24T06:28:21.749684+0000","last_undegraded":"2023-11-24T06:28:21.749684+0000","last_fullsized":"2023-11-24T06:28:21.749684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:18.305749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361169+0000","last_change":"2023-11-24T06:28:20.740157+0000","last_active":"2023-11-24T06:28:22.361169+0000","last_peered":"2023-11-24T06:28:22.361169+0000","last_clean":"2023-11-24T06:28:22.361169+0000","last_became_active":"2023-11-24T06:28:20.739896+0000","last_became_peered":"2023-11-24T06:28:20.739896+0000","last_unstale":"2023-11-24T06:28:22.361169+0000","last_undegraded":"2023-11-24T06:28:22.361169+0000","last_fullsized":"2023-11-24T06:28:22.361169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:41.168494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363756+0000","last_change":"2023-11-24T06:28:20.744520+0000","last_active":"2023-11-24T06:28:22.363756+0000","last_peered":"2023-11-24T06:28:22.363756+0000","last_clean":"2023-11-24T06:28:22.363756+0000","last_became_active":"2023-11-24T06:28:20.744131+0000","last_became_peered":"2023-11-24T06:28:20.744131+0000","last_unstale":"2023-11-24T06:28:22.363756+0000","last_undegraded":"2023-11-24T06:28:22.363756+0000","last_fullsized":"2023-11-24T06:28:22.363756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:52.718291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749170+0000","last_change":"2023-11-24T06:28:20.749484+0000","last_active":"2023-11-24T06:28:21.749170+0000","last_peered":"2023-11-24T06:28:21.749170+0000","last_clean":"2023-11-24T06:28:21.749170+0000","last_became_active":"2023-11-24T06:28:20.749186+0000","last_became_peered":"2023-11-24T06:28:20.749186+0000","last_unstale":"2023-11-24T06:28:21.749170+0000","last_undegraded":"2023-11-24T06:28:21.749170+0000","last_fullsized":"2023-11-24T06:28:21.749170+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:48.588430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748803+0000","last_change":"2023-11-24T06:28:20.755151+0000","last_active":"2023-11-24T06:28:21.748803+0000","last_peered":"2023-11-24T06:28:21.748803+0000","last_clean":"2023-11-24T06:28:21.748803+0000","last_became_active":"2023-11-24T06:28:20.754783+0000","last_became_peered":"2023-11-24T06:28:20.754783+0000","last_unstale":"2023-11-24T06:28:21.748803+0000","last_undegraded":"2023-11-24T06:28:21.748803+0000","last_fullsized":"2023-11-24T06:28:21.748803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:52.033759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748672+0000","last_change":"2023-11-24T06:28:20.751616+0000","last_active":"2023-11-24T06:28:21.748672+0000","last_peered":"2023-11-24T06:28:21.748672+0000","last_clean":"2023-11-24T06:28:21.748672+0000","last_became_active":"2023-11-24T06:28:20.751347+0000","last_became_peered":"2023-11-24T06:28:20.751347+0000","last_unstale":"2023-11-24T06:28:21.748672+0000","last_undegraded":"2023-11-24T06:28:21.748672+0000","last_fullsized":"2023-11-24T06:28:21.748672+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:52.718291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364306+0000","last_change":"2023-11-24T06:28:20.752060+0000","last_active":"2023-11-24T06:28:22.364306+0000","last_peered":"2023-11-24T06:28:22.364306+0000","last_clean":"2023-11-24T06:28:22.364306+0000","last_became_active":"2023-11-24T06:28:20.751597+0000","last_became_peered":"2023-11-24T06:28:20.751597+0000","last_unstale":"2023-11-24T06:28:22.364306+0000","last_undegraded":"2023-11-24T06:28:22.364306+0000","last_fullsized":"2023-11-24T06:28:22.364306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:51.206064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363421+0000","last_change":"2023-11-24T06:28:20.751828+0000","last_active":"2023-11-24T06:28:22.363421+0000","last_peered":"2023-11-24T06:28:22.363421+0000","last_clean":"2023-11-24T06:28:22.363421+0000","last_became_active":"2023-11-24T06:28:20.751496+0000","last_became_peered":"2023-11-24T06:28:20.751496+0000","last_unstale":"2023-11-24T06:28:22.363421+0000","last_undegraded":"2023-11-24T06:28:22.363421+0000","last_fullsized":"2023-11-24T06:28:22.363421+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:00.075286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361993+0000","last_change":"2023-11-24T06:28:20.742826+0000","last_active":"2023-11-24T06:28:22.361993+0000","last_peered":"2023-11-24T06:28:22.361993+0000","last_clean":"2023-11-24T06:28:22.361993+0000","last_became_active":"2023-11-24T06:28:20.742479+0000","last_became_peered":"2023-11-24T06:28:20.742479+0000","last_unstale":"2023-11-24T06:28:22.361993+0000","last_undegraded":"2023-11-24T06:28:22.361993+0000","last_fullsized":"2023-11-24T06:28:22.361993+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:06.929632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363090+0000","last_change":"2023-11-24T06:28:20.752827+0000","last_active":"2023-11-24T06:28:22.363090+0000","last_peered":"2023-11-24T06:28:22.363090+0000","last_clean":"2023-11-24T06:28:22.363090+0000","last_became_active":"2023-11-24T06:28:20.752553+0000","last_became_peered":"2023-11-24T06:28:20.752553+0000","last_unstale":"2023-11-24T06:28:22.363090+0000","last_undegraded":"2023-11-24T06:28:22.363090+0000","last_fullsized":"2023-11-24T06:28:22.363090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:34.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362788+0000","last_change":"2023-11-24T06:28:20.747287+0000","last_active":"2023-11-24T06:28:22.362788+0000","last_peered":"2023-11-24T06:28:22.362788+0000","last_clean":"2023-11-24T06:28:22.362788+0000","last_became_active":"2023-11-24T06:28:20.746913+0000","last_became_peered":"2023-11-24T06:28:20.746913+0000","last_unstale":"2023-11-24T06:28:22.362788+0000","last_undegraded":"2023-11-24T06:28:22.362788+0000","last_fullsized":"2023-11-24T06:28:22.362788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:08.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361398+0000","last_change":"2023-11-24T06:28:20.755636+0000","last_active":"2023-11-24T06:28:22.361398+0000","last_peered":"2023-11-24T06:28:22.361398+0000","last_clean":"2023-11-24T06:28:22.361398+0000","last_became_active":"2023-11-24T06:28:20.755151+0000","last_became_peered":"2023-11-24T06:28:20.755151+0000","last_unstale":"2023-11-24T06:28:22.361398+0000","last_undegraded":"2023-11-24T06:28:22.361398+0000","last_fullsized":"2023-11-24T06:28:22.361398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:34.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747983+0000","last_change":"2023-11-24T06:28:20.738640+0000","last_active":"2023-11-24T06:28:21.747983+0000","last_peered":"2023-11-24T06:28:21.747983+0000","last_clean":"2023-11-24T06:28:21.747983+0000","last_became_active":"2023-11-24T06:28:20.737115+0000","last_became_peered":"2023-11-24T06:28:20.737115+0000","last_unstale":"2023-11-24T06:28:21.747983+0000","last_undegraded":"2023-11-24T06:28:21.747983+0000","last_fullsized":"2023-11-24T06:28:21.747983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:41.832504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742269+0000","last_change":"2023-11-24T06:28:20.754501+0000","last_active":"2023-11-24T06:28:21.742269+0000","last_peered":"2023-11-24T06:28:21.742269+0000","last_clean":"2023-11-24T06:28:21.742269+0000","last_became_active":"2023-11-24T06:28:20.749001+0000","last_became_peered":"2023-11-24T06:28:20.749001+0000","last_unstale":"2023-11-24T06:28:21.742269+0000","last_undegraded":"2023-11-24T06:28:21.742269+0000","last_fullsized":"2023-11-24T06:28:21.742269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:35.498445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364527+0000","last_change":"2023-11-24T06:28:20.749900+0000","last_active":"2023-11-24T06:28:22.364527+0000","last_peered":"2023-11-24T06:28:22.364527+0000","last_clean":"2023-11-24T06:28:22.364527+0000","last_became_active":"2023-11-24T06:28:20.749612+0000","last_became_peered":"2023-11-24T06:28:20.749612+0000","last_unstale":"2023-11-24T06:28:22.364527+0000","last_undegraded":"2023-11-24T06:28:22.364527+0000","last_fullsized":"2023-11-24T06:28:22.364527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:02.550632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742049+0000","last_change":"2023-11-24T06:28:20.757641+0000","last_active":"2023-11-24T06:28:21.742049+0000","last_peered":"2023-11-24T06:28:21.742049+0000","last_clean":"2023-11-24T06:28:21.742049+0000","last_became_active":"2023-11-24T06:28:20.757307+0000","last_became_peered":"2023-11-24T06:28:20.757307+0000","last_unstale":"2023-11-24T06:28:21.742049+0000","last_undegraded":"2023-11-24T06:28:21.742049+0000","last_fullsized":"2023-11-24T06:28:21.742049+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:36.858522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749834+0000","last_change":"2023-11-24T06:28:20.738500+0000","last_active":"2023-11-24T06:28:21.749834+0000","last_peered":"2023-11-24T06:28:21.749834+0000","last_clean":"2023-11-24T06:28:21.749834+0000","last_became_active":"2023-11-24T06:28:20.738051+0000","last_became_peered":"2023-11-24T06:28:20.738051+0000","last_unstale":"2023-11-24T06:28:21.749834+0000","last_undegraded":"2023-11-24T06:28:21.749834+0000","last_fullsized":"2023-11-24T06:28:21.749834+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:46.026504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363095+0000","last_change":"2023-11-24T06:28:20.743275+0000","last_active":"2023-11-24T06:28:22.363095+0000","last_peered":"2023-11-24T06:28:22.363095+0000","last_clean":"2023-11-24T06:28:22.363095+0000","last_became_active":"2023-11-24T06:28:20.742978+0000","last_became_peered":"2023-11-24T06:28:20.742978+0000","last_unstale":"2023-11-24T06:28:22.363095+0000","last_undegraded":"2023-11-24T06:28:22.363095+0000","last_fullsized":"2023-11-24T06:28:22.363095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:12.693448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747735+0000","last_change":"2023-11-24T06:28:20.751681+0000","last_active":"2023-11-24T06:28:21.747735+0000","last_peered":"2023-11-24T06:28:21.747735+0000","last_clean":"2023-11-24T06:28:21.747735+0000","last_became_active":"2023-11-24T06:28:20.751402+0000","last_became_peered":"2023-11-24T06:28:20.751402+0000","last_unstale":"2023-11-24T06:28:21.747735+0000","last_undegraded":"2023-11-24T06:28:21.747735+0000","last_fullsized":"2023-11-24T06:28:21.747735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:36.384991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361179+0000","last_change":"2023-11-24T06:28:20.754007+0000","last_active":"2023-11-24T06:28:22.361179+0000","last_peered":"2023-11-24T06:28:22.361179+0000","last_clean":"2023-11-24T06:28:22.361179+0000","last_became_active":"2023-11-24T06:28:20.752557+0000","last_became_peered":"2023-11-24T06:28:20.752557+0000","last_unstale":"2023-11-24T06:28:22.361179+0000","last_undegraded":"2023-11-24T06:28:22.361179+0000","last_fullsized":"2023-11-24T06:28:22.361179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:51.904162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748122+0000","last_change":"2023-11-24T06:28:20.757581+0000","last_active":"2023-11-24T06:28:21.748122+0000","last_peered":"2023-11-24T06:28:21.748122+0000","last_clean":"2023-11-24T06:28:21.748122+0000","last_became_active":"2023-11-24T06:28:20.757233+0000","last_became_peered":"2023-11-24T06:28:20.757233+0000","last_unstale":"2023-11-24T06:28:21.748122+0000","last_undegraded":"2023-11-24T06:28:21.748122+0000","last_fullsized":"2023-11-24T06:28:21.748122+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:23.123298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361983+0000","last_change":"2023-11-24T06:28:20.747847+0000","last_active":"2023-11-24T06:28:22.361983+0000","last_peered":"2023-11-24T06:28:22.361983+0000","last_clean":"2023-11-24T06:28:22.361983+0000","last_became_active":"2023-11-24T06:28:20.747551+0000","last_became_peered":"2023-11-24T06:28:20.747551+0000","last_unstale":"2023-11-24T06:28:22.361983+0000","last_undegraded":"2023-11-24T06:28:22.361983+0000","last_fullsized":"2023-11-24T06:28:22.361983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:39.039302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.770328+0000","last_change":"2023-11-24T06:28:19.748835+0000","last_active":"2023-11-24T06:28:20.770328+0000","last_peered":"2023-11-24T06:28:20.770328+0000","last_clean":"2023-11-24T06:28:20.770328+0000","last_became_active":"2023-11-24T06:28:19.748412+0000","last_became_peered":"2023-11-24T06:28:19.748412+0000","last_unstale":"2023-11-24T06:28:20.770328+0000","last_undegraded":"2023-11-24T06:28:20.770328+0000","last_fullsized":"2023-11-24T06:28:20.770328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:48.325368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362430+0000","last_change":"2023-11-24T06:28:20.748498+0000","last_active":"2023-11-24T06:28:22.362430+0000","last_peered":"2023-11-24T06:28:22.362430+0000","last_clean":"2023-11-24T06:28:22.362430+0000","last_became_active":"2023-11-24T06:28:20.748101+0000","last_became_peered":"2023-11-24T06:28:20.748101+0000","last_unstale":"2023-11-24T06:28:22.362430+0000","last_undegraded":"2023-11-24T06:28:22.362430+0000","last_fullsized":"2023-11-24T06:28:22.362430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:12.693448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.771276+0000","last_change":"2023-11-24T06:28:19.742903+0000","last_active":"2023-11-24T06:28:20.771276+0000","last_peered":"2023-11-24T06:28:20.771276+0000","last_clean":"2023-11-24T06:28:20.771276+0000","last_became_active":"2023-11-24T06:28:19.742598+0000","last_became_peered":"2023-11-24T06:28:19.742598+0000","last_unstale":"2023-11-24T06:28:20.771276+0000","last_undegraded":"2023-11-24T06:28:20.771276+0000","last_fullsized":"2023-11-24T06:28:20.771276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:29.191620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363955+0000","last_change":"2023-11-24T06:28:20.751153+0000","last_active":"2023-11-24T06:28:22.363955+0000","last_peered":"2023-11-24T06:28:22.363955+0000","last_clean":"2023-11-24T06:28:22.363955+0000","last_became_active":"2023-11-24T06:28:20.749651+0000","last_became_peered":"2023-11-24T06:28:20.749651+0000","last_unstale":"2023-11-24T06:28:22.363955+0000","last_undegraded":"2023-11-24T06:28:22.363955+0000","last_fullsized":"2023-11-24T06:28:22.363955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:23.869661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.773975+0000","last_change":"2023-11-24T06:28:19.733747+0000","last_active":"2023-11-24T06:28:20.773975+0000","last_peered":"2023-11-24T06:28:20.773975+0000","last_clean":"2023-11-24T06:28:20.773975+0000","last_became_active":"2023-11-24T06:28:19.733405+0000","last_became_peered":"2023-11-24T06:28:19.733405+0000","last_unstale":"2023-11-24T06:28:20.773975+0000","last_undegraded":"2023-11-24T06:28:20.773975+0000","last_fullsized":"2023-11-24T06:28:20.773975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:22.109728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.742130+0000","last_change":"2023-11-24T06:28:20.757546+0000","last_active":"2023-11-24T06:28:21.742130+0000","last_peered":"2023-11-24T06:28:21.742130+0000","last_clean":"2023-11-24T06:28:21.742130+0000","last_became_active":"2023-11-24T06:28:20.757229+0000","last_became_peered":"2023-11-24T06:28:20.757229+0000","last_unstale":"2023-11-24T06:28:21.742130+0000","last_undegraded":"2023-11-24T06:28:21.742130+0000","last_fullsized":"2023-11-24T06:28:21.742130+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:16.647539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.772837+0000","last_change":"2023-11-24T06:28:19.733959+0000","last_active":"2023-11-24T06:28:20.772837+0000","last_peered":"2023-11-24T06:28:20.772837+0000","last_clean":"2023-11-24T06:28:20.772837+0000","last_became_active":"2023-11-24T06:28:19.733633+0000","last_became_peered":"2023-11-24T06:28:19.733633+0000","last_unstale":"2023-11-24T06:28:20.772837+0000","last_undegraded":"2023-11-24T06:28:20.772837+0000","last_fullsized":"2023-11-24T06:28:20.772837+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:07:48.392066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748219+0000","last_change":"2023-11-24T06:28:20.759715+0000","last_active":"2023-11-24T06:28:21.748219+0000","last_peered":"2023-11-24T06:28:21.748219+0000","last_clean":"2023-11-24T06:28:21.748219+0000","last_became_active":"2023-11-24T06:28:20.757687+0000","last_became_peered":"2023-11-24T06:28:20.757687+0000","last_unstale":"2023-11-24T06:28:21.748219+0000","last_undegraded":"2023-11-24T06:28:21.748219+0000","last_fullsized":"2023-11-24T06:28:21.748219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:46.026504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.768459+0000","last_change":"2023-11-24T06:28:19.740944+0000","last_active":"2023-11-24T06:28:20.768459+0000","last_peered":"2023-11-24T06:28:20.768459+0000","last_clean":"2023-11-24T06:28:20.768459+0000","last_became_active":"2023-11-24T06:28:19.740201+0000","last_became_peered":"2023-11-24T06:28:19.740201+0000","last_unstale":"2023-11-24T06:28:20.768459+0000","last_undegraded":"2023-11-24T06:28:20.768459+0000","last_fullsized":"2023-11-24T06:28:20.768459+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:07:48.392066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749401+0000","last_change":"2023-11-24T06:28:20.747089+0000","last_active":"2023-11-24T06:28:21.749401+0000","last_peered":"2023-11-24T06:28:21.749401+0000","last_clean":"2023-11-24T06:28:21.749401+0000","last_became_active":"2023-11-24T06:28:20.742220+0000","last_became_peered":"2023-11-24T06:28:20.742220+0000","last_unstale":"2023-11-24T06:28:21.749401+0000","last_undegraded":"2023-11-24T06:28:21.749401+0000","last_fullsized":"2023-11-24T06:28:21.749401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:36.384991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.767921+0000","last_change":"2023-11-24T06:28:19.737597+0000","last_active":"2023-11-24T06:28:20.767921+0000","last_peered":"2023-11-24T06:28:20.767921+0000","last_clean":"2023-11-24T06:28:20.767921+0000","last_became_active":"2023-11-24T06:28:19.737301+0000","last_became_peered":"2023-11-24T06:28:19.737301+0000","last_unstale":"2023-11-24T06:28:20.767921+0000","last_undegraded":"2023-11-24T06:28:20.767921+0000","last_fullsized":"2023-11-24T06:28:20.767921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:07.528212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361176+0000","last_change":"2023-11-24T06:28:20.752076+0000","last_active":"2023-11-24T06:28:22.361176+0000","last_peered":"2023-11-24T06:28:22.361176+0000","last_clean":"2023-11-24T06:28:22.361176+0000","last_became_active":"2023-11-24T06:28:20.751527+0000","last_became_peered":"2023-11-24T06:28:20.751527+0000","last_unstale":"2023-11-24T06:28:22.361176+0000","last_undegraded":"2023-11-24T06:28:22.361176+0000","last_fullsized":"2023-11-24T06:28:22.361176+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:51.904162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.728185+0000","last_change":"2023-11-24T06:28:19.732965+0000","last_active":"2023-11-24T06:28:20.728185+0000","last_peered":"2023-11-24T06:28:20.728185+0000","last_clean":"2023-11-24T06:28:20.728185+0000","last_became_active":"2023-11-24T06:28:19.731006+0000","last_became_peered":"2023-11-24T06:28:19.731006+0000","last_unstale":"2023-11-24T06:28:20.728185+0000","last_undegraded":"2023-11-24T06:28:20.728185+0000","last_fullsized":"2023-11-24T06:28:20.728185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:40.818934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741644+0000","last_change":"2023-11-24T06:28:20.755312+0000","last_active":"2023-11-24T06:28:21.741644+0000","last_peered":"2023-11-24T06:28:21.741644+0000","last_clean":"2023-11-24T06:28:21.741644+0000","last_became_active":"2023-11-24T06:28:20.753599+0000","last_became_peered":"2023-11-24T06:28:20.753599+0000","last_unstale":"2023-11-24T06:28:21.741644+0000","last_undegraded":"2023-11-24T06:28:21.741644+0000","last_fullsized":"2023-11-24T06:28:21.741644+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:41.967768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.734014+0000","last_change":"2023-11-24T06:28:19.740715+0000","last_active":"2023-11-24T06:28:20.734014+0000","last_peered":"2023-11-24T06:28:20.734014+0000","last_clean":"2023-11-24T06:28:20.734014+0000","last_became_active":"2023-11-24T06:28:19.738599+0000","last_became_peered":"2023-11-24T06:28:19.738599+0000","last_unstale":"2023-11-24T06:28:20.734014+0000","last_undegraded":"2023-11-24T06:28:20.734014+0000","last_fullsized":"2023-11-24T06:28:20.734014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:02.155451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749992+0000","last_change":"2023-11-24T06:28:20.741845+0000","last_active":"2023-11-24T06:28:21.749992+0000","last_peered":"2023-11-24T06:28:21.749992+0000","last_clean":"2023-11-24T06:28:21.749992+0000","last_became_active":"2023-11-24T06:28:20.741520+0000","last_became_peered":"2023-11-24T06:28:20.741520+0000","last_unstale":"2023-11-24T06:28:21.749992+0000","last_undegraded":"2023-11-24T06:28:21.749992+0000","last_fullsized":"2023-11-24T06:28:21.749992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:00.075286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.729679+0000","last_change":"2023-11-24T06:28:19.742211+0000","last_active":"2023-11-24T06:28:20.729679+0000","last_peered":"2023-11-24T06:28:20.729679+0000","last_clean":"2023-11-24T06:28:20.729679+0000","last_became_active":"2023-11-24T06:28:19.741525+0000","last_became_peered":"2023-11-24T06:28:19.741525+0000","last_unstale":"2023-11-24T06:28:20.729679+0000","last_undegraded":"2023-11-24T06:28:20.729679+0000","last_fullsized":"2023-11-24T06:28:20.729679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:17.292179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363780+0000","last_change":"2023-11-24T06:28:20.752528+0000","last_active":"2023-11-24T06:28:22.363780+0000","last_peered":"2023-11-24T06:28:22.363780+0000","last_clean":"2023-11-24T06:28:22.363780+0000","last_became_active":"2023-11-24T06:28:20.752084+0000","last_became_peered":"2023-11-24T06:28:20.752084+0000","last_unstale":"2023-11-24T06:28:22.363780+0000","last_undegraded":"2023-11-24T06:28:22.363780+0000","last_fullsized":"2023-11-24T06:28:22.363780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:21.555443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.723909+0000","last_change":"2023-11-24T06:28:19.735685+0000","last_active":"2023-11-24T06:28:20.723909+0000","last_peered":"2023-11-24T06:28:20.723909+0000","last_clean":"2023-11-24T06:28:20.723909+0000","last_became_active":"2023-11-24T06:28:19.735419+0000","last_became_peered":"2023-11-24T06:28:19.735419+0000","last_unstale":"2023-11-24T06:28:20.723909+0000","last_undegraded":"2023-11-24T06:28:20.723909+0000","last_fullsized":"2023-11-24T06:28:20.723909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:07.528212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.364602+0000","last_change":"2023-11-24T06:28:20.748284+0000","last_active":"2023-11-24T06:28:22.364602+0000","last_peered":"2023-11-24T06:28:22.364602+0000","last_clean":"2023-11-24T06:28:22.364602+0000","last_became_active":"2023-11-24T06:28:20.748000+0000","last_became_peered":"2023-11-24T06:28:20.748000+0000","last_unstale":"2023-11-24T06:28:22.364602+0000","last_undegraded":"2023-11-24T06:28:22.364602+0000","last_fullsized":"2023-11-24T06:28:22.364602+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:23.440271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.770416+0000","last_change":"2023-11-24T06:28:19.742398+0000","last_active":"2023-11-24T06:28:20.770416+0000","last_peered":"2023-11-24T06:28:20.770416+0000","last_clean":"2023-11-24T06:28:20.770416+0000","last_became_active":"2023-11-24T06:28:19.739648+0000","last_became_peered":"2023-11-24T06:28:19.739648+0000","last_unstale":"2023-11-24T06:28:20.770416+0000","last_undegraded":"2023-11-24T06:28:20.770416+0000","last_fullsized":"2023-11-24T06:28:20.770416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:40.474217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.749587+0000","last_change":"2023-11-24T06:28:20.745174+0000","last_active":"2023-11-24T06:28:21.749587+0000","last_peered":"2023-11-24T06:28:21.749587+0000","last_clean":"2023-11-24T06:28:21.749587+0000","last_became_active":"2023-11-24T06:28:20.744317+0000","last_became_peered":"2023-11-24T06:28:20.744317+0000","last_unstale":"2023-11-24T06:28:21.749587+0000","last_undegraded":"2023-11-24T06:28:21.749587+0000","last_fullsized":"2023-11-24T06:28:21.749587+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:26:23.123298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.776450+0000","last_change":"2023-11-24T06:28:19.732266+0000","last_active":"2023-11-24T06:28:20.776450+0000","last_peered":"2023-11-24T06:28:20.776450+0000","last_clean":"2023-11-24T06:28:20.776450+0000","last_became_active":"2023-11-24T06:28:19.731925+0000","last_became_peered":"2023-11-24T06:28:19.731925+0000","last_unstale":"2023-11-24T06:28:20.776450+0000","last_undegraded":"2023-11-24T06:28:20.776450+0000","last_fullsized":"2023-11-24T06:28:20.776450+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:47.574860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741618+0000","last_change":"2023-11-24T06:28:20.749699+0000","last_active":"2023-11-24T06:28:21.741618+0000","last_peered":"2023-11-24T06:28:21.741618+0000","last_clean":"2023-11-24T06:28:21.741618+0000","last_became_active":"2023-11-24T06:28:20.749273+0000","last_became_peered":"2023-11-24T06:28:20.749273+0000","last_unstale":"2023-11-24T06:28:21.741618+0000","last_undegraded":"2023-11-24T06:28:21.741618+0000","last_fullsized":"2023-11-24T06:28:21.741618+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:23.869661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.769745+0000","last_change":"2023-11-24T06:28:19.740983+0000","last_active":"2023-11-24T06:28:20.769745+0000","last_peered":"2023-11-24T06:28:20.769745+0000","last_clean":"2023-11-24T06:28:20.769745+0000","last_became_active":"2023-11-24T06:28:19.740647+0000","last_became_peered":"2023-11-24T06:28:19.740647+0000","last_unstale":"2023-11-24T06:28:20.769745+0000","last_undegraded":"2023-11-24T06:28:20.769745+0000","last_fullsized":"2023-11-24T06:28:20.769745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:11:28.316281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363921+0000","last_change":"2023-11-24T06:28:20.758815+0000","last_active":"2023-11-24T06:28:22.363921+0000","last_peered":"2023-11-24T06:28:22.363921+0000","last_clean":"2023-11-24T06:28:22.363921+0000","last_became_active":"2023-11-24T06:28:20.757556+0000","last_became_peered":"2023-11-24T06:28:20.757556+0000","last_unstale":"2023-11-24T06:28:22.363921+0000","last_undegraded":"2023-11-24T06:28:22.363921+0000","last_fullsized":"2023-11-24T06:28:22.363921+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:03:52.033759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.739778+0000","last_change":"2023-11-24T06:28:19.741817+0000","last_active":"2023-11-24T06:28:20.739778+0000","last_peered":"2023-11-24T06:28:20.739778+0000","last_clean":"2023-11-24T06:28:20.739778+0000","last_became_active":"2023-11-24T06:28:19.741553+0000","last_became_peered":"2023-11-24T06:28:19.741553+0000","last_unstale":"2023-11-24T06:28:20.739778+0000","last_undegraded":"2023-11-24T06:28:20.739778+0000","last_fullsized":"2023-11-24T06:28:20.739778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:07.764462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748482+0000","last_change":"2023-11-24T06:28:20.757476+0000","last_active":"2023-11-24T06:28:21.748482+0000","last_peered":"2023-11-24T06:28:21.748482+0000","last_clean":"2023-11-24T06:28:21.748482+0000","last_became_active":"2023-11-24T06:28:20.757215+0000","last_became_peered":"2023-11-24T06:28:20.757215+0000","last_unstale":"2023-11-24T06:28:21.748482+0000","last_undegraded":"2023-11-24T06:28:21.748482+0000","last_fullsized":"2023-11-24T06:28:21.748482+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:00.075286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.727760+0000","last_change":"2023-11-24T06:28:19.737019+0000","last_active":"2023-11-24T06:28:20.727760+0000","last_peered":"2023-11-24T06:28:20.727760+0000","last_clean":"2023-11-24T06:28:20.727760+0000","last_became_active":"2023-11-24T06:28:19.736587+0000","last_became_peered":"2023-11-24T06:28:19.736587+0000","last_unstale":"2023-11-24T06:28:20.727760+0000","last_undegraded":"2023-11-24T06:28:20.727760+0000","last_fullsized":"2023-11-24T06:28:20.727760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:34.484876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361269+0000","last_change":"2023-11-24T06:28:20.756359+0000","last_active":"2023-11-24T06:28:22.361269+0000","last_peered":"2023-11-24T06:28:22.361269+0000","last_clean":"2023-11-24T06:28:22.361269+0000","last_became_active":"2023-11-24T06:28:20.756086+0000","last_became_peered":"2023-11-24T06:28:20.756086+0000","last_unstale":"2023-11-24T06:28:22.361269+0000","last_undegraded":"2023-11-24T06:28:22.361269+0000","last_fullsized":"2023-11-24T06:28:22.361269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:19.935685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.774855+0000","last_change":"2023-11-24T06:28:19.728784+0000","last_active":"2023-11-24T06:28:20.774855+0000","last_peered":"2023-11-24T06:28:20.774855+0000","last_clean":"2023-11-24T06:28:20.774855+0000","last_became_active":"2023-11-24T06:28:19.728379+0000","last_became_peered":"2023-11-24T06:28:19.728379+0000","last_unstale":"2023-11-24T06:28:20.774855+0000","last_undegraded":"2023-11-24T06:28:20.774855+0000","last_fullsized":"2023-11-24T06:28:20.774855+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:35.371421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363620+0000","last_change":"2023-11-24T06:28:20.748159+0000","last_active":"2023-11-24T06:28:22.363620+0000","last_peered":"2023-11-24T06:28:22.363620+0000","last_clean":"2023-11-24T06:28:22.363620+0000","last_became_active":"2023-11-24T06:28:20.747300+0000","last_became_peered":"2023-11-24T06:28:20.747300+0000","last_unstale":"2023-11-24T06:28:22.363620+0000","last_undegraded":"2023-11-24T06:28:22.363620+0000","last_fullsized":"2023-11-24T06:28:22.363620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:39.039302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.773830+0000","last_change":"2023-11-24T06:28:19.738232+0000","last_active":"2023-11-24T06:28:20.773830+0000","last_peered":"2023-11-24T06:28:20.773830+0000","last_clean":"2023-11-24T06:28:20.773830+0000","last_became_active":"2023-11-24T06:28:19.737493+0000","last_became_peered":"2023-11-24T06:28:19.737493+0000","last_unstale":"2023-11-24T06:28:20.773830+0000","last_undegraded":"2023-11-24T06:28:20.773830+0000","last_fullsized":"2023-11-24T06:28:20.773830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:38.025733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.750413+0000","last_change":"2023-11-24T06:28:20.752165+0000","last_active":"2023-11-24T06:28:21.750413+0000","last_peered":"2023-11-24T06:28:21.750413+0000","last_clean":"2023-11-24T06:28:21.750413+0000","last_became_active":"2023-11-24T06:28:20.751192+0000","last_became_peered":"2023-11-24T06:28:20.751192+0000","last_unstale":"2023-11-24T06:28:21.750413+0000","last_undegraded":"2023-11-24T06:28:21.750413+0000","last_fullsized":"2023-11-24T06:28:21.750413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:52.718291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.769292+0000","last_change":"2023-11-24T06:28:19.740458+0000","last_active":"2023-11-24T06:28:20.769292+0000","last_peered":"2023-11-24T06:28:20.769292+0000","last_clean":"2023-11-24T06:28:20.769292+0000","last_became_active":"2023-11-24T06:28:19.740072+0000","last_became_peered":"2023-11-24T06:28:19.740072+0000","last_unstale":"2023-11-24T06:28:20.769292+0000","last_undegraded":"2023-11-24T06:28:20.769292+0000","last_fullsized":"2023-11-24T06:28:20.769292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:14.621798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.741778+0000","last_change":"2023-11-24T06:28:20.754060+0000","last_active":"2023-11-24T06:28:21.741778+0000","last_peered":"2023-11-24T06:28:21.741778+0000","last_clean":"2023-11-24T06:28:21.741778+0000","last_became_active":"2023-11-24T06:28:20.753777+0000","last_became_peered":"2023-11-24T06:28:20.753777+0000","last_unstale":"2023-11-24T06:28:21.741778+0000","last_undegraded":"2023-11-24T06:28:21.741778+0000","last_fullsized":"2023-11-24T06:28:21.741778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:29.453645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.767249+0000","last_change":"2023-11-24T06:28:19.742059+0000","last_active":"2023-11-24T06:28:20.767249+0000","last_peered":"2023-11-24T06:28:20.767249+0000","last_clean":"2023-11-24T06:28:20.767249+0000","last_became_active":"2023-11-24T06:28:19.741784+0000","last_became_peered":"2023-11-24T06:28:19.741784+0000","last_unstale":"2023-11-24T06:28:20.767249+0000","last_undegraded":"2023-11-24T06:28:20.767249+0000","last_fullsized":"2023-11-24T06:28:20.767249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:40.154924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361968+0000","last_change":"2023-11-24T06:28:20.749842+0000","last_active":"2023-11-24T06:28:22.361968+0000","last_peered":"2023-11-24T06:28:22.361968+0000","last_clean":"2023-11-24T06:28:22.361968+0000","last_became_active":"2023-11-24T06:28:20.749578+0000","last_became_peered":"2023-11-24T06:28:20.749578+0000","last_unstale":"2023-11-24T06:28:22.361968+0000","last_undegraded":"2023-11-24T06:28:22.361968+0000","last_fullsized":"2023-11-24T06:28:22.361968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:30.205190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.735299+0000","last_change":"2023-11-24T06:28:19.736327+0000","last_active":"2023-11-24T06:28:20.735299+0000","last_peered":"2023-11-24T06:28:20.735299+0000","last_clean":"2023-11-24T06:28:20.735299+0000","last_became_active":"2023-11-24T06:28:19.736017+0000","last_became_peered":"2023-11-24T06:28:19.736017+0000","last_unstale":"2023-11-24T06:28:20.735299+0000","last_undegraded":"2023-11-24T06:28:20.735299+0000","last_fullsized":"2023-11-24T06:28:20.735299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:05.916062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.360985+0000","last_change":"2023-11-24T06:28:20.749162+0000","last_active":"2023-11-24T06:28:22.360985+0000","last_peered":"2023-11-24T06:28:22.360985+0000","last_clean":"2023-11-24T06:28:22.360985+0000","last_became_active":"2023-11-24T06:28:20.748814+0000","last_became_peered":"2023-11-24T06:28:20.748814+0000","last_unstale":"2023-11-24T06:28:22.360985+0000","last_undegraded":"2023-11-24T06:28:22.360985+0000","last_fullsized":"2023-11-24T06:28:22.360985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:15.635368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.771526+0000","last_change":"2023-11-24T06:28:19.732595+0000","last_active":"2023-11-24T06:28:20.771526+0000","last_peered":"2023-11-24T06:28:20.771526+0000","last_clean":"2023-11-24T06:28:20.771526+0000","last_became_active":"2023-11-24T06:28:19.732320+0000","last_became_peered":"2023-11-24T06:28:19.732320+0000","last_unstale":"2023-11-24T06:28:20.771526+0000","last_undegraded":"2023-11-24T06:28:20.771526+0000","last_fullsized":"2023-11-24T06:28:20.771526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:50.890592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361338+0000","last_change":"2023-11-24T06:28:20.742637+0000","last_active":"2023-11-24T06:28:22.361338+0000","last_peered":"2023-11-24T06:28:22.361338+0000","last_clean":"2023-11-24T06:28:22.361338+0000","last_became_active":"2023-11-24T06:28:20.742277+0000","last_became_peered":"2023-11-24T06:28:20.742277+0000","last_unstale":"2023-11-24T06:28:22.361338+0000","last_undegraded":"2023-11-24T06:28:22.361338+0000","last_fullsized":"2023-11-24T06:28:22.361338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:57.469275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.736299+0000","last_change":"2023-11-24T06:28:19.745300+0000","last_active":"2023-11-24T06:28:20.736299+0000","last_peered":"2023-11-24T06:28:20.736299+0000","last_clean":"2023-11-24T06:28:20.736299+0000","last_became_active":"2023-11-24T06:28:19.744997+0000","last_became_peered":"2023-11-24T06:28:19.744997+0000","last_unstale":"2023-11-24T06:28:20.736299+0000","last_undegraded":"2023-11-24T06:28:20.736299+0000","last_fullsized":"2023-11-24T06:28:20.736299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:05.916062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361396+0000","last_change":"2023-11-24T06:28:20.746645+0000","last_active":"2023-11-24T06:28:22.361396+0000","last_peered":"2023-11-24T06:28:22.361396+0000","last_clean":"2023-11-24T06:28:22.361396+0000","last_became_active":"2023-11-24T06:28:20.742228+0000","last_became_peered":"2023-11-24T06:28:20.742228+0000","last_unstale":"2023-11-24T06:28:22.361396+0000","last_undegraded":"2023-11-24T06:28:22.361396+0000","last_fullsized":"2023-11-24T06:28:22.361396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:03.169021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.728527+0000","last_change":"2023-11-24T06:28:19.741555+0000","last_active":"2023-11-24T06:28:20.728527+0000","last_peered":"2023-11-24T06:28:20.728527+0000","last_clean":"2023-11-24T06:28:20.728527+0000","last_became_active":"2023-11-24T06:28:19.739895+0000","last_became_peered":"2023-11-24T06:28:19.739895+0000","last_unstale":"2023-11-24T06:28:20.728527+0000","last_undegraded":"2023-11-24T06:28:20.728527+0000","last_fullsized":"2023-11-24T06:28:20.728527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:55:19.561292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.747734+0000","last_change":"2023-11-24T06:28:20.750118+0000","last_active":"2023-11-24T06:28:21.747734+0000","last_peered":"2023-11-24T06:28:21.747734+0000","last_clean":"2023-11-24T06:28:21.747734+0000","last_became_active":"2023-11-24T06:28:20.749804+0000","last_became_peered":"2023-11-24T06:28:20.749804+0000","last_unstale":"2023-11-24T06:28:21.747734+0000","last_undegraded":"2023-11-24T06:28:21.747734+0000","last_fullsized":"2023-11-24T06:28:21.747734+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:48.588430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.726227+0000","last_change":"2023-11-24T06:28:19.739558+0000","last_active":"2023-11-24T06:28:20.726227+0000","last_peered":"2023-11-24T06:28:20.726227+0000","last_clean":"2023-11-24T06:28:20.726227+0000","last_became_active":"2023-11-24T06:28:19.736344+0000","last_became_peered":"2023-11-24T06:28:19.736344+0000","last_unstale":"2023-11-24T06:28:20.726227+0000","last_undegraded":"2023-11-24T06:28:20.726227+0000","last_fullsized":"2023-11-24T06:28:20.726227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:40.954198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361860+0000","last_change":"2023-11-24T06:28:20.745248+0000","last_active":"2023-11-24T06:28:22.361860+0000","last_peered":"2023-11-24T06:28:22.361860+0000","last_clean":"2023-11-24T06:28:22.361860+0000","last_became_active":"2023-11-24T06:28:20.744949+0000","last_became_peered":"2023-11-24T06:28:20.744949+0000","last_unstale":"2023-11-24T06:28:22.361860+0000","last_undegraded":"2023-11-24T06:28:22.361860+0000","last_fullsized":"2023-11-24T06:28:22.361860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:49.338938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.769325+0000","last_change":"2023-11-24T06:28:19.739351+0000","last_active":"2023-11-24T06:28:20.769325+0000","last_peered":"2023-11-24T06:28:20.769325+0000","last_clean":"2023-11-24T06:28:20.769325+0000","last_became_active":"2023-11-24T06:28:19.737557+0000","last_became_peered":"2023-11-24T06:28:19.737557+0000","last_unstale":"2023-11-24T06:28:20.769325+0000","last_undegraded":"2023-11-24T06:28:20.769325+0000","last_fullsized":"2023-11-24T06:28:20.769325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:20.569170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.750531+0000","last_change":"2023-11-24T06:28:20.756413+0000","last_active":"2023-11-24T06:28:21.750531+0000","last_peered":"2023-11-24T06:28:21.750531+0000","last_clean":"2023-11-24T06:28:21.750531+0000","last_became_active":"2023-11-24T06:28:20.756144+0000","last_became_peered":"2023-11-24T06:28:20.756144+0000","last_unstale":"2023-11-24T06:28:21.750531+0000","last_undegraded":"2023-11-24T06:28:21.750531+0000","last_fullsized":"2023-11-24T06:28:21.750531+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:21.555443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362096+0000","last_change":"2023-11-24T06:28:19.733326+0000","last_active":"2023-11-24T06:28:22.362096+0000","last_peered":"2023-11-24T06:28:22.362096+0000","last_clean":"2023-11-24T06:28:22.362096+0000","last_became_active":"2023-11-24T06:28:19.733070+0000","last_became_peered":"2023-11-24T06:28:19.733070+0000","last_unstale":"2023-11-24T06:28:22.362096+0000","last_undegraded":"2023-11-24T06:28:22.362096+0000","last_fullsized":"2023-11-24T06:28:22.362096+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:05.916062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361892+0000","last_change":"2023-11-24T06:28:20.741782+0000","last_active":"2023-11-24T06:28:22.361892+0000","last_peered":"2023-11-24T06:28:22.361892+0000","last_clean":"2023-11-24T06:28:22.361892+0000","last_became_active":"2023-11-24T06:28:20.741508+0000","last_became_peered":"2023-11-24T06:28:20.741508+0000","last_unstale":"2023-11-24T06:28:22.361892+0000","last_undegraded":"2023-11-24T06:28:22.361892+0000","last_fullsized":"2023-11-24T06:28:22.361892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:12.693448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.740853+0000","last_change":"2023-11-24T06:28:19.741328+0000","last_active":"2023-11-24T06:28:20.740853+0000","last_peered":"2023-11-24T06:28:20.740853+0000","last_clean":"2023-11-24T06:28:20.740853+0000","last_became_active":"2023-11-24T06:28:19.740211+0000","last_became_peered":"2023-11-24T06:28:19.740211+0000","last_unstale":"2023-11-24T06:28:20.740853+0000","last_undegraded":"2023-11-24T06:28:20.740853+0000","last_fullsized":"2023-11-24T06:28:20.740853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:11.679878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361842+0000","last_change":"2023-11-24T06:28:20.746488+0000","last_active":"2023-11-24T06:28:22.361842+0000","last_peered":"2023-11-24T06:28:22.361842+0000","last_clean":"2023-11-24T06:28:22.361842+0000","last_became_active":"2023-11-24T06:28:20.746196+0000","last_became_peered":"2023-11-24T06:28:20.746196+0000","last_unstale":"2023-11-24T06:28:22.361842+0000","last_undegraded":"2023-11-24T06:28:22.361842+0000","last_fullsized":"2023-11-24T06:28:22.361842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:08.778032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.724893+0000","last_change":"2023-11-24T06:28:19.738778+0000","last_active":"2023-11-24T06:28:20.724893+0000","last_peered":"2023-11-24T06:28:20.724893+0000","last_clean":"2023-11-24T06:28:20.724893+0000","last_became_active":"2023-11-24T06:28:19.738482+0000","last_became_peered":"2023-11-24T06:28:19.738482+0000","last_unstale":"2023-11-24T06:28:20.724893+0000","last_undegraded":"2023-11-24T06:28:20.724893+0000","last_fullsized":"2023-11-24T06:28:20.724893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:18.922115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.363974+0000","last_change":"2023-11-24T06:28:20.746401+0000","last_active":"2023-11-24T06:28:22.363974+0000","last_peered":"2023-11-24T06:28:22.363974+0000","last_clean":"2023-11-24T06:28:22.363974+0000","last_became_active":"2023-11-24T06:28:20.745992+0000","last_became_peered":"2023-11-24T06:28:20.745992+0000","last_unstale":"2023-11-24T06:28:22.363974+0000","last_undegraded":"2023-11-24T06:28:22.363974+0000","last_fullsized":"2023-11-24T06:28:22.363974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:21.582740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.739931+0000","last_change":"2023-11-24T06:28:19.737632+0000","last_active":"2023-11-24T06:28:20.739931+0000","last_peered":"2023-11-24T06:28:20.739931+0000","last_clean":"2023-11-24T06:28:20.739931+0000","last_became_active":"2023-11-24T06:28:19.736632+0000","last_became_peered":"2023-11-24T06:28:19.736632+0000","last_unstale":"2023-11-24T06:28:20.739931+0000","last_undegraded":"2023-11-24T06:28:20.739931+0000","last_fullsized":"2023-11-24T06:28:20.739931+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:33.275487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.362995+0000","last_change":"2023-11-24T06:28:20.753196+0000","last_active":"2023-11-24T06:28:22.362995+0000","last_peered":"2023-11-24T06:28:22.362995+0000","last_clean":"2023-11-24T06:28:22.362995+0000","last_became_active":"2023-11-24T06:28:20.752928+0000","last_became_peered":"2023-11-24T06:28:20.752928+0000","last_unstale":"2023-11-24T06:28:22.362995+0000","last_undegraded":"2023-11-24T06:28:22.362995+0000","last_fullsized":"2023-11-24T06:28:22.362995+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:46.026504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.728819+0000","last_change":"2023-11-24T06:28:19.744231+0000","last_active":"2023-11-24T06:28:20.728819+0000","last_peered":"2023-11-24T06:28:20.728819+0000","last_clean":"2023-11-24T06:28:20.728819+0000","last_became_active":"2023-11-24T06:28:19.743719+0000","last_became_peered":"2023-11-24T06:28:19.743719+0000","last_unstale":"2023-11-24T06:28:20.728819+0000","last_undegraded":"2023-11-24T06:28:20.728819+0000","last_fullsized":"2023-11-24T06:28:20.728819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:44:59.965561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:22.361501+0000","last_change":"2023-11-24T06:28:20.742551+0000","last_active":"2023-11-24T06:28:22.361501+0000","last_peered":"2023-11-24T06:28:22.361501+0000","last_clean":"2023-11-24T06:28:22.361501+0000","last_became_active":"2023-11-24T06:28:20.742211+0000","last_became_peered":"2023-11-24T06:28:20.742211+0000","last_unstale":"2023-11-24T06:28:22.361501+0000","last_undegraded":"2023-11-24T06:28:22.361501+0000","last_fullsized":"2023-11-24T06:28:22.361501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:19.935685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.726724+0000","last_change":"2023-11-24T06:28:19.737801+0000","last_active":"2023-11-24T06:28:20.726724+0000","last_peered":"2023-11-24T06:28:20.726724+0000","last_clean":"2023-11-24T06:28:20.726724+0000","last_became_active":"2023-11-24T06:28:19.736632+0000","last_became_peered":"2023-11-24T06:28:19.736632+0000","last_unstale":"2023-11-24T06:28:20.726724+0000","last_undegraded":"2023-11-24T06:28:20.726724+0000","last_fullsized":"2023-11-24T06:28:20.726724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:15.633969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:28:21.748625+0000","last_change":"2023-11-24T06:28:20.748776+0000","last_active":"2023-11-24T06:28:21.748625+0000","last_peered":"2023-11-24T06:28:21.748625+0000","last_clean":"2023-11-24T06:28:21.748625+0000","last_became_active":"2023-11-24T06:28:20.745731+0000","last_became_peered":"2023-11-24T06:28:20.745731+0000","last_unstale":"2023-11-24T06:28:21.748625+0000","last_undegraded":"2023-11-24T06:28:21.748625+0000","last_fullsized":"2023-11-24T06:28:21.748625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:19.688317+0000","last_clean_scrub_stamp":"2023-11-24T06:28:19.688317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:14.252801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.736666+0000","last_change":"2023-11-24T06:28:19.745039+0000","last_active":"2023-11-24T06:28:20.736666+0000","last_peered":"2023-11-24T06:28:20.736666+0000","last_clean":"2023-11-24T06:28:20.736666+0000","last_became_active":"2023-11-24T06:28:19.744548+0000","last_became_peered":"2023-11-24T06:28:19.744548+0000","last_unstale":"2023-11-24T06:28:20.736666+0000","last_undegraded":"2023-11-24T06:28:20.736666+0000","last_fullsized":"2023-11-24T06:28:20.736666+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:28:18.674747+0000","last_clean_scrub_stamp":"2023-11-24T06:28:18.674747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:07.528212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:28:20.742551+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:28:20.742551+0000","last_peered":"2023-11-24T06:28:20.742551+0000","last_clean":"2023-11-24T06:28:20.742551+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:28:20.742551+0000","last_undegraded":"2023-11-24T06:28:20.742551+0000","last_fullsized":"2023-11-24T06:28:20.742551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":742397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":4.7359999999999998,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.8179999999999996,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.0409999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.749},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":6.3319999999999999,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":0.99199999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":5.3380000000000001,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.3700000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.008},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.4259999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.607,"15min":1.607},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":6.2649999999999997,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":1.1000000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.252},{"interface":"front","average":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":6.7359999999999998,"5min":6.7359999999999998,"15min":6.7359999999999998},"last":1.319}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.905,"5min":1.905,"15min":1.905},"min":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"max":{"1min":7.6449999999999996,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.359},{"interface":"front","average":{"1min":1.73,"5min":1.73,"15min":1.73},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":5.7270000000000003,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":1.1879999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.0649999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.103},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.99399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.149},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":3.4670000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":0.97599999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":3.04,"5min":3.04,"15min":3.04},"last":0.90300000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.875},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.774,"5min":3.774,"15min":3.774},"last":1.03}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":1.204},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":3.649,"5min":3.649,"15min":3.649},"last":1.0189999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.96199999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.115},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.90800000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.24}]},{"osd":3,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.3360000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.554,"15min":1.554},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.085,"5min":3.085,"15min":3.085},"last":1.179}]},{"osd":6,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.4790000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.05}]},{"osd":7,"last update":"Fri Nov 24 06:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.6720000000000002,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.294},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.5289999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":152043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.84099999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":1.458},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.02}]},{"osd":2,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.694}]},{"osd":3,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.589}]},{"osd":5,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.5950000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.226},{"interface":"front","average":{"1min":1.581,"5min":1.581,"15min":1.581},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":3.52,"5min":3.52,"15min":3.52},"last":1.6399999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.7490000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":1.5289999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:27:51 2023","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":1.837}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":153573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":1.1080000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.786,"5min":1.786,"15min":1.786},"last":1.073}]},{"osd":2,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":3.5910000000000002,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":0.96099999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.081,"5min":3.081,"15min":3.081},"last":1.587},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.758,"5min":2.758,"15min":2.758},"last":1.9139999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.6960000000000002,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.472},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":8.8879999999999999,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.8180000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.423},"min":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.9810000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":1.861},{"interface":"front","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":3.798,"5min":3.798,"15min":3.798},"last":1.7370000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":389120,"data_stored":152009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.86799999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.1819999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.5049999999999999,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.028},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.2170000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.0589999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.587}]},{"osd":5,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":3.7410000000000001,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.516}]},{"osd":6,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.681},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":4.1210000000000004,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.2370000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:27:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":3.8119999999999998,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.456},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.2929999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":401408,"data_stored":153573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.0629999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.5069999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.1890000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.407},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.109}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.728}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":2.0910000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.7969999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.8220000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":742307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.081}]},{"osd":4,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.036},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.171}]},{"osd":5,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.2},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.3109999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.2440000000000002,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.53},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.4950000000000001,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.278}]},{"osd":7,"last update":"Fri Nov 24 06:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.3799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:28:31.174 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:28:31.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:28:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.448+0000 7f3dbe1b3640 1 Processor -- start 2023-11-24T06:28:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.449+0000 7f3dbe1b3640 1 -- start start 2023-11-24T06:28:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.450+0000 7f3dbe1b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3db80b1f30 0x7f3db80b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.450+0000 7f3dbe1b3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80d0050 con 0x7f3db815d190 2023-11-24T06:28:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.450+0000 7f3dbe1b3640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80d02a0 con 0x7f3db80d1d00 2023-11-24T06:28:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.450+0000 7f3dbe1b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80d0580 con 0x7f3db80b1f30 2023-11-24T06:28:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.451+0000 7f3dbd9b2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3db80b1f30 0x7f3db80b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.451+0000 7f3dbd1b1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3db815d190 0x7f3db80d15f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60774/0 (socket says 172.21.15.16:60774) 2023-11-24T06:28:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.451+0000 7f3dbd9b2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3db80b1f30 0x7f3db80b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50396/0 (socket says 172.21.15.16:50396) 2023-11-24T06:28:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.451+0000 7f3dbd1b1640 1 -- 172.21.15.16:0/3657689292 learned_addr learned my addr 172.21.15.16:0/3657689292 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.451+0000 7f3dbd9b2640 1 -- 172.21.15.16:0/3657689292 >> v1:172.21.15.173:6789/0 conn(0x7f3db80d1d00 legacy=0x7f3db80b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.451+0000 7f3dbd9b2640 1 -- 172.21.15.16:0/3657689292 >> v1:172.21.15.16:6789/0 conn(0x7f3db815d190 legacy=0x7f3db80d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.451+0000 7f3dbd9b2640 1 -- 172.21.15.16:0/3657689292 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db80d0860 con 0x7f3db80b1f30 2023-11-24T06:28:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.453+0000 7f3dbd9b2640 1 --2- 172.21.15.16:0/3657689292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80b1f30 0x7f3db80b0890 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009dc0 tx=0x7f3db002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1926d17e22cb88 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.453+0000 7f3dabfff640 1 -- 172.21.15.16:0/3657689292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db002cc40 con 0x7f3db80b1f30 2023-11-24T06:28:31.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.453+0000 7f3dabfff640 1 -- 172.21.15.16:0/3657689292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db002cda0 con 0x7f3db80b1f30 2023-11-24T06:28:31.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.453+0000 7f3dabfff640 1 -- 172.21.15.16:0/3657689292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0034660 con 0x7f3db80b1f30 2023-11-24T06:28:31.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.453+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/3657689292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80b1f30 msgr2=0x7f3db80b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.453+0000 7f3dbe1b3640 1 --2- 172.21.15.16:0/3657689292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80b1f30 0x7f3db80b0890 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009dc0 tx=0x7f3db002be90 comp rx=0 tx=0).stop 2023-11-24T06:28:31.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.454+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/3657689292 shutdown_connections 2023-11-24T06:28:31.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.454+0000 7f3dbe1b3640 1 --2- 172.21.15.16:0/3657689292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db80b1f30 0x7f3db80b0890 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.454+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/3657689292 >> 172.21.15.16:0/3657689292 conn(0x7f3db80d3010 msgr2=0x7f3db80d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:31.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.454+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/3657689292 shutdown_connections 2023-11-24T06:28:31.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.454+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/3657689292 wait complete. 2023-11-24T06:28:31.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.455+0000 7f3dbe1b3640 1 Processor -- start 2023-11-24T06:28:31.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.456+0000 7f3dbe1b3640 1 -- start start 2023-11-24T06:28:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.456+0000 7f3dbe1b3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3db80b1f30 0x7f3db815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.456+0000 7f3dbe1b3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db80d1d00 0x7f3db80b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbe1b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db815d190 0x7f3db8158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbe1b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80b6b90 con 0x7f3db815d190 2023-11-24T06:28:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbe1b3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81e66c0 con 0x7f3db80d1d00 2023-11-24T06:28:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbe1b3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80b6e10 con 0x7f3db80b1f30 2023-11-24T06:28:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbd9b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db815d190 0x7f3db8158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbd9b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db815d190 0x7f3db8158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50410/0 (socket says 172.21.15.16:50410) 2023-11-24T06:28:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbd9b2640 1 -- 172.21.15.16:0/1892572657 learned_addr learned my addr 172.21.15.16:0/1892572657 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbd1b1640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3db80b1f30 0x7f3db815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbd9b2640 1 -- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3db80b1f30 msgr2=0x7f3db815ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbc9b0640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db80d1d00 0x7f3db80b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbd9b2640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3db80b1f30 0x7f3db815ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbd9b2640 1 -- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db80d1d00 msgr2=0x7f3db80b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbd9b2640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db80d1d00 0x7f3db80b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.457+0000 7f3dbd9b2640 1 -- 172.21.15.16:0/1892572657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db0009a70 con 0x7f3db815d190 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.458+0000 7f3dbd9b2640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db815d190 0x7f3db8158850 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3db0037040 tx=0x7f3db0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.458+0000 7f3da9ffb640 1 -- 172.21.15.16:0/1892572657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db003e610 con 0x7f3db815d190 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.458+0000 7f3da9ffb640 1 -- 172.21.15.16:0/1892572657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db003e770 con 0x7f3db815d190 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.458+0000 7f3da9ffb640 1 -- 172.21.15.16:0/1892572657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db003eb00 con 0x7f3db815d190 2023-11-24T06:28:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.458+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db80b70f0 con 0x7f3db815d190 2023-11-24T06:28:31.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.458+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db815f180 con 0x7f3db815d190 2023-11-24T06:28:31.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.460+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db815f340 con 0x7f3db815d190 2023-11-24T06:28:31.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.460+0000 7f3da9ffb640 1 -- 172.21.15.16:0/1892572657 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3db003b050 con 0x7f3db815d190 2023-11-24T06:28:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.461+0000 7f3da9ffb640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d9c06ae40 0x7f3d9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.461+0000 7f3da9ffb640 1 -- 172.21.15.16:0/1892572657 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3db00ac0b0 con 0x7f3db815d190 2023-11-24T06:28:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.462+0000 7f3dbd1b1640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d9c06ae40 0x7f3d9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.463+0000 7f3dbd1b1640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d9c06ae40 0x7f3d9c06d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3db4005880 tx=0x7f3db4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.466+0000 7f3da9ffb640 1 -- 172.21.15.16:0/1892572657 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3db00770c0 con 0x7f3db815d190 2023-11-24T06:28:31.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.735+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3db8067450 con 0x7f3db815d190 2023-11-24T06:28:31.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.740+0000 7f3da9ffb640 1 -- 172.21.15.16:0/1892572657 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3db007a6e0 con 0x7f3db815d190 2023-11-24T06:28:31.744 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-24T06:28:31.744 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA== 2023-11-24T06:28:31.744 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2023-11-24T06:28:31.744 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2023-11-24T06:28:31.744 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:28:31.744 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-11-24T06:28:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d9c06ae40 msgr2=0x7f3d9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d9c06ae40 0x7f3d9c06d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3db4005880 tx=0x7f3db4009040 comp rx=0 tx=0).stop 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db815d190 msgr2=0x7f3db8158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db815d190 0x7f3db8158850 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3db0037040 tx=0x7f3db0004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 shutdown_connections 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3d9c06ae40 0x7f3d9c06d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db815d190 0x7f3db8158850 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3db80d1d00 0x7f3db80b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 --2- 172.21.15.16:0/1892572657 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3db80b1f30 0x7f3db815ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 >> 172.21.15.16:0/1892572657 conn(0x7f3db80d3010 msgr2=0x7f3db80b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 shutdown_connections 2023-11-24T06:28:31.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.743+0000 7f3dbe1b3640 1 -- 172.21.15.16:0/1892572657 wait complete. 2023-11-24T06:28:31.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:31.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.987+0000 7f4f32e09640 1 Processor -- start 2023-11-24T06:28:31.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.987+0000 7f4f32e09640 1 -- start start 2023-11-24T06:28:31.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.989+0000 7f4f32e09640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f2c0cf9a0 0x7f4f2c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.989+0000 7f4f32e09640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c070c40 con 0x7f4f2c0ae5a0 2023-11-24T06:28:31.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.989+0000 7f4f32e09640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c06c470 con 0x7f4f2c0b3ae0 2023-11-24T06:28:31.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.989+0000 7f4f32e09640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f2c071450 con 0x7f4f2c0cf9a0 2023-11-24T06:28:31.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.989+0000 7f4f30ba5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4f2c0ae5a0 0x7f4f2c06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60778/0 (socket says 172.21.15.16:60778) 2023-11-24T06:28:31.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.989+0000 7f4f30ba5640 1 -- 172.21.15.16:0/3159405922 learned_addr learned my addr 172.21.15.16:0/3159405922 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:31.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.989+0000 7f4f313a6640 1 --2- 172.21.15.16:0/3159405922 >> v2:172.21.15.16:3300/0 conn(0x7f4f2c0cf9a0 0x7f4f2c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.990+0000 7f4f2b7fe640 1 -- 172.21.15.16:0/3159405922 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590636289 0 0) 0x7f4f2c070c40 con 0x7f4f2c0ae5a0 2023-11-24T06:28:31.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.990+0000 7f4f2b7fe640 1 -- 172.21.15.16:0/3159405922 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f1c003520 con 0x7f4f2c0ae5a0 2023-11-24T06:28:31.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.990+0000 7f4f313a6640 1 -- 172.21.15.16:0/3159405922 >> v1:172.21.15.173:6789/0 conn(0x7f4f2c0b3ae0 legacy=0x7f4f2c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:31.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.990+0000 7f4f313a6640 1 -- 172.21.15.16:0/3159405922 >> v1:172.21.15.16:6789/0 conn(0x7f4f2c0ae5a0 legacy=0x7f4f2c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.990+0000 7f4f313a6640 1 -- 172.21.15.16:0/3159405922 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c06c750 con 0x7f4f2c0cf9a0 2023-11-24T06:28:31.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.992+0000 7f4f313a6640 1 --2- 172.21.15.16:0/3159405922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0cf9a0 0x7f4f2c15e430 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009d90 tx=0x7f4f2402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406a11c565c23a9d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:31.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.992+0000 7f4f2b7fe640 1 -- 172.21.15.16:0/3159405922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2402cd00 con 0x7f4f2c0cf9a0 2023-11-24T06:28:31.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.992+0000 7f4f2b7fe640 1 -- 172.21.15.16:0/3159405922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2402ce60 con 0x7f4f2c0cf9a0 2023-11-24T06:28:31.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.992+0000 7f4f2b7fe640 1 -- 172.21.15.16:0/3159405922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24034660 con 0x7f4f2c0cf9a0 2023-11-24T06:28:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.992+0000 7f4f32e09640 1 -- 172.21.15.16:0/3159405922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0cf9a0 msgr2=0x7f4f2c15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.992+0000 7f4f32e09640 1 --2- 172.21.15.16:0/3159405922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0cf9a0 0x7f4f2c15e430 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009d90 tx=0x7f4f2402bf50 comp rx=0 tx=0).stop 2023-11-24T06:28:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.992+0000 7f4f32e09640 1 -- 172.21.15.16:0/3159405922 shutdown_connections 2023-11-24T06:28:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.992+0000 7f4f32e09640 1 --2- 172.21.15.16:0/3159405922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0cf9a0 0x7f4f2c15e430 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.992+0000 7f4f32e09640 1 -- 172.21.15.16:0/3159405922 >> 172.21.15.16:0/3159405922 conn(0x7f4f2c06a800 msgr2=0x7f4f2c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.993+0000 7f4f32e09640 1 -- 172.21.15.16:0/3159405922 shutdown_connections 2023-11-24T06:28:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.993+0000 7f4f32e09640 1 -- 172.21.15.16:0/3159405922 wait complete. 2023-11-24T06:28:31.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.994+0000 7f4f32e09640 1 Processor -- start 2023-11-24T06:28:31.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.994+0000 7f4f32e09640 1 -- start start 2023-11-24T06:28:31.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.995+0000 7f4f32e09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0ae5a0 0x7f4f2c157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.995+0000 7f4f32e09640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f2c0b3ae0 0x7f4f2c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.995+0000 7f4f32e09640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f2c0cf9a0 0x7f4f2c1f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:31.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.995+0000 7f4f32e09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c0cbe40 con 0x7f4f2c0ae5a0 2023-11-24T06:28:31.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.995+0000 7f4f32e09640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c0cc0e0 con 0x7f4f2c0b3ae0 2023-11-24T06:28:31.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.995+0000 7f4f32e09640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c0cc3c0 con 0x7f4f2c0cf9a0 2023-11-24T06:28:31.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.996+0000 7f4f30ba5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0ae5a0 0x7f4f2c157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:31.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.996+0000 7f4f30ba5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0ae5a0 0x7f4f2c157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50434/0 (socket says 172.21.15.16:50434) 2023-11-24T06:28:31.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.996+0000 7f4f30ba5640 1 -- 172.21.15.16:0/2274682352 learned_addr learned my addr 172.21.15.16:0/2274682352 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:32.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.996+0000 7f4f30ba5640 1 -- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f2c0cf9a0 msgr2=0x7f4f2c1f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.996+0000 7f4f30ba5640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f2c0cf9a0 0x7f4f2c1f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.996+0000 7f4f30ba5640 1 -- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f2c0b3ae0 msgr2=0x7f4f2c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.996+0000 7f4f30ba5640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f2c0b3ae0 0x7f4f2c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.996+0000 7f4f30ba5640 1 -- 172.21.15.16:0/2274682352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f24009a70 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.997+0000 7f4f30ba5640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0ae5a0 0x7f4f2c157ee0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000c500 tx=0x7f4f2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.997+0000 7f4f297fa640 1 -- 172.21.15.16:0/2274682352 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20005ce0 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.997+0000 7f4f297fa640 1 -- 172.21.15.16:0/2274682352 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f20005e40 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.997+0000 7f4f297fa640 1 -- 172.21.15.16:0/2274682352 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20007330 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.997+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c0cc6a0 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.997+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c1f7070 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.999+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2c0ccbc0 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:31.999+0000 7f4f297fa640 1 -- 172.21.15.16:0/2274682352 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f20007550 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.000+0000 7f4f297fa640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4f1006ae40 0x7f4f1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.001+0000 7f4f297fa640 1 -- 172.21.15.16:0/2274682352 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f20088cc0 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.001+0000 7f4f2bfff640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4f1006ae40 0x7f4f1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.002+0000 7f4f2bfff640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4f1006ae40 0x7f4f1006d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c1f6270 tx=0x7f4f180079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.005+0000 7f4f297fa640 1 -- 172.21.15.16:0/2274682352 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f2c0ccbc0 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.239+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f2c0ccbc0 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.240+0000 7f4f297fa640 1 -- 172.21.15.16:0/2274682352 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f4f20017020 con 0x7f4f2c0ae5a0 2023-11-24T06:28:32.244 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:32.244 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/147489979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":147489979},{"type":"v1","addr":"172.21.15.173:6836","nonce":147489979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1330874839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1330874839},{"type":"v1","addr":"172.21.15.173:6837","nonce":1330874839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2341098684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2341098684},{"type":"v1","addr":"172.21.15.16:6835","nonce":2341098684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:28:20.189904+0000","modified":"2023-11-24T06:28:23.468425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6834/4049479925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4049479925},{"type":"v1","addr":"172.21.15.16:6834","nonce":4049479925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:28:32.244 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 16 2023-11-24T06:28:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.243+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4f1006ae40 msgr2=0x7f4f1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.243+0000 7f4f32e09640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4f1006ae40 0x7f4f1006d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c1f6270 tx=0x7f4f180079b0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.243+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0ae5a0 msgr2=0x7f4f2c157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.243+0000 7f4f32e09640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0ae5a0 0x7f4f2c157ee0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000c500 tx=0x7f4f2000c9c0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.244+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 shutdown_connections 2023-11-24T06:28:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.244+0000 7f4f32e09640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4f1006ae40 0x7f4f1006d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.244+0000 7f4f32e09640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f2c0cf9a0 0x7f4f2c1f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.244+0000 7f4f32e09640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f2c0b3ae0 0x7f4f2c0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.244+0000 7f4f32e09640 1 --2- 172.21.15.16:0/2274682352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f2c0ae5a0 0x7f4f2c157ee0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.244+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 >> 172.21.15.16:0/2274682352 conn(0x7f4f2c06a800 msgr2=0x7f4f2c15e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.244+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 shutdown_connections 2023-11-24T06:28:32.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.244+0000 7f4f32e09640 1 -- 172.21.15.16:0/2274682352 wait complete. 2023-11-24T06:28:32.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:28:20.189904+0000', 'modified': '2023-11-24T06:28:23.468425+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5211}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5211': {'gid': 5211, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.16:6834/4049479925', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 4049479925}, {'type': 'v1', 'addr': '172.21.15.16:6834', 'nonce': 4049479925}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:28:32.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:28:32.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T06:28:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.902+0000 7f23a359e640 1 Processor -- start 2023-11-24T06:28:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.902+0000 7f23a359e640 1 -- start start 2023-11-24T06:28:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.903+0000 7f23a359e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f23a4143c40 0x7f23a4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23a40717d0 0x7f23a40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a359e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a4070e50 con 0x7f23a40cca40 2023-11-24T06:28:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a40723c0 con 0x7f23a40717d0 2023-11-24T06:28:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a359e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a40cce40 con 0x7f23a4143c40 2023-11-24T06:28:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a1d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23a40717d0 0x7f23a40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a1d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23a40717d0 0x7f23a40cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46720/0 (socket says 172.21.15.16:46720) 2023-11-24T06:28:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a1d9b640 1 -- 172.21.15.16:0/1369825771 learned_addr learned my addr 172.21.15.16:0/1369825771 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a1d9b640 1 -- 172.21.15.16:0/1369825771 >> v2:172.21.15.173:3300/0 conn(0x7f23a4143c40 msgr2=0x7f23a4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a259c640 1 --2- 172.21.15.16:0/1369825771 >> v2:172.21.15.173:3300/0 conn(0x7f23a4143c40 0x7f23a4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/1369825771 >> v2:172.21.15.173:3300/0 conn(0x7f23a4143c40 0x7f23a4071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a1d9b640 1 -- 172.21.15.16:0/1369825771 >> v1:172.21.15.173:6789/0 conn(0x7f23a40cca40 legacy=0x7f23a415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.904+0000 7f23a1d9b640 1 -- 172.21.15.16:0/1369825771 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a406e170 con 0x7f23a40717d0 2023-11-24T06:28:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.906+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/1369825771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40717d0 0x7f23a40cc500 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f23a406e410 tx=0x7f239802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1c67efbd694fb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.906+0000 7f23a159a640 1 -- 172.21.15.16:0/1369825771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980317e0 con 0x7f23a40717d0 2023-11-24T06:28:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.906+0000 7f23a159a640 1 -- 172.21.15.16:0/1369825771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398031940 con 0x7f23a40717d0 2023-11-24T06:28:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.906+0000 7f23a159a640 1 -- 172.21.15.16:0/1369825771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398031cd0 con 0x7f23a40717d0 2023-11-24T06:28:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.907+0000 7f23a359e640 1 -- 172.21.15.16:0/1369825771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40717d0 msgr2=0x7f23a40cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.907+0000 7f23a359e640 1 --2- 172.21.15.16:0/1369825771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40717d0 0x7f23a40cc500 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f23a406e410 tx=0x7f239802b880 comp rx=0 tx=0).stop 2023-11-24T06:28:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.907+0000 7f23a359e640 1 -- 172.21.15.16:0/1369825771 shutdown_connections 2023-11-24T06:28:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.907+0000 7f23a359e640 1 --2- 172.21.15.16:0/1369825771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40717d0 0x7f23a40cc500 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.907+0000 7f23a359e640 1 --2- 172.21.15.16:0/1369825771 >> v2:172.21.15.173:3300/0 conn(0x7f23a4143c40 0x7f23a4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.907+0000 7f23a359e640 1 -- 172.21.15.16:0/1369825771 >> 172.21.15.16:0/1369825771 conn(0x7f23a406a800 msgr2=0x7f23a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.907+0000 7f23a359e640 1 -- 172.21.15.16:0/1369825771 shutdown_connections 2023-11-24T06:28:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.907+0000 7f23a359e640 1 -- 172.21.15.16:0/1369825771 wait complete. 2023-11-24T06:28:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.909+0000 7f23a359e640 1 Processor -- start 2023-11-24T06:28:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.909+0000 7f23a359e640 1 -- start start 2023-11-24T06:28:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.910+0000 7f23a359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a40717d0 0x7f23a4159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a40cca40 0x7f23a41e3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4143c40 0x7f23a41e4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a41e0270 con 0x7f23a4143c40 2023-11-24T06:28:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a41e4ef0 con 0x7f23a40717d0 2023-11-24T06:28:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a41e4650 con 0x7f23a40cca40 2023-11-24T06:28:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4143c40 0x7f23a41e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4143c40 0x7f23a41e4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46724/0 (socket says 172.21.15.16:46724) 2023-11-24T06:28:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a2d9d640 1 -- 172.21.15.16:0/1007256364 learned_addr learned my addr 172.21.15.16:0/1007256364 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a259c640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a40717d0 0x7f23a4159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a40cca40 0x7f23a41e3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a2d9d640 1 -- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a40cca40 msgr2=0x7f23a41e3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a2d9d640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a40cca40 0x7f23a41e3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a2d9d640 1 -- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a40717d0 msgr2=0x7f23a4159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a2d9d640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a40717d0 0x7f23a4159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.911+0000 7f23a2d9d640 1 -- 172.21.15.16:0/1007256364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980094d0 con 0x7f23a4143c40 2023-11-24T06:28:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.912+0000 7f23a2d9d640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4143c40 0x7f23a41e4040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f239c00c390 tx=0x7f239c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.912+0000 7f2392ffd640 1 -- 172.21.15.16:0/1007256364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c0035f0 con 0x7f23a4143c40 2023-11-24T06:28:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.912+0000 7f2392ffd640 1 -- 172.21.15.16:0/1007256364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239c003750 con 0x7f23a4143c40 2023-11-24T06:28:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.912+0000 7f2392ffd640 1 -- 172.21.15.16:0/1007256364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c003b00 con 0x7f23a4143c40 2023-11-24T06:28:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.912+0000 7f23a359e640 1 -- 172.21.15.16:0/1007256364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a41e4930 con 0x7f23a4143c40 2023-11-24T06:28:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.912+0000 7f23a359e640 1 -- 172.21.15.16:0/1007256364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a41e5930 con 0x7f23a4143c40 2023-11-24T06:28:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.915+0000 7f2392ffd640 1 -- 172.21.15.16:0/1007256364 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f239c003c60 con 0x7f23a4143c40 2023-11-24T06:28:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.916+0000 7f2392ffd640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238406ae40 0x7f238406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.916+0000 7f2392ffd640 1 -- 172.21.15.16:0/1007256364 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f239c088790 con 0x7f23a4143c40 2023-11-24T06:28:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.916+0000 7f23a259c640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238406ae40 0x7f238406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.916+0000 7f23ab797740 1 Processor -- start 2023-11-24T06:28:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.917+0000 7f23ab797740 1 -- start start 2023-11-24T06:28:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.917+0000 7f23a259c640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238406ae40 0x7f238406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f23940070b0 tx=0x7f2394007040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23ab797740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442c71130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23ab797740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c46c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23ab797740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c471b0 0x55d442c89730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23ab797740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d442ba4880 con 0x55d442c70d50 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23ab797740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d442bb3ae0 con 0x55d442c40d70 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23ab797740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d442bb11d0 con 0x55d442c471b0 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442c71130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442c71130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46738/0 (socket says 172.21.15.16:46738) 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23a259c640 1 -- 172.21.15.16:0/292633108 learned_addr learned my addr 172.21.15.16:0/292633108 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/292633108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c46c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23a259c640 1 -- 172.21.15.16:0/292633108 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c471b0 msgr2=0x55d442c89730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23a259c640 1 --2- 172.21.15.16:0/292633108 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c471b0 0x55d442c89730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23a259c640 1 -- 172.21.15.16:0/292633108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 msgr2=0x55d442c46c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23a259c640 1 --2- 172.21.15.16:0/292633108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c46c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.918+0000 7f23a259c640 1 -- 172.21.15.16:0/292633108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d442b74860 con 0x55d442c70d50 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.919+0000 7f23a259c640 1 --2- 172.21.15.16:0/292633108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442c71130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f23940076e0 tx=0x7f2394011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0181bb07d593d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.919+0000 7f2383fff640 1 -- 172.21.15.16:0/292633108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23940106a0 con 0x55d442c70d50 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.919+0000 7f2383fff640 1 -- 172.21.15.16:0/292633108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394010800 con 0x55d442c70d50 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.919+0000 7f2383fff640 1 -- 172.21.15.16:0/292633108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394010bb0 con 0x55d442c70d50 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.920+0000 7f23ab797740 1 -- 172.21.15.16:0/292633108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 msgr2=0x55d442c71130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.920+0000 7f23ab797740 1 --2- 172.21.15.16:0/292633108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442c71130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f23940076e0 tx=0x7f2394011000 comp rx=0 tx=0).stop 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.920+0000 7f23ab797740 1 -- 172.21.15.16:0/292633108 shutdown_connections 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.920+0000 7f23ab797740 1 --2- 172.21.15.16:0/292633108 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c471b0 0x55d442c89730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.920+0000 7f23ab797740 1 --2- 172.21.15.16:0/292633108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c46c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.920+0000 7f23ab797740 1 --2- 172.21.15.16:0/292633108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442c71130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.920+0000 7f23ab797740 1 -- 172.21.15.16:0/292633108 >> 172.21.15.16:0/292633108 conn(0x55d442c07c70 msgr2=0x55d442c8bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.920+0000 7f23ab797740 1 -- 172.21.15.16:0/292633108 shutdown_connections 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.920+0000 7f23ab797740 1 -- 172.21.15.16:0/292633108 wait complete. 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.921+0000 7f23ab797740 1 -- start start 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.921+0000 7f23ab797740 1 Processor -- start 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23ab797740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c72930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23ab797740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442ca3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23ab797740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c62280 0x55d442ca3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23ab797740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d442bacac0 con 0x55d442c70d50 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23ab797740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d442b492f0 con 0x55d442c40d70 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23ab797740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d442b66c70 con 0x55d442c62280 2023-11-24T06:28:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23ab797740 10 client.0 _ll_register_callbacks cb 0x55d442b6b3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23a1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442ca3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23a1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442ca3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46754/0 (socket says 172.21.15.16:46754) 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.922+0000 7f23a1d9b640 1 -- 172.21.15.16:0/141209159 learned_addr learned my addr 172.21.15.16:0/141209159 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23a2d9d640 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c62280 0x55d442ca3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23a259c640 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c72930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23a1d9b640 1 -- 172.21.15.16:0/141209159 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c62280 msgr2=0x55d442ca3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c62280 0x55d442ca3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23a1d9b640 1 -- 172.21.15.16:0/141209159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 msgr2=0x55d442c72930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c72930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23a1d9b640 1 -- 172.21.15.16:0/141209159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2394008040 con 0x55d442c70d50 2023-11-24T06:28:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442ca3570 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23980090b0 tx=0x7f2398004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2317f51d675b58ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23817fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23817fa640 1 -- 172.21.15.16:0/141209159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398032880 con 0x55d442c70d50 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23817fa640 1 -- 172.21.15.16:0/141209159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2398040040 con 0x55d442c70d50 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.923+0000 7f23ab797740 1 -- 172.21.15.16:0/141209159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d442ba5aa0 con 0x55d442c70d50 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23817fa640 1 -- 172.21.15.16:0/141209159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23980329e0 con 0x55d442c70d50 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23ab797740 10 client.5421 fetch_fsmap learned FSMap version 16 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23ab797740 1 -- 172.21.15.16:0/141209159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d442bab140 con 0x55d442c70d50 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23817fa640 1 -- 172.21.15.16:0/141209159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d442ba5aa0 con 0x55d442c70d50 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23817fa640 1 -- 172.21.15.16:0/141209159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f239804d060 con 0x55d442c70d50 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23ab797740 10 client.5421 fetch_fsmap finished waiting for FSMap version 16 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23ab797740 10 client.5421 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:28:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23ab797740 20 client.5421 populate_metadata read hostname 'smithi016' 2023-11-24T06:28:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23ab797740 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] conn(0x55d442c60580 0x55d442c60960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23ab797740 4 client.5421 mds_command: new command op to 5211 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:28:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.924+0000 7f23ab797740 1 -- 172.21.15.16:0/141209159 --> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d442ba57c0 con 0x55d442c60580 2023-11-24T06:28:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.925+0000 7f23a259c640 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] conn(0x55d442c60580 0x55d442c60960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.925+0000 7f23a259c640 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] conn(0x55d442c60580 0x55d442c60960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49b4a070bc8b7a7 server_cookie=3f9efd53bf97fad7 in_seq=0 out_seq=0 2023-11-24T06:28:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.925+0000 7f23817fa640 10 client.5421 ms_handle_connect on v2:172.21.15.16:6833/4049479925 2023-11-24T06:28:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23817fa640 1 -- 172.21.15.16:0/141209159 <== mds.0 v2:172.21.15.16:6833/4049479925 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d442ba57c0 con 0x55d442c60580 2023-11-24T06:28:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23817fa640 10 client.5421 handle_command_reply: tid=0 2023-11-24T06:28:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23ab797740 1 client.5421 shutdown 2023-11-24T06:28:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23ab797740 1 -- 172.21.15.16:0/141209159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 msgr2=0x55d442ca3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23ab797740 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442ca3570 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23980090b0 tx=0x7f2398004730 comp rx=0 tx=0).stop 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23ab797740 1 -- 172.21.15.16:0/141209159 shutdown_connections 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23ab797740 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] conn(0x55d442c60580 0x55d442c60960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23ab797740 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c62280 0x55d442ca3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23ab797740 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c70d50 0x55d442ca3570 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23817fa640 0 client.5421 ms_handle_reset on v2:172.21.15.16:6833/4049479925 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.926+0000 7f23ab797740 1 --2- 172.21.15.16:0/141209159 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c72930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.927+0000 7f23ab797740 1 -- 172.21.15.16:0/141209159 >> 172.21.15.16:0/141209159 conn(0x55d442c07c70 msgr2=0x55d442c8b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.927+0000 7f23ab797740 1 -- 172.21.15.16:0/141209159 shutdown_connections 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.927+0000 7f23ab797740 1 -- 172.21.15.16:0/141209159 wait complete. 2023-11-24T06:28:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.927+0000 7f23ab797740 20 client.5421 trim_cache size 0 max 16384 2023-11-24T06:28:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.944+0000 7f23ab797740 1 Processor -- start 2023-11-24T06:28:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.944+0000 7f23ab797740 1 -- start start 2023-11-24T06:28:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.945+0000 7f23ab797740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c8b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.945+0000 7f23ab797740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c62280 0x55d442c447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.945+0000 7f23ab797740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 0x55d442c44d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.945+0000 7f23ab797740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d44290ff20 con 0x55d442c62280 2023-11-24T06:28:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.945+0000 7f23ab797740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d442c6e940 con 0x55d442c40d70 2023-11-24T06:28:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.945+0000 7f23ab797740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d442c42dc0 con 0x55d442c70d50 2023-11-24T06:28:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.945+0000 7f23a1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c62280 0x55d442c447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.945+0000 7f23a1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c62280 0x55d442c447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46756/0 (socket says 172.21.15.16:46756) 2023-11-24T06:28:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.945+0000 7f23a1d9b640 1 -- 172.21.15.16:0/169617773 learned_addr learned my addr 172.21.15.16:0/169617773 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:32.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f23a1d9b640 1 -- 172.21.15.16:0/169617773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 msgr2=0x55d442c44d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/169617773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 0x55d442c44d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f23a259c640 1 --2- 172.21.15.16:0/169617773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c8b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f23a1d9b640 1 -- 172.21.15.16:0/169617773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 msgr2=0x55d442c8b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/169617773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c8b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f23a1d9b640 1 -- 172.21.15.16:0/169617773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d442b48b80 con 0x55d442c62280 2023-11-24T06:28:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/169617773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c62280 0x55d442c447d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2398000c00 tx=0x7f2398042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd62a22cb7b8d35 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f2390ff9640 1 -- 172.21.15.16:0/169617773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398031270 con 0x55d442c62280 2023-11-24T06:28:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f2390ff9640 1 -- 172.21.15.16:0/169617773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398032680 con 0x55d442c62280 2023-11-24T06:28:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.946+0000 7f2390ff9640 1 -- 172.21.15.16:0/169617773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398032a10 con 0x55d442c62280 2023-11-24T06:28:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.947+0000 7f23ab797740 1 -- 172.21.15.16:0/169617773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c62280 msgr2=0x55d442c447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.947+0000 7f23ab797740 1 --2- 172.21.15.16:0/169617773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c62280 0x55d442c447d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2398000c00 tx=0x7f2398042000 comp rx=0 tx=0).stop 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.947+0000 7f23ab797740 1 -- 172.21.15.16:0/169617773 shutdown_connections 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.947+0000 7f23ab797740 1 --2- 172.21.15.16:0/169617773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 0x55d442c44d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.947+0000 7f23ab797740 1 --2- 172.21.15.16:0/169617773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c62280 0x55d442c447d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.947+0000 7f23ab797740 1 --2- 172.21.15.16:0/169617773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c40d70 0x55d442c8b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.947+0000 7f23ab797740 1 -- 172.21.15.16:0/169617773 >> 172.21.15.16:0/169617773 conn(0x55d442c07c70 msgr2=0x55d442c45d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.947+0000 7f23ab797740 1 -- 172.21.15.16:0/169617773 shutdown_connections 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.947+0000 7f23ab797740 1 -- 172.21.15.16:0/169617773 wait complete. 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.948+0000 7f23ab797740 1 -- start start 2023-11-24T06:28:32.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.948+0000 7f23ab797740 1 Processor -- start 2023-11-24T06:28:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23ab797740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c40d70 0x55d442ce3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23ab797740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c62280 0x55d442c5fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23ab797740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 0x55d442c881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23ab797740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d442c71e40 con 0x55d442c40d70 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23ab797740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d442c5f750 con 0x55d442c62280 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23ab797740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d442ce4170 con 0x55d442c70d50 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23ab797740 10 client.0 _ll_register_callbacks cb 0x7f23980096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c40d70 0x55d442ce3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23a259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c40d70 0x55d442ce3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46770/0 (socket says 172.21.15.16:46770) 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.949+0000 7f23a259c640 1 -- 172.21.15.16:0/514628134 learned_addr learned my addr 172.21.15.16:0/514628134 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c62280 0x55d442c5fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23a2d9d640 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 0x55d442c881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23a259c640 1 -- 172.21.15.16:0/514628134 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 msgr2=0x55d442c881a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23a259c640 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 0x55d442c881a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23a259c640 1 -- 172.21.15.16:0/514628134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c62280 msgr2=0x55d442c5fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23a259c640 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c62280 0x55d442c5fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23a259c640 1 -- 172.21.15.16:0/514628134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d442bab140 con 0x55d442c40d70 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23a2d9d640 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 0x55d442c881a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23a259c640 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c40d70 0x55d442ce3c30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2394020040 tx=0x7f2394015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b9b4f9114b687d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23817fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.950+0000 7f23817fa640 1 -- 172.21.15.16:0/514628134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239402e040 con 0x55d442c40d70 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23817fa640 1 -- 172.21.15.16:0/514628134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23940337b0 con 0x55d442c40d70 2023-11-24T06:28:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23ab797740 1 -- 172.21.15.16:0/514628134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2394010800 con 0x55d442c40d70 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23817fa640 1 -- 172.21.15.16:0/514628134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2394007ca0 con 0x55d442c40d70 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23ab797740 10 client.5427 fetch_fsmap learned FSMap version 16 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23ab797740 1 -- 172.21.15.16:0/514628134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55d442b6e170 con 0x55d442c40d70 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23817fa640 1 -- 172.21.15.16:0/514628134 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394010800 con 0x55d442c40d70 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23817fa640 1 -- 172.21.15.16:0/514628134 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2394003990 con 0x55d442c40d70 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23ab797740 10 client.5427 fetch_fsmap finished waiting for FSMap version 16 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23ab797740 10 client.5427 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23ab797740 20 client.5427 populate_metadata read hostname 'smithi016' 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23ab797740 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] conn(0x55d442c8bab0 0x55d442c8be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:32.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23ab797740 4 client.5427 mds_command: new command op to 5211 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23ab797740 1 -- 172.21.15.16:0/514628134 --> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d4428ad6c0 con 0x55d442c8bab0 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.951+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] conn(0x55d442c8bab0 0x55d442c8be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.952+0000 7f23a1d9b640 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] conn(0x55d442c8bab0 0x55d442c8be90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c966df75dc327ac server_cookie=27cf6243a13b9e25 in_seq=0 out_seq=0 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.952+0000 7f23817fa640 10 client.5427 ms_handle_connect on v2:172.21.15.16:6833/4049479925 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.952+0000 7f23817fa640 1 -- 172.21.15.16:0/514628134 <== mds.0 v2:172.21.15.16:6833/4049479925 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55d4428ad6c0 con 0x55d442c8bab0 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.952+0000 7f23817fa640 10 client.5427 handle_command_reply: tid=0 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.952+0000 7f23ab797740 1 client.5427 shutdown 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.952+0000 7f23ab797740 1 -- 172.21.15.16:0/514628134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c40d70 msgr2=0x55d442ce3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.952+0000 7f23ab797740 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c40d70 0x55d442ce3c30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2394020040 tx=0x7f2394015000 comp rx=0 tx=0).stop 2023-11-24T06:28:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23ab797740 1 -- 172.21.15.16:0/514628134 shutdown_connections 2023-11-24T06:28:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23ab797740 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.16:6833/4049479925,v1:172.21.15.16:6834/4049479925] conn(0x55d442c8bab0 0x55d442c8be90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23ab797740 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55d442c70d50 0x55d442c881a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23ab797740 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d442c62280 0x55d442c5fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23ab797740 1 --2- 172.21.15.16:0/514628134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d442c40d70 0x55d442ce3c30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23817fa640 0 client.5427 ms_handle_reset on v2:172.21.15.16:6833/4049479925 2023-11-24T06:28:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23ab797740 1 -- 172.21.15.16:0/514628134 >> 172.21.15.16:0/514628134 conn(0x55d442c07c70 msgr2=0x55d442c72690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23ab797740 1 -- 172.21.15.16:0/514628134 shutdown_connections 2023-11-24T06:28:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23ab797740 1 -- 172.21.15.16:0/514628134 wait complete. 2023-11-24T06:28:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23ab797740 20 client.5427 trim_cache size 0 max 16384 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6dc1668b-58b0-47dd-b8dc-4164e5a228ab", 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5211, 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 12.219815874, 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 16, 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 34, 2023-11-24T06:28:32.971 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 34, 2023-11-24T06:28:32.972 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 21.456673624 2023-11-24T06:28:32.972 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:28:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23a359e640 1 -- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238406ae40 msgr2=0x7f238406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23a359e640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238406ae40 0x7f238406d300 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f23940070b0 tx=0x7f2394007040 comp rx=0 tx=0).stop 2023-11-24T06:28:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23a359e640 1 -- 172.21.15.16:0/1007256364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4143c40 msgr2=0x7f23a41e4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.953+0000 7f23a359e640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4143c40 0x7f23a41e4040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f239c00c390 tx=0x7f239c00c850 comp rx=0 tx=0).stop 2023-11-24T06:28:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.954+0000 7f23a359e640 1 -- 172.21.15.16:0/1007256364 shutdown_connections 2023-11-24T06:28:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.954+0000 7f23a359e640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238406ae40 0x7f238406d300 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.954+0000 7f23a359e640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4143c40 0x7f23a41e4040 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.954+0000 7f23a359e640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a40cca40 0x7f23a41e3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.954+0000 7f23a359e640 1 --2- 172.21.15.16:0/1007256364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a40717d0 0x7f23a4159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.954+0000 7f23a359e640 1 -- 172.21.15.16:0/1007256364 >> 172.21.15.16:0/1007256364 conn(0x7f23a406a800 msgr2=0x7f23a4141230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.954+0000 7f23a359e640 1 -- 172.21.15.16:0/1007256364 shutdown_connections 2023-11-24T06:28:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:32.954+0000 7f23a359e640 1 -- 172.21.15.16:0/1007256364 wait complete. 2023-11-24T06:28:32.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:28:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.188+0000 7f32eaf1a640 1 Processor -- start 2023-11-24T06:28:33.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.188+0000 7f32eaf1a640 1 -- start start 2023-11-24T06:28:33.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.189+0000 7f32eaf1a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32e4143c40 0x7f32e4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.189+0000 7f32eaf1a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f32e40717d0 0x7f32e40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.189+0000 7f32eaf1a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e4070e50 con 0x7f32e40cca40 2023-11-24T06:28:33.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.189+0000 7f32eaf1a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e40723c0 con 0x7f32e40717d0 2023-11-24T06:28:33.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.189+0000 7f32eaf1a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40cce70 con 0x7f32e4143c40 2023-11-24T06:28:33.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.189+0000 7f32ea719640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32e40cca40 0x7f32e415eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34880/0 (socket says 172.21.15.16:34880) 2023-11-24T06:28:33.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.189+0000 7f32ea719640 1 -- 172.21.15.16:0/4181916961 learned_addr learned my addr 172.21.15.16:0/4181916961 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e9f18640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32e4143c40 0x7f32e4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e9717640 1 --2- 172.21.15.16:0/4181916961 >> v2:172.21.15.173:3300/0 conn(0x7f32e40717d0 0x7f32e40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562534654 0 0) 0x7f32e4070e50 con 0x7f32e40cca40 2023-11-24T06:28:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d0003520 con 0x7f32e40cca40 2023-11-24T06:28:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4292521310 0 0) 0x7f32d0003520 con 0x7f32e40cca40 2023-11-24T06:28:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32e4143c40 msgr2=0x7f32e4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32e4143c40 0x7f32e4071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 >> v2:172.21.15.173:3300/0 conn(0x7f32e40717d0 msgr2=0x7f32e40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 --2- 172.21.15.16:0/4181916961 >> v2:172.21.15.173:3300/0 conn(0x7f32e40717d0 0x7f32e40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e406e170 con 0x7f32e40cca40 2023-11-24T06:28:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f32dc0019f0 con 0x7f32e40cca40 2023-11-24T06:28:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 >> v1:172.21.15.16:6789/0 conn(0x7f32e40cca40 legacy=0x7f32e415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d0003970 0x7f32d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d00047c0 0x7f32d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32d0005180 0x7f32d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40723c0 con 0x7f32d0003970 2023-11-24T06:28:33.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40cce70 con 0x7f32d0005180 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.190+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d000c0a0 con 0x7f32d00047c0 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9f18640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32e4143c40 0x7f32e4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9717640 1 --2- 172.21.15.16:0/4181916961 >> v2:172.21.15.173:3300/0 conn(0x7f32e40717d0 0x7f32e40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9f18640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d0003970 0x7f32d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32ea719640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32d0005180 0x7f32d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9717640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d00047c0 0x7f32d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9f18640 1 -- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d00047c0 msgr2=0x7f32d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9f18640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d00047c0 0x7f32d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9f18640 1 -- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32d0005180 msgr2=0x7f32d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9f18640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32d0005180 0x7f32d000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9f18640 1 -- 172.21.15.16:0/4181916961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d0003520 con 0x7f32d0003970 2023-11-24T06:28:33.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.191+0000 7f32e9717640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d00047c0 0x7f32d0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.193+0000 7f32e9f18640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d0003970 0x7f32d0003d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f32e0006fd0 tx=0x7f32d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6015ca976cf97f73 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.193+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0033600 con 0x7f32d0003970 2023-11-24T06:28:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.193+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e002f050 con 0x7f32d0003970 2023-11-24T06:28:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.193+0000 7f32e8f16640 1 -- 172.21.15.16:0/4181916961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0033980 con 0x7f32d0003970 2023-11-24T06:28:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.193+0000 7f32eaf1a640 1 -- 172.21.15.16:0/4181916961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d0003970 msgr2=0x7f32d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.193+0000 7f32eaf1a640 1 --2- 172.21.15.16:0/4181916961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d0003970 0x7f32d0003d90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f32e0006fd0 tx=0x7f32d00057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.193+0000 7f32ea719640 1 -- 172.21.15.16:0/4181916961 reap_dead start 2023-11-24T06:28:33.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.194+0000 7f32eaf1a640 1 -- 172.21.15.16:0/4181916961 shutdown_connections 2023-11-24T06:28:33.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.194+0000 7f32eaf1a640 1 -- 172.21.15.16:0/4181916961 >> 172.21.15.16:0/4181916961 conn(0x7f32e406a800 msgr2=0x7f32e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:33.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.194+0000 7f32eaf1a640 1 -- 172.21.15.16:0/4181916961 shutdown_connections 2023-11-24T06:28:33.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.194+0000 7f32eaf1a640 1 -- 172.21.15.16:0/4181916961 wait complete. 2023-11-24T06:28:33.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.196+0000 7f32eaf1a640 1 Processor -- start 2023-11-24T06:28:33.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.196+0000 7f32eaf1a640 1 -- start start 2023-11-24T06:28:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32eaf1a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d0003970 0x7f32e4138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32eaf1a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32e41f5e30 0x7f32e4134e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32eaf1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e413a7c0 0x7f32e4142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32eaf1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e415aeb0 con 0x7f32e413a7c0 2023-11-24T06:28:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32eaf1a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e413b0e0 con 0x7f32e41f5e30 2023-11-24T06:28:33.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32eaf1a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e413b3c0 con 0x7f32d0003970 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32ea719640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e413a7c0 0x7f32e4142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32ea719640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e413a7c0 0x7f32e4142140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46796/0 (socket says 172.21.15.16:46796) 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32ea719640 1 -- 172.21.15.16:0/708586488 learned_addr learned my addr 172.21.15.16:0/708586488 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32e9f18640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d0003970 0x7f32e4138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.197+0000 7f32ea719640 1 -- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d0003970 msgr2=0x7f32e4138ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32ea719640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d0003970 0x7f32e4138ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32ea719640 1 -- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32e41f5e30 msgr2=0x7f32e4134e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32ea719640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32e41f5e30 0x7f32e4134e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32ea719640 1 -- 172.21.15.16:0/708586488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e413b6a0 con 0x7f32e413a7c0 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32ea719640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e413a7c0 0x7f32e4142140 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f32e40706c0 tx=0x7f32dc007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32da7fc640 1 -- 172.21.15.16:0/708586488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc00e9c0 con 0x7f32e413a7c0 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32da7fc640 1 -- 172.21.15.16:0/708586488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc00eb60 con 0x7f32e413a7c0 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e40b1fa0 con 0x7f32e413a7c0 2023-11-24T06:28:33.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32da7fc640 1 -- 172.21.15.16:0/708586488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc013420 con 0x7f32e413a7c0 2023-11-24T06:28:33.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.198+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e413ac40 con 0x7f32e413a7c0 2023-11-24T06:28:33.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.199+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e40b24c0 con 0x7f32e413a7c0 2023-11-24T06:28:33.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.200+0000 7f32da7fc640 1 -- 172.21.15.16:0/708586488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32dc013580 con 0x7f32e413a7c0 2023-11-24T06:28:33.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.201+0000 7f32da7fc640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32c806ae70 0x7f32c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:33.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.201+0000 7f32da7fc640 1 -- 172.21.15.16:0/708586488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32dc0898f0 con 0x7f32e413a7c0 2023-11-24T06:28:33.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.202+0000 7f32e9f18640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32c806ae70 0x7f32c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:33.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.203+0000 7f32e9f18640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32c806ae70 0x7f32c806d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f32e0036000 tx=0x7f32e0035040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.205+0000 7f32da7fc640 1 -- 172.21.15.16:0/708586488 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32e40b24c0 con 0x7f32e413a7c0 2023-11-24T06:28:33.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.442+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f32e4067450 con 0x7f32e413a7c0 2023-11-24T06:28:33.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.442+0000 7f32da7fc640 1 -- 172.21.15.16:0/708586488 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f32dc054900 con 0x7f32e413a7c0 2023-11-24T06:28:33.446 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:28:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32c806ae70 msgr2=0x7f32c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32c806ae70 0x7f32c806d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f32e0036000 tx=0x7f32e0035040 comp rx=0 tx=0).stop 2023-11-24T06:28:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e413a7c0 msgr2=0x7f32e4142140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e413a7c0 0x7f32e4142140 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f32e40706c0 tx=0x7f32dc007af0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 shutdown_connections 2023-11-24T06:28:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32c806ae70 0x7f32c806d330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32e413a7c0 0x7f32e4142140 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f32e41f5e30 0x7f32e4134e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 --2- 172.21.15.16:0/708586488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32d0003970 0x7f32e4138ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.445+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 >> 172.21.15.16:0/708586488 conn(0x7f32e406a800 msgr2=0x7f32e406d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.446+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 shutdown_connections 2023-11-24T06:28:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:33.446+0000 7f32eaf1a640 1 -- 172.21.15.16:0/708586488 wait complete. 2023-11-24T06:28:33.467 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:28:33.467 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:28:33.467 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-24T06:28:33.467 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:28:33.468 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:33.468 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:28:33.468 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:28:33.468 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:28:33.468 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:28:33.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:33.468 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-24T06:28:33.526 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-24T06:28:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42230sec preferred_lft 42230sec 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8cfd:d2ff:fee1:5b18/64 scope link 2023-11-24T06:28:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:33.529 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:28:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8c89:e0ff:fefc:a3fb/64 scope link 2023-11-24T06:28:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:33.529 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:28:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether ea:e3:8b:37:17:ee brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e8e3:8bff:fe37:17ee/64 scope link 2023-11-24T06:28:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:33.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:33.530 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:28:33.629 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-24T06:28:33.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:33.634 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-24T06:28:33.688 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-24T06:28:33.688 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:28:33.689 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:28:33.689 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:28:33.690 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:28:33.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:33.693 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:28:33.693 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:28:33.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:28:33.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:28:33.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:28:33.697 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:28:33.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:33.894 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-24T06:28:33.949 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:28:33.949 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:28:33.950 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:28:33.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:33.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-24T06:28:34.056 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:28:34.056 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:34.056 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:34.116 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:28:34.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:34.117 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:34.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:28:34.282 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:34.344 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:28:34.344 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:28:34.345 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:28:34.345 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:28:34.345 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:28:34.345 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:28:34.345 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:28:34.345 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:28:34.346 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:28:34.346 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:28:34.346 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:28:34.346 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:34.346 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:28:34.346 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:28:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:28:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:28:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:28:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:34.347 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:34.348 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:34.348 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:34.348 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:28:34.348 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:28:34.349 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:28:34.349 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:28:34.350 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:28:34.350 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:28:34.350 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:28:34.350 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:34.350 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:34.350 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:34.351 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:34.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:34.353 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:28:34.409 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:28:34.409 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:28:34.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:28:34.496 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:34.540 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:28:34.541 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:28:34.541 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:28:34.541 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:28:34.541 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:28:34.541 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:28:34.542 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:28:34.542 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:28:34.542 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:28:34.542 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:28:34.542 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:28:34.542 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:34.543 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:28:34.543 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:28:34.543 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:28:34.543 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:28:34.543 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:34.544 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:28:34.544 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:34.544 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:34.544 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:34.544 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:34.544 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:28:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:28:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:28:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:28:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:28:34.545 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:28:34.546 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:28:34.546 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:34.546 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:28:34.546 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:34.546 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:28:34.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:34.550 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:28:34.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-24T06:28:34.596+0000 7fa4ea03a480 -1 init, newargv = 0x556efb5e1bb0 newargc=15 2023-11-24T06:28:34.600 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[101477]: starting ceph client 2023-11-24T06:28:34.607 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[101477]: starting fuse 2023-11-24T06:28:34.636 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-24T06:28:34.636 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:28:34.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:28:34.636 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-24T06:28:34.636 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-24T06:28:34.636 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-24T06:28:34.637 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-24T06:28:34.638 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-24T06:28:34.843 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.101477.asok 2023-11-24T06:28:34.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:28:34.862 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.101477.asok 2023-11-24T06:28:34.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:34.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.101477.asok status 2023-11-24T06:28:35.114 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:28:35.114 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-24T06:28:35.114 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:28:35.114 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:28:35.114 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-24T06:28:35.114 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "101477", 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5442, 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5442 2023-11-24T06:28:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4222399762 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 4222399762 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5442 192.168.0.1:0/4222399762", 2023-11-24T06:28:35.116 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/4222399762", 2023-11-24T06:28:35.117 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-24T06:28:35.117 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 16, 2023-11-24T06:28:35.117 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 34, 2023-11-24T06:28:35.117 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:28:35.117 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-24T06:28:35.117 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-24T06:28:35.117 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:28:35.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:35.129 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-24T06:28:35.129 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:35.187 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-24T06:28:35.188 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:35.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:35.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:35.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:28:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.515+0000 7f028dad1640 1 Processor -- start 2023-11-24T06:28:35.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.515+0000 7f028dad1640 1 -- start start 2023-11-24T06:28:35.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.517+0000 7f028dad1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0288143c40 0x7f0288071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.517+0000 7f028dad1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f02880717d0 0x7f02880cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.517+0000 7f028dad1640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0288070e50 con 0x7f02880cca40 2023-11-24T06:28:35.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.517+0000 7f028dad1640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02880723c0 con 0x7f02880717d0 2023-11-24T06:28:35.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.517+0000 7f028dad1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02880cce70 con 0x7f0288143c40 2023-11-24T06:28:35.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.518+0000 7f028cacf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0288143c40 0x7f0288071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.518+0000 7f0287fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f02880717d0 0x7f02880cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.518+0000 7f0287fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f02880717d0 0x7f02880cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:34430/0 (socket says 172.21.15.16:34430) 2023-11-24T06:28:35.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.518+0000 7f028cacf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0288143c40 0x7f0288071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:54362/0 (socket says 172.21.15.16:54362) 2023-11-24T06:28:35.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.518+0000 7f0287fff640 1 -- 172.21.15.16:0/3924443409 learned_addr learned my addr 172.21.15.16:0/3924443409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:35.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.519+0000 7f028cacf640 1 -- 172.21.15.16:0/3924443409 >> v2:172.21.15.173:3300/0 conn(0x7f02880717d0 msgr2=0x7f02880cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:35.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.519+0000 7f028cacf640 1 --2- 172.21.15.16:0/3924443409 >> v2:172.21.15.173:3300/0 conn(0x7f02880717d0 0x7f02880cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.519+0000 7f028cacf640 1 -- 172.21.15.16:0/3924443409 >> v1:172.21.15.173:6789/0 conn(0x7f02880cca40 legacy=0x7f028815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.519+0000 7f028cacf640 1 -- 172.21.15.16:0/3924443409 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028806e170 con 0x7f0288143c40 2023-11-24T06:28:35.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.520+0000 7f028cacf640 1 --2- 172.21.15.16:0/3924443409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0288143c40 0x7f0288071290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f027c0097a0 tx=0x7f027c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6504669673fb2e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.520+0000 7f02877fe640 1 -- 172.21.15.16:0/3924443409 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c02e070 con 0x7f0288143c40 2023-11-24T06:28:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.520+0000 7f02877fe640 1 -- 172.21.15.16:0/3924443409 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027c0315a0 con 0x7f0288143c40 2023-11-24T06:28:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.520+0000 7f02877fe640 1 -- 172.21.15.16:0/3924443409 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c031930 con 0x7f0288143c40 2023-11-24T06:28:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.521+0000 7f028dad1640 1 -- 172.21.15.16:0/3924443409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0288143c40 msgr2=0x7f0288071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.521+0000 7f028dad1640 1 --2- 172.21.15.16:0/3924443409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0288143c40 0x7f0288071290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f027c0097a0 tx=0x7f027c02b840 comp rx=0 tx=0).stop 2023-11-24T06:28:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.521+0000 7f028dad1640 1 -- 172.21.15.16:0/3924443409 shutdown_connections 2023-11-24T06:28:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.521+0000 7f028dad1640 1 --2- 172.21.15.16:0/3924443409 >> v2:172.21.15.173:3300/0 conn(0x7f02880717d0 0x7f02880cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.521+0000 7f028dad1640 1 --2- 172.21.15.16:0/3924443409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0288143c40 0x7f0288071290 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.521+0000 7f028dad1640 1 -- 172.21.15.16:0/3924443409 >> 172.21.15.16:0/3924443409 conn(0x7f028806a800 msgr2=0x7f028806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:35.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.521+0000 7f028dad1640 1 -- 172.21.15.16:0/3924443409 shutdown_connections 2023-11-24T06:28:35.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.521+0000 7f028dad1640 1 -- 172.21.15.16:0/3924443409 wait complete. 2023-11-24T06:28:35.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.524+0000 7f028dad1640 1 Processor -- start 2023-11-24T06:28:35.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.524+0000 7f028dad1640 1 -- start start 2023-11-24T06:28:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.525+0000 7f028dad1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02880717d0 0x7f028815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.525+0000 7f028dad1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02880cca40 0x7f02881f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.525+0000 7f028dad1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288143c40 0x7f0288141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.525+0000 7f028dad1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02881432e0 con 0x7f0288143c40 2023-11-24T06:28:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.525+0000 7f028dad1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02881e3f00 con 0x7f02880717d0 2023-11-24T06:28:35.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.525+0000 7f028dad1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02881e1940 con 0x7f02880cca40 2023-11-24T06:28:35.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.526+0000 7f028d2d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288143c40 0x7f0288141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.526+0000 7f028d2d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288143c40 0x7f0288141fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46806/0 (socket says 172.21.15.16:46806) 2023-11-24T06:28:35.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.526+0000 7f028d2d0640 1 -- 172.21.15.16:0/3014926182 learned_addr learned my addr 172.21.15.16:0/3014926182 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:35.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.526+0000 7f028cacf640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02880717d0 0x7f028815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.526+0000 7f0287fff640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02880cca40 0x7f02881f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f028d2d0640 1 -- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02880cca40 msgr2=0x7f02881f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f028d2d0640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02880cca40 0x7f02881f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f028d2d0640 1 -- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02880717d0 msgr2=0x7f028815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f028d2d0640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02880717d0 0x7f028815a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f028d2d0640 1 -- 172.21.15.16:0/3014926182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c009400 con 0x7f0288143c40 2023-11-24T06:28:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f028d2d0640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288143c40 0x7f0288141fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f028000c7a0 tx=0x7f028000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f02857fa640 1 -- 172.21.15.16:0/3014926182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280003b70 con 0x7f0288143c40 2023-11-24T06:28:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f02857fa640 1 -- 172.21.15.16:0/3014926182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280003cf0 con 0x7f0288143c40 2023-11-24T06:28:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02881e3c20 con 0x7f0288143c40 2023-11-24T06:28:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f02857fa640 1 -- 172.21.15.16:0/3014926182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000aa30 con 0x7f0288143c40 2023-11-24T06:28:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.527+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02881e15d0 con 0x7f0288143c40 2023-11-24T06:28:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.529+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0288067450 con 0x7f0288143c40 2023-11-24T06:28:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.530+0000 7f02857fa640 1 -- 172.21.15.16:0/3014926182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f028000ab90 con 0x7f0288143c40 2023-11-24T06:28:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.531+0000 7f02857fa640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f026c06ae40 0x7f026c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.531+0000 7f02857fa640 1 -- 172.21.15.16:0/3014926182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02800892c0 con 0x7f0288143c40 2023-11-24T06:28:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.532+0000 7f028cacf640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f026c06ae40 0x7f026c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.533+0000 7f028cacf640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f026c06ae40 0x7f026c06d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f027c036000 tx=0x7f027c03d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.537+0000 7f02857fa640 1 -- 172.21.15.16:0/3014926182 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0280054400 con 0x7f0288143c40 2023-11-24T06:28:35.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.774+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f02881e1a80 con 0x7f0288143c40 2023-11-24T06:28:35.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.775+0000 7f02857fa640 1 -- 172.21.15.16:0/3014926182 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0280003e60 con 0x7f0288143c40 2023-11-24T06:28:35.778 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:28:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.777+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f026c06ae40 msgr2=0x7f026c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.777+0000 7f028dad1640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f026c06ae40 0x7f026c06d300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f027c036000 tx=0x7f027c03d040 comp rx=0 tx=0).stop 2023-11-24T06:28:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.777+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288143c40 msgr2=0x7f0288141fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.777+0000 7f028dad1640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288143c40 0x7f0288141fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f028000c7a0 tx=0x7f028000cc60 comp rx=0 tx=0).stop 2023-11-24T06:28:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.778+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 shutdown_connections 2023-11-24T06:28:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.778+0000 7f028dad1640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f026c06ae40 0x7f026c06d300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.778+0000 7f028dad1640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0288143c40 0x7f0288141fc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.778+0000 7f028dad1640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02880cca40 0x7f02881f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.778+0000 7f028dad1640 1 --2- 172.21.15.16:0/3014926182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02880717d0 0x7f028815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.778+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 >> 172.21.15.16:0/3014926182 conn(0x7f028806a800 msgr2=0x7f02880af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.778+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 shutdown_connections 2023-11-24T06:28:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:35.778+0000 7f028dad1640 1 -- 172.21.15.16:0/3014926182 wait complete. 2023-11-24T06:28:35.800 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:28:35.800 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:28:35.800 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2023-11-24T06:28:35.800 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:28:35.800 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:35.800 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:28:35.801 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:28:35.801 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:28:35.801 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:28:35.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:35.801 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2023-11-24T06:28:35.864 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:35.865 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42288sec preferred_lft 42288sec 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:28:35.866 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::c4e1:59ff:feae:2f56/64 scope link 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ea:3b:9f:a2:9b:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::e83b:9fff:fea2:9ba6/64 scope link 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::64de:a3ff:fee0:a827/64 scope link 2023-11-24T06:28:35.867 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:28:35.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:35.868 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:28:35.962 INFO:teuthology.orchestra.run.smithi173.stdout:1 2023-11-24T06:28:35.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:35.966 DEBUG:teuthology.orchestra.run.smithi173:> route 2023-11-24T06:28:36.022 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2023-11-24T06:28:36.022 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:28:36.022 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:28:36.022 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:28:36.022 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:28:36.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:36.023 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:28:36.023 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:28:36.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:28:36.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:28:36.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:28:36.023 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:28:36.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:36.203 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2023-11-24T06:28:36.252 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:28:36.252 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:28:36.253 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:28:36.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:36.253 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2023-11-24T06:28:36.347 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:28:36.347 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:36.347 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:36.400 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:28:36.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:28:36.400 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:36.451 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:28:36.547 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:36.606 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:28:36.606 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:28:36.606 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:28:36.606 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:28:36.606 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:28:36.606 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:28:36.606 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:28:36.606 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:28:36.606 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:36.607 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:36.608 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:36.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:36.609 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:28:36.663 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:28:36.663 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:28:36.669 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:28:36.748 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:28:36.793 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:28:36.793 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:28:36.793 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:28:36.794 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:28:36.795 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:28:36.796 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:28:36.796 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:28:36.796 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:28:36.796 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:36.796 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:28:36.796 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:36.796 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:28:36.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:36.797 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:28:36.864 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:2023-11-24T06:28:36.862+0000 7f2904666480 -1 init, newargv = 0x5594cd56ba40 newargc=15 2023-11-24T06:28:36.865 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[95554]: starting ceph client 2023-11-24T06:28:36.876 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[95554]: starting fuse 2023-11-24T06:28:36.888 INFO:teuthology.orchestra.run.smithi173.stdout:39 2023-11-24T06:28:36.888 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-11-24T06:28:36.888 DEBUG:teuthology.orchestra.run.smithi173:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:28:36.888 DEBUG:teuthology.orchestra.run.smithi173:> import glob 2023-11-24T06:28:36.888 DEBUG:teuthology.orchestra.run.smithi173:> import re 2023-11-24T06:28:36.888 DEBUG:teuthology.orchestra.run.smithi173:> import os 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> import subprocess 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> def _find_admin_socket(client_name): 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> files = glob.glob(asok_path) 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> # Given a non-glob path, it better be there 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> if "*" not in asok_path: 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> assert(len(files) == 1) 2023-11-24T06:28:36.889 DEBUG:teuthology.orchestra.run.smithi173:> return files[0] 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> for f in files: 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> contents = proc_f.read() 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> if mountpoint in contents: 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> return f 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:28:36.890 DEBUG:teuthology.orchestra.run.smithi173:> print(_find_admin_socket("client.1")) 2023-11-24T06:28:36.891 DEBUG:teuthology.orchestra.run.smithi173:> ' 2023-11-24T06:28:37.091 INFO:teuthology.orchestra.run.smithi173.stdout:/var/run/ceph/ceph-client.1.95554.asok 2023-11-24T06:28:37.095 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:28:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:28:37.108 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.95554.asok 2023-11-24T06:28:37.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:37.108 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.95554.asok status 2023-11-24T06:28:37.347 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-11-24T06:28:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "metadata": { 2023-11-24T06:28:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:28:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:28:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "entity_id": "1", 2023-11-24T06:28:37.347 INFO:teuthology.orchestra.run.smithi173.stdout: "hostname": "smithi173", 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "pid": "95554", 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "root": "/" 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_count": 0, 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_pinned_count": 0, 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "id": 4975, 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "inst": { 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "name": { 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "client", 2023-11-24T06:28:37.348 INFO:teuthology.orchestra.run.smithi173.stdout: "num": 4975 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 2212980399 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:28:37.349 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:28:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 2212980399 2023-11-24T06:28:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:28:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: "inst_str": "client.4975 192.168.0.1:0/2212980399", 2023-11-24T06:28:37.350 INFO:teuthology.orchestra.run.smithi173.stdout: "addr_str": "192.168.0.1:0/2212980399", 2023-11-24T06:28:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "inode_count": 1, 2023-11-24T06:28:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_epoch": 16, 2023-11-24T06:28:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch": 34, 2023-11-24T06:28:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:28:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "blocklisted": false, 2023-11-24T06:28:37.351 INFO:teuthology.orchestra.run.smithi173.stdout: "fs_name": "cephfs" 2023-11-24T06:28:37.351 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-11-24T06:28:37.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:37.362 DEBUG:teuthology.orchestra.run.smithi173:> stat --file-system '--printf=%T 2023-11-24T06:28:37.362 DEBUG:teuthology.orchestra.run.smithi173:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:37.416 INFO:teuthology.orchestra.run.smithi173.stdout:fuseblk 2023-11-24T06:28:37.416 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:37.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:37.417 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:37.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:28:37.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.752+0000 7fadcffff640 1 Processor -- start 2023-11-24T06:28:37.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.752+0000 7fadcffff640 1 -- start start 2023-11-24T06:28:37.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.755+0000 7fadcffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadd0162430 0x7fadd00b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.755+0000 7fadcffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fadd0162810 0x7fadd0071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.755+0000 7fadcffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd00702e0 con 0x7fadd00b3650 2023-11-24T06:28:37.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.755+0000 7fadcffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd00705c0 con 0x7fadd0162430 2023-11-24T06:28:37.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.755+0000 7fadcffff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd00708a0 con 0x7fadd0162810 2023-11-24T06:28:37.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.756+0000 7fadceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadd0162430 0x7fadd00b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.756+0000 7fadceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadd0162430 0x7fadd00b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46816/0 (socket says 172.21.15.16:46816) 2023-11-24T06:28:37.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.756+0000 7fadceffd640 1 -- 172.21.15.16:0/3998210118 learned_addr learned my addr 172.21.15.16:0/3998210118 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:37.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.756+0000 7fadc67fc640 1 --2- 172.21.15.16:0/3998210118 >> v2:172.21.15.173:3300/0 conn(0x7fadd0162810 0x7fadd0071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.756+0000 7fadceffd640 1 -- 172.21.15.16:0/3998210118 >> v2:172.21.15.173:3300/0 conn(0x7fadd0162810 msgr2=0x7fadd0071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:37.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.756+0000 7fadceffd640 1 --2- 172.21.15.16:0/3998210118 >> v2:172.21.15.173:3300/0 conn(0x7fadd0162810 0x7fadd0071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.756+0000 7fadceffd640 1 -- 172.21.15.16:0/3998210118 >> v1:172.21.15.16:6789/0 conn(0x7fadd00b3650 legacy=0x7fadd01650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.756+0000 7fadceffd640 1 -- 172.21.15.16:0/3998210118 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd0070b80 con 0x7fadd0162430 2023-11-24T06:28:37.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.757+0000 7fadceffd640 1 --2- 172.21.15.16:0/3998210118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd0162430 0x7fadd00b3110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fadd015c090 tx=0x7fadc002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a152c406fc3ef1b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:37.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadce7fc640 1 -- 172.21.15.16:0/3998210118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0030860 con 0x7fadd0162430 2023-11-24T06:28:37.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadce7fc640 1 -- 172.21.15.16:0/3998210118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc00309c0 con 0x7fadd0162430 2023-11-24T06:28:37.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadce7fc640 1 -- 172.21.15.16:0/3998210118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0030d50 con 0x7fadd0162430 2023-11-24T06:28:37.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadcffff640 1 -- 172.21.15.16:0/3998210118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd0162430 msgr2=0x7fadd00b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadcffff640 1 --2- 172.21.15.16:0/3998210118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd0162430 0x7fadd00b3110 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fadd015c090 tx=0x7fadc002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadcffff640 1 -- 172.21.15.16:0/3998210118 shutdown_connections 2023-11-24T06:28:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadcffff640 1 --2- 172.21.15.16:0/3998210118 >> v2:172.21.15.173:3300/0 conn(0x7fadd0162810 0x7fadd0071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadcffff640 1 --2- 172.21.15.16:0/3998210118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd0162430 0x7fadd00b3110 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadcffff640 1 -- 172.21.15.16:0/3998210118 >> 172.21.15.16:0/3998210118 conn(0x7fadd006a800 msgr2=0x7fadd006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.758+0000 7fadcffff640 1 -- 172.21.15.16:0/3998210118 shutdown_connections 2023-11-24T06:28:37.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.759+0000 7fadcffff640 1 -- 172.21.15.16:0/3998210118 wait complete. 2023-11-24T06:28:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.760+0000 7fadcffff640 1 Processor -- start 2023-11-24T06:28:37.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.760+0000 7fadcffff640 1 -- start start 2023-11-24T06:28:37.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.761+0000 7fadcffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadd00b3650 0x7fadd0153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.761+0000 7fadcffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015a990 0x7fadd00af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.761+0000 7fadcffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadd015ae10 0x7fadd0158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.761+0000 7fadcffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd015bed0 con 0x7fadd015a990 2023-11-24T06:28:37.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.761+0000 7fadcffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd015b2c0 con 0x7fadd00b3650 2023-11-24T06:28:37.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.761+0000 7fadcffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd015b5a0 con 0x7fadd015ae10 2023-11-24T06:28:37.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.762+0000 7fadc67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015a990 0x7fadd00af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.762+0000 7fadc67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015a990 0x7fadd00af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46822/0 (socket says 172.21.15.16:46822) 2023-11-24T06:28:37.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.762+0000 7fadc67fc640 1 -- 172.21.15.16:0/2845730805 learned_addr learned my addr 172.21.15.16:0/2845730805 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.762+0000 7fadc67fc640 1 -- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadd015ae10 msgr2=0x7fadd0158a10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:28:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.762+0000 7fadc67fc640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadd015ae10 0x7fadd0158a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.762+0000 7fadc67fc640 1 -- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadd00b3650 msgr2=0x7fadd0153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.762+0000 7fadc67fc640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadd00b3650 0x7fadd0153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.762+0000 7fadc67fc640 1 -- 172.21.15.16:0/2845730805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00094d0 con 0x7fadd015a990 2023-11-24T06:28:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.762+0000 7fadc67fc640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015a990 0x7fadd00af570 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fadb8007e60 tx=0x7fadb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.763+0000 7fadc7fff640 1 -- 172.21.15.16:0/2845730805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8003b90 con 0x7fadd015a990 2023-11-24T06:28:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.763+0000 7fadc7fff640 1 -- 172.21.15.16:0/2845730805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb800ee90 con 0x7fadd015a990 2023-11-24T06:28:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.763+0000 7fadc7fff640 1 -- 172.21.15.16:0/2845730805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8004030 con 0x7fadd015a990 2023-11-24T06:28:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.763+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd015b880 con 0x7fadd015a990 2023-11-24T06:28:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.763+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd00afab0 con 0x7fadd015a990 2023-11-24T06:28:37.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.764+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd0067450 con 0x7fadd015a990 2023-11-24T06:28:37.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.765+0000 7fadc7fff640 1 -- 172.21.15.16:0/2845730805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadb8004190 con 0x7fadd015a990 2023-11-24T06:28:37.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.766+0000 7fadc7fff640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fadb006ae70 0x7fadb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:37.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.766+0000 7fadc7fff640 1 -- 172.21.15.16:0/2845730805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadb80899e0 con 0x7fadd015a990 2023-11-24T06:28:37.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.766+0000 7fadceffd640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fadb006ae70 0x7fadb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:37.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.768+0000 7fadceffd640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fadb006ae70 0x7fadb006d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadc0036000 tx=0x7fadc003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:37.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:37.770+0000 7fadc7fff640 1 -- 172.21.15.16:0/2845730805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadb8054aa0 con 0x7fadd015a990 2023-11-24T06:28:38.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.005+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fadd00b2ef0 con 0x7fadd015a990 2023-11-24T06:28:38.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.060+0000 7fadc7fff640 1 -- 172.21.15.16:0/2845730805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 136+0+0 (secure 0 0 0) 0x7fadb80580c0 con 0x7fadd015a990 2023-11-24T06:28:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fadb006ae70 msgr2=0x7fadb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fadb006ae70 0x7fadb006d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadc0036000 tx=0x7fadc003b040 comp rx=0 tx=0).stop 2023-11-24T06:28:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015a990 msgr2=0x7fadd00af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015a990 0x7fadd00af570 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fadb8007e60 tx=0x7fadb800e4b0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 shutdown_connections 2023-11-24T06:28:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fadb006ae70 0x7fadb006d330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadd015ae10 0x7fadd0158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd015a990 0x7fadd00af570 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 --2- 172.21.15.16:0/2845730805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadd00b3650 0x7fadd0153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 >> 172.21.15.16:0/2845730805 conn(0x7fadd006a800 msgr2=0x7fadd006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 shutdown_connections 2023-11-24T06:28:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.063+0000 7fadcffff640 1 -- 172.21.15.16:0/2845730805 wait complete. 2023-11-24T06:28:38.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:28:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.315+0000 7fc4a48b3640 1 Processor -- start 2023-11-24T06:28:38.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.315+0000 7fc4a48b3640 1 -- start start 2023-11-24T06:28:38.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.316+0000 7fc4a48b3640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc4a013cfa0 0x7fc4a015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.316+0000 7fc4a48b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4a013fcb0 0x7fc4a00b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.316+0000 7fc4a48b3640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a00cdec0 con 0x7fc4a00cd530 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.316+0000 7fc4a48b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a00ce1a0 con 0x7fc4a013fcb0 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.316+0000 7fc4a48b3640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a00ce480 con 0x7fc4a013cfa0 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.317+0000 7fc49f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4a013fcb0 0x7fc4a00b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.317+0000 7fc49f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4a013fcb0 0x7fc4a00b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46838/0 (socket says 172.21.15.16:46838) 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.317+0000 7fc49f59e640 1 -- 172.21.15.16:0/3891378192 learned_addr learned my addr 172.21.15.16:0/3891378192 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.317+0000 7fc49ed9d640 1 --2- 172.21.15.16:0/3891378192 >> v2:172.21.15.173:3300/0 conn(0x7fc4a013cfa0 0x7fc4a015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.317+0000 7fc49f59e640 1 -- 172.21.15.16:0/3891378192 >> v2:172.21.15.173:3300/0 conn(0x7fc4a013cfa0 msgr2=0x7fc4a015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.317+0000 7fc49f59e640 1 --2- 172.21.15.16:0/3891378192 >> v2:172.21.15.173:3300/0 conn(0x7fc4a013cfa0 0x7fc4a015afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.317+0000 7fc49f59e640 1 -- 172.21.15.16:0/3891378192 >> v1:172.21.15.173:6789/0 conn(0x7fc4a00cd530 legacy=0x7fc4a013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:38.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.317+0000 7fc49f59e640 1 -- 172.21.15.16:0/3891378192 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a00ce760 con 0x7fc4a013fcb0 2023-11-24T06:28:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.318+0000 7fc49f59e640 1 --2- 172.21.15.16:0/3891378192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013fcb0 0x7fc4a00b2040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc4a006cf00 tx=0x7fc49802e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bd80d6d9b5f978 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.319+0000 7fc49e59c640 1 -- 172.21.15.16:0/3891378192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49802ea00 con 0x7fc4a013fcb0 2023-11-24T06:28:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.319+0000 7fc49e59c640 1 -- 172.21.15.16:0/3891378192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49802eb60 con 0x7fc4a013fcb0 2023-11-24T06:28:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.319+0000 7fc49e59c640 1 -- 172.21.15.16:0/3891378192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498009600 con 0x7fc4a013fcb0 2023-11-24T06:28:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.319+0000 7fc4a48b3640 1 -- 172.21.15.16:0/3891378192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013fcb0 msgr2=0x7fc4a00b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.319+0000 7fc4a48b3640 1 --2- 172.21.15.16:0/3891378192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013fcb0 0x7fc4a00b2040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc4a006cf00 tx=0x7fc49802e060 comp rx=0 tx=0).stop 2023-11-24T06:28:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.319+0000 7fc4a48b3640 1 -- 172.21.15.16:0/3891378192 shutdown_connections 2023-11-24T06:28:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.319+0000 7fc4a48b3640 1 --2- 172.21.15.16:0/3891378192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013fcb0 0x7fc4a00b2040 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.319+0000 7fc4a48b3640 1 --2- 172.21.15.16:0/3891378192 >> v2:172.21.15.173:3300/0 conn(0x7fc4a013cfa0 0x7fc4a015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.320+0000 7fc4a48b3640 1 -- 172.21.15.16:0/3891378192 >> 172.21.15.16:0/3891378192 conn(0x7fc4a006a800 msgr2=0x7fc4a006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.320+0000 7fc4a48b3640 1 -- 172.21.15.16:0/3891378192 shutdown_connections 2023-11-24T06:28:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.320+0000 7fc4a48b3640 1 -- 172.21.15.16:0/3891378192 wait complete. 2023-11-24T06:28:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.322+0000 7fc4a48b3640 1 Processor -- start 2023-11-24T06:28:38.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.322+0000 7fc4a48b3640 1 -- start start 2023-11-24T06:28:38.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.323+0000 7fc4a48b3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a00cd530 0x7fc4a01e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.323+0000 7fc4a48b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013cfa0 0x7fc4a0154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.323+0000 7fc4a48b3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4a013fcb0 0x7fc4a00ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.323+0000 7fc4a48b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a01f7740 con 0x7fc4a013cfa0 2023-11-24T06:28:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.323+0000 7fc4a48b3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a01e5120 con 0x7fc4a00cd530 2023-11-24T06:28:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.323+0000 7fc4a48b3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a01f6b30 con 0x7fc4a013fcb0 2023-11-24T06:28:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.323+0000 7fc497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013cfa0 0x7fc4a0154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.323+0000 7fc497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013cfa0 0x7fc4a0154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46842/0 (socket says 172.21.15.16:46842) 2023-11-24T06:28:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.323+0000 7fc497fff640 1 -- 172.21.15.16:0/1751624942 learned_addr learned my addr 172.21.15.16:0/1751624942 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:38.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc49ed9d640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a00cd530 0x7fc4a01e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc497fff640 1 -- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4a013fcb0 msgr2=0x7fc4a00ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc497fff640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4a013fcb0 0x7fc4a00ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc497fff640 1 -- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a00cd530 msgr2=0x7fc4a01e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc497fff640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a00cd530 0x7fc4a01e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc497fff640 1 -- 172.21.15.16:0/1751624942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49800bd40 con 0x7fc4a013cfa0 2023-11-24T06:28:38.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc49ed9d640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a00cd530 0x7fc4a01e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc497fff640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013cfa0 0x7fc4a0154e10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc48800c410 tx=0x7fc48800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc4977fe640 1 -- 172.21.15.16:0/1751624942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4880035f0 con 0x7fc4a013cfa0 2023-11-24T06:28:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc4977fe640 1 -- 172.21.15.16:0/1751624942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488003750 con 0x7fc4a013cfa0 2023-11-24T06:28:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc4977fe640 1 -- 172.21.15.16:0/1751624942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488003b40 con 0x7fc4a013cfa0 2023-11-24T06:28:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a01f6e10 con 0x7fc4a013cfa0 2023-11-24T06:28:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.324+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a01f73d0 con 0x7fc4a013cfa0 2023-11-24T06:28:38.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.326+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a01f78f0 con 0x7fc4a013cfa0 2023-11-24T06:28:38.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.327+0000 7fc4977fe640 1 -- 172.21.15.16:0/1751624942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc488003ca0 con 0x7fc4a013cfa0 2023-11-24T06:28:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.328+0000 7fc4977fe640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48406ae70 0x7fc48406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.328+0000 7fc4977fe640 1 -- 172.21.15.16:0/1751624942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc488088750 con 0x7fc4a013cfa0 2023-11-24T06:28:38.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.329+0000 7fc49ed9d640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48406ae70 0x7fc48406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.330+0000 7fc49ed9d640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48406ae70 0x7fc48406d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc490008890 tx=0x7fc490008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:38.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.333+0000 7fc4977fe640 1 -- 172.21.15.16:0/1751624942 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc4a01f78f0 con 0x7fc4a013cfa0 2023-11-24T06:28:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.549+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4a00b2580 con 0x7fc48406ae70 2023-11-24T06:28:38.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.554+0000 7fc4977fe640 1 -- 172.21.15.16:0/1751624942 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc49000c040 con 0x7fc48406ae70 2023-11-24T06:28:38.556 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:28:38.557 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:28:38.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephfs" 2023-11-24T06:28:38.557 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:28:38.557 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:28:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.555+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48406ae70 msgr2=0x7fc48406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.555+0000 7fc4a48b3640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48406ae70 0x7fc48406d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc490008890 tx=0x7fc490008040 comp rx=0 tx=0).stop 2023-11-24T06:28:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.555+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013cfa0 msgr2=0x7fc4a0154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.555+0000 7fc4a48b3640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013cfa0 0x7fc4a0154e10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc48800c410 tx=0x7fc48800c8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.556+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 shutdown_connections 2023-11-24T06:28:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.556+0000 7fc4a48b3640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48406ae70 0x7fc48406d330 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.556+0000 7fc4a48b3640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4a013fcb0 0x7fc4a00ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.556+0000 7fc4a48b3640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a013cfa0 0x7fc4a0154e10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.556+0000 7fc4a48b3640 1 --2- 172.21.15.16:0/1751624942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a00cd530 0x7fc4a01e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.556+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 >> 172.21.15.16:0/1751624942 conn(0x7fc4a006a800 msgr2=0x7fc4a00aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.556+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 shutdown_connections 2023-11-24T06:28:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.556+0000 7fc4a48b3640 1 -- 172.21.15.16:0/1751624942 wait complete. 2023-11-24T06:28:38.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:28:38.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.812+0000 7f3b211c1640 1 Processor -- start 2023-11-24T06:28:38.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.812+0000 7f3b211c1640 1 -- start start 2023-11-24T06:28:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.814+0000 7f3b211c1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3b1c13cda0 0x7f3b1c15da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.814+0000 7f3b211c1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3b1c15dfa0 0x7f3b1c0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.814+0000 7f3b211c1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c071b50 con 0x7f3b1c0cdef0 2023-11-24T06:28:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.814+0000 7f3b211c1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c0700f0 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.814+0000 7f3b211c1640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c15e3e0 con 0x7f3b1c13cda0 2023-11-24T06:28:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.814+0000 7f3b209c0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3b1c0cdef0 0x7f3b1c15a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34894/0 (socket says 172.21.15.16:34894) 2023-11-24T06:28:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.814+0000 7f3b209c0640 1 -- 172.21.15.16:0/2764226404 learned_addr learned my addr 172.21.15.16:0/2764226404 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:38.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.814+0000 7f3b13fff640 1 --2- 172.21.15.16:0/2764226404 >> v2:172.21.15.16:3300/0 conn(0x7f3b1c15dfa0 0x7f3b1c0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.815+0000 7f3b1bfff640 1 --2- 172.21.15.16:0/2764226404 >> v2:172.21.15.173:3300/0 conn(0x7f3b1c13cda0 0x7f3b1c15da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.815+0000 7f3b1b7fe640 1 -- 172.21.15.16:0/2764226404 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265726969 0 0) 0x7f3b1c071b50 con 0x7f3b1c0cdef0 2023-11-24T06:28:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.815+0000 7f3b1b7fe640 1 -- 172.21.15.16:0/2764226404 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b08003520 con 0x7f3b1c0cdef0 2023-11-24T06:28:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.815+0000 7f3b13fff640 1 -- 172.21.15.16:0/2764226404 >> v2:172.21.15.173:3300/0 conn(0x7f3b1c13cda0 msgr2=0x7f3b1c15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.815+0000 7f3b13fff640 1 --2- 172.21.15.16:0/2764226404 >> v2:172.21.15.173:3300/0 conn(0x7f3b1c13cda0 0x7f3b1c15da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.815+0000 7f3b13fff640 1 -- 172.21.15.16:0/2764226404 >> v1:172.21.15.16:6789/0 conn(0x7f3b1c0cdef0 legacy=0x7f3b1c15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.815+0000 7f3b13fff640 1 -- 172.21.15.16:0/2764226404 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c0ce720 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.816+0000 7f3b13fff640 1 --2- 172.21.15.16:0/2764226404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 0x7f3b1c0cd9b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c06df10 tx=0x7f3b0402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97248c23644996b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.817+0000 7f3b1b7fe640 1 -- 172.21.15.16:0/2764226404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0402e070 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.817+0000 7f3b1b7fe640 1 -- 172.21.15.16:0/2764226404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b040315a0 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.817+0000 7f3b1b7fe640 1 -- 172.21.15.16:0/2764226404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04031910 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.817+0000 7f3b211c1640 1 -- 172.21.15.16:0/2764226404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 msgr2=0x7f3b1c0cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.817+0000 7f3b211c1640 1 --2- 172.21.15.16:0/2764226404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 0x7f3b1c0cd9b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c06df10 tx=0x7f3b0402b840 comp rx=0 tx=0).stop 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.817+0000 7f3b211c1640 1 -- 172.21.15.16:0/2764226404 shutdown_connections 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.817+0000 7f3b211c1640 1 --2- 172.21.15.16:0/2764226404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 0x7f3b1c0cd9b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.817+0000 7f3b211c1640 1 --2- 172.21.15.16:0/2764226404 >> v2:172.21.15.173:3300/0 conn(0x7f3b1c13cda0 0x7f3b1c15da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.817+0000 7f3b211c1640 1 -- 172.21.15.16:0/2764226404 >> 172.21.15.16:0/2764226404 conn(0x7f3b1c06a3e0 msgr2=0x7f3b1c06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.818+0000 7f3b211c1640 1 -- 172.21.15.16:0/2764226404 shutdown_connections 2023-11-24T06:28:38.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.818+0000 7f3b211c1640 1 -- 172.21.15.16:0/2764226404 wait complete. 2023-11-24T06:28:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.819+0000 7f3b211c1640 1 Processor -- start 2023-11-24T06:28:38.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.820+0000 7f3b211c1640 1 -- start start 2023-11-24T06:28:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.820+0000 7f3b211c1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c0cdef0 0x7f3b1c1e3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.820+0000 7f3b211c1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b1c13cda0 0x7f3b1c13b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.820+0000 7f3b211c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 0x7f3b1c1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.820+0000 7f3b211c1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0b2900 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.820+0000 7f3b211c1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0b1ff0 con 0x7f3b1c0cdef0 2023-11-24T06:28:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.820+0000 7f3b211c1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1e3e60 con 0x7f3b1c13cda0 2023-11-24T06:28:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.821+0000 7f3b209c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 0x7f3b1c1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.821+0000 7f3b209c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 0x7f3b1c1590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46852/0 (socket says 172.21.15.16:46852) 2023-11-24T06:28:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.821+0000 7f3b209c0640 1 -- 172.21.15.16:0/1271346392 learned_addr learned my addr 172.21.15.16:0/1271346392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.821+0000 7f3b209c0640 1 -- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b1c13cda0 msgr2=0x7f3b1c13b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.821+0000 7f3b13fff640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b1c13cda0 0x7f3b1c13b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.821+0000 7f3b209c0640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b1c13cda0 0x7f3b1c13b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.821+0000 7f3b209c0640 1 -- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c0cdef0 msgr2=0x7f3b1c1e3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.821+0000 7f3b209c0640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c0cdef0 0x7f3b1c1e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.821+0000 7f3b209c0640 1 -- 172.21.15.16:0/1271346392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04009400 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.822+0000 7f3b209c0640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 0x7f3b1c1590a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400caa0 tx=0x7f3b1400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.822+0000 7f3b197fa640 1 -- 172.21.15.16:0/1271346392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400ac00 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.822+0000 7f3b197fa640 1 -- 172.21.15.16:0/1271346392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1400ad60 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.822+0000 7f3b197fa640 1 -- 172.21.15.16:0/1271346392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14012620 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.822+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c0b22b0 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.822+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c13c770 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.823+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c067450 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.824+0000 7f3b197fa640 1 -- 172.21.15.16:0/1271346392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b14007070 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.825+0000 7f3b197fa640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3afc06ae40 0x7f3afc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.825+0000 7f3b197fa640 1 -- 172.21.15.16:0/1271346392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b140896a0 con 0x7f3b1c15dfa0 2023-11-24T06:28:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.826+0000 7f3b1bfff640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3afc06ae40 0x7f3afc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:38.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.827+0000 7f3b1bfff640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3afc06ae40 0x7f3afc06d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0098f0 tx=0x7f3b0c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:38.830+0000 7f3b197fa640 1 -- 172.21.15.16:0/1271346392 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b140546b0 con 0x7f3b1c15dfa0 2023-11-24T06:28:39.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.024+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3b1c0b2a40 con 0x7f3b1c15dfa0 2023-11-24T06:28:39.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.025+0000 7f3b197fa640 1 -- 172.21.15.16:0/1271346392 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3b1c0b2a40 con 0x7f3b1c15dfa0 2023-11-24T06:28:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3afc06ae40 msgr2=0x7f3afc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3afc06ae40 0x7f3afc06d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0098f0 tx=0x7f3b0c008040 comp rx=0 tx=0).stop 2023-11-24T06:28:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 msgr2=0x7f3b1c1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 0x7f3b1c1590a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400caa0 tx=0x7f3b1400cf60 comp rx=0 tx=0).stop 2023-11-24T06:28:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 shutdown_connections 2023-11-24T06:28:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3afc06ae40 0x7f3afc06d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b1c15dfa0 0x7f3b1c1590a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b1c13cda0 0x7f3b1c13b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 --2- 172.21.15.16:0/1271346392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c0cdef0 0x7f3b1c1e3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 >> 172.21.15.16:0/1271346392 conn(0x7f3b1c06a3e0 msgr2=0x7f3b1c0afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.028+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 shutdown_connections 2023-11-24T06:28:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.029+0000 7f3b211c1640 1 -- 172.21.15.16:0/1271346392 wait complete. 2023-11-24T06:28:39.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp5320 2023-11-24T06:28:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.280+0000 7fd5826a3640 1 Processor -- start 2023-11-24T06:28:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.280+0000 7fd5826a3640 1 -- start start 2023-11-24T06:28:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.282+0000 7fd5826a3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd57c0adf20 0x7fd57c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.282+0000 7fd5826a3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 0x7fd57c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.282+0000 7fd5826a3640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd57c0cc980 0x7fd57c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.282+0000 7fd5826a3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57c15f970 con 0x7fd57c0adf20 2023-11-24T06:28:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.282+0000 7fd5826a3640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57c15f4e0 con 0x7fd57c0cc980 2023-11-24T06:28:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.282+0000 7fd5826a3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0cce10 con 0x7fd57c15bf10 2023-11-24T06:28:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.282+0000 7fd5816a1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd57c0adf20 0x7fd57c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.282+0000 7fd5816a1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd57c0adf20 0x7fd57c070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46866/0 (socket says 172.21.15.16:46866) 2023-11-24T06:28:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.282+0000 7fd5816a1640 1 -- 172.21.15.16:0/2625300694 learned_addr learned my addr 172.21.15.16:0/2625300694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:39.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.283+0000 7fd581ea2640 1 --2- 172.21.15.16:0/2625300694 >> v2:172.21.15.173:3300/0 conn(0x7fd57c0cc980 0x7fd57c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.283+0000 7fd580ea0640 1 --2- 172.21.15.16:0/2625300694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 0x7fd57c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.283+0000 7fd5816a1640 1 -- 172.21.15.16:0/2625300694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 msgr2=0x7fd57c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.283+0000 7fd5816a1640 1 --2- 172.21.15.16:0/2625300694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 0x7fd57c0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.283+0000 7fd5816a1640 1 -- 172.21.15.16:0/2625300694 >> v2:172.21.15.173:3300/0 conn(0x7fd57c0cc980 msgr2=0x7fd57c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:39.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.283+0000 7fd5816a1640 1 --2- 172.21.15.16:0/2625300694 >> v2:172.21.15.173:3300/0 conn(0x7fd57c0cc980 0x7fd57c143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.283+0000 7fd5816a1640 1 -- 172.21.15.16:0/2625300694 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c071130 con 0x7fd57c0adf20 2023-11-24T06:28:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.283+0000 7fd580ea0640 1 --2- 172.21.15.16:0/2625300694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 0x7fd57c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.284+0000 7fd5816a1640 1 --2- 172.21.15.16:0/2625300694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 0x7fd57c070a30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd57c06e490 tx=0x7fd57802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f4d4feb9c5cbd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.284+0000 7fd573fff640 1 -- 172.21.15.16:0/2625300694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5780317e0 con 0x7fd57c0adf20 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.284+0000 7fd573fff640 1 -- 172.21.15.16:0/2625300694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd578031940 con 0x7fd57c0adf20 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.284+0000 7fd573fff640 1 -- 172.21.15.16:0/2625300694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578031cd0 con 0x7fd57c0adf20 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.285+0000 7fd5826a3640 1 -- 172.21.15.16:0/2625300694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 msgr2=0x7fd57c070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.285+0000 7fd5826a3640 1 --2- 172.21.15.16:0/2625300694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 0x7fd57c070a30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd57c06e490 tx=0x7fd57802b880 comp rx=0 tx=0).stop 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.285+0000 7fd5826a3640 1 -- 172.21.15.16:0/2625300694 shutdown_connections 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.285+0000 7fd5826a3640 1 --2- 172.21.15.16:0/2625300694 >> v2:172.21.15.173:3300/0 conn(0x7fd57c0cc980 0x7fd57c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.285+0000 7fd5826a3640 1 --2- 172.21.15.16:0/2625300694 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 0x7fd57c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.285+0000 7fd5826a3640 1 --2- 172.21.15.16:0/2625300694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 0x7fd57c070a30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.285+0000 7fd5826a3640 1 -- 172.21.15.16:0/2625300694 >> 172.21.15.16:0/2625300694 conn(0x7fd57c06a7d0 msgr2=0x7fd57c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.285+0000 7fd5826a3640 1 -- 172.21.15.16:0/2625300694 shutdown_connections 2023-11-24T06:28:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.285+0000 7fd5826a3640 1 -- 172.21.15.16:0/2625300694 wait complete. 2023-11-24T06:28:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.287+0000 7fd5826a3640 1 Processor -- start 2023-11-24T06:28:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.287+0000 7fd5826a3640 1 -- start start 2023-11-24T06:28:39.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.288+0000 7fd5826a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 0x7fd57c0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.288+0000 7fd5826a3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd57c0cc980 0x7fd57c15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.288+0000 7fd5826a3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 0x7fd57c0ab520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.288+0000 7fd5826a3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c1f8250 con 0x7fd57c0adf20 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.288+0000 7fd5826a3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c1f7f70 con 0x7fd57c0cc980 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.288+0000 7fd5826a3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0b2010 con 0x7fd57c15bf10 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd5816a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 0x7fd57c0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd5816a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 0x7fd57c0b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46882/0 (socket says 172.21.15.16:46882) 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd5816a1640 1 -- 172.21.15.16:0/835929394 learned_addr learned my addr 172.21.15.16:0/835929394 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd580ea0640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd57c0cc980 0x7fd57c15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd581ea2640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 0x7fd57c0ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd5816a1640 1 -- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 msgr2=0x7fd57c0ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd5816a1640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 0x7fd57c0ab520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd5816a1640 1 -- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd57c0cc980 msgr2=0x7fd57c15b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd5816a1640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd57c0cc980 0x7fd57c15b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd5816a1640 1 -- 172.21.15.16:0/835929394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780094d0 con 0x7fd57c0adf20 2023-11-24T06:28:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.289+0000 7fd5816a1640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 0x7fd57c0b2af0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd578000c00 tx=0x7fd5780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.290+0000 7fd571ffb640 1 -- 172.21.15.16:0/835929394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578032880 con 0x7fd57c0adf20 2023-11-24T06:28:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.290+0000 7fd571ffb640 1 -- 172.21.15.16:0/835929394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5780329e0 con 0x7fd57c0adf20 2023-11-24T06:28:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.290+0000 7fd571ffb640 1 -- 172.21.15.16:0/835929394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578032d90 con 0x7fd57c0adf20 2023-11-24T06:28:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.290+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c0b22f0 con 0x7fd57c0adf20 2023-11-24T06:28:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.290+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c0b3030 con 0x7fd57c0adf20 2023-11-24T06:28:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.291+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c0b2810 con 0x7fd57c0adf20 2023-11-24T06:28:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.292+0000 7fd571ffb640 1 -- 172.21.15.16:0/835929394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd57804e450 con 0x7fd57c0adf20 2023-11-24T06:28:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.293+0000 7fd571ffb640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd56006ae40 0x7fd56006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.293+0000 7fd571ffb640 1 -- 172.21.15.16:0/835929394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5780ab890 con 0x7fd57c0adf20 2023-11-24T06:28:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.294+0000 7fd580ea0640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd56006ae40 0x7fd56006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.295+0000 7fd580ea0640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd56006ae40 0x7fd56006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd57c071750 tx=0x7fd56c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.298+0000 7fd571ffb640 1 -- 172.21.15.16:0/835929394 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd57c0b2810 con 0x7fd57c0adf20 2023-11-24T06:28:39.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.517+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp5320", "target": ["mon-mgr", ""]}) v1 -- 0x7fd57c0b1e90 con 0x7fd56006ae40 2023-11-24T06:28:39.535 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:28:39.532+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:28:39.535 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:28:39.532+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:28:39.535 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:28:39.532+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:28:39.536 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:28:39.532+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:28:39.536 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:28:39.532+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:28:39.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.566+0000 7fd571ffb640 1 -- 172.21.15.16:0/835929394 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd56c00a650 con 0x7fd56006ae40 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd56006ae40 msgr2=0x7fd56006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd56006ae40 0x7fd56006d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd57c071750 tx=0x7fd56c008040 comp rx=0 tx=0).stop 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 msgr2=0x7fd57c0b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 0x7fd57c0b2af0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd578000c00 tx=0x7fd5780046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 shutdown_connections 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd56006ae40 0x7fd56006d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd57c15bf10 0x7fd57c0ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd57c0cc980 0x7fd57c15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 --2- 172.21.15.16:0/835929394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd57c0adf20 0x7fd57c0b2af0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.569+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 >> 172.21.15.16:0/835929394 conn(0x7fd57c06a7d0 msgr2=0x7fd57c15cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.570+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 shutdown_connections 2023-11-24T06:28:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.570+0000 7fd5826a3640 1 -- 172.21.15.16:0/835929394 wait complete. 2023-11-24T06:28:39.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create cephfs subvol_grp5320 subvol_snap6143 2023-11-24T06:28:39.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.825+0000 7f2bd47fb640 1 Processor -- start 2023-11-24T06:28:39.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.826+0000 7f2bd47fb640 1 -- start start 2023-11-24T06:28:39.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.827+0000 7f2bd47fb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bcc0cd240 0x7f2bcc0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.828+0000 7f2bd47fb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bcc06cea0 con 0x7f2bcc06c6b0 2023-11-24T06:28:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.828+0000 7f2bd47fb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bcc06cbc0 con 0x7f2bcc09a170 2023-11-24T06:28:39.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.828+0000 7f2bd47fb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bcc06f350 con 0x7f2bcc0cd240 2023-11-24T06:28:39.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.828+0000 7f2bd2597640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bcc0cd240 0x7f2bcc0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.828+0000 7f2bd2597640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bcc0cd240 0x7f2bcc0b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46890/0 (socket says 172.21.15.16:46890) 2023-11-24T06:28:39.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.828+0000 7f2bd2597640 1 -- 172.21.15.16:0/16110909 learned_addr learned my addr 172.21.15.16:0/16110909 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:39.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.829+0000 7f2bd2597640 1 -- 172.21.15.16:0/16110909 >> v1:172.21.15.173:6789/0 conn(0x7f2bcc09a170 legacy=0x7f2bcc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:39.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.829+0000 7f2bd2597640 1 -- 172.21.15.16:0/16110909 >> v1:172.21.15.16:6789/0 conn(0x7f2bcc06c6b0 legacy=0x7f2bcc099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.829+0000 7f2bd1595640 1 -- 172.21.15.16:0/16110909 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711493685 0 0) 0x7f2bcc06cea0 con 0x7f2bcc06c6b0 2023-11-24T06:28:39.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.829+0000 7f2bd2597640 1 -- 172.21.15.16:0/16110909 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc06f630 con 0x7f2bcc0cd240 2023-11-24T06:28:39.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.830+0000 7f2bd2597640 1 --2- 172.21.15.16:0/16110909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc0cd240 0x7f2bcc0b3820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009870 tx=0x7f2bc802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd615a66cfa4c23 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.830+0000 7f2bd1595640 1 -- 172.21.15.16:0/16110909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8030860 con 0x7f2bcc0cd240 2023-11-24T06:28:39.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.830+0000 7f2bd1595640 1 -- 172.21.15.16:0/16110909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc80309c0 con 0x7f2bcc0cd240 2023-11-24T06:28:39.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.830+0000 7f2bd1595640 1 -- 172.21.15.16:0/16110909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8030d50 con 0x7f2bcc0cd240 2023-11-24T06:28:39.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.831+0000 7f2bd47fb640 1 -- 172.21.15.16:0/16110909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc0cd240 msgr2=0x7f2bcc0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.831+0000 7f2bd47fb640 1 --2- 172.21.15.16:0/16110909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc0cd240 0x7f2bcc0b3820 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009870 tx=0x7f2bc802b9a0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.831+0000 7f2bd47fb640 1 -- 172.21.15.16:0/16110909 shutdown_connections 2023-11-24T06:28:39.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.831+0000 7f2bd47fb640 1 --2- 172.21.15.16:0/16110909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc0cd240 0x7f2bcc0b3820 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.831+0000 7f2bd47fb640 1 -- 172.21.15.16:0/16110909 >> 172.21.15.16:0/16110909 conn(0x7f2bcc0b2d00 msgr2=0x7f2bcc0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:39.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.831+0000 7f2bd47fb640 1 -- 172.21.15.16:0/16110909 shutdown_connections 2023-11-24T06:28:39.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.831+0000 7f2bd47fb640 1 -- 172.21.15.16:0/16110909 wait complete. 2023-11-24T06:28:39.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.833+0000 7f2bd47fb640 1 Processor -- start 2023-11-24T06:28:39.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.833+0000 7f2bd47fb640 1 -- start start 2023-11-24T06:28:39.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.834+0000 7f2bd47fb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bcc06c6b0 0x7f2bcc0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.834+0000 7f2bd47fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc09a170 0x7f2bcc15db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.834+0000 7f2bd47fb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2bcc0cd240 0x7f2bcc1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.834+0000 7f2bd47fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc15e2c0 con 0x7f2bcc09a170 2023-11-24T06:28:39.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.834+0000 7f2bd47fb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc0cac90 con 0x7f2bcc06c6b0 2023-11-24T06:28:39.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.834+0000 7f2bd47fb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc15e0d0 con 0x7f2bcc0cd240 2023-11-24T06:28:39.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.834+0000 7f2bd1d96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc09a170 0x7f2bcc15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.834+0000 7f2bd1d96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc09a170 0x7f2bcc15db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46904/0 (socket says 172.21.15.16:46904) 2023-11-24T06:28:39.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.834+0000 7f2bd1d96640 1 -- 172.21.15.16:0/1782829209 learned_addr learned my addr 172.21.15.16:0/1782829209 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:39.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bd2597640 1 --2- 172.21.15.16:0/1782829209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bcc06c6b0 0x7f2bcc0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bd2d98640 1 --2- 172.21.15.16:0/1782829209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2bcc0cd240 0x7f2bcc1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bd1d96640 1 -- 172.21.15.16:0/1782829209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2bcc0cd240 msgr2=0x7f2bcc1581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bd1d96640 1 --2- 172.21.15.16:0/1782829209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2bcc0cd240 0x7f2bcc1581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bd1d96640 1 -- 172.21.15.16:0/1782829209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bcc06c6b0 msgr2=0x7f2bcc0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bd1d96640 1 --2- 172.21.15.16:0/1782829209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2bcc06c6b0 0x7f2bcc0c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bd1d96640 1 -- 172.21.15.16:0/1782829209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80094d0 con 0x7f2bcc09a170 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bd1d96640 1 --2- 172.21.15.16:0/1782829209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bcc09a170 0x7f2bcc15db90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c520 tx=0x7f2bc000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bbeffd640 1 -- 172.21.15.16:0/1782829209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00035f0 con 0x7f2bcc09a170 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bbeffd640 1 -- 172.21.15.16:0/1782829209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc0003770 con 0x7f2bcc09a170 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bbeffd640 1 -- 172.21.15.16:0/1782829209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0003b20 con 0x7f2bcc09a170 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.835+0000 7f2bd47fb640 1 -- 172.21.15.16:0/1782829209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bcc0b5250 con 0x7f2bcc09a170 2023-11-24T06:28:39.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.836+0000 7f2bd47fb640 1 -- 172.21.15.16:0/1782829209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc0b5810 con 0x7f2bcc09a170 2023-11-24T06:28:39.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.837+0000 7f2bd47fb640 1 -- 172.21.15.16:0/1782829209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bcc06d0d0 con 0x7f2bcc09a170 2023-11-24T06:28:39.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.838+0000 7f2bbeffd640 1 -- 172.21.15.16:0/1782829209 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2bc0003c80 con 0x7f2bcc09a170 2023-11-24T06:28:39.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.839+0000 7f2bbeffd640 1 --2- 172.21.15.16:0/1782829209 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2bb006ae40 0x7f2bb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:39.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.839+0000 7f2bbeffd640 1 -- 172.21.15.16:0/1782829209 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bc0088c30 con 0x7f2bcc09a170 2023-11-24T06:28:39.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.840+0000 7f2bd2597640 1 --2- 172.21.15.16:0/1782829209 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2bb006ae40 0x7f2bb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:39.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.841+0000 7f2bd2597640 1 --2- 172.21.15.16:0/1782829209 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2bb006ae40 0x7f2bb006d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8036000 tx=0x7f2bc803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:39.843+0000 7f2bbeffd640 1 -- 172.21.15.16:0/1782829209 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2bc0053cd0 con 0x7f2bcc09a170 2023-11-24T06:28:40.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.065+0000 7f2bd47fb640 1 -- 172.21.15.16:0/1782829209 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "cephfs", "group_name": "subvol_grp5320", "snap_name": "subvol_snap6143", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bcc0cd620 con 0x7f2bb006ae40 2023-11-24T06:28:40.072 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:28:40.069+0000 7fd2b5afe640 -1 mgr.server reply reply (38) Function not implemented subvolume group snapshots are not supported 2023-11-24T06:28:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.070+0000 7f2bbeffd640 1 -- 172.21.15.16:0/1782829209 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: -38 subvolume group snapshots are not supported) v1 ==== 51+0+0 (secure 0 0 0) 0x7f2bc8030cb0 con 0x7f2bb006ae40 2023-11-24T06:28:40.073 INFO:teuthology.orchestra.run.smithi016.stderr:Error ENOSYS: subvolume group snapshots are not supported 2023-11-24T06:28:40.087 DEBUG:teuthology.orchestra.run:got remote process result: 38 2023-11-24T06:28:40.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp5320 2023-11-24T06:28:40.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.320+0000 7f8f5f59e640 1 Processor -- start 2023-11-24T06:28:40.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.320+0000 7f8f5f59e640 1 -- start start 2023-11-24T06:28:40.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.322+0000 7f8f5f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f600ccb50 0x7f8f60070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.322+0000 7f8f5f59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8f6015bf10 0x7f8f60143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.322+0000 7f8f5f59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6015f4e0 con 0x7f8f600adf20 2023-11-24T06:28:40.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.322+0000 7f8f5f59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6015f970 con 0x7f8f600ccb50 2023-11-24T06:28:40.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.322+0000 7f8f5f59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f60071130 con 0x7f8f6015bf10 2023-11-24T06:28:40.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.323+0000 7f8f5dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f600ccb50 0x7f8f60070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.323+0000 7f8f5dd9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f600ccb50 0x7f8f60070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46906/0 (socket says 172.21.15.16:46906) 2023-11-24T06:28:40.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.323+0000 7f8f5dd9b640 1 -- 172.21.15.16:0/4139918491 learned_addr learned my addr 172.21.15.16:0/4139918491 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:40.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.323+0000 7f8f5dd9b640 1 -- 172.21.15.16:0/4139918491 >> v2:172.21.15.173:3300/0 conn(0x7f8f6015bf10 msgr2=0x7f8f60143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:40.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.323+0000 7f8f5dd9b640 1 --2- 172.21.15.16:0/4139918491 >> v2:172.21.15.173:3300/0 conn(0x7f8f6015bf10 0x7f8f60143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.323+0000 7f8f5dd9b640 1 -- 172.21.15.16:0/4139918491 >> v1:172.21.15.173:6789/0 conn(0x7f8f600adf20 legacy=0x7f8f600cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:40.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.323+0000 7f8f5dd9b640 1 -- 172.21.15.16:0/4139918491 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6015ebb0 con 0x7f8f600ccb50 2023-11-24T06:28:40.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.324+0000 7f8f5dd9b640 1 --2- 172.21.15.16:0/4139918491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 0x7f8f60070a30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f6006ff20 tx=0x7f8f5402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce88afafa585583 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:40.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.325+0000 7f8f5d59a640 1 -- 172.21.15.16:0/4139918491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540317e0 con 0x7f8f600ccb50 2023-11-24T06:28:40.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.325+0000 7f8f5d59a640 1 -- 172.21.15.16:0/4139918491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f54031940 con 0x7f8f600ccb50 2023-11-24T06:28:40.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.325+0000 7f8f5d59a640 1 -- 172.21.15.16:0/4139918491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54031cd0 con 0x7f8f600ccb50 2023-11-24T06:28:40.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.325+0000 7f8f5f59e640 1 -- 172.21.15.16:0/4139918491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 msgr2=0x7f8f60070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.325+0000 7f8f5f59e640 1 --2- 172.21.15.16:0/4139918491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 0x7f8f60070a30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f6006ff20 tx=0x7f8f5402b880 comp rx=0 tx=0).stop 2023-11-24T06:28:40.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.325+0000 7f8f5f59e640 1 -- 172.21.15.16:0/4139918491 shutdown_connections 2023-11-24T06:28:40.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.325+0000 7f8f5f59e640 1 --2- 172.21.15.16:0/4139918491 >> v2:172.21.15.173:3300/0 conn(0x7f8f6015bf10 0x7f8f60143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.325+0000 7f8f5f59e640 1 --2- 172.21.15.16:0/4139918491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 0x7f8f60070a30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.325+0000 7f8f5f59e640 1 -- 172.21.15.16:0/4139918491 >> 172.21.15.16:0/4139918491 conn(0x7f8f6006a7d0 msgr2=0x7f8f6006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.326+0000 7f8f5f59e640 1 -- 172.21.15.16:0/4139918491 shutdown_connections 2023-11-24T06:28:40.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.326+0000 7f8f5f59e640 1 -- 172.21.15.16:0/4139918491 wait complete. 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.327+0000 7f8f5f59e640 1 Processor -- start 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.327+0000 7f8f5f59e640 1 -- start start 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.328+0000 7f8f5f59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f600adf20 0x7f8f6013fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.328+0000 7f8f5f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 0x7f8f6013d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.328+0000 7f8f5f59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f6015bf10 0x7f8f6015d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.328+0000 7f8f5f59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601f2ac0 con 0x7f8f600ccb50 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.328+0000 7f8f5f59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600b22e0 con 0x7f8f600adf20 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.328+0000 7f8f5f59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600b25c0 con 0x7f8f6015bf10 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.329+0000 7f8f5dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 0x7f8f6013d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.329+0000 7f8f5dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 0x7f8f6013d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46910/0 (socket says 172.21.15.16:46910) 2023-11-24T06:28:40.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.329+0000 7f8f5dd9b640 1 -- 172.21.15.16:0/2546844529 learned_addr learned my addr 172.21.15.16:0/2546844529 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.329+0000 7f8f5dd9b640 1 -- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f6015bf10 msgr2=0x7f8f6015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.329+0000 7f8f5dd9b640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f6015bf10 0x7f8f6015d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.329+0000 7f8f5dd9b640 1 -- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f600adf20 msgr2=0x7f8f6013fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.329+0000 7f8f5dd9b640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f600adf20 0x7f8f6013fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.329+0000 7f8f5dd9b640 1 -- 172.21.15.16:0/2546844529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f540094d0 con 0x7f8f600ccb50 2023-11-24T06:28:40.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.329+0000 7f8f5dd9b640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 0x7f8f6013d540 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.330+0000 7f8f4effd640 1 -- 172.21.15.16:0/2546844529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54032880 con 0x7f8f600ccb50 2023-11-24T06:28:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.330+0000 7f8f4effd640 1 -- 172.21.15.16:0/2546844529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f540329e0 con 0x7f8f600ccb50 2023-11-24T06:28:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.330+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f600b28a0 con 0x7f8f600ccb50 2023-11-24T06:28:40.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.330+0000 7f8f4effd640 1 -- 172.21.15.16:0/2546844529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54032d90 con 0x7f8f600ccb50 2023-11-24T06:28:40.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.330+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6013fa20 con 0x7f8f600ccb50 2023-11-24T06:28:40.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.331+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f600b2dc0 con 0x7f8f600ccb50 2023-11-24T06:28:40.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.332+0000 7f8f4effd640 1 -- 172.21.15.16:0/2546844529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f5404d3f0 con 0x7f8f600ccb50 2023-11-24T06:28:40.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.333+0000 7f8f4effd640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f4006ae40 0x7f8f4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:40.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.334+0000 7f8f4effd640 1 -- 172.21.15.16:0/2546844529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f540ab530 con 0x7f8f600ccb50 2023-11-24T06:28:40.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.334+0000 7f8f5e59c640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f4006ae40 0x7f8f4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:40.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.335+0000 7f8f5e59c640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f4006ae40 0x7f8f4006d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f50007fd0 tx=0x7f8f500079a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:40.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.338+0000 7f8f4effd640 1 -- 172.21.15.16:0/2546844529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f540765d0 con 0x7f8f600ccb50 2023-11-24T06:28:40.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.556+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp5320", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f6015c380 con 0x7f8f4006ae40 2023-11-24T06:28:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.567+0000 7f8f4effd640 1 -- 172.21.15.16:0/2546844529 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8f50003950 con 0x7f8f4006ae40 2023-11-24T06:28:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f4006ae40 msgr2=0x7f8f4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f4006ae40 0x7f8f4006d300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8f50007fd0 tx=0x7f8f500079a0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 msgr2=0x7f8f6013d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 0x7f8f6013d540 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f540046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 shutdown_connections 2023-11-24T06:28:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f4006ae40 0x7f8f4006d300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f6015bf10 0x7f8f6015d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f600ccb50 0x7f8f6013d540 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 --2- 172.21.15.16:0/2546844529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f600adf20 0x7f8f6013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 >> 172.21.15.16:0/2546844529 conn(0x7f8f6006a7d0 msgr2=0x7f8f600704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 shutdown_connections 2023-11-24T06:28:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:40.569+0000 7f8f5f59e640 1 -- 172.21.15.16:0/2546844529 wait complete. 2023-11-24T06:28:40.593 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:28:40.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:28:40.711 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:28:40.832 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:28:40.832 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:28:40.832 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:40.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.134+0000 7f86b6d90640 1 Processor -- start 2023-11-24T06:28:41.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.134+0000 7f86b6d90640 1 -- start start 2023-11-24T06:28:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.136+0000 7f86b6d90640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86b0143ae0 0x7f86b00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.136+0000 7f86b6d90640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f86b00cf7d0 0x7f86b015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.136+0000 7f86b6d90640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b00cfc30 con 0x7f86b015e1e0 2023-11-24T06:28:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.136+0000 7f86b6d90640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b00b1ee0 con 0x7f86b0143ae0 2023-11-24T06:28:41.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.136+0000 7f86b6d90640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b00b2780 con 0x7f86b00cf7d0 2023-11-24T06:28:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.136+0000 7f86b4b2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86b0143ae0 0x7f86b00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.136+0000 7f86b4b2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86b0143ae0 0x7f86b00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46918/0 (socket says 172.21.15.16:46918) 2023-11-24T06:28:41.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.136+0000 7f86b4b2c640 1 -- 172.21.15.16:0/3830065783 learned_addr learned my addr 172.21.15.16:0/3830065783 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.137+0000 7f86b4b2c640 1 -- 172.21.15.16:0/3830065783 >> v2:172.21.15.173:3300/0 conn(0x7f86b00cf7d0 msgr2=0x7f86b015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.137+0000 7f86affff640 1 --2- 172.21.15.16:0/3830065783 >> v2:172.21.15.173:3300/0 conn(0x7f86b00cf7d0 0x7f86b015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.137+0000 7f86b4b2c640 1 --2- 172.21.15.16:0/3830065783 >> v2:172.21.15.173:3300/0 conn(0x7f86b00cf7d0 0x7f86b015dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.137+0000 7f86b4b2c640 1 -- 172.21.15.16:0/3830065783 >> v1:172.21.15.173:6789/0 conn(0x7f86b015e1e0 legacy=0x7f86b015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.137+0000 7f86b4b2c640 1 -- 172.21.15.16:0/3830065783 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b015e990 con 0x7f86b0143ae0 2023-11-24T06:28:41.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.138+0000 7f86b4b2c640 1 --2- 172.21.15.16:0/3830065783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0143ae0 0x7f86b00cf290 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f86b00cb830 tx=0x7f86a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96fc09949e00ade server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.138+0000 7f86af7fe640 1 -- 172.21.15.16:0/3830065783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4031870 con 0x7f86b0143ae0 2023-11-24T06:28:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.138+0000 7f86af7fe640 1 -- 172.21.15.16:0/3830065783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a40319d0 con 0x7f86b0143ae0 2023-11-24T06:28:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.138+0000 7f86af7fe640 1 -- 172.21.15.16:0/3830065783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4031d60 con 0x7f86b0143ae0 2023-11-24T06:28:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.139+0000 7f86b6d90640 1 -- 172.21.15.16:0/3830065783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0143ae0 msgr2=0x7f86b00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.139+0000 7f86b6d90640 1 --2- 172.21.15.16:0/3830065783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0143ae0 0x7f86b00cf290 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f86b00cb830 tx=0x7f86a402b910 comp rx=0 tx=0).stop 2023-11-24T06:28:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.139+0000 7f86b6d90640 1 -- 172.21.15.16:0/3830065783 shutdown_connections 2023-11-24T06:28:41.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.139+0000 7f86b6d90640 1 --2- 172.21.15.16:0/3830065783 >> v2:172.21.15.173:3300/0 conn(0x7f86b00cf7d0 0x7f86b015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.139+0000 7f86b6d90640 1 --2- 172.21.15.16:0/3830065783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b0143ae0 0x7f86b00cf290 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.139+0000 7f86b6d90640 1 -- 172.21.15.16:0/3830065783 >> 172.21.15.16:0/3830065783 conn(0x7f86b006a800 msgr2=0x7f86b006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:41.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.139+0000 7f86b6d90640 1 -- 172.21.15.16:0/3830065783 shutdown_connections 2023-11-24T06:28:41.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.139+0000 7f86b6d90640 1 -- 172.21.15.16:0/3830065783 wait complete. 2023-11-24T06:28:41.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.141+0000 7f86b6d90640 1 Processor -- start 2023-11-24T06:28:41.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.141+0000 7f86b6d90640 1 -- start start 2023-11-24T06:28:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.142+0000 7f86b6d90640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b00cf7d0 0x7f86b01657f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.142+0000 7f86b6d90640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86b0143ae0 0x7f86b015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.142+0000 7f86b6d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015e1e0 0x7f86b0140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.142+0000 7f86b6d90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00a84c0 con 0x7f86b015e1e0 2023-11-24T06:28:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.142+0000 7f86b6d90640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b01429a0 con 0x7f86b00cf7d0 2023-11-24T06:28:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.142+0000 7f86b6d90640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b0165ef0 con 0x7f86b0143ae0 2023-11-24T06:28:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.142+0000 7f86b532d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015e1e0 0x7f86b0140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.142+0000 7f86b532d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015e1e0 0x7f86b0140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46930/0 (socket says 172.21.15.16:46930) 2023-11-24T06:28:41.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.142+0000 7f86b532d640 1 -- 172.21.15.16:0/3608263611 learned_addr learned my addr 172.21.15.16:0/3608263611 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86b4b2c640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b00cf7d0 0x7f86b01657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86affff640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86b0143ae0 0x7f86b015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86b532d640 1 -- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86b0143ae0 msgr2=0x7f86b015ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86b532d640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86b0143ae0 0x7f86b015ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86b532d640 1 -- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b00cf7d0 msgr2=0x7f86b01657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86b532d640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b00cf7d0 0x7f86b01657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86b532d640 1 -- 172.21.15.16:0/3608263611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a40094d0 con 0x7f86b015e1e0 2023-11-24T06:28:41.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86affff640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86b0143ae0 0x7f86b015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86b532d640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015e1e0 0x7f86b0140a40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b970 tx=0x7f86a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86ad7fa640 1 -- 172.21.15.16:0/3608263611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a800cb40 con 0x7f86b015e1e0 2023-11-24T06:28:41.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86ad7fa640 1 -- 172.21.15.16:0/3608263611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a800cca0 con 0x7f86b015e1e0 2023-11-24T06:28:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86ad7fa640 1 -- 172.21.15.16:0/3608263611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8014680 con 0x7f86b015e1e0 2023-11-24T06:28:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.143+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b01661d0 con 0x7f86b015e1e0 2023-11-24T06:28:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.144+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b00a89f0 con 0x7f86b015e1e0 2023-11-24T06:28:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.145+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b00b2bf0 con 0x7f86b015e1e0 2023-11-24T06:28:41.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.146+0000 7f86ad7fa640 1 -- 172.21.15.16:0/3608263611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f86a801d3b0 con 0x7f86b015e1e0 2023-11-24T06:28:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.147+0000 7f86ad7fa640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f869406aed0 0x7f869406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.147+0000 7f86ad7fa640 1 -- 172.21.15.16:0/3608263611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a8089480 con 0x7f86b015e1e0 2023-11-24T06:28:41.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.148+0000 7f86b4b2c640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f869406aed0 0x7f869406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.149+0000 7f86b4b2c640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f869406aed0 0x7f869406d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86a4036000 tx=0x7f86a403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.151+0000 7f86ad7fa640 1 -- 172.21.15.16:0/3608263611 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86b00b2bf0 con 0x7f86b015e1e0 2023-11-24T06:28:41.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.345+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86b00b2bf0 con 0x7f86b015e1e0 2023-11-24T06:28:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.347+0000 7f86ad7fa640 1 -- 172.21.15.16:0/3608263611 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f86b00b2bf0 con 0x7f86b015e1e0 2023-11-24T06:28:41.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:41.351 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:21.731503+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:28:18.648898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:28:19.200832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f869406aed0 msgr2=0x7f869406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f869406aed0 0x7f869406d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86a4036000 tx=0x7f86a403c040 comp rx=0 tx=0).stop 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015e1e0 msgr2=0x7f86b0140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015e1e0 0x7f86b0140a40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f86a800b970 tx=0x7f86a800be30 comp rx=0 tx=0).stop 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 shutdown_connections 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f869406aed0 0x7f869406d390 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86b015e1e0 0x7f86b0140a40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86b0143ae0 0x7f86b015ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 --2- 172.21.15.16:0/3608263611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86b00cf7d0 0x7f86b01657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 >> 172.21.15.16:0/3608263611 conn(0x7f86b006a800 msgr2=0x7f86b00702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.350+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 shutdown_connections 2023-11-24T06:28:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.351+0000 7f86b6d90640 1 -- 172.21.15.16:0/3608263611 wait complete. 2023-11-24T06:28:41.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:28:18.648898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:28:19.200832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:41.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.605+0000 7f134c3e5640 1 Processor -- start 2023-11-24T06:28:41.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.605+0000 7f134c3e5640 1 -- start start 2023-11-24T06:28:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.606+0000 7f134c3e5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13440d17e0 0x7f13440d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.606+0000 7f134c3e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13440b3890 0x7f13440b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.606+0000 7f134c3e5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13440d2180 con 0x7f134415d610 2023-11-24T06:28:41.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.606+0000 7f134c3e5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13440d1390 con 0x7f13440d17e0 2023-11-24T06:28:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.606+0000 7f134c3e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13440b3400 con 0x7f13440b3890 2023-11-24T06:28:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.607+0000 7f1349980640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13440d17e0 0x7f13440d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.607+0000 7f1349980640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13440d17e0 0x7f13440d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46946/0 (socket says 172.21.15.16:46946) 2023-11-24T06:28:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.607+0000 7f134a181640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f134415d610 0x7f13440ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34910/0 (socket says 172.21.15.16:34910) 2023-11-24T06:28:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.607+0000 7f1349980640 1 -- 172.21.15.16:0/128621962 learned_addr learned my addr 172.21.15.16:0/128621962 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.607+0000 7f1349980640 1 -- 172.21.15.16:0/128621962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13440b3890 msgr2=0x7f13440b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.607+0000 7f1349980640 1 --2- 172.21.15.16:0/128621962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13440b3890 0x7f13440b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.607+0000 7f1349980640 1 -- 172.21.15.16:0/128621962 >> v1:172.21.15.16:6789/0 conn(0x7f134415d610 legacy=0x7f13440ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.607+0000 7f1349980640 1 -- 172.21.15.16:0/128621962 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13440b27f0 con 0x7f13440d17e0 2023-11-24T06:28:41.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.607+0000 7f134917f640 1 -- 172.21.15.16:0/128621962 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587552025 0 0) 0x7f13440d2180 con 0x7f134415d610 2023-11-24T06:28:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.608+0000 7f1349980640 1 --2- 172.21.15.16:0/128621962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13440d17e0 0x7f13440d1c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1334009870 tx=0x7f133402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4423aa1ff289eeb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134917f640 1 -- 172.21.15.16:0/128621962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1334030840 con 0x7f13440d17e0 2023-11-24T06:28:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134917f640 1 -- 172.21.15.16:0/128621962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13340309a0 con 0x7f13440d17e0 2023-11-24T06:28:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134917f640 1 -- 172.21.15.16:0/128621962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1334030d30 con 0x7f13440d17e0 2023-11-24T06:28:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134c3e5640 1 -- 172.21.15.16:0/128621962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13440d17e0 msgr2=0x7f13440d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134c3e5640 1 --2- 172.21.15.16:0/128621962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13440d17e0 0x7f13440d1c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1334009870 tx=0x7f133402b980 comp rx=0 tx=0).stop 2023-11-24T06:28:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134c3e5640 1 -- 172.21.15.16:0/128621962 shutdown_connections 2023-11-24T06:28:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134c3e5640 1 --2- 172.21.15.16:0/128621962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13440b3890 0x7f13440b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134c3e5640 1 --2- 172.21.15.16:0/128621962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13440d17e0 0x7f13440d1c40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134c3e5640 1 -- 172.21.15.16:0/128621962 >> 172.21.15.16:0/128621962 conn(0x7f134409c540 msgr2=0x7f13440d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134c3e5640 1 -- 172.21.15.16:0/128621962 shutdown_connections 2023-11-24T06:28:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.609+0000 7f134c3e5640 1 -- 172.21.15.16:0/128621962 wait complete. 2023-11-24T06:28:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.611+0000 7f134c3e5640 1 Processor -- start 2023-11-24T06:28:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.611+0000 7f134c3e5640 1 -- start start 2023-11-24T06:28:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.612+0000 7f134c3e5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13440b3890 0x7f1344155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.612+0000 7f134c3e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13440d17e0 0x7f1344155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.612+0000 7f134c3e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134415d610 0x7f13440b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.612+0000 7f134c3e5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440b86d0 con 0x7f134415d610 2023-11-24T06:28:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.612+0000 7f134c3e5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440b89b0 con 0x7f13440b3890 2023-11-24T06:28:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.612+0000 7f134c3e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13440b8c90 con 0x7f13440d17e0 2023-11-24T06:28:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134415d610 0x7f13440b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134415d610 0x7f13440b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46958/0 (socket says 172.21.15.16:46958) 2023-11-24T06:28:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a982640 1 -- 172.21.15.16:0/2228230396 learned_addr learned my addr 172.21.15.16:0/2228230396 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f1349980640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13440d17e0 0x7f1344155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a181640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13440b3890 0x7f1344155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a982640 1 -- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13440d17e0 msgr2=0x7f1344155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a982640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13440d17e0 0x7f1344155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a982640 1 -- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13440b3890 msgr2=0x7f1344155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a982640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13440b3890 0x7f1344155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a982640 1 -- 172.21.15.16:0/2228230396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13340094d0 con 0x7f134415d610 2023-11-24T06:28:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.613+0000 7f134a982640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134415d610 0x7f13440b77c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f133c009fd0 tx=0x7f133c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.614+0000 7f133affd640 1 -- 172.21.15.16:0/2228230396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c011830 con 0x7f134415d610 2023-11-24T06:28:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.614+0000 7f133affd640 1 -- 172.21.15.16:0/2228230396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c011990 con 0x7f134415d610 2023-11-24T06:28:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.614+0000 7f133affd640 1 -- 172.21.15.16:0/2228230396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c011d20 con 0x7f134415d610 2023-11-24T06:28:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.614+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13440b8f70 con 0x7f134415d610 2023-11-24T06:28:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.614+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1344152af0 con 0x7f134415d610 2023-11-24T06:28:41.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.615+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13441536f0 con 0x7f134415d610 2023-11-24T06:28:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.616+0000 7f133affd640 1 -- 172.21.15.16:0/2228230396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f133c01b460 con 0x7f134415d610 2023-11-24T06:28:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.617+0000 7f133affd640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f132806aed0 0x7f132806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.617+0000 7f133affd640 1 -- 172.21.15.16:0/2228230396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f133c089520 con 0x7f134415d610 2023-11-24T06:28:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.618+0000 7f134a181640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f132806aed0 0x7f132806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:41.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.619+0000 7f134a181640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f132806aed0 0x7f132806d390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f13400012e0 tx=0x7f1340005eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:41.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.621+0000 7f133affd640 1 -- 172.21.15.16:0/2228230396 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13441536f0 con 0x7f134415d610 2023-11-24T06:28:41.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.817+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13441536f0 con 0x7f134415d610 2023-11-24T06:28:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.818+0000 7f133affd640 1 -- 172.21.15.16:0/2228230396 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13441536f0 con 0x7f134415d610 2023-11-24T06:28:41.821 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:28:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.820+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f132806aed0 msgr2=0x7f132806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f132806aed0 0x7f132806d390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f13400012e0 tx=0x7f1340005eb0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134415d610 msgr2=0x7f13440b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134415d610 0x7f13440b77c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f133c009fd0 tx=0x7f133c00b8e0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 shutdown_connections 2023-11-24T06:28:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f132806aed0 0x7f132806d390 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f134415d610 0x7f13440b77c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13440d17e0 0x7f1344155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 --2- 172.21.15.16:0/2228230396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13440b3890 0x7f1344155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 >> 172.21.15.16:0/2228230396 conn(0x7f134409c540 msgr2=0x7f13440b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 shutdown_connections 2023-11-24T06:28:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:41.821+0000 7f134c3e5640 1 -- 172.21.15.16:0/2228230396 wait complete. 2023-11-24T06:28:41.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.079+0000 7fdb71a9f640 1 Processor -- start 2023-11-24T06:28:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.080+0000 7fdb71a9f640 1 -- start start 2023-11-24T06:28:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.081+0000 7fdb71a9f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb6c15d4b0 0x7fdb6c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.081+0000 7fdb71a9f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdb6c0cf9a0 0x7fdb6c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.081+0000 7fdb71a9f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6c0d0320 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.081+0000 7fdb71a9f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6c0b2730 con 0x7fdb6c15d4b0 2023-11-24T06:28:42.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.081+0000 7fdb71a9f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6c0d1ec0 con 0x7fdb6c0cf9a0 2023-11-24T06:28:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.081+0000 7fdb70a9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb6c15d4b0 0x7fdb6c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.081+0000 7fdb7129e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdb6c0cddf0 0x7fdb6c0ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34920/0 (socket says 172.21.15.16:34920) 2023-11-24T06:28:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.081+0000 7fdb7129e640 1 -- 172.21.15.16:0/521115731 learned_addr learned my addr 172.21.15.16:0/521115731 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.082+0000 7fdb6bfff640 1 --2- 172.21.15.16:0/521115731 >> v2:172.21.15.173:3300/0 conn(0x7fdb6c0cf9a0 0x7fdb6c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.082+0000 7fdb70a9d640 1 -- 172.21.15.16:0/521115731 >> v2:172.21.15.173:3300/0 conn(0x7fdb6c0cf9a0 msgr2=0x7fdb6c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:42.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.082+0000 7fdb70a9d640 1 --2- 172.21.15.16:0/521115731 >> v2:172.21.15.173:3300/0 conn(0x7fdb6c0cf9a0 0x7fdb6c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.082+0000 7fdb70a9d640 1 -- 172.21.15.16:0/521115731 >> v1:172.21.15.16:6789/0 conn(0x7fdb6c0cddf0 legacy=0x7fdb6c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.082+0000 7fdb70a9d640 1 -- 172.21.15.16:0/521115731 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6c0b29d0 con 0x7fdb6c15d4b0 2023-11-24T06:28:42.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.083+0000 7fdb70a9d640 1 --2- 172.21.15.16:0/521115731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c15d4b0 0x7fdb6c0d1550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c08b580 tx=0x7fdb6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c00131921ccf6d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.083+0000 7fdb6b7fe640 1 -- 172.21.15.16:0/521115731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60031830 con 0x7fdb6c15d4b0 2023-11-24T06:28:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.083+0000 7fdb6b7fe640 1 -- 172.21.15.16:0/521115731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb60031990 con 0x7fdb6c15d4b0 2023-11-24T06:28:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.083+0000 7fdb6b7fe640 1 -- 172.21.15.16:0/521115731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60031d20 con 0x7fdb6c15d4b0 2023-11-24T06:28:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.084+0000 7fdb71a9f640 1 -- 172.21.15.16:0/521115731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c15d4b0 msgr2=0x7fdb6c0d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.084+0000 7fdb71a9f640 1 --2- 172.21.15.16:0/521115731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c15d4b0 0x7fdb6c0d1550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c08b580 tx=0x7fdb6002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.084+0000 7fdb71a9f640 1 -- 172.21.15.16:0/521115731 shutdown_connections 2023-11-24T06:28:42.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.084+0000 7fdb71a9f640 1 --2- 172.21.15.16:0/521115731 >> v2:172.21.15.173:3300/0 conn(0x7fdb6c0cf9a0 0x7fdb6c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.084+0000 7fdb71a9f640 1 --2- 172.21.15.16:0/521115731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c15d4b0 0x7fdb6c0d1550 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.084+0000 7fdb71a9f640 1 -- 172.21.15.16:0/521115731 >> 172.21.15.16:0/521115731 conn(0x7fdb6c0d3000 msgr2=0x7fdb6c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:42.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.084+0000 7fdb71a9f640 1 -- 172.21.15.16:0/521115731 shutdown_connections 2023-11-24T06:28:42.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.084+0000 7fdb71a9f640 1 -- 172.21.15.16:0/521115731 wait complete. 2023-11-24T06:28:42.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.086+0000 7fdb71a9f640 1 Processor -- start 2023-11-24T06:28:42.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.086+0000 7fdb71a9f640 1 -- start start 2023-11-24T06:28:42.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.087+0000 7fdb71a9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c0cddf0 0x7fdb6c1f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.087+0000 7fdb71a9f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6c0cf9a0 0x7fdb6c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.087+0000 7fdb71a9f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb6c15d4b0 0x7fdb6c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.087+0000 7fdb71a9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c09d9d0 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.087+0000 7fdb71a9f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c09dcb0 con 0x7fdb6c15d4b0 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.087+0000 7fdb71a9f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6c09df90 con 0x7fdb6c0cf9a0 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.087+0000 7fdb70a9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c0cddf0 0x7fdb6c1f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.087+0000 7fdb70a9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c0cddf0 0x7fdb6c1f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46980/0 (socket says 172.21.15.16:46980) 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.087+0000 7fdb70a9d640 1 -- 172.21.15.16:0/3397293320 learned_addr learned my addr 172.21.15.16:0/3397293320 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb7129e640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb6c15d4b0 0x7fdb6c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb6bfff640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6c0cf9a0 0x7fdb6c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb70a9d640 1 -- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6c0cf9a0 msgr2=0x7fdb6c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb70a9d640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6c0cf9a0 0x7fdb6c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb70a9d640 1 -- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb6c15d4b0 msgr2=0x7fdb6c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb70a9d640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb6c15d4b0 0x7fdb6c0ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb70a9d640 1 -- 172.21.15.16:0/3397293320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb60009400 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb6bfff640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6c0cf9a0 0x7fdb6c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb70a9d640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c0cddf0 0x7fdb6c1f2590 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdb60000c00 tx=0x7fdb600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb697fa640 1 -- 172.21.15.16:0/3397293320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb600317d0 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.088+0000 7fdb697fa640 1 -- 172.21.15.16:0/3397293320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6003aa60 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.089+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6c15da10 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.089+0000 7fdb697fa640 1 -- 172.21.15.16:0/3397293320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6003ae10 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.089+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6c09e550 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.090+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6c067450 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.091+0000 7fdb697fa640 1 -- 172.21.15.16:0/3397293320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fdb6004d450 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.092+0000 7fdb697fa640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb5006af00 0x7fdb5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.092+0000 7fdb697fa640 1 -- 172.21.15.16:0/3397293320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb600ab8e0 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.092+0000 7fdb6bfff640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb5006af00 0x7fdb5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.094+0000 7fdb6bfff640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb5006af00 0x7fdb5006d3c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0a0f20 tx=0x7fdb58008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.097+0000 7fdb697fa640 1 -- 172.21.15.16:0/3397293320 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb60076820 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.292+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb6c0cd3a0 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.293+0000 7fdb697fa640 1 -- 172.21.15.16:0/3397293320 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb60079e40 con 0x7fdb6c0cddf0 2023-11-24T06:28:42.296 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:28:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb5006af00 msgr2=0x7fdb5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb5006af00 0x7fdb5006d3c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0a0f20 tx=0x7fdb58008040 comp rx=0 tx=0).stop 2023-11-24T06:28:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c0cddf0 msgr2=0x7fdb6c1f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c0cddf0 0x7fdb6c1f2590 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdb60000c00 tx=0x7fdb600046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 shutdown_connections 2023-11-24T06:28:42.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb5006af00 0x7fdb5006d3c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb6c15d4b0 0x7fdb6c0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6c0cf9a0 0x7fdb6c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 --2- 172.21.15.16:0/3397293320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6c0cddf0 0x7fdb6c1f2590 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 >> 172.21.15.16:0/3397293320 conn(0x7fdb6c0d3000 msgr2=0x7fdb6c06bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 shutdown_connections 2023-11-24T06:28:42.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.296+0000 7fdb71a9f640 1 -- 172.21.15.16:0/3397293320 wait complete. 2023-11-24T06:28:42.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:42.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.543+0000 7f659eba0640 1 Processor -- start 2023-11-24T06:28:42.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.543+0000 7f659eba0640 1 -- start start 2023-11-24T06:28:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659eba0640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f65980cf9a0 0x7f65980cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659eba0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65980b31d0 0x7f65980b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659eba0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65980d0320 con 0x7f659815d4b0 2023-11-24T06:28:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659eba0640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65980d1c60 con 0x7f65980cf9a0 2023-11-24T06:28:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659eba0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65980b2730 con 0x7f65980b31d0 2023-11-24T06:28:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659db9e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f659815d4b0 0x7f65980d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32924/0 (socket says 172.21.15.16:32924) 2023-11-24T06:28:42.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659db9e640 1 -- 172.21.15.16:0/3158953508 learned_addr learned my addr 172.21.15.16:0/3158953508 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:42.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791323281 0 0) 0x7f65980d0320 con 0x7f659815d4b0 2023-11-24T06:28:42.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6584003520 con 0x7f659815d4b0 2023-11-24T06:28:42.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.545+0000 7f659e39f640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65980b31d0 0x7f65980b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659d39d640 1 --2- 172.21.15.16:0/3158953508 >> v2:172.21.15.173:3300/0 conn(0x7f65980cf9a0 0x7f65980cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3543193953 0 0) 0x7f6584003520 con 0x7f659815d4b0 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65980b31d0 msgr2=0x7f65980b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65980b31d0 0x7f65980b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 >> v2:172.21.15.173:3300/0 conn(0x7f65980cf9a0 msgr2=0x7f65980cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 --2- 172.21.15.16:0/3158953508 >> v2:172.21.15.173:3300/0 conn(0x7f65980cf9a0 0x7f65980cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980d1ec0 con 0x7f659815d4b0 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f6594001670 con 0x7f659815d4b0 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 >> v1:172.21.15.16:6789/0 conn(0x7f659815d4b0 legacy=0x7f65980d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6584003970 0x7f6584003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65980b31d0 0x7f65840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f6584005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65980d1c60 con 0x7f6584004d00 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65980b2730 con 0x7f6584003970 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.546+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658400ba60 con 0x7f65980b31d0 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.547+0000 7f659e39f640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f6584005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.547+0000 7f659db9e640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6584003970 0x7f6584003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.547+0000 7f659d39d640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65980b31d0 0x7f65840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.547+0000 7f659e39f640 1 -- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65980b31d0 msgr2=0x7f65840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.547+0000 7f659e39f640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65980b31d0 0x7f65840047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.547+0000 7f659e39f640 1 -- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6584003970 msgr2=0x7f6584003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.547+0000 7f659e39f640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6584003970 0x7f6584003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.547+0000 7f659e39f640 1 -- 172.21.15.16:0/3158953508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584003520 con 0x7f6584004d00 2023-11-24T06:28:42.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.547+0000 7f659db9e640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6584003970 0x7f6584003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.549+0000 7f659e39f640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f6584005180 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6590007360 tx=0x7f65840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8955dd53a3160d0d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.549+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659002ccb0 con 0x7f6584004d00 2023-11-24T06:28:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.549+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659002ce10 con 0x7f6584004d00 2023-11-24T06:28:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.549+0000 7f659cb9c640 1 -- 172.21.15.16:0/3158953508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590034660 con 0x7f6584004d00 2023-11-24T06:28:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.549+0000 7f659eba0640 1 -- 172.21.15.16:0/3158953508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 msgr2=0x7f6584005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.549+0000 7f659eba0640 1 --2- 172.21.15.16:0/3158953508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f6584005180 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6590007360 tx=0x7f65840057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.550+0000 7f659e39f640 1 -- 172.21.15.16:0/3158953508 reap_dead start 2023-11-24T06:28:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.550+0000 7f659eba0640 1 -- 172.21.15.16:0/3158953508 shutdown_connections 2023-11-24T06:28:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.550+0000 7f659eba0640 1 -- 172.21.15.16:0/3158953508 >> 172.21.15.16:0/3158953508 conn(0x7f65980d3000 msgr2=0x7f65980d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.550+0000 7f659eba0640 1 -- 172.21.15.16:0/3158953508 shutdown_connections 2023-11-24T06:28:42.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.550+0000 7f659eba0640 1 -- 172.21.15.16:0/3158953508 wait complete. 2023-11-24T06:28:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.552+0000 7f659eba0640 1 Processor -- start 2023-11-24T06:28:42.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.552+0000 7f659eba0640 1 -- start start 2023-11-24T06:28:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.553+0000 7f659eba0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f659814b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.553+0000 7f659eba0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6598159fb0 0x7f6598148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.553+0000 7f659eba0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f659806c0b0 0x7f659806c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.553+0000 7f659eba0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65980cbbc0 con 0x7f6584004d00 2023-11-24T06:28:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.553+0000 7f659eba0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659806caa0 con 0x7f659806c0b0 2023-11-24T06:28:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.553+0000 7f659eba0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65980c92b0 con 0x7f6598159fb0 2023-11-24T06:28:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.553+0000 7f659db9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f659814b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.553+0000 7f659db9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f659814b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50752/0 (socket says 172.21.15.16:50752) 2023-11-24T06:28:42.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.553+0000 7f659db9e640 1 -- 172.21.15.16:0/261531993 learned_addr learned my addr 172.21.15.16:0/261531993 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f659d39d640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6598159fb0 0x7f6598148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f659db9e640 1 -- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6598159fb0 msgr2=0x7f6598148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f659db9e640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6598159fb0 0x7f6598148580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f659db9e640 1 -- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f659806c0b0 msgr2=0x7f659806c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f659e39f640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f659806c0b0 0x7f659806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f659db9e640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f659806c0b0 0x7f659806c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f659db9e640 1 -- 172.21.15.16:0/261531993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659814be20 con 0x7f6584004d00 2023-11-24T06:28:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f659db9e640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f659814b8e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f659400e870 tx=0x7f659400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f658e7fc640 1 -- 172.21.15.16:0/261531993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400c870 con 0x7f6584004d00 2023-11-24T06:28:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f658e7fc640 1 -- 172.21.15.16:0/261531993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659400c9d0 con 0x7f6584004d00 2023-11-24T06:28:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f658e7fc640 1 -- 172.21.15.16:0/261531993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400cd40 con 0x7f6584004d00 2023-11-24T06:28:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.554+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65980c9550 con 0x7f6584004d00 2023-11-24T06:28:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.555+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6598148ac0 con 0x7f6584004d00 2023-11-24T06:28:42.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.556+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6598067450 con 0x7f6584004d00 2023-11-24T06:28:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.557+0000 7f658e7fc640 1 -- 172.21.15.16:0/261531993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f659401d460 con 0x7f6584004d00 2023-11-24T06:28:42.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.558+0000 7f658e7fc640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f657c06af00 0x7f657c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.558+0000 7f658e7fc640 1 -- 172.21.15.16:0/261531993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6594089650 con 0x7f6584004d00 2023-11-24T06:28:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.558+0000 7f659d39d640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f657c06af00 0x7f657c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.559+0000 7f659d39d640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f657c06af00 0x7f657c06d3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f65980ce3b0 tx=0x7f6588008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:42.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.562+0000 7f658e7fc640 1 -- 172.21.15.16:0/261531993 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65940546a0 con 0x7f6584004d00 2023-11-24T06:28:42.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.758+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65980b8dd0 con 0x7f6584004d00 2023-11-24T06:28:42.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.759+0000 7f658e7fc640 1 -- 172.21.15.16:0/261531993 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6594057cc0 con 0x7f6584004d00 2023-11-24T06:28:42.762 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.761+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f657c06af00 msgr2=0x7f657c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.761+0000 7f659eba0640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f657c06af00 0x7f657c06d3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f65980ce3b0 tx=0x7f6588008040 comp rx=0 tx=0).stop 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.761+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 msgr2=0x7f659814b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.761+0000 7f659eba0640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f659814b8e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f659400e870 tx=0x7f659400ed30 comp rx=0 tx=0).stop 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.762+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 shutdown_connections 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.762+0000 7f659eba0640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f657c06af00 0x7f657c06d3c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.762+0000 7f659eba0640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f659806c0b0 0x7f659806c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.762+0000 7f659eba0640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6598159fb0 0x7f6598148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.762+0000 7f659eba0640 1 --2- 172.21.15.16:0/261531993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6584004d00 0x7f659814b8e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.762+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 >> 172.21.15.16:0/261531993 conn(0x7f65980d3000 msgr2=0x7f65980b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.762+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 shutdown_connections 2023-11-24T06:28:42.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:42.762+0000 7f659eba0640 1 -- 172.21.15.16:0/261531993 wait complete. 2023-11-24T06:28:42.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:43.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.014+0000 7fbebc941640 1 Processor -- start 2023-11-24T06:28:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.014+0000 7fbebc941640 1 -- start start 2023-11-24T06:28:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.016+0000 7fbebc941640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbeb813df80 0x7fbeb815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.016+0000 7fbebc941640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbeb80cf9a0 0x7fbeb80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.016+0000 7fbebc941640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb806c470 con 0x7fbeb813e360 2023-11-24T06:28:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.016+0000 7fbebc941640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb806c750 con 0x7fbeb813df80 2023-11-24T06:28:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.016+0000 7fbebc941640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb806ca30 con 0x7fbeb80cf9a0 2023-11-24T06:28:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.016+0000 7fbeb659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbeb813df80 0x7fbeb815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.016+0000 7fbeb659c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbeb813df80 0x7fbeb815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50758/0 (socket says 172.21.15.16:50758) 2023-11-24T06:28:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.016+0000 7fbeb659c640 1 -- 172.21.15.16:0/80518626 learned_addr learned my addr 172.21.15.16:0/80518626 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.017+0000 7fbeb659c640 1 -- 172.21.15.16:0/80518626 >> v2:172.21.15.173:3300/0 conn(0x7fbeb80cf9a0 msgr2=0x7fbeb80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.017+0000 7fbeb659c640 1 --2- 172.21.15.16:0/80518626 >> v2:172.21.15.173:3300/0 conn(0x7fbeb80cf9a0 0x7fbeb80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.017+0000 7fbeb659c640 1 -- 172.21.15.16:0/80518626 >> v1:172.21.15.173:6789/0 conn(0x7fbeb813e360 legacy=0x7fbeb80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.017+0000 7fbeb659c640 1 -- 172.21.15.16:0/80518626 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb806cd10 con 0x7fbeb813df80 2023-11-24T06:28:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.018+0000 7fbeb659c640 1 --2- 172.21.15.16:0/80518626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 0x7fbeb815bee0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbeb80ccb10 tx=0x7fbea802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff1fc83302abb58 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.018+0000 7fbeb559a640 1 -- 172.21.15.16:0/80518626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8031870 con 0x7fbeb813df80 2023-11-24T06:28:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.018+0000 7fbeb559a640 1 -- 172.21.15.16:0/80518626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea80319d0 con 0x7fbeb813df80 2023-11-24T06:28:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.018+0000 7fbeb559a640 1 -- 172.21.15.16:0/80518626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8031d60 con 0x7fbeb813df80 2023-11-24T06:28:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.019+0000 7fbebc941640 1 -- 172.21.15.16:0/80518626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 msgr2=0x7fbeb815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.019+0000 7fbebc941640 1 --2- 172.21.15.16:0/80518626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 0x7fbeb815bee0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbeb80ccb10 tx=0x7fbea802b910 comp rx=0 tx=0).stop 2023-11-24T06:28:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.019+0000 7fbebc941640 1 -- 172.21.15.16:0/80518626 shutdown_connections 2023-11-24T06:28:43.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.019+0000 7fbebc941640 1 --2- 172.21.15.16:0/80518626 >> v2:172.21.15.173:3300/0 conn(0x7fbeb80cf9a0 0x7fbeb80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.019+0000 7fbebc941640 1 --2- 172.21.15.16:0/80518626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 0x7fbeb815bee0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.019+0000 7fbebc941640 1 -- 172.21.15.16:0/80518626 >> 172.21.15.16:0/80518626 conn(0x7fbeb806a800 msgr2=0x7fbeb806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:43.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.019+0000 7fbebc941640 1 -- 172.21.15.16:0/80518626 shutdown_connections 2023-11-24T06:28:43.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.019+0000 7fbebc941640 1 -- 172.21.15.16:0/80518626 wait complete. 2023-11-24T06:28:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.021+0000 7fbebc941640 1 Processor -- start 2023-11-24T06:28:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.021+0000 7fbebc941640 1 -- start start 2023-11-24T06:28:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.022+0000 7fbebc941640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbeb80cf9a0 0x7fbeb813d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.022+0000 7fbebc941640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 0x7fbeb8152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.022+0000 7fbebc941640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb813e360 0x7fbeb8153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.022+0000 7fbebc941640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81f3170 con 0x7fbeb813df80 2023-11-24T06:28:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.022+0000 7fbebc941640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb8153610 con 0x7fbeb80cf9a0 2023-11-24T06:28:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.022+0000 7fbebc941640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb81538f0 con 0x7fbeb813e360 2023-11-24T06:28:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.022+0000 7fbeb5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 0x7fbeb8152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.022+0000 7fbeb5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 0x7fbeb8152ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50766/0 (socket says 172.21.15.16:50766) 2023-11-24T06:28:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.022+0000 7fbeb5d9b640 1 -- 172.21.15.16:0/3785921057 learned_addr learned my addr 172.21.15.16:0/3785921057 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbeb659c640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbeb80cf9a0 0x7fbeb813d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbeb6d9d640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb813e360 0x7fbeb8153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbeb5d9b640 1 -- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb813e360 msgr2=0x7fbeb8153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbeb5d9b640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb813e360 0x7fbeb8153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbeb5d9b640 1 -- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbeb80cf9a0 msgr2=0x7fbeb813d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbeb5d9b640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbeb80cf9a0 0x7fbeb813d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbeb5d9b640 1 -- 172.21.15.16:0/3785921057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80094d0 con 0x7fbeb813df80 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbeb6d9d640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb813e360 0x7fbeb8153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbeb5d9b640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 0x7fbeb8152ae0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00c400 tx=0x7fbeac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbea6ffd640 1 -- 172.21.15.16:0/3785921057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0035f0 con 0x7fbeb813df80 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbea6ffd640 1 -- 172.21.15.16:0/3785921057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac003750 con 0x7fbeb813df80 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb8153bd0 con 0x7fbeb813df80 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbea6ffd640 1 -- 172.21.15.16:0/3785921057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac003b40 con 0x7fbeb813df80 2023-11-24T06:28:43.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.023+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb81e5400 con 0x7fbeb813df80 2023-11-24T06:28:43.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.025+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb80675d0 con 0x7fbeb813df80 2023-11-24T06:28:43.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.026+0000 7fbea6ffd640 1 -- 172.21.15.16:0/3785921057 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fbeac003ca0 con 0x7fbeb813df80 2023-11-24T06:28:43.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.026+0000 7fbea6ffd640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe9806aed0 0x7fbe9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.027+0000 7fbea6ffd640 1 -- 172.21.15.16:0/3785921057 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbeac088cb0 con 0x7fbeb813df80 2023-11-24T06:28:43.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.027+0000 7fbeb659c640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe9806aed0 0x7fbe9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.028+0000 7fbeb659c640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe9806aed0 0x7fbe9806d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbea8036000 tx=0x7fbea803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.031+0000 7fbea6ffd640 1 -- 172.21.15.16:0/3785921057 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbeac053d20 con 0x7fbeb813df80 2023-11-24T06:28:43.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.239+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbeb81f3310 con 0x7fbeb813df80 2023-11-24T06:28:43.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.240+0000 7fbea6ffd640 1 -- 172.21.15.16:0/3785921057 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbeac017020 con 0x7fbeb813df80 2023-11-24T06:28:43.243 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:28:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe9806aed0 msgr2=0x7fbe9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe9806aed0 0x7fbe9806d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbea8036000 tx=0x7fbea803c040 comp rx=0 tx=0).stop 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 msgr2=0x7fbeb8152ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 0x7fbeb8152ae0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00c400 tx=0x7fbeac00c8c0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 shutdown_connections 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbe9806aed0 0x7fbe9806d390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbeb813e360 0x7fbeb8153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb813df80 0x7fbeb8152ae0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 --2- 172.21.15.16:0/3785921057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbeb80cf9a0 0x7fbeb813d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 >> 172.21.15.16:0/3785921057 conn(0x7fbeb806a800 msgr2=0x7fbeb8070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 shutdown_connections 2023-11-24T06:28:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.243+0000 7fbebc941640 1 -- 172.21.15.16:0/3785921057 wait complete. 2023-11-24T06:28:43.266 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-24T06:28:43.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:43.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:28:43.366 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[101477]: fuse finished with error 0 and tester_r 0 2023-11-24T06:28:43.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:28:43.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:43.375 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:28:43.560 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:28:43.560 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:43.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:43.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.842+0000 7f734da78640 1 Processor -- start 2023-11-24T06:28:43.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.843+0000 7f734da78640 1 -- start start 2023-11-24T06:28:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.844+0000 7f734da78640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f734815d3c0 0x7f73480d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.844+0000 7f734da78640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73480b4110 0x7f73480cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.844+0000 7f734da78640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73480b2c70 con 0x7f73480cfa40 2023-11-24T06:28:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.844+0000 7f734da78640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73480d03f0 con 0x7f734815d3c0 2023-11-24T06:28:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.844+0000 7f734da78640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480cf6d0 con 0x7f73480b4110 2023-11-24T06:28:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.845+0000 7f734d277640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73480b4110 0x7f73480cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.845+0000 7f734d277640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73480b4110 0x7f73480cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:57670/0 (socket says 172.21.15.16:57670) 2023-11-24T06:28:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.845+0000 7f734d277640 1 -- 172.21.15.16:0/2398643204 learned_addr learned my addr 172.21.15.16:0/2398643204 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:43.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.845+0000 7f734ca76640 1 --2- 172.21.15.16:0/2398643204 >> v2:172.21.15.173:3300/0 conn(0x7f734815d3c0 0x7f73480d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.845+0000 7f7347fff640 1 --1- 172.21.15.16:0/2398643204 >> v1:172.21.15.173:6789/0 conn(0x7f73480cfa40 0x7f73480cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:50600/0 (socket says 172.21.15.16:50600) 2023-11-24T06:28:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.846+0000 7f734d277640 1 -- 172.21.15.16:0/2398643204 >> v2:172.21.15.173:3300/0 conn(0x7f734815d3c0 msgr2=0x7f73480d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.846+0000 7f734d277640 1 --2- 172.21.15.16:0/2398643204 >> v2:172.21.15.173:3300/0 conn(0x7f734815d3c0 0x7f73480d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.846+0000 7f734d277640 1 -- 172.21.15.16:0/2398643204 >> v1:172.21.15.173:6789/0 conn(0x7f73480cfa40 legacy=0x7f73480cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.846+0000 7f734d277640 1 -- 172.21.15.16:0/2398643204 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480cf3f0 con 0x7f73480b4110 2023-11-24T06:28:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.847+0000 7f734d277640 1 --2- 172.21.15.16:0/2398643204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73480b4110 0x7f73480cfeb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7340009870 tx=0x7f734002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95b0e0b3481f337a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.847+0000 7f73477fe640 1 -- 172.21.15.16:0/2398643204 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340031640 con 0x7f73480b4110 2023-11-24T06:28:43.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f73477fe640 1 -- 172.21.15.16:0/2398643204 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73400317a0 con 0x7f73480b4110 2023-11-24T06:28:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f73477fe640 1 -- 172.21.15.16:0/2398643204 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340031b30 con 0x7f73480b4110 2023-11-24T06:28:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f734da78640 1 -- 172.21.15.16:0/2398643204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73480b4110 msgr2=0x7f73480cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f734da78640 1 --2- 172.21.15.16:0/2398643204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73480b4110 0x7f73480cfeb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7340009870 tx=0x7f734002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f734da78640 1 -- 172.21.15.16:0/2398643204 shutdown_connections 2023-11-24T06:28:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f734da78640 1 --2- 172.21.15.16:0/2398643204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73480b4110 0x7f73480cfeb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f734da78640 1 --2- 172.21.15.16:0/2398643204 >> v2:172.21.15.173:3300/0 conn(0x7f734815d3c0 0x7f73480d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f734da78640 1 -- 172.21.15.16:0/2398643204 >> 172.21.15.16:0/2398643204 conn(0x7f73480d3050 msgr2=0x7f73480d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:43.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f734da78640 1 -- 172.21.15.16:0/2398643204 shutdown_connections 2023-11-24T06:28:43.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.848+0000 7f734da78640 1 -- 172.21.15.16:0/2398643204 wait complete. 2023-11-24T06:28:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.850+0000 7f734da78640 1 Processor -- start 2023-11-24T06:28:43.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.850+0000 7f734da78640 1 -- start start 2023-11-24T06:28:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.851+0000 7f734da78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73480b4110 0x7f7348152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.851+0000 7f734da78640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73480cfa40 0x7f7348160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.851+0000 7f734da78640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f734815d3c0 0x7f73480c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.851+0000 7f734da78640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73481608c0 con 0x7f73480b4110 2023-11-24T06:28:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.851+0000 7f734da78640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480b6610 con 0x7f73480cfa40 2023-11-24T06:28:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.851+0000 7f734da78640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73481f6810 con 0x7f734815d3c0 2023-11-24T06:28:43.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734ca76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73480b4110 0x7f7348152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734ca76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73480b4110 0x7f7348152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50776/0 (socket says 172.21.15.16:50776) 2023-11-24T06:28:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734ca76640 1 -- 172.21.15.16:0/978640253 learned_addr learned my addr 172.21.15.16:0/978640253 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f7347fff640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73480cfa40 0x7f7348160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734d277640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f734815d3c0 0x7f73480c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734ca76640 1 -- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f734815d3c0 msgr2=0x7f73480c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734ca76640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f734815d3c0 0x7f73480c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734ca76640 1 -- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73480cfa40 msgr2=0x7f7348160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734ca76640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73480cfa40 0x7f7348160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:43.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734ca76640 1 -- 172.21.15.16:0/978640253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73400094d0 con 0x7f73480b4110 2023-11-24T06:28:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734d277640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f734815d3c0 0x7f73480c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.852+0000 7f734ca76640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73480b4110 0x7f7348152190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f733800efc0 tx=0x7f733800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.853+0000 7f73457fa640 1 -- 172.21.15.16:0/978640253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338009b80 con 0x7f73480b4110 2023-11-24T06:28:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.853+0000 7f73457fa640 1 -- 172.21.15.16:0/978640253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733800f040 con 0x7f73480b4110 2023-11-24T06:28:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.853+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73481f6af0 con 0x7f73480b4110 2023-11-24T06:28:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.853+0000 7f73457fa640 1 -- 172.21.15.16:0/978640253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380146a0 con 0x7f73480b4110 2023-11-24T06:28:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.853+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73481f70b0 con 0x7f73480b4110 2023-11-24T06:28:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.854+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734808ba80 con 0x7f73480b4110 2023-11-24T06:28:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.856+0000 7f73457fa640 1 -- 172.21.15.16:0/978640253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f7338014800 con 0x7f73480b4110 2023-11-24T06:28:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.856+0000 7f73457fa640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f733406af00 0x7f733406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.856+0000 7f73457fa640 1 -- 172.21.15.16:0/978640253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f733808a610 con 0x7f73480b4110 2023-11-24T06:28:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.857+0000 7f7347fff640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f733406af00 0x7f733406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.858+0000 7f7347fff640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f733406af00 0x7f733406d3c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f733c007fd0 tx=0x7f733c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:43.861+0000 7f73457fa640 1 -- 172.21.15.16:0/978640253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7338055550 con 0x7f73480b4110 2023-11-24T06:28:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.059+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73480c6b80 con 0x7f73480b4110 2023-11-24T06:28:44.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.063+0000 7f73457fa640 1 -- 172.21.15.16:0/978640253 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f7338058b70 con 0x7f73480b4110 2023-11-24T06:28:44.067 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:44.067 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:21.731503+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:28:18.648898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:28:19.200832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.065+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f733406af00 msgr2=0x7f733406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.065+0000 7f734da78640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f733406af00 0x7f733406d3c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f733c007fd0 tx=0x7f733c009040 comp rx=0 tx=0).stop 2023-11-24T06:28:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.065+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73480b4110 msgr2=0x7f7348152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.066+0000 7f734da78640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73480b4110 0x7f7348152190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f733800efc0 tx=0x7f733800c490 comp rx=0 tx=0).stop 2023-11-24T06:28:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.066+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 shutdown_connections 2023-11-24T06:28:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.066+0000 7f734da78640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f733406af00 0x7f733406d3c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.066+0000 7f734da78640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f734815d3c0 0x7f73480c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.066+0000 7f734da78640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73480cfa40 0x7f7348160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.066+0000 7f734da78640 1 --2- 172.21.15.16:0/978640253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73480b4110 0x7f7348152190 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.066+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 >> 172.21.15.16:0/978640253 conn(0x7f73480d3050 msgr2=0x7f73480b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:44.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.066+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 shutdown_connections 2023-11-24T06:28:44.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.066+0000 7f734da78640 1 -- 172.21.15.16:0/978640253 wait complete. 2023-11-24T06:28:44.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:28:18.648898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:28:19.200832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:44.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.321+0000 7fb391373640 1 Processor -- start 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.323+0000 7fb391373640 1 -- start start 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.323+0000 7fb391373640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb38c0adf20 0x7fb38c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.323+0000 7fb391373640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.324+0000 7fb391373640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c15f4e0 con 0x7fb38c0cc980 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.324+0000 7fb391373640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c15f970 con 0x7fb38c0adf20 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.324+0000 7fb391373640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0cce10 con 0x7fb38c15bf10 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.324+0000 7fb383fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.324+0000 7fb383fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:57682/0 (socket says 172.21.15.16:57682) 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.324+0000 7fb383fff640 1 -- 172.21.15.16:0/2102104948 learned_addr learned my addr 172.21.15.16:0/2102104948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.326+0000 7fb38bfff640 1 --2- 172.21.15.16:0/2102104948 >> v2:172.21.15.173:3300/0 conn(0x7fb38c0adf20 0x7fb38c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.326+0000 7fb383fff640 1 -- 172.21.15.16:0/2102104948 >> v2:172.21.15.173:3300/0 conn(0x7fb38c0adf20 msgr2=0x7fb38c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.326+0000 7fb383fff640 1 --2- 172.21.15.16:0/2102104948 >> v2:172.21.15.173:3300/0 conn(0x7fb38c0adf20 0x7fb38c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.326+0000 7fb383fff640 1 -- 172.21.15.16:0/2102104948 >> v1:172.21.15.173:6789/0 conn(0x7fb38c0cc980 legacy=0x7fb38c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.326+0000 7fb383fff640 1 -- 172.21.15.16:0/2102104948 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c071130 con 0x7fb38c15bf10 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.327+0000 7fb383fff640 1 --2- 172.21.15.16:0/2102104948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0cc440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb374009870 tx=0x7fb37402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd8babb743425fc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.327+0000 7fb38b7fe640 1 -- 172.21.15.16:0/2102104948 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374031640 con 0x7fb38c15bf10 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.327+0000 7fb38b7fe640 1 -- 172.21.15.16:0/2102104948 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3740317a0 con 0x7fb38c15bf10 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.327+0000 7fb38b7fe640 1 -- 172.21.15.16:0/2102104948 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374031b30 con 0x7fb38c15bf10 2023-11-24T06:28:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.329+0000 7fb391373640 1 -- 172.21.15.16:0/2102104948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 msgr2=0x7fb38c0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.329+0000 7fb391373640 1 --2- 172.21.15.16:0/2102104948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0cc440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb374009870 tx=0x7fb37402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.329+0000 7fb391373640 1 -- 172.21.15.16:0/2102104948 shutdown_connections 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.329+0000 7fb391373640 1 --2- 172.21.15.16:0/2102104948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0cc440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.329+0000 7fb391373640 1 --2- 172.21.15.16:0/2102104948 >> v2:172.21.15.173:3300/0 conn(0x7fb38c0adf20 0x7fb38c070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.329+0000 7fb391373640 1 -- 172.21.15.16:0/2102104948 >> 172.21.15.16:0/2102104948 conn(0x7fb38c06a7d0 msgr2=0x7fb38c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.329+0000 7fb391373640 1 -- 172.21.15.16:0/2102104948 shutdown_connections 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.329+0000 7fb391373640 1 -- 172.21.15.16:0/2102104948 wait complete. 2023-11-24T06:28:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.331+0000 7fb391373640 1 Processor -- start 2023-11-24T06:28:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.331+0000 7fb391373640 1 -- start start 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb391373640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb38c0adf20 0x7fb38c139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb391373640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c0cc980 0x7fb38c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb391373640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb391373640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c139560 con 0x7fb38c0cc980 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb391373640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0c67f0 con 0x7fb38c0adf20 2023-11-24T06:28:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb391373640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0c6ad0 con 0x7fb38c15bf10 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c0cc980 0x7fb38c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb383fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c0cc980 0x7fb38c15a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50788/0 (socket says 172.21.15.16:50788) 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb383fff640 1 -- 172.21.15.16:0/546529443 learned_addr learned my addr 172.21.15.16:0/546529443 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb38bfff640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb38c0adf20 0x7fb38c139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb390b72640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb383fff640 1 -- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 msgr2=0x7fb38c0c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb383fff640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb383fff640 1 -- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb38c0adf20 msgr2=0x7fb38c139020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb383fff640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb38c0adf20 0x7fb38c139020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb383fff640 1 -- 172.21.15.16:0/546529443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3740094d0 con 0x7fb38c0cc980 2023-11-24T06:28:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb383fff640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c0cc980 0x7fb38c15a910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb3740090b0 tx=0x7fb3740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb3897fa640 1 -- 172.21.15.16:0/546529443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37403a9f0 con 0x7fb38c0cc980 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb3897fa640 1 -- 172.21.15.16:0/546529443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37403ab70 con 0x7fb38c0cc980 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.332+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0c6db0 con 0x7fb38c0cc980 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.333+0000 7fb3897fa640 1 -- 172.21.15.16:0/546529443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3740443f0 con 0x7fb38c0cc980 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.333+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c0c7370 con 0x7fb38c0cc980 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.334+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c0c7560 con 0x7fb38c0cc980 2023-11-24T06:28:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.335+0000 7fb3897fa640 1 -- 172.21.15.16:0/546529443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb37402f050 con 0x7fb38c0cc980 2023-11-24T06:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.336+0000 7fb3897fa640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb36c06aed0 0x7fb36c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.336+0000 7fb3897fa640 1 -- 172.21.15.16:0/546529443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3740ab880 con 0x7fb38c0cc980 2023-11-24T06:28:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.336+0000 7fb38bfff640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb36c06aed0 0x7fb36c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.338+0000 7fb38bfff640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb36c06aed0 0x7fb36c06d390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009a30 tx=0x7fb37c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.340+0000 7fb3897fa640 1 -- 172.21.15.16:0/546529443 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3740766a0 con 0x7fb38c0cc980 2023-11-24T06:28:44.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.548+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb38c067450 con 0x7fb38c0cc980 2023-11-24T06:28:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.550+0000 7fb3897fa640 1 -- 172.21.15.16:0/546529443 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb374079cc0 con 0x7fb38c0cc980 2023-11-24T06:28:44.552 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb36c06aed0 msgr2=0x7fb36c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb36c06aed0 0x7fb36c06d390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009a30 tx=0x7fb37c008040 comp rx=0 tx=0).stop 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c0cc980 msgr2=0x7fb38c15a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c0cc980 0x7fb38c15a910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb3740090b0 tx=0x7fb3740046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 shutdown_connections 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb36c06aed0 0x7fb36c06d390 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb38c15bf10 0x7fb38c0c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c0cc980 0x7fb38c15a910 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 --2- 172.21.15.16:0/546529443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb38c0adf20 0x7fb38c139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.552+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 >> 172.21.15.16:0/546529443 conn(0x7fb38c06a7d0 msgr2=0x7fb38c0cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:44.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.553+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 shutdown_connections 2023-11-24T06:28:44.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.553+0000 7fb391373640 1 -- 172.21.15.16:0/546529443 wait complete. 2023-11-24T06:28:44.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.794+0000 7f2b1d5c9640 1 Processor -- start 2023-11-24T06:28:44.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.794+0000 7f2b1d5c9640 1 -- start start 2023-11-24T06:28:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.796+0000 7f2b1d5c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b1813e360 0x7f2b180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.796+0000 7f2b1d5c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1815c5f0 con 0x7f2b1815f9c0 2023-11-24T06:28:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.796+0000 7f2b1d5c9640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1806c470 con 0x7f2b1813df80 2023-11-24T06:28:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.796+0000 7f2b1d5c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1806c750 con 0x7f2b1813e360 2023-11-24T06:28:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.796+0000 7f2b167fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b1813e360 0x7f2b180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.796+0000 7f2b167fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b1813e360 0x7f2b180cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50794/0 (socket says 172.21.15.16:50794) 2023-11-24T06:28:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.796+0000 7f2b167fc640 1 -- 172.21.15.16:0/61288800 learned_addr learned my addr 172.21.15.16:0/61288800 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.796+0000 7f2b177fe640 1 --1- 172.21.15.16:0/61288800 >> v1:172.21.15.16:6789/0 conn(0x7f2b1815f9c0 0x7f2b180cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32932/0 (socket says 172.21.15.16:32932) 2023-11-24T06:28:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.797+0000 7f2b167fc640 1 -- 172.21.15.16:0/61288800 >> v1:172.21.15.173:6789/0 conn(0x7f2b1813df80 legacy=0x7f2b1815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:44.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.797+0000 7f2b167fc640 1 -- 172.21.15.16:0/61288800 >> v1:172.21.15.16:6789/0 conn(0x7f2b1815f9c0 legacy=0x7f2b180cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.797+0000 7f2b167fc640 1 -- 172.21.15.16:0/61288800 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1806ca30 con 0x7f2b1813e360 2023-11-24T06:28:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.798+0000 7f2b167fc640 1 --2- 172.21.15.16:0/61288800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 0x7f2b180cf290 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009870 tx=0x7f2b0c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68ae31d36613bff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.798+0000 7f2b15ffb640 1 -- 172.21.15.16:0/61288800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c030860 con 0x7f2b1813e360 2023-11-24T06:28:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.798+0000 7f2b15ffb640 1 -- 172.21.15.16:0/61288800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c0309c0 con 0x7f2b1813e360 2023-11-24T06:28:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.798+0000 7f2b15ffb640 1 -- 172.21.15.16:0/61288800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c030d50 con 0x7f2b1813e360 2023-11-24T06:28:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.799+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/61288800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 msgr2=0x7f2b180cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.799+0000 7f2b1d5c9640 1 --2- 172.21.15.16:0/61288800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 0x7f2b180cf290 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009870 tx=0x7f2b0c02b9a0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.799+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/61288800 shutdown_connections 2023-11-24T06:28:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.799+0000 7f2b1d5c9640 1 --2- 172.21.15.16:0/61288800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 0x7f2b180cf290 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.799+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/61288800 >> 172.21.15.16:0/61288800 conn(0x7f2b1806a800 msgr2=0x7f2b1806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.799+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/61288800 shutdown_connections 2023-11-24T06:28:44.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.799+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/61288800 wait complete. 2023-11-24T06:28:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.801+0000 7f2b1d5c9640 1 Processor -- start 2023-11-24T06:28:44.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.801+0000 7f2b1d5c9640 1 -- start start 2023-11-24T06:28:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.802+0000 7f2b1d5c9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b1813df80 0x7f2b1813d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.802+0000 7f2b1d5c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 0x7f2b1814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.802+0000 7f2b1d5c9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b1815f9c0 0x7f2b1813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.802+0000 7f2b1d5c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b18151f70 con 0x7f2b1813e360 2023-11-24T06:28:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.802+0000 7f2b1d5c9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1813db10 con 0x7f2b1815f9c0 2023-11-24T06:28:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.802+0000 7f2b1d5c9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1813d090 con 0x7f2b1813df80 2023-11-24T06:28:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 0x7f2b1814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 0x7f2b1814d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50810/0 (socket says 172.21.15.16:50810) 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b167fc640 1 -- 172.21.15.16:0/1576967833 learned_addr learned my addr 172.21.15.16:0/1576967833 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b177fe640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b1815f9c0 0x7f2b1813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b167fc640 1 -- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b1813df80 msgr2=0x7f2b1813d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b167fc640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b1813df80 0x7f2b1813d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b167fc640 1 -- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b1815f9c0 msgr2=0x7f2b1813cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b167fc640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b1815f9c0 0x7f2b1813cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b167fc640 1 -- 172.21.15.16:0/1576967833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c0094d0 con 0x7f2b1813e360 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b177fe640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b1815f9c0 0x7f2b1813cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.803+0000 7f2b167fc640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 0x7f2b1814d8c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009870 tx=0x7f2b0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.804+0000 7f2aff7fe640 1 -- 172.21.15.16:0/1576967833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c030810 con 0x7f2b1813e360 2023-11-24T06:28:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.804+0000 7f2aff7fe640 1 -- 172.21.15.16:0/1576967833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c043400 con 0x7f2b1813e360 2023-11-24T06:28:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.804+0000 7f2aff7fe640 1 -- 172.21.15.16:0/1576967833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0437b0 con 0x7f2b1813e360 2023-11-24T06:28:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.804+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1814e9c0 con 0x7f2b1813e360 2023-11-24T06:28:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.804+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1814e6e0 con 0x7f2b1813e360 2023-11-24T06:28:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.805+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b180675d0 con 0x7f2b1813e360 2023-11-24T06:28:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.806+0000 7f2aff7fe640 1 -- 172.21.15.16:0/1576967833 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2b0c043910 con 0x7f2b1813e360 2023-11-24T06:28:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.807+0000 7f2aff7fe640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2b0406af00 0x7f2b0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.807+0000 7f2aff7fe640 1 -- 172.21.15.16:0/1576967833 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b0c03e070 con 0x7f2b1813e360 2023-11-24T06:28:44.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.808+0000 7f2b16ffd640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2b0406af00 0x7f2b0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:44.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.809+0000 7f2b16ffd640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2b0406af00 0x7f2b0406d3c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2b08006fd0 tx=0x7f2b08006bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:44.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:44.811+0000 7f2aff7fe640 1 -- 172.21.15.16:0/1576967833 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b0c077ae0 con 0x7f2b1813e360 2023-11-24T06:28:45.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.007+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b1806df00 con 0x7f2b1813e360 2023-11-24T06:28:45.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.008+0000 7f2aff7fe640 1 -- 172.21.15.16:0/1576967833 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b0c034020 con 0x7f2b1813e360 2023-11-24T06:28:45.010 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:28:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.010+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2b0406af00 msgr2=0x7f2b0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.010+0000 7f2b1d5c9640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2b0406af00 0x7f2b0406d3c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2b08006fd0 tx=0x7f2b08006bc0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.010+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 msgr2=0x7f2b1814d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.010+0000 7f2b1d5c9640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 0x7f2b1814d8c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009870 tx=0x7f2b0c004730 comp rx=0 tx=0).stop 2023-11-24T06:28:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.011+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 shutdown_connections 2023-11-24T06:28:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.011+0000 7f2b1d5c9640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2b0406af00 0x7f2b0406d3c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.011+0000 7f2b1d5c9640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b1815f9c0 0x7f2b1813cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.011+0000 7f2b1d5c9640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1813e360 0x7f2b1814d8c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.011+0000 7f2b1d5c9640 1 --2- 172.21.15.16:0/1576967833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b1813df80 0x7f2b1813d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.011+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 >> 172.21.15.16:0/1576967833 conn(0x7f2b1806a800 msgr2=0x7f2b1815bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.011+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 shutdown_connections 2023-11-24T06:28:45.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.011+0000 7f2b1d5c9640 1 -- 172.21.15.16:0/1576967833 wait complete. 2023-11-24T06:28:45.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:45.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.266+0000 7ff60e6da640 1 Processor -- start 2023-11-24T06:28:45.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.266+0000 7ff60e6da640 1 -- start start 2023-11-24T06:28:45.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.268+0000 7ff60e6da640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff608143810 0x7ff608071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.268+0000 7ff60e6da640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff60815d770 0x7ff60815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.268+0000 7ff60e6da640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6080ae620 con 0x7ff6080ae170 2023-11-24T06:28:45.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.268+0000 7ff60e6da640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff608143bf0 con 0x7ff60815d770 2023-11-24T06:28:45.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.268+0000 7ff60e6da640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60806c730 con 0x7ff608143810 2023-11-24T06:28:45.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.269+0000 7ff60ded9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff60815d770 0x7ff60815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.269+0000 7ff60d6d8640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff608143810 0x7ff608071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.269+0000 7ff60ced7640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7ff6080ae170 0x7ff60815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:50620/0 (socket says 172.21.15.16:50620) 2023-11-24T06:28:45.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.269+0000 7ff60ced7640 1 -- 172.21.15.16:0/2478464091 learned_addr learned my addr 172.21.15.16:0/2478464091 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:45.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.270+0000 7ff60ded9640 1 -- 172.21.15.16:0/2478464091 >> v2:172.21.15.173:3300/0 conn(0x7ff608143810 msgr2=0x7ff608071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:45.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.270+0000 7ff60ded9640 1 --2- 172.21.15.16:0/2478464091 >> v2:172.21.15.173:3300/0 conn(0x7ff608143810 0x7ff608071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.270+0000 7ff60ded9640 1 -- 172.21.15.16:0/2478464091 >> v1:172.21.15.173:6789/0 conn(0x7ff6080ae170 legacy=0x7ff60815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.270+0000 7ff60ded9640 1 -- 172.21.15.16:0/2478464091 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080724d0 con 0x7ff60815d770 2023-11-24T06:28:45.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.270+0000 7ff60d6d8640 1 --2- 172.21.15.16:0/2478464091 >> v2:172.21.15.173:3300/0 conn(0x7ff608143810 0x7ff608071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:45.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.271+0000 7ff60ded9640 1 --2- 172.21.15.16:0/2478464091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 0x7ff60815dbc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cec40 tx=0x7ff60002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c447257324c328 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:45.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.272+0000 7ff5fffff640 1 -- 172.21.15.16:0/2478464091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60002cb60 con 0x7ff60815d770 2023-11-24T06:28:45.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.272+0000 7ff5fffff640 1 -- 172.21.15.16:0/2478464091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60002ccc0 con 0x7ff60815d770 2023-11-24T06:28:45.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.272+0000 7ff5fffff640 1 -- 172.21.15.16:0/2478464091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600034660 con 0x7ff60815d770 2023-11-24T06:28:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.272+0000 7ff60e6da640 1 -- 172.21.15.16:0/2478464091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 msgr2=0x7ff60815dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.272+0000 7ff60e6da640 1 --2- 172.21.15.16:0/2478464091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 0x7ff60815dbc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cec40 tx=0x7ff60002bdb0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.273+0000 7ff60e6da640 1 -- 172.21.15.16:0/2478464091 shutdown_connections 2023-11-24T06:28:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.273+0000 7ff60e6da640 1 --2- 172.21.15.16:0/2478464091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 0x7ff60815dbc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.273+0000 7ff60e6da640 1 --2- 172.21.15.16:0/2478464091 >> v2:172.21.15.173:3300/0 conn(0x7ff608143810 0x7ff608071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.273+0000 7ff60e6da640 1 -- 172.21.15.16:0/2478464091 >> 172.21.15.16:0/2478464091 conn(0x7ff60806a7d0 msgr2=0x7ff60806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.273+0000 7ff60e6da640 1 -- 172.21.15.16:0/2478464091 shutdown_connections 2023-11-24T06:28:45.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.273+0000 7ff60e6da640 1 -- 172.21.15.16:0/2478464091 wait complete. 2023-11-24T06:28:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.275+0000 7ff60e6da640 1 Processor -- start 2023-11-24T06:28:45.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.275+0000 7ff60e6da640 1 -- start start 2023-11-24T06:28:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60e6da640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6080ae170 0x7ff6081f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60e6da640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff608143810 0x7ff608145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60e6da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 0x7ff60812f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60e6da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff608145630 con 0x7ff60815d770 2023-11-24T06:28:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60e6da640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6080b1b60 con 0x7ff6080ae170 2023-11-24T06:28:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60e6da640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6081464f0 con 0x7ff608143810 2023-11-24T06:28:45.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60ded9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 0x7ff60812f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60ded9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 0x7ff60812f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50832/0 (socket says 172.21.15.16:50832) 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60ded9640 1 -- 172.21.15.16:0/1063526728 learned_addr learned my addr 172.21.15.16:0/1063526728 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60d6d8640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6080ae170 0x7ff6081f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.276+0000 7ff60ced7640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff608143810 0x7ff608145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff60ded9640 1 -- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff608143810 msgr2=0x7ff608145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff60ded9640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff608143810 0x7ff608145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff60ded9640 1 -- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6080ae170 msgr2=0x7ff6081f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff60ded9640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6080ae170 0x7ff6081f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff60ded9640 1 -- 172.21.15.16:0/1063526728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff604009400 con 0x7ff60815d770 2023-11-24T06:28:45.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff60ded9640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 0x7ff60812f6a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff60002c3c0 tx=0x7ff600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff5fdffb640 1 -- 172.21.15.16:0/1063526728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60003e610 con 0x7ff60815d770 2023-11-24T06:28:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff5fdffb640 1 -- 172.21.15.16:0/1063526728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60003e770 con 0x7ff60815d770 2023-11-24T06:28:45.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff5fdffb640 1 -- 172.21.15.16:0/1063526728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60003eb20 con 0x7ff60815d770 2023-11-24T06:28:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.277+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff600009a70 con 0x7ff60815d770 2023-11-24T06:28:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.278+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff608146ab0 con 0x7ff60815d770 2023-11-24T06:28:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.279+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6080b1d20 con 0x7ff60815d770 2023-11-24T06:28:45.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.280+0000 7ff5fdffb640 1 -- 172.21.15.16:0/1063526728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff60003b050 con 0x7ff60815d770 2023-11-24T06:28:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.281+0000 7ff5fdffb640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff5ec06aed0 0x7ff5ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.281+0000 7ff5fdffb640 1 -- 172.21.15.16:0/1063526728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6000abfc0 con 0x7ff60815d770 2023-11-24T06:28:45.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.282+0000 7ff60d6d8640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff5ec06aed0 0x7ff5ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.283+0000 7ff60d6d8640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff5ec06aed0 0x7ff5ec06d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff604009c60 tx=0x7ff604008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:45.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.285+0000 7ff5fdffb640 1 -- 172.21.15.16:0/1063526728 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff600076f80 con 0x7ff60815d770 2023-11-24T06:28:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.483+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff608067450 con 0x7ff60815d770 2023-11-24T06:28:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.485+0000 7ff5fdffb640 1 -- 172.21.15.16:0/1063526728 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff60007a5a0 con 0x7ff60815d770 2023-11-24T06:28:45.488 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:28:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff5ec06aed0 msgr2=0x7ff5ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff5ec06aed0 0x7ff5ec06d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff604009c60 tx=0x7ff604008040 comp rx=0 tx=0).stop 2023-11-24T06:28:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 msgr2=0x7ff60812f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 0x7ff60812f6a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff60002c3c0 tx=0x7ff600004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 shutdown_connections 2023-11-24T06:28:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff5ec06aed0 0x7ff5ec06d390 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff60815d770 0x7ff60812f6a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff608143810 0x7ff608145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 --2- 172.21.15.16:0/1063526728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6080ae170 0x7ff6081f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.487+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 >> 172.21.15.16:0/1063526728 conn(0x7ff60806a7d0 msgr2=0x7ff6080ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.488+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 shutdown_connections 2023-11-24T06:28:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.488+0000 7ff60e6da640 1 -- 172.21.15.16:0/1063526728 wait complete. 2023-11-24T06:28:45.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:28:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.736+0000 7f8d24883640 1 Processor -- start 2023-11-24T06:28:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.736+0000 7f8d24883640 1 -- start start 2023-11-24T06:28:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.737+0000 7f8d24883640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d2015d610 0x7f8d200ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.737+0000 7f8d24883640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d200ced60 0x7f8d200d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.737+0000 7f8d24883640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d200b3400 con 0x7f8d200d1d20 2023-11-24T06:28:45.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.737+0000 7f8d24883640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d200d1390 con 0x7f8d200ced60 2023-11-24T06:28:45.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.737+0000 7f8d24883640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200d21b0 con 0x7f8d2015d610 2023-11-24T06:28:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.738+0000 7f8d1ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8d200d1d20 0x7f8d200cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32936/0 (socket says 172.21.15.16:32936) 2023-11-24T06:28:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.738+0000 7f8d1ed9d640 1 -- 172.21.15.16:0/3387639017 learned_addr learned my addr 172.21.15.16:0/3387639017 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.738+0000 7f8d1dd9b640 1 --2- 172.21.15.16:0/3387639017 >> v2:172.21.15.16:3300/0 conn(0x7f8d200ced60 0x7f8d200d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.738+0000 7f8d1e59c640 1 --2- 172.21.15.16:0/3387639017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d2015d610 0x7f8d200ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.738+0000 7f8d1d59a640 1 -- 172.21.15.16:0/3387639017 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956055199 0 0) 0x7f8d200b3400 con 0x7f8d200d1d20 2023-11-24T06:28:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.738+0000 7f8d1dd9b640 1 -- 172.21.15.16:0/3387639017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d2015d610 msgr2=0x7f8d200ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.738+0000 7f8d1dd9b640 1 --2- 172.21.15.16:0/3387639017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d2015d610 0x7f8d200ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.738+0000 7f8d1dd9b640 1 -- 172.21.15.16:0/3387639017 >> v1:172.21.15.16:6789/0 conn(0x7f8d200d1d20 legacy=0x7f8d200cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.738+0000 7f8d1dd9b640 1 -- 172.21.15.16:0/3387639017 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200b27f0 con 0x7f8d200ced60 2023-11-24T06:28:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.740+0000 7f8d1dd9b640 1 --2- 172.21.15.16:0/3387639017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d200ced60 0x7f8d200d17e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d140097a0 tx=0x7f8d1402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ec127ec1b8272e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.740+0000 7f8d1d59a640 1 -- 172.21.15.16:0/3387639017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14031830 con 0x7f8d200ced60 2023-11-24T06:28:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.740+0000 7f8d1d59a640 1 -- 172.21.15.16:0/3387639017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d14031990 con 0x7f8d200ced60 2023-11-24T06:28:45.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.740+0000 7f8d1d59a640 1 -- 172.21.15.16:0/3387639017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14031d20 con 0x7f8d200ced60 2023-11-24T06:28:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.741+0000 7f8d24883640 1 -- 172.21.15.16:0/3387639017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d200ced60 msgr2=0x7f8d200d17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.741+0000 7f8d24883640 1 --2- 172.21.15.16:0/3387639017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d200ced60 0x7f8d200d17e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d140097a0 tx=0x7f8d1402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.741+0000 7f8d24883640 1 -- 172.21.15.16:0/3387639017 shutdown_connections 2023-11-24T06:28:45.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.741+0000 7f8d24883640 1 --2- 172.21.15.16:0/3387639017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d200ced60 0x7f8d200d17e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.741+0000 7f8d24883640 1 --2- 172.21.15.16:0/3387639017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d2015d610 0x7f8d200ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.741+0000 7f8d24883640 1 -- 172.21.15.16:0/3387639017 >> 172.21.15.16:0/3387639017 conn(0x7f8d2009c540 msgr2=0x7f8d200d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.741+0000 7f8d24883640 1 -- 172.21.15.16:0/3387639017 shutdown_connections 2023-11-24T06:28:45.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.741+0000 7f8d24883640 1 -- 172.21.15.16:0/3387639017 wait complete. 2023-11-24T06:28:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.743+0000 7f8d24883640 1 Processor -- start 2023-11-24T06:28:45.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.743+0000 7f8d24883640 1 -- start start 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.744+0000 7f8d24883640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d200ced60 0x7f8d20152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.744+0000 7f8d24883640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d200d1d20 0x7f8d20149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.744+0000 7f8d24883640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2015d610 0x7f8d200b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.744+0000 7f8d24883640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20149b70 con 0x7f8d2015d610 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.744+0000 7f8d24883640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20156120 con 0x7f8d200d1d20 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.744+0000 7f8d24883640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200b87b0 con 0x7f8d200ced60 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.744+0000 7f8d1ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2015d610 0x7f8d200b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2015d610 0x7f8d200b78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50846/0 (socket says 172.21.15.16:50846) 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1ed9d640 1 -- 172.21.15.16:0/1308015330 learned_addr learned my addr 172.21.15.16:0/1308015330 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1e59c640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d200ced60 0x7f8d20152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1dd9b640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d200d1d20 0x7f8d20149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1ed9d640 1 -- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d200ced60 msgr2=0x7f8d20152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1ed9d640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d200ced60 0x7f8d20152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1ed9d640 1 -- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d200d1d20 msgr2=0x7f8d20149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1ed9d640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d200d1d20 0x7f8d20149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1ed9d640 1 -- 172.21.15.16:0/1308015330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d14009400 con 0x7f8d2015d610 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.745+0000 7f8d1ed9d640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2015d610 0x7f8d200b78a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800c7e0 tx=0x7f8d1800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.746+0000 7f8d0effd640 1 -- 172.21.15.16:0/1308015330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1800a640 con 0x7f8d2015d610 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.746+0000 7f8d0effd640 1 -- 172.21.15.16:0/1308015330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1800a7c0 con 0x7f8d2015d610 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.746+0000 7f8d0effd640 1 -- 172.21.15.16:0/1308015330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1800ab70 con 0x7f8d2015d610 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.746+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d200b8a90 con 0x7f8d2015d610 2023-11-24T06:28:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.746+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d201528e0 con 0x7f8d2015d610 2023-11-24T06:28:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.747+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d200b8fb0 con 0x7f8d2015d610 2023-11-24T06:28:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.748+0000 7f8d0effd640 1 -- 172.21.15.16:0/1308015330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8d1800acd0 con 0x7f8d2015d610 2023-11-24T06:28:45.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.749+0000 7f8d0effd640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d0006aed0 0x7f8d0006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.750+0000 7f8d0effd640 1 -- 172.21.15.16:0/1308015330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d18088c20 con 0x7f8d2015d610 2023-11-24T06:28:45.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.750+0000 7f8d1e59c640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d0006aed0 0x7f8d0006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.751+0000 7f8d1e59c640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d0006aed0 0x7f8d0006d390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d100079a0 tx=0x7f8d10008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.754+0000 7f8d0effd640 1 -- 172.21.15.16:0/1308015330 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d200b8fb0 con 0x7f8d2015d610 2023-11-24T06:28:45.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.966+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8d200b8fb0 con 0x7f8d2015d610 2023-11-24T06:28:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.967+0000 7f8d0effd640 1 -- 172.21.15.16:0/1308015330 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8d200b8fb0 con 0x7f8d2015d610 2023-11-24T06:28:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:28:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d0006aed0 msgr2=0x7f8d0006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d0006aed0 0x7f8d0006d390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d100079a0 tx=0x7f8d10008040 comp rx=0 tx=0).stop 2023-11-24T06:28:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2015d610 msgr2=0x7f8d200b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2015d610 0x7f8d200b78a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800c7e0 tx=0x7f8d1800cca0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 shutdown_connections 2023-11-24T06:28:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d0006aed0 0x7f8d0006d390 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2015d610 0x7f8d200b78a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d200d1d20 0x7f8d20149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 --2- 172.21.15.16:0/1308015330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d200ced60 0x7f8d20152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 >> 172.21.15.16:0/1308015330 conn(0x7f8d2009c540 msgr2=0x7f8d200d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 shutdown_connections 2023-11-24T06:28:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:45.970+0000 7f8d24883640 1 -- 172.21.15.16:0/1308015330 wait complete. 2023-11-24T06:28:45.992 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi173.front.sepia.ceph.com... 2023-11-24T06:28:45.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:45.992 DEBUG:teuthology.orchestra.run.smithi173:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:28:46.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[95554]: fuse finished with error 0 and tester_r 0 2023-11-24T06:28:46.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:28:46.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:46.096 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:28:46.268 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:28:46.268 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:46.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:28:46.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:28:46.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:46.335 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:28:46.398 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:28:46.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:46.399 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:28:46.399 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:46.459 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:28:46.459 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:28:46.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:28:46.459 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:28:46.513 INFO:teuthology.orchestra.run.smithi173.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:28:46.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:28:46.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.745+0000 7f99fa34c640 1 Processor -- start 2023-11-24T06:28:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.745+0000 7f99fa34c640 1 -- start start 2023-11-24T06:28:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.746+0000 7f99fa34c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99f40cd240 0x7f99f40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.746+0000 7f99fa34c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f406cea0 con 0x7f99f409a170 2023-11-24T06:28:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.746+0000 7f99fa34c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f406cbc0 con 0x7f99f406c6b0 2023-11-24T06:28:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.746+0000 7f99fa34c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f406f350 con 0x7f99f40cd240 2023-11-24T06:28:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.746+0000 7f99f3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99f40cd240 0x7f99f40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.746+0000 7f99f88e9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f99f409a170 0x7f99f40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32944/0 (socket says 172.21.15.16:32944) 2023-11-24T06:28:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.746+0000 7f99f3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99f40cd240 0x7f99f40b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50852/0 (socket says 172.21.15.16:50852) 2023-11-24T06:28:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.746+0000 7f99f88e9640 1 -- 172.21.15.16:0/3042267125 learned_addr learned my addr 172.21.15.16:0/3042267125 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.747+0000 7f99f3fff640 1 -- 172.21.15.16:0/3042267125 >> v1:172.21.15.173:6789/0 conn(0x7f99f406c6b0 legacy=0x7f99f4099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.747+0000 7f99f3fff640 1 -- 172.21.15.16:0/3042267125 >> v1:172.21.15.16:6789/0 conn(0x7f99f409a170 legacy=0x7f99f40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.747+0000 7f99f3fff640 1 -- 172.21.15.16:0/3042267125 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f406f630 con 0x7f99f40cd240 2023-11-24T06:28:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.747+0000 7f99f2ffd640 1 -- 172.21.15.16:0/3042267125 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29436925 0 0) 0x7f99f406cea0 con 0x7f99f409a170 2023-11-24T06:28:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.748+0000 7f99f3fff640 1 --2- 172.21.15.16:0/3042267125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 0x7f99f40b3820 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009d90 tx=0x7f99e402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823607bac994fc84 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.748+0000 7f99f2ffd640 1 -- 172.21.15.16:0/3042267125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e402cc60 con 0x7f99f40cd240 2023-11-24T06:28:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.748+0000 7f99f2ffd640 1 -- 172.21.15.16:0/3042267125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e402cdc0 con 0x7f99f40cd240 2023-11-24T06:28:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.749+0000 7f99f2ffd640 1 -- 172.21.15.16:0/3042267125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4034660 con 0x7f99f40cd240 2023-11-24T06:28:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.749+0000 7f99fa34c640 1 -- 172.21.15.16:0/3042267125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 msgr2=0x7f99f40b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.749+0000 7f99fa34c640 1 --2- 172.21.15.16:0/3042267125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 0x7f99f40b3820 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009d90 tx=0x7f99e402beb0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.749+0000 7f99fa34c640 1 -- 172.21.15.16:0/3042267125 shutdown_connections 2023-11-24T06:28:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.749+0000 7f99fa34c640 1 --2- 172.21.15.16:0/3042267125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 0x7f99f40b3820 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.749+0000 7f99fa34c640 1 -- 172.21.15.16:0/3042267125 >> 172.21.15.16:0/3042267125 conn(0x7f99f40b2d00 msgr2=0x7f99f40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.749+0000 7f99fa34c640 1 -- 172.21.15.16:0/3042267125 shutdown_connections 2023-11-24T06:28:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.749+0000 7f99fa34c640 1 -- 172.21.15.16:0/3042267125 wait complete. 2023-11-24T06:28:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.751+0000 7f99fa34c640 1 Processor -- start 2023-11-24T06:28:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.751+0000 7f99fa34c640 1 -- start start 2023-11-24T06:28:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.752+0000 7f99fa34c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99f406c6b0 0x7f99f40c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.752+0000 7f99fa34c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f409a170 0x7f99f40c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.752+0000 7f99fa34c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 0x7f99f40b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.752+0000 7f99fa34c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40c5ed0 con 0x7f99f40cd240 2023-11-24T06:28:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.752+0000 7f99fa34c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40c4e30 con 0x7f99f409a170 2023-11-24T06:28:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.752+0000 7f99fa34c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40c5110 con 0x7f99f406c6b0 2023-11-24T06:28:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f88e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 0x7f99f40b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f88e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 0x7f99f40b6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50858/0 (socket says 172.21.15.16:50858) 2023-11-24T06:28:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f88e9640 1 -- 172.21.15.16:0/2360722917 learned_addr learned my addr 172.21.15.16:0/2360722917 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f37fe640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f409a170 0x7f99f40c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f88e9640 1 -- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99f406c6b0 msgr2=0x7f99f40c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f88e9640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99f406c6b0 0x7f99f40c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f88e9640 1 -- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f409a170 msgr2=0x7f99f40c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f88e9640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f409a170 0x7f99f40c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f88e9640 1 -- 172.21.15.16:0/2360722917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e4009a70 con 0x7f99f40cd240 2023-11-24T06:28:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f37fe640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f409a170 0x7f99f40c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f88e9640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 0x7f99f40b6590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c410 tx=0x7f99ec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f0ff9640 1 -- 172.21.15.16:0/2360722917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0074e0 con 0x7f99f40cd240 2023-11-24T06:28:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f0ff9640 1 -- 172.21.15.16:0/2360722917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec007660 con 0x7f99f40cd240 2023-11-24T06:28:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.753+0000 7f99f0ff9640 1 -- 172.21.15.16:0/2360722917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec004a20 con 0x7f99f40cd240 2023-11-24T06:28:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.754+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f40c53f0 con 0x7f99f40cd240 2023-11-24T06:28:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.754+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f40c59b0 con 0x7f99f40cd240 2023-11-24T06:28:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.755+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f40b3c80 con 0x7f99f40cd240 2023-11-24T06:28:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.756+0000 7f99f0ff9640 1 -- 172.21.15.16:0/2360722917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f99ec004b80 con 0x7f99f40cd240 2023-11-24T06:28:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.757+0000 7f99f0ff9640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f99d406aed0 0x7f99d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.757+0000 7f99f0ff9640 1 -- 172.21.15.16:0/2360722917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99ec089370 con 0x7f99f40cd240 2023-11-24T06:28:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.757+0000 7f99f3fff640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f99d406aed0 0x7f99d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.758+0000 7f99f3fff640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f99d406aed0 0x7f99d406d390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f99e4036000 tx=0x7f99e402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.761+0000 7f99f0ff9640 1 -- 172.21.15.16:0/2360722917 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f99ec0544c0 con 0x7f99f40cd240 2023-11-24T06:28:46.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.995+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99f40675d0 con 0x7f99f40cd240 2023-11-24T06:28:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.996+0000 7f99f0ff9640 1 -- 172.21.15.16:0/2360722917 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f99ec057ae0 con 0x7f99f40cd240 2023-11-24T06:28:46.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:46.998 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/147489979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":147489979},{"type":"v1","addr":"172.21.15.173:6836","nonce":147489979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1330874839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1330874839},{"type":"v1","addr":"172.21.15.173:6837","nonce":1330874839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2341098684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2341098684},{"type":"v1","addr":"172.21.15.16:6835","nonce":2341098684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:28:20.189904+0000","modified":"2023-11-24T06:28:23.468425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6834/4049479925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4049479925},{"type":"v1","addr":"172.21.15.16:6834","nonce":4049479925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:28:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2023-11-24T06:28:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f99d406aed0 msgr2=0x7f99d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f99d406aed0 0x7f99d406d390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f99e4036000 tx=0x7f99e402e000 comp rx=0 tx=0).stop 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 msgr2=0x7f99f40b6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 0x7f99f40b6590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c410 tx=0x7f99ec00c8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 shutdown_connections 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f99d406aed0 0x7f99d406d390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99f40cd240 0x7f99f40b6590 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f409a170 0x7f99f40c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 --2- 172.21.15.16:0/2360722917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99f406c6b0 0x7f99f40c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 >> 172.21.15.16:0/2360722917 conn(0x7f99f40b2d00 msgr2=0x7f99f40cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 shutdown_connections 2023-11-24T06:28:47.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:46.999+0000 7f99fa34c640 1 -- 172.21.15.16:0/2360722917 wait complete. 2023-11-24T06:28:47.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:47.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.242+0000 7f384d367640 1 Processor -- start 2023-11-24T06:28:47.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.242+0000 7f384d367640 1 -- start start 2023-11-24T06:28:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.243+0000 7f384d367640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3848160890 0x7f38480d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.243+0000 7f384d367640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f38480c6be0 0x7f38480cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.244+0000 7f384d367640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38480cf6a0 con 0x7f3848160c70 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.244+0000 7f384d367640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38480d1430 con 0x7f38480c6be0 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.244+0000 7f384d367640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38480b4370 con 0x7f3848160890 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.244+0000 7f38477fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3848160c70 0x7f38480d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32958/0 (socket says 172.21.15.16:32958) 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.244+0000 7f38477fe640 1 -- 172.21.15.16:0/1647954264 learned_addr learned my addr 172.21.15.16:0/1647954264 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.244+0000 7f3847fff640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3848160890 0x7f38480d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.244+0000 7f384cb66640 1 --2- 172.21.15.16:0/1647954264 >> v2:172.21.15.173:3300/0 conn(0x7f38480c6be0 0x7f38480cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.244+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96009417 0 0) 0x7f38480cf6a0 con 0x7f3848160c70 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.244+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3830003520 con 0x7f3848160c70 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1940208099 0 0) 0x7f3830003520 con 0x7f3848160c70 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3848160890 msgr2=0x7f38480d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3848160890 0x7f38480d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 >> v2:172.21.15.173:3300/0 conn(0x7f38480c6be0 msgr2=0x7f38480cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 --2- 172.21.15.16:0/1647954264 >> v2:172.21.15.173:3300/0 conn(0x7f38480c6be0 0x7f38480cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480ce7c0 con 0x7f3848160c70 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f383c001670 con 0x7f3848160c70 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 >> v1:172.21.15.16:6789/0 conn(0x7f3848160c70 legacy=0x7f38480d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3848160890 0x7f3830003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3830004d00 0x7f3830005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38480d1430 con 0x7f3830003f40 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38480b4370 con 0x7f3830004d00 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383000bab0 con 0x7f3848160890 2023-11-24T06:28:47.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.245+0000 7f38477fe640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.246+0000 7f384cb66640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3830004d00 0x7f3830005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.246+0000 7f3847fff640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3848160890 0x7f3830003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.246+0000 7f38477fe640 1 -- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3848160890 msgr2=0x7f3830003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.246+0000 7f38477fe640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3848160890 0x7f3830003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.246+0000 7f38477fe640 1 -- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3830004d00 msgr2=0x7f3830005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.246+0000 7f38477fe640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3830004d00 0x7f3830005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.246+0000 7f38477fe640 1 -- 172.21.15.16:0/1647954264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3830003520 con 0x7f3830003f40 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.246+0000 7f3847fff640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3848160890 0x7f3830003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.247+0000 7f38477fe640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38300047c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f383c00c0e0 tx=0x7f38300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695d69a2c21064cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.247+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c005ad0 con 0x7f3830003f40 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.247+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c005c50 con 0x7f3830003f40 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.247+0000 7f3846ffd640 1 -- 172.21.15.16:0/1647954264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c02c7f0 con 0x7f3830003f40 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.248+0000 7f384d367640 1 -- 172.21.15.16:0/1647954264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 msgr2=0x7f38300047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.248+0000 7f384d367640 1 --2- 172.21.15.16:0/1647954264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38300047c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f383c00c0e0 tx=0x7f38300057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.248+0000 7f384cb66640 1 -- 172.21.15.16:0/1647954264 reap_dead start 2023-11-24T06:28:47.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.248+0000 7f384d367640 1 -- 172.21.15.16:0/1647954264 shutdown_connections 2023-11-24T06:28:47.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.248+0000 7f384d367640 1 -- 172.21.15.16:0/1647954264 >> 172.21.15.16:0/1647954264 conn(0x7f38480d3090 msgr2=0x7f38480d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:47.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.248+0000 7f384d367640 1 -- 172.21.15.16:0/1647954264 shutdown_connections 2023-11-24T06:28:47.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.248+0000 7f384d367640 1 -- 172.21.15.16:0/1647954264 wait complete. 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.250+0000 7f384d367640 1 Processor -- start 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.250+0000 7f384d367640 1 -- start start 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.251+0000 7f384d367640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38480b8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.251+0000 7f384d367640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38480b7950 0x7f38480b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.251+0000 7f384d367640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384815da30 0x7f384815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.251+0000 7f384d367640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38480b5fe0 con 0x7f3830003f40 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.251+0000 7f384d367640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384815c990 con 0x7f384815da30 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.251+0000 7f384d367640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384815cc70 con 0x7f38480b7950 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.251+0000 7f3847fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38480b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.251+0000 7f3847fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38480b8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50886/0 (socket says 172.21.15.16:50886) 2023-11-24T06:28:47.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.251+0000 7f3847fff640 1 -- 172.21.15.16:0/121001320 learned_addr learned my addr 172.21.15.16:0/121001320 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f384cb66640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384815da30 0x7f384815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f38477fe640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38480b7950 0x7f38480b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f3847fff640 1 -- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38480b7950 msgr2=0x7f38480b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f3847fff640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38480b7950 0x7f38480b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f3847fff640 1 -- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384815da30 msgr2=0x7f384815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f3847fff640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384815da30 0x7f384815deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f3847fff640 1 -- 172.21.15.16:0/121001320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384815cf50 con 0x7f3830003f40 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f3847fff640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38480b8860 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f383800f350 tx=0x7f383800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f3844ff9640 1 -- 172.21.15.16:0/121001320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838003b90 con 0x7f3830003f40 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f3844ff9640 1 -- 172.21.15.16:0/121001320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3838011040 con 0x7f3830003f40 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f3844ff9640 1 -- 172.21.15.16:0/121001320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838004020 con 0x7f3830003f40 2023-11-24T06:28:47.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.252+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384815d230 con 0x7f3830003f40 2023-11-24T06:28:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.253+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38480b8310 con 0x7f3830003f40 2023-11-24T06:28:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.254+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3848067450 con 0x7f3830003f40 2023-11-24T06:28:47.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.255+0000 7f3844ff9640 1 -- 172.21.15.16:0/121001320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3838004180 con 0x7f3830003f40 2023-11-24T06:28:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.255+0000 7f3844ff9640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f383406aed0 0x7f383406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.256+0000 7f3844ff9640 1 -- 172.21.15.16:0/121001320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3838089f60 con 0x7f3830003f40 2023-11-24T06:28:47.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.256+0000 7f38477fe640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f383406aed0 0x7f383406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.257+0000 7f38477fe640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f383406aed0 0x7f383406d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f383c034000 tx=0x7f383c007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.260+0000 7f3844ff9640 1 -- 172.21.15.16:0/121001320 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3838054d80 con 0x7f3830003f40 2023-11-24T06:28:47.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.454+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f384808c7e0 con 0x7f3830003f40 2023-11-24T06:28:47.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.458+0000 7f3844ff9640 1 -- 172.21.15.16:0/121001320 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f38380583a0 con 0x7f3830003f40 2023-11-24T06:28:47.460 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:47.460 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:21.731503+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:28:18.648898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:28:19.200832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:47.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f383406aed0 msgr2=0x7f383406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f383406aed0 0x7f383406d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f383c034000 tx=0x7f383c007000 comp rx=0 tx=0).stop 2023-11-24T06:28:47.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 msgr2=0x7f38480b8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38480b8860 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f383800f350 tx=0x7f383800f810 comp rx=0 tx=0).stop 2023-11-24T06:28:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 shutdown_connections 2023-11-24T06:28:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f383406aed0 0x7f383406d390 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384815da30 0x7f384815deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38480b7950 0x7f38480b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 --2- 172.21.15.16:0/121001320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3830003f40 0x7f38480b8860 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.460+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 >> 172.21.15.16:0/121001320 conn(0x7f38480d3090 msgr2=0x7f38480b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.461+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 shutdown_connections 2023-11-24T06:28:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.461+0000 7f384d367640 1 -- 172.21.15.16:0/121001320 wait complete. 2023-11-24T06:28:47.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:28:18.648898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:28:19.200832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:47.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:47.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.708+0000 7f95ea610640 1 Processor -- start 2023-11-24T06:28:47.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.708+0000 7f95ea610640 1 -- start start 2023-11-24T06:28:47.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.709+0000 7f95ea610640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40cf9a0 0x7f95e40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.709+0000 7f95ea610640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e40d0320 con 0x7f95e415d4b0 2023-11-24T06:28:47.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.709+0000 7f95ea610640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e40d1c60 con 0x7f95e40b31d0 2023-11-24T06:28:47.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.709+0000 7f95ea610640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40b2730 con 0x7f95e40cf9a0 2023-11-24T06:28:47.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.710+0000 7f95e960e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f95e415d4b0 0x7f95e40d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32962/0 (socket says 172.21.15.16:32962) 2023-11-24T06:28:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.710+0000 7f95e960e640 1 -- 172.21.15.16:0/2632877691 learned_addr learned my addr 172.21.15.16:0/2632877691 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.710+0000 7f95e8e0d640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40cf9a0 0x7f95e40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.710+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039420300 0 0) 0x7f95e40d0320 con 0x7f95e415d4b0 2023-11-24T06:28:47.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95cc003520 con 0x7f95e415d4b0 2023-11-24T06:28:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2110978919 0 0) 0x7f95cc003520 con 0x7f95e415d4b0 2023-11-24T06:28:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40cf9a0 msgr2=0x7f95e40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40cf9a0 0x7f95e40cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 >> v1:172.21.15.173:6789/0 conn(0x7f95e40b31d0 legacy=0x7f95e40b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40d1ec0 con 0x7f95e415d4b0 2023-11-24T06:28:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f95d40019f0 con 0x7f95e415d4b0 2023-11-24T06:28:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 >> v1:172.21.15.16:6789/0 conn(0x7f95e415d4b0 legacy=0x7f95e40d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95cc004390 0x7f95cc004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40cf9a0 0x7f95cc0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40b2730 con 0x7f95cc0039a0 2023-11-24T06:28:47.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc00b9b0 con 0x7f95cc004390 2023-11-24T06:28:47.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc00bb70 con 0x7f95e40cf9a0 2023-11-24T06:28:47.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.711+0000 7f95e960e640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.712+0000 7f95e9e0f640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40cf9a0 0x7f95cc0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.712+0000 7f95e8e0d640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95cc004390 0x7f95cc004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.712+0000 7f95e960e640 1 -- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40cf9a0 msgr2=0x7f95cc0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:47.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.712+0000 7f95e960e640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40cf9a0 0x7f95cc0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.712+0000 7f95e960e640 1 -- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95cc004390 msgr2=0x7f95cc004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.712+0000 7f95e960e640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95cc004390 0x7f95cc004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.712+0000 7f95e960e640 1 -- 172.21.15.16:0/2632877691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40d1ec0 con 0x7f95cc0039a0 2023-11-24T06:28:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.714+0000 7f95e960e640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95cc003dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f95d400a900 tx=0x7f95cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc1ee1956e84d3c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.714+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4035670 con 0x7f95cc0039a0 2023-11-24T06:28:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.714+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d40357d0 con 0x7f95cc0039a0 2023-11-24T06:28:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.714+0000 7f95e3fff640 1 -- 172.21.15.16:0/2632877691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4035b80 con 0x7f95cc0039a0 2023-11-24T06:28:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.715+0000 7f95ea610640 1 -- 172.21.15.16:0/2632877691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 msgr2=0x7f95cc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.715+0000 7f95ea610640 1 --2- 172.21.15.16:0/2632877691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95cc003dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f95d400a900 tx=0x7f95cc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.715+0000 7f95e9e0f640 1 -- 172.21.15.16:0/2632877691 reap_dead start 2023-11-24T06:28:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.715+0000 7f95ea610640 1 -- 172.21.15.16:0/2632877691 shutdown_connections 2023-11-24T06:28:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.715+0000 7f95ea610640 1 -- 172.21.15.16:0/2632877691 >> 172.21.15.16:0/2632877691 conn(0x7f95e40d3000 msgr2=0x7f95e40d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.715+0000 7f95ea610640 1 -- 172.21.15.16:0/2632877691 shutdown_connections 2023-11-24T06:28:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.715+0000 7f95ea610640 1 -- 172.21.15.16:0/2632877691 wait complete. 2023-11-24T06:28:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.717+0000 7f95ea610640 1 Processor -- start 2023-11-24T06:28:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.717+0000 7f95ea610640 1 -- start start 2023-11-24T06:28:47.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.718+0000 7f95ea610640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95e41f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.718+0000 7f95ea610640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40c7f40 0x7f95e40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.718+0000 7f95ea610640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e40b7f00 0x7f95e415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.718+0000 7f95ea610640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40c7b20 con 0x7f95cc0039a0 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.718+0000 7f95ea610640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40b8380 con 0x7f95e40b7f00 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.718+0000 7f95ea610640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4152850 con 0x7f95e40c7f40 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.718+0000 7f95e960e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95e41f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.718+0000 7f95e960e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95e41f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50898/0 (socket says 172.21.15.16:50898) 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.718+0000 7f95e960e640 1 -- 172.21.15.16:0/970913687 learned_addr learned my addr 172.21.15.16:0/970913687 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e960e640 1 -- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40c7f40 msgr2=0x7f95e40b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e960e640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40c7f40 0x7f95e40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e9e0f640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e40b7f00 0x7f95e415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e960e640 1 -- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e40b7f00 msgr2=0x7f95e415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e960e640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e40b7f00 0x7f95e415e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e960e640 1 -- 172.21.15.16:0/970913687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40c6ea0 con 0x7f95cc0039a0 2023-11-24T06:28:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e960e640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95e41f23a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f95d402cc60 tx=0x7f95d4001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e1ffb640 1 -- 172.21.15.16:0/970913687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4035940 con 0x7f95cc0039a0 2023-11-24T06:28:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e1ffb640 1 -- 172.21.15.16:0/970913687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4035ac0 con 0x7f95cc0039a0 2023-11-24T06:28:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95e1ffb640 1 -- 172.21.15.16:0/970913687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4035e70 con 0x7f95cc0039a0 2023-11-24T06:28:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e40c7180 con 0x7f95cc0039a0 2023-11-24T06:28:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.719+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e40c7740 con 0x7f95cc0039a0 2023-11-24T06:28:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.720+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e4067450 con 0x7f95cc0039a0 2023-11-24T06:28:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.722+0000 7f95e1ffb640 1 -- 172.21.15.16:0/970913687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f95d4032070 con 0x7f95cc0039a0 2023-11-24T06:28:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.723+0000 7f95e1ffb640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f95d006af00 0x7f95d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.723+0000 7f95e1ffb640 1 -- 172.21.15.16:0/970913687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95d40aacc0 con 0x7f95cc0039a0 2023-11-24T06:28:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.724+0000 7f95e8e0d640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f95d006af00 0x7f95d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.725+0000 7f95e8e0d640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f95d006af00 0x7f95d006d3c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f95e40c9060 tx=0x7f95d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.728+0000 7f95e1ffb640 1 -- 172.21.15.16:0/970913687 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95d4075e10 con 0x7f95cc0039a0 2023-11-24T06:28:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.923+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95e40c7d90 con 0x7f95cc0039a0 2023-11-24T06:28:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.924+0000 7f95e1ffb640 1 -- 172.21.15.16:0/970913687 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95d4079430 con 0x7f95cc0039a0 2023-11-24T06:28:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:28:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.926+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f95d006af00 msgr2=0x7f95d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.926+0000 7f95ea610640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f95d006af00 0x7f95d006d3c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f95e40c9060 tx=0x7f95d8008040 comp rx=0 tx=0).stop 2023-11-24T06:28:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.926+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 msgr2=0x7f95e41f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.926+0000 7f95ea610640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95e41f23a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f95d402cc60 tx=0x7f95d4001360 comp rx=0 tx=0).stop 2023-11-24T06:28:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.927+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 shutdown_connections 2023-11-24T06:28:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.927+0000 7f95ea610640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f95d006af00 0x7f95d006d3c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.927+0000 7f95ea610640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95e40b7f00 0x7f95e415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.927+0000 7f95ea610640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95e40c7f40 0x7f95e40b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.927+0000 7f95ea610640 1 --2- 172.21.15.16:0/970913687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95cc0039a0 0x7f95e41f23a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.927+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 >> 172.21.15.16:0/970913687 conn(0x7f95e40d3000 msgr2=0x7f95e406c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.927+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 shutdown_connections 2023-11-24T06:28:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:47.927+0000 7f95ea610640 1 -- 172.21.15.16:0/970913687 wait complete. 2023-11-24T06:28:47.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:28:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.179+0000 7fa9dc141640 1 Processor -- start 2023-11-24T06:28:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.179+0000 7fa9dc141640 1 -- start start 2023-11-24T06:28:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.180+0000 7fa9dc141640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa9d40d3910 0x7fa9d40d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.181+0000 7fa9dc141640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d4160e10 0x7fa9d40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.181+0000 7fa9dc141640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d40d4160 con 0x7fa9d40cfcf0 2023-11-24T06:28:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.181+0000 7fa9dc141640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d40ceff0 con 0x7fa9d40d3910 2023-11-24T06:28:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.181+0000 7fa9dc141640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40cf410 con 0x7fa9d4160e10 2023-11-24T06:28:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.181+0000 7fa9d96dc640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa9d40d3910 0x7fa9d40d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.181+0000 7fa9da6de640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d4160e10 0x7fa9d40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.182+0000 7fa9d96dc640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa9d40d3910 0x7fa9d40d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:34492/0 (socket says 172.21.15.16:34492) 2023-11-24T06:28:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.182+0000 7fa9d96dc640 1 -- 172.21.15.16:0/1885985225 learned_addr learned my addr 172.21.15.16:0/1885985225 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.182+0000 7fa9d96dc640 1 -- 172.21.15.16:0/1885985225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d4160e10 msgr2=0x7fa9d40d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.182+0000 7fa9d96dc640 1 --2- 172.21.15.16:0/1885985225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d4160e10 0x7fa9d40d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.182+0000 7fa9d96dc640 1 -- 172.21.15.16:0/1885985225 >> v1:172.21.15.173:6789/0 conn(0x7fa9d40cfcf0 legacy=0x7fa9d4160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.182+0000 7fa9d96dc640 1 -- 172.21.15.16:0/1885985225 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40b2420 con 0x7fa9d40d3910 2023-11-24T06:28:48.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.184+0000 7fa9d96dc640 1 --2- 172.21.15.16:0/1885985225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9d40d3910 0x7fa9d40d1100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40098a0 tx=0x7fa9c402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae77ee45cdb5174 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.185+0000 7fa9d8edb640 1 -- 172.21.15.16:0/1885985225 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4031640 con 0x7fa9d40d3910 2023-11-24T06:28:48.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.185+0000 7fa9d8edb640 1 -- 172.21.15.16:0/1885985225 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c40317a0 con 0x7fa9d40d3910 2023-11-24T06:28:48.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.185+0000 7fa9d8edb640 1 -- 172.21.15.16:0/1885985225 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4031b30 con 0x7fa9d40d3910 2023-11-24T06:28:48.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.185+0000 7fa9dc141640 1 -- 172.21.15.16:0/1885985225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9d40d3910 msgr2=0x7fa9d40d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.185+0000 7fa9dc141640 1 --2- 172.21.15.16:0/1885985225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9d40d3910 0x7fa9d40d1100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40098a0 tx=0x7fa9c402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.185+0000 7fa9dc141640 1 -- 172.21.15.16:0/1885985225 shutdown_connections 2023-11-24T06:28:48.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.185+0000 7fa9dc141640 1 --2- 172.21.15.16:0/1885985225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d4160e10 0x7fa9d40d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.185+0000 7fa9dc141640 1 --2- 172.21.15.16:0/1885985225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9d40d3910 0x7fa9d40d1100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.186+0000 7fa9dc141640 1 -- 172.21.15.16:0/1885985225 >> 172.21.15.16:0/1885985225 conn(0x7fa9d409c530 msgr2=0x7fa9d40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.186+0000 7fa9dc141640 1 -- 172.21.15.16:0/1885985225 shutdown_connections 2023-11-24T06:28:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.186+0000 7fa9dc141640 1 -- 172.21.15.16:0/1885985225 wait complete. 2023-11-24T06:28:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.188+0000 7fa9dc141640 1 Processor -- start 2023-11-24T06:28:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.188+0000 7fa9dc141640 1 -- start start 2023-11-24T06:28:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.189+0000 7fa9dc141640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9d40cfcf0 0x7fa9d40b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.189+0000 7fa9dc141640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d40d3910 0x7fa9d4149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.189+0000 7fa9dc141640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d4160e10 0x7fa9d40b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.189+0000 7fa9dc141640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4149bb0 con 0x7fa9d4160e10 2023-11-24T06:28:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.189+0000 7fa9dc141640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c87a0 con 0x7fa9d40cfcf0 2023-11-24T06:28:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.189+0000 7fa9dc141640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d40c7e70 con 0x7fa9d40d3910 2023-11-24T06:28:48.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9da6de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d4160e10 0x7fa9d40b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9da6de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d4160e10 0x7fa9d40b6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50902/0 (socket says 172.21.15.16:50902) 2023-11-24T06:28:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9da6de640 1 -- 172.21.15.16:0/2844087127 learned_addr learned my addr 172.21.15.16:0/2844087127 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9d96dc640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d40d3910 0x7fa9d4149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9d9edd640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9d40cfcf0 0x7fa9d40b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9da6de640 1 -- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d40d3910 msgr2=0x7fa9d4149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9da6de640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d40d3910 0x7fa9d4149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9da6de640 1 -- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9d40cfcf0 msgr2=0x7fa9d40b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9da6de640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9d40cfcf0 0x7fa9d40b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.190+0000 7fa9da6de640 1 -- 172.21.15.16:0/2844087127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c40094d0 con 0x7fa9d4160e10 2023-11-24T06:28:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.191+0000 7fa9da6de640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d4160e10 0x7fa9d40b6ad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00efc0 tx=0x7fa9cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.191+0000 7fa9ca7fc640 1 -- 172.21.15.16:0/2844087127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc009b20 con 0x7fa9d4160e10 2023-11-24T06:28:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.191+0000 7fa9ca7fc640 1 -- 172.21.15.16:0/2844087127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc00f040 con 0x7fa9d4160e10 2023-11-24T06:28:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.191+0000 7fa9ca7fc640 1 -- 172.21.15.16:0/2844087127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0146c0 con 0x7fa9d4160e10 2023-11-24T06:28:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.191+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d40c8150 con 0x7fa9d4160e10 2023-11-24T06:28:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.191+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d40b6050 con 0x7fa9d4160e10 2023-11-24T06:28:48.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.193+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d40c8910 con 0x7fa9d4160e10 2023-11-24T06:28:48.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.193+0000 7fa9ca7fc640 1 -- 172.21.15.16:0/2844087127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa9cc014820 con 0x7fa9d4160e10 2023-11-24T06:28:48.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.194+0000 7fa9ca7fc640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa9b806aed0 0x7fa9b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.194+0000 7fa9ca7fc640 1 -- 172.21.15.16:0/2844087127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9cc08a530 con 0x7fa9d4160e10 2023-11-24T06:28:48.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.195+0000 7fa9d9edd640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa9b806aed0 0x7fa9b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.196+0000 7fa9d9edd640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa9b806aed0 0x7fa9b806d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0000f60 tx=0x7fa9d0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.199+0000 7fa9ca7fc640 1 -- 172.21.15.16:0/2844087127 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9cc0554e0 con 0x7fa9d4160e10 2023-11-24T06:28:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.391+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9d40b6240 con 0x7fa9d4160e10 2023-11-24T06:28:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.392+0000 7fa9ca7fc640 1 -- 172.21.15.16:0/2844087127 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa9cc009e60 con 0x7fa9d4160e10 2023-11-24T06:28:48.395 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:28:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa9b806aed0 msgr2=0x7fa9b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa9b806aed0 0x7fa9b806d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0000f60 tx=0x7fa9d0009040 comp rx=0 tx=0).stop 2023-11-24T06:28:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d4160e10 msgr2=0x7fa9d40b6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d4160e10 0x7fa9d40b6ad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00efc0 tx=0x7fa9cc00c490 comp rx=0 tx=0).stop 2023-11-24T06:28:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 shutdown_connections 2023-11-24T06:28:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa9b806aed0 0x7fa9b806d390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d4160e10 0x7fa9d40b6ad0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9d40d3910 0x7fa9d4149670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 --2- 172.21.15.16:0/2844087127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9d40cfcf0 0x7fa9d40b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 >> 172.21.15.16:0/2844087127 conn(0x7fa9d409c530 msgr2=0x7fa9d40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.395+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 shutdown_connections 2023-11-24T06:28:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.396+0000 7fa9dc141640 1 -- 172.21.15.16:0/2844087127 wait complete. 2023-11-24T06:28:48.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:28:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.643+0000 7f4a0ffff640 1 Processor -- start 2023-11-24T06:28:48.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.643+0000 7f4a0ffff640 1 -- start start 2023-11-24T06:28:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.645+0000 7f4a0ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a100cf9a0 0x7f4a100cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.645+0000 7f4a0ffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a100d0320 con 0x7f4a1015d4b0 2023-11-24T06:28:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.645+0000 7f4a0ffff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a100d1c60 con 0x7f4a100b31d0 2023-11-24T06:28:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.645+0000 7f4a0ffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a100b2730 con 0x7f4a100cf9a0 2023-11-24T06:28:48.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.645+0000 7f4a0e7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a100cf9a0 0x7f4a100cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.645+0000 7f4a0e7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a100cf9a0 0x7f4a100cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50904/0 (socket says 172.21.15.16:50904) 2023-11-24T06:28:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.645+0000 7f4a0e7fc640 1 -- 172.21.15.16:0/1805457969 learned_addr learned my addr 172.21.15.16:0/1805457969 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.645+0000 7f4a0effd640 1 --1- 172.21.15.16:0/1805457969 >> v1:172.21.15.16:6789/0 conn(0x7f4a1015d4b0 0x7f4a100d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32972/0 (socket says 172.21.15.16:32972) 2023-11-24T06:28:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.646+0000 7f4a0e7fc640 1 -- 172.21.15.16:0/1805457969 >> v1:172.21.15.173:6789/0 conn(0x7f4a100b31d0 legacy=0x7f4a100b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.646+0000 7f4a0e7fc640 1 -- 172.21.15.16:0/1805457969 >> v1:172.21.15.16:6789/0 conn(0x7f4a1015d4b0 legacy=0x7f4a100d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.646+0000 7f4a0e7fc640 1 -- 172.21.15.16:0/1805457969 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a100d1ec0 con 0x7f4a100cf9a0 2023-11-24T06:28:48.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.646+0000 7f4a0dffb640 1 -- 172.21.15.16:0/1805457969 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766949188 0 0) 0x7f4a100d0320 con 0x7f4a1015d4b0 2023-11-24T06:28:48.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.647+0000 7f4a0e7fc640 1 --2- 172.21.15.16:0/1805457969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 0x7f4a100cfde0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009870 tx=0x7f49f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d24912322d884e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.647+0000 7f4a0dffb640 1 -- 172.21.15.16:0/1805457969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8030860 con 0x7f4a100cf9a0 2023-11-24T06:28:48.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.647+0000 7f4a0dffb640 1 -- 172.21.15.16:0/1805457969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f80309c0 con 0x7f4a100cf9a0 2023-11-24T06:28:48.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.647+0000 7f4a0dffb640 1 -- 172.21.15.16:0/1805457969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8030d50 con 0x7f4a100cf9a0 2023-11-24T06:28:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.648+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1805457969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 msgr2=0x7f4a100cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.648+0000 7f4a0ffff640 1 --2- 172.21.15.16:0/1805457969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 0x7f4a100cfde0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009870 tx=0x7f49f802b9a0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.648+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1805457969 shutdown_connections 2023-11-24T06:28:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.648+0000 7f4a0ffff640 1 --2- 172.21.15.16:0/1805457969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 0x7f4a100cfde0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.648+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1805457969 >> 172.21.15.16:0/1805457969 conn(0x7f4a100d3000 msgr2=0x7f4a100d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.648+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1805457969 shutdown_connections 2023-11-24T06:28:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.648+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1805457969 wait complete. 2023-11-24T06:28:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.650+0000 7f4a0ffff640 1 Processor -- start 2023-11-24T06:28:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.650+0000 7f4a0ffff640 1 -- start start 2023-11-24T06:28:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.651+0000 7f4a0ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a100b31d0 0x7f4a101f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.651+0000 7f4a0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 0x7f4a100caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.651+0000 7f4a0ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a1015d4b0 0x7f4a1009fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.651+0000 7f4a0ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100ca630 con 0x7f4a100cf9a0 2023-11-24T06:28:48.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.651+0000 7f4a0ffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100a0800 con 0x7f4a1015d4b0 2023-11-24T06:28:48.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.651+0000 7f4a0ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a100a0ae0 con 0x7f4a100b31d0 2023-11-24T06:28:48.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.651+0000 7f4a0e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 0x7f4a100caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.651+0000 7f4a0e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 0x7f4a100caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50906/0 (socket says 172.21.15.16:50906) 2023-11-24T06:28:48.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.651+0000 7f4a0e7fc640 1 -- 172.21.15.16:0/1712319289 learned_addr learned my addr 172.21.15.16:0/1712319289 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:48.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.652+0000 7f4a0effd640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a100b31d0 0x7f4a101f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.652+0000 7f4a0f7fe640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a1015d4b0 0x7f4a1009fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.652+0000 7f4a0e7fc640 1 -- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a100b31d0 msgr2=0x7f4a101f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.652+0000 7f4a0e7fc640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a100b31d0 0x7f4a101f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.652+0000 7f4a0e7fc640 1 -- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a1015d4b0 msgr2=0x7f4a1009fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.652+0000 7f4a0e7fc640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a1015d4b0 0x7f4a1009fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.652+0000 7f4a0e7fc640 1 -- 172.21.15.16:0/1712319289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f80094d0 con 0x7f4a100cf9a0 2023-11-24T06:28:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.652+0000 7f4a0e7fc640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 0x7f4a100caa50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009870 tx=0x7f49f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.653+0000 7f4a077fe640 1 -- 172.21.15.16:0/1712319289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8039d60 con 0x7f4a100cf9a0 2023-11-24T06:28:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.653+0000 7f4a077fe640 1 -- 172.21.15.16:0/1712319289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f8043400 con 0x7f4a100cf9a0 2023-11-24T06:28:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.653+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a100ca250 con 0x7f4a100cf9a0 2023-11-24T06:28:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.653+0000 7f4a077fe640 1 -- 172.21.15.16:0/1712319289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f80437b0 con 0x7f4a100cf9a0 2023-11-24T06:28:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.653+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a10154e40 con 0x7f4a100cf9a0 2023-11-24T06:28:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.654+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a100ca770 con 0x7f4a100cf9a0 2023-11-24T06:28:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.655+0000 7f4a077fe640 1 -- 172.21.15.16:0/1712319289 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f49f8043910 con 0x7f4a100cf9a0 2023-11-24T06:28:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.656+0000 7f4a077fe640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49f006aed0 0x7f49f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.656+0000 7f4a077fe640 1 -- 172.21.15.16:0/1712319289 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49f803e070 con 0x7f4a100cf9a0 2023-11-24T06:28:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.656+0000 7f4a0effd640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49f006aed0 0x7f49f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.657+0000 7f4a0effd640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49f006aed0 0x7f49f006d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4a000012e0 tx=0x7f4a00005eb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:48.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.661+0000 7f4a077fe640 1 -- 172.21.15.16:0/1712319289 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a100ca770 con 0x7f4a100cf9a0 2023-11-24T06:28:48.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.854+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a100ca770 con 0x7f4a100cf9a0 2023-11-24T06:28:48.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.855+0000 7f4a077fe640 1 -- 172.21.15.16:0/1712319289 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a100ca770 con 0x7f4a100cf9a0 2023-11-24T06:28:48.857 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:28:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49f006aed0 msgr2=0x7f49f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49f006aed0 0x7f49f006d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4a000012e0 tx=0x7f4a00005eb0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 msgr2=0x7f4a100caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 0x7f4a100caa50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009870 tx=0x7f49f8004730 comp rx=0 tx=0).stop 2023-11-24T06:28:48.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 shutdown_connections 2023-11-24T06:28:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49f006aed0 0x7f49f006d390 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a1015d4b0 0x7f4a1009fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a100cf9a0 0x7f4a100caa50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 --2- 172.21.15.16:0/1712319289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a100b31d0 0x7f4a101f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 >> 172.21.15.16:0/1712319289 conn(0x7f4a100d3000 msgr2=0x7f4a1006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:48.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 shutdown_connections 2023-11-24T06:28:48.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:48.858+0000 7f4a0ffff640 1 -- 172.21.15.16:0/1712319289 wait complete. 2023-11-24T06:28:48.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.100+0000 7fd3ddacf640 1 Processor -- start 2023-11-24T06:28:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.101+0000 7fd3ddacf640 1 -- start start 2023-11-24T06:28:49.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.102+0000 7fd3ddacf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d813df80 0x7fd3d815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.102+0000 7fd3ddacf640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd3d80cf9a0 0x7fd3d80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.102+0000 7fd3ddacf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d806c470 con 0x7fd3d813e360 2023-11-24T06:28:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.102+0000 7fd3ddacf640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d806c750 con 0x7fd3d80cf9a0 2023-11-24T06:28:49.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.102+0000 7fd3ddacf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d806ca30 con 0x7fd3d813df80 2023-11-24T06:28:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.103+0000 7fd3d6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd3d813e360 0x7fd3d80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32984/0 (socket says 172.21.15.16:32984) 2023-11-24T06:28:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.103+0000 7fd3d6ffd640 1 -- 172.21.15.16:0/227638612 learned_addr learned my addr 172.21.15.16:0/227638612 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.103+0000 7fd3d7fff640 1 --2- 172.21.15.16:0/227638612 >> v2:172.21.15.173:3300/0 conn(0x7fd3d80cf9a0 0x7fd3d80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.103+0000 7fd3d77fe640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d813df80 0x7fd3d815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.103+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237717173 0 0) 0x7fd3d806c470 con 0x7fd3d813e360 2023-11-24T06:28:49.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.103+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3c4003520 con 0x7fd3d813e360 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3255077744 0 0) 0x7fd3c4003520 con 0x7fd3d813e360 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d813df80 msgr2=0x7fd3d815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d813df80 0x7fd3d815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 >> v2:172.21.15.173:3300/0 conn(0x7fd3d80cf9a0 msgr2=0x7fd3d80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 --2- 172.21.15.16:0/227638612 >> v2:172.21.15.173:3300/0 conn(0x7fd3d80cf9a0 0x7fd3d80cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d806cd10 con 0x7fd3d813e360 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fd3c0001670 con 0x7fd3d813e360 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 >> v1:172.21.15.16:6789/0 conn(0x7fd3d813e360 legacy=0x7fd3d80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c4003970 0x7fd3c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3c40047c0 0x7fd3c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d813df80 0x7fd3c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d806c750 con 0x7fd3c4003970 2023-11-24T06:28:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d806ca30 con 0x7fd3c40047c0 2023-11-24T06:28:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c400b980 con 0x7fd3d813df80 2023-11-24T06:28:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d7fff640 1 --2- 172.21.15.16:0/227638612 >> v2:172.21.15.173:3300/0 conn(0x7fd3d80cf9a0 0x7fd3d80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:28:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.104+0000 7fd3d77fe640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c4003970 0x7fd3c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.105+0000 7fd3d6ffd640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3c40047c0 0x7fd3c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.105+0000 7fd3d77fe640 1 -- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d813df80 msgr2=0x7fd3c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.105+0000 7fd3d77fe640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d813df80 0x7fd3c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.105+0000 7fd3d77fe640 1 -- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3c40047c0 msgr2=0x7fd3c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.105+0000 7fd3d77fe640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3c40047c0 0x7fd3c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.105+0000 7fd3d77fe640 1 -- 172.21.15.16:0/227638612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c4003520 con 0x7fd3c4003970 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.106+0000 7fd3d77fe640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c4003970 0x7fd3c4003d90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009fc0 tx=0x7fd3c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7aa771c92730a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.107+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c802cc40 con 0x7fd3c4003970 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.107+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c800f040 con 0x7fd3c4003970 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.107+0000 7fd3d67fc640 1 -- 172.21.15.16:0/227638612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8034440 con 0x7fd3c4003970 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.107+0000 7fd3ddacf640 1 -- 172.21.15.16:0/227638612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c4003970 msgr2=0x7fd3c4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.107+0000 7fd3ddacf640 1 --2- 172.21.15.16:0/227638612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c4003970 0x7fd3c4003d90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009fc0 tx=0x7fd3c40057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.107+0000 7fd3d7fff640 1 -- 172.21.15.16:0/227638612 reap_dead start 2023-11-24T06:28:49.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.107+0000 7fd3ddacf640 1 -- 172.21.15.16:0/227638612 shutdown_connections 2023-11-24T06:28:49.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.107+0000 7fd3ddacf640 1 -- 172.21.15.16:0/227638612 >> 172.21.15.16:0/227638612 conn(0x7fd3d806a800 msgr2=0x7fd3d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:49.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.108+0000 7fd3ddacf640 1 -- 172.21.15.16:0/227638612 shutdown_connections 2023-11-24T06:28:49.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.108+0000 7fd3ddacf640 1 -- 172.21.15.16:0/227638612 wait complete. 2023-11-24T06:28:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.110+0000 7fd3ddacf640 1 Processor -- start 2023-11-24T06:28:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.110+0000 7fd3ddacf640 1 -- start start 2023-11-24T06:28:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3ddacf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3c4003970 0x7fd3d813d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3ddacf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8150fb0 0x7fd3d81f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3ddacf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d81f0ff0 0x7fd3d80b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3ddacf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d813dda0 con 0x7fd3d8150fb0 2023-11-24T06:28:49.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3ddacf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d81f15c0 con 0x7fd3c4003970 2023-11-24T06:28:49.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3ddacf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d81f18a0 con 0x7fd3d81f0ff0 2023-11-24T06:28:49.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8150fb0 0x7fd3d81f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8150fb0 0x7fd3d81f0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50932/0 (socket says 172.21.15.16:50932) 2023-11-24T06:28:49.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d6ffd640 1 -- 172.21.15.16:0/495334779 learned_addr learned my addr 172.21.15.16:0/495334779 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:49.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d7fff640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d81f0ff0 0x7fd3d80b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d6ffd640 1 -- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d81f0ff0 msgr2=0x7fd3d80b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d6ffd640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d81f0ff0 0x7fd3d80b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d6ffd640 1 -- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3c4003970 msgr2=0x7fd3d813d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d6ffd640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3c4003970 0x7fd3d813d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d6ffd640 1 -- 172.21.15.16:0/495334779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d81507b0 con 0x7fd3d8150fb0 2023-11-24T06:28:49.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.112+0000 7fd3d7fff640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d81f0ff0 0x7fd3d80b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:49.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.113+0000 7fd3d6ffd640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8150fb0 0x7fd3d81f0ab0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000e780 tx=0x7fd3c000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:49.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.113+0000 7fd3cffff640 1 -- 172.21.15.16:0/495334779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000ee90 con 0x7fd3d8150fb0 2023-11-24T06:28:49.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.113+0000 7fd3cffff640 1 -- 172.21.15.16:0/495334779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c000c870 con 0x7fd3d8150fb0 2023-11-24T06:28:49.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.113+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d8150a90 con 0x7fd3d8150fb0 2023-11-24T06:28:49.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.113+0000 7fd3cffff640 1 -- 172.21.15.16:0/495334779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000cc20 con 0x7fd3d8150fb0 2023-11-24T06:28:49.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.113+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d8151430 con 0x7fd3d8150fb0 2023-11-24T06:28:49.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.115+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d8150cd0 con 0x7fd3d8150fb0 2023-11-24T06:28:49.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.115+0000 7fd3cffff640 1 -- 172.21.15.16:0/495334779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fd3c001d460 con 0x7fd3d8150fb0 2023-11-24T06:28:49.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.116+0000 7fd3cffff640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b806af00 0x7fd3b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.116+0000 7fd3cffff640 1 -- 172.21.15.16:0/495334779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3c0089640 con 0x7fd3d8150fb0 2023-11-24T06:28:49.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.117+0000 7fd3d77fe640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b806af00 0x7fd3b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.118+0000 7fd3d77fe640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b806af00 0x7fd3b806d3c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8036000 tx=0x7fd3c802e070 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.121+0000 7fd3cffff640 1 -- 172.21.15.16:0/495334779 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3d8150cd0 con 0x7fd3d8150fb0 2023-11-24T06:28:49.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.355+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3d8150cd0 con 0x7fd3d8150fb0 2023-11-24T06:28:49.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.356+0000 7fd3cffff640 1 -- 172.21.15.16:0/495334779 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fd3d8150cd0 con 0x7fd3d8150fb0 2023-11-24T06:28:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:49.361 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/147489979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":147489979},{"type":"v1","addr":"172.21.15.173:6836","nonce":147489979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1330874839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1330874839},{"type":"v1","addr":"172.21.15.173:6837","nonce":1330874839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2341098684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2341098684},{"type":"v1","addr":"172.21.15.16:6835","nonce":2341098684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:28:20.189904+0000","modified":"2023-11-24T06:28:23.468425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6834/4049479925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4049479925},{"type":"v1","addr":"172.21.15.16:6834","nonce":4049479925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:28:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2023-11-24T06:28:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b806af00 msgr2=0x7fd3b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b806af00 0x7fd3b806d3c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8036000 tx=0x7fd3c802e070 comp rx=0 tx=0).stop 2023-11-24T06:28:49.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8150fb0 msgr2=0x7fd3d81f0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8150fb0 0x7fd3d81f0ab0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000e780 tx=0x7fd3c000ec40 comp rx=0 tx=0).stop 2023-11-24T06:28:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 shutdown_connections 2023-11-24T06:28:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd3b806af00 0x7fd3b806d3c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d81f0ff0 0x7fd3d80b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d8150fb0 0x7fd3d81f0ab0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 --2- 172.21.15.16:0/495334779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3c4003970 0x7fd3d813d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 >> 172.21.15.16:0/495334779 conn(0x7fd3d806a800 msgr2=0x7fd3d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 shutdown_connections 2023-11-24T06:28:49.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.360+0000 7fd3ddacf640 1 -- 172.21.15.16:0/495334779 wait complete. 2023-11-24T06:28:49.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.595+0000 7fdeb0e85640 1 Processor -- start 2023-11-24T06:28:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.595+0000 7fdeb0e85640 1 -- start start 2023-11-24T06:28:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.597+0000 7fdeb0e85640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdeac15afb0 0x7fdeac13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.597+0000 7fdeb0e85640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdeac13fae0 0x7fdeac0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.597+0000 7fdeb0e85640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac13ff30 con 0x7fdeac13cfa0 2023-11-24T06:28:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.597+0000 7fdeb0e85640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac0cdc40 con 0x7fdeac13fae0 2023-11-24T06:28:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.597+0000 7fdeb0e85640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac0cdec0 con 0x7fdeac15afb0 2023-11-24T06:28:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.597+0000 7fdeabfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdeac13fae0 0x7fdeac0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.597+0000 7fdeabfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdeac13fae0 0x7fdeac0b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50936/0 (socket says 172.21.15.16:50936) 2023-11-24T06:28:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.597+0000 7fdeabfff640 1 -- 172.21.15.16:0/38652488 learned_addr learned my addr 172.21.15.16:0/38652488 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.597+0000 7fdea3fff640 1 --2- 172.21.15.16:0/38652488 >> v2:172.21.15.173:3300/0 conn(0x7fdeac15afb0 0x7fdeac13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.598+0000 7fdeabfff640 1 -- 172.21.15.16:0/38652488 >> v2:172.21.15.173:3300/0 conn(0x7fdeac15afb0 msgr2=0x7fdeac13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.598+0000 7fdeabfff640 1 --2- 172.21.15.16:0/38652488 >> v2:172.21.15.173:3300/0 conn(0x7fdeac15afb0 0x7fdeac13f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.598+0000 7fdeabfff640 1 -- 172.21.15.16:0/38652488 >> v1:172.21.15.173:6789/0 conn(0x7fdeac13cfa0 legacy=0x7fdeac0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.598+0000 7fdeabfff640 1 -- 172.21.15.16:0/38652488 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0ce1a0 con 0x7fdeac13fae0 2023-11-24T06:28:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.599+0000 7fdeabfff640 1 --2- 172.21.15.16:0/38652488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 0x7fdeac0b2040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0b3fd0 tx=0x7fdea402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565ab7c762459558 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.599+0000 7fdeaaffd640 1 -- 172.21.15.16:0/38652488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea402ea00 con 0x7fdeac13fae0 2023-11-24T06:28:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.599+0000 7fdeaaffd640 1 -- 172.21.15.16:0/38652488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea402eb60 con 0x7fdeac13fae0 2023-11-24T06:28:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.599+0000 7fdeaaffd640 1 -- 172.21.15.16:0/38652488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4009600 con 0x7fdeac13fae0 2023-11-24T06:28:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.600+0000 7fdeb0e85640 1 -- 172.21.15.16:0/38652488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 msgr2=0x7fdeac0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.600+0000 7fdeb0e85640 1 --2- 172.21.15.16:0/38652488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 0x7fdeac0b2040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0b3fd0 tx=0x7fdea402e060 comp rx=0 tx=0).stop 2023-11-24T06:28:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.600+0000 7fdeb0e85640 1 -- 172.21.15.16:0/38652488 shutdown_connections 2023-11-24T06:28:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.600+0000 7fdeb0e85640 1 --2- 172.21.15.16:0/38652488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 0x7fdeac0b2040 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.600+0000 7fdeb0e85640 1 --2- 172.21.15.16:0/38652488 >> v2:172.21.15.173:3300/0 conn(0x7fdeac15afb0 0x7fdeac13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.600+0000 7fdeb0e85640 1 -- 172.21.15.16:0/38652488 >> 172.21.15.16:0/38652488 conn(0x7fdeac06a800 msgr2=0x7fdeac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.600+0000 7fdeb0e85640 1 -- 172.21.15.16:0/38652488 shutdown_connections 2023-11-24T06:28:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.600+0000 7fdeb0e85640 1 -- 172.21.15.16:0/38652488 wait complete. 2023-11-24T06:28:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.602+0000 7fdeb0e85640 1 Processor -- start 2023-11-24T06:28:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.602+0000 7fdeb0e85640 1 -- start start 2023-11-24T06:28:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.603+0000 7fdeb0e85640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac13cfa0 0x7fdeac1f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.603+0000 7fdeb0e85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 0x7fdeac155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.603+0000 7fdeb0e85640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15afb0 0x7fdeac155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.603+0000 7fdeb0e85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac13e7b0 con 0x7fdeac13fae0 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.603+0000 7fdeb0e85640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac13ea90 con 0x7fdeac13cfa0 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.603+0000 7fdeb0e85640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac13ed70 con 0x7fdeac15afb0 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdea3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 0x7fdeac155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdea3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 0x7fdeac155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50952/0 (socket says 172.21.15.16:50952) 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdea3fff640 1 -- 172.21.15.16:0/2346826898 learned_addr learned my addr 172.21.15.16:0/2346826898 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdeabfff640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15afb0 0x7fdeac155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdea3fff640 1 -- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15afb0 msgr2=0x7fdeac155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdea3fff640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15afb0 0x7fdeac155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdea3fff640 1 -- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac13cfa0 msgr2=0x7fdeac1f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdeab7fe640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac13cfa0 0x7fdeac1f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdea3fff640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac13cfa0 0x7fdeac1f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdea3fff640 1 -- 172.21.15.16:0/2346826898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea400bd40 con 0x7fdeac13fae0 2023-11-24T06:28:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.604+0000 7fdea3fff640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 0x7fdeac155020 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fde9400d5f0 tx=0x7fde9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.605+0000 7fdea8ff9640 1 -- 172.21.15.16:0/2346826898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde940125f0 con 0x7fdeac13fae0 2023-11-24T06:28:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.605+0000 7fdea8ff9640 1 -- 172.21.15.16:0/2346826898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde94012750 con 0x7fdeac13fae0 2023-11-24T06:28:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.605+0000 7fdea8ff9640 1 -- 172.21.15.16:0/2346826898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94012b00 con 0x7fdeac13fae0 2023-11-24T06:28:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.605+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac13f050 con 0x7fdeac13fae0 2023-11-24T06:28:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.605+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac13e460 con 0x7fdeac13fae0 2023-11-24T06:28:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.606+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac0cc640 con 0x7fdeac13fae0 2023-11-24T06:28:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.607+0000 7fdea8ff9640 1 -- 172.21.15.16:0/2346826898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fde94012c60 con 0x7fdeac13fae0 2023-11-24T06:28:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.608+0000 7fdea8ff9640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde8c06af00 0x7fde8c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.608+0000 7fdea8ff9640 1 -- 172.21.15.16:0/2346826898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde94089bd0 con 0x7fdeac13fae0 2023-11-24T06:28:49.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.609+0000 7fdeab7fe640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde8c06af00 0x7fde8c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:49.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.610+0000 7fdeab7fe640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde8c06af00 0x7fde8c06d3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0015d0 tx=0x7fde9c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.612+0000 7fdea8ff9640 1 -- 172.21.15.16:0/2346826898 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde94054b90 con 0x7fdeac13fae0 2023-11-24T06:28:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.805+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdeac067450 con 0x7fdeac13fae0 2023-11-24T06:28:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.806+0000 7fdea8ff9640 1 -- 172.21.15.16:0/2346826898 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fde940581b0 con 0x7fdeac13fae0 2023-11-24T06:28:49.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:49.809 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:21.731503+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:28:18.648898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:28:19.200832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde8c06af00 msgr2=0x7fde8c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde8c06af00 0x7fde8c06d3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0015d0 tx=0x7fde9c009040 comp rx=0 tx=0).stop 2023-11-24T06:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 msgr2=0x7fdeac155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 0x7fdeac155020 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fde9400d5f0 tx=0x7fde9400dab0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 shutdown_connections 2023-11-24T06:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde8c06af00 0x7fde8c06d3c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdeac15afb0 0x7fdeac155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeac13fae0 0x7fdeac155020 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 --2- 172.21.15.16:0/2346826898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdeac13cfa0 0x7fdeac1f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 >> 172.21.15.16:0/2346826898 conn(0x7fdeac06a800 msgr2=0x7fdeac153970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 shutdown_connections 2023-11-24T06:28:49.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:49.809+0000 7fdeb0e85640 1 -- 172.21.15.16:0/2346826898 wait complete. 2023-11-24T06:28:49.831 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:28:49.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.050+0000 7fc984190640 1 Processor -- start 2023-11-24T06:28:50.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.050+0000 7fc984190640 1 -- start start 2023-11-24T06:28:50.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc984190640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc97c13df80 0x7fc97c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc984190640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc97c0cf9a0 0x7fc97c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc984190640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c06c470 con 0x7fc97c13e360 2023-11-24T06:28:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc984190640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c06c750 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc984190640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c06ca30 con 0x7fc97c13df80 2023-11-24T06:28:50.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc98272d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc97c0cf9a0 0x7fc97c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc98272d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc97c0cf9a0 0x7fc97c0cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50968/0 (socket says 172.21.15.16:50968) 2023-11-24T06:28:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc98272d640 1 -- 172.21.15.16:0/4256151950 learned_addr learned my addr 172.21.15.16:0/4256151950 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc98172b640 1 --1- 172.21.15.16:0/4256151950 >> v1:172.21.15.16:6789/0 conn(0x7fc97c13e360 0x7fc97c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32994/0 (socket says 172.21.15.16:32994) 2023-11-24T06:28:50.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.052+0000 7fc981f2c640 1 --2- 172.21.15.16:0/4256151950 >> v2:172.21.15.173:3300/0 conn(0x7fc97c13df80 0x7fc97c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.053+0000 7fc98272d640 1 -- 172.21.15.16:0/4256151950 >> v2:172.21.15.173:3300/0 conn(0x7fc97c13df80 msgr2=0x7fc97c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.053+0000 7fc98272d640 1 --2- 172.21.15.16:0/4256151950 >> v2:172.21.15.173:3300/0 conn(0x7fc97c13df80 0x7fc97c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.053+0000 7fc98272d640 1 -- 172.21.15.16:0/4256151950 >> v1:172.21.15.16:6789/0 conn(0x7fc97c13e360 legacy=0x7fc97c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.053+0000 7fc98272d640 1 -- 172.21.15.16:0/4256151950 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c06cd10 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.053+0000 7fc980f2a640 1 -- 172.21.15.16:0/4256151950 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491187603 0 0) 0x7fc97c06c470 con 0x7fc97c13e360 2023-11-24T06:28:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.054+0000 7fc98272d640 1 --2- 172.21.15.16:0/4256151950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 0x7fc97c0cd840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc97c072770 tx=0x7fc97402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c7d0f1e3b1a3a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.054+0000 7fc980f2a640 1 -- 172.21.15.16:0/4256151950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974030860 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.054+0000 7fc980f2a640 1 -- 172.21.15.16:0/4256151950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9740309c0 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.054+0000 7fc980f2a640 1 -- 172.21.15.16:0/4256151950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974030d50 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.055+0000 7fc984190640 1 -- 172.21.15.16:0/4256151950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 msgr2=0x7fc97c0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.055+0000 7fc984190640 1 --2- 172.21.15.16:0/4256151950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 0x7fc97c0cd840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc97c072770 tx=0x7fc97402b9a0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.055+0000 7fc984190640 1 -- 172.21.15.16:0/4256151950 shutdown_connections 2023-11-24T06:28:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.055+0000 7fc984190640 1 --2- 172.21.15.16:0/4256151950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 0x7fc97c0cd840 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.055+0000 7fc984190640 1 --2- 172.21.15.16:0/4256151950 >> v2:172.21.15.173:3300/0 conn(0x7fc97c13df80 0x7fc97c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.055+0000 7fc984190640 1 -- 172.21.15.16:0/4256151950 >> 172.21.15.16:0/4256151950 conn(0x7fc97c06a800 msgr2=0x7fc97c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.055+0000 7fc984190640 1 -- 172.21.15.16:0/4256151950 shutdown_connections 2023-11-24T06:28:50.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.055+0000 7fc984190640 1 -- 172.21.15.16:0/4256151950 wait complete. 2023-11-24T06:28:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.057+0000 7fc984190640 1 Processor -- start 2023-11-24T06:28:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.057+0000 7fc984190640 1 -- start start 2023-11-24T06:28:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc984190640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 0x7fc97c156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc984190640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc97c13df80 0x7fc97c158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc984190640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc97c13e360 0x7fc97c1f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc984190640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c159300 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc984190640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c1f0710 con 0x7fc97c13df80 2023-11-24T06:28:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc984190640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c156e00 con 0x7fc97c13e360 2023-11-24T06:28:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc981f2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 0x7fc97c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc981f2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 0x7fc97c156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50980/0 (socket says 172.21.15.16:50980) 2023-11-24T06:28:50.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc981f2c640 1 -- 172.21.15.16:0/1421483452 learned_addr learned my addr 172.21.15.16:0/1421483452 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc981f2c640 1 -- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc97c13e360 msgr2=0x7fc97c1f01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc98172b640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc97c13df80 0x7fc97c158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.058+0000 7fc98272d640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc97c13e360 0x7fc97c1f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc981f2c640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc97c13e360 0x7fc97c1f01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc981f2c640 1 -- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc97c13df80 msgr2=0x7fc97c158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc981f2c640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc97c13df80 0x7fc97c158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc981f2c640 1 -- 172.21.15.16:0/1421483452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740094d0 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc981f2c640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 0x7fc97c156740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc970007ef0 tx=0x7fc97000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc97a7fc640 1 -- 172.21.15.16:0/1421483452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970014620 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc97a7fc640 1 -- 172.21.15.16:0/1421483452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc970014780 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc97a7fc640 1 -- 172.21.15.16:0/1421483452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970014b30 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c1570e0 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.059+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c1f0990 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.061+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c1f0b50 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.061+0000 7fc97a7fc640 1 -- 172.21.15.16:0/1421483452 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc970014c90 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.063+0000 7fc97a7fc640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc96006aed0 0x7fc96006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.063+0000 7fc97a7fc640 1 -- 172.21.15.16:0/1421483452 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc97008a140 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.064+0000 7fc98172b640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc96006aed0 0x7fc96006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.065+0000 7fc98172b640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc96006aed0 0x7fc96006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc968007fd0 tx=0x7fc968009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.067+0000 7fc97a7fc640 1 -- 172.21.15.16:0/1421483452 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc970055100 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.303+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc97c0675d0 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.304+0000 7fc97a7fc640 1 -- 172.21.15.16:0/1421483452 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fc970058720 con 0x7fc97c0cf9a0 2023-11-24T06:28:50.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:50.307 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/147489979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":147489979},{"type":"v1","addr":"172.21.15.173:6836","nonce":147489979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1330874839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1330874839},{"type":"v1","addr":"172.21.15.173:6837","nonce":1330874839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2341098684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2341098684},{"type":"v1","addr":"172.21.15.16:6835","nonce":2341098684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:28:20.189904+0000","modified":"2023-11-24T06:28:23.468425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6834/4049479925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4049479925},{"type":"v1","addr":"172.21.15.16:6834","nonce":4049479925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:28:50.308 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2023-11-24T06:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc96006aed0 msgr2=0x7fc96006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc96006aed0 0x7fc96006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc968007fd0 tx=0x7fc968009040 comp rx=0 tx=0).stop 2023-11-24T06:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 msgr2=0x7fc97c156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 0x7fc97c156740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc970007ef0 tx=0x7fc97000e4b0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 shutdown_connections 2023-11-24T06:28:50.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc96006aed0 0x7fc96006d390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc97c13e360 0x7fc97c1f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc97c13df80 0x7fc97c158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 --2- 172.21.15.16:0/1421483452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97c0cf9a0 0x7fc97c156740 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 >> 172.21.15.16:0/1421483452 conn(0x7fc97c06a800 msgr2=0x7fc97c153fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 shutdown_connections 2023-11-24T06:28:50.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.307+0000 7fc984190640 1 -- 172.21.15.16:0/1421483452 wait complete. 2023-11-24T06:28:50.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.555+0000 7fedf24ee640 1 Processor -- start 2023-11-24T06:28:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.556+0000 7fedf24ee640 1 -- start start 2023-11-24T06:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.557+0000 7fedf24ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedec13df80 0x7fedec15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.557+0000 7fedf24ee640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fedec13e360 0x7fedec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.557+0000 7fedf24ee640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedec06c470 con 0x7fedec15f9c0 2023-11-24T06:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.557+0000 7fedf24ee640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedec06c750 con 0x7fedec13df80 2023-11-24T06:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.557+0000 7fedf24ee640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedec06ca30 con 0x7fedec13e360 2023-11-24T06:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.557+0000 7fedebfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedec13df80 0x7fedec15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.557+0000 7fedebfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedec13df80 0x7fedec15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50988/0 (socket says 172.21.15.16:50988) 2023-11-24T06:28:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.557+0000 7fedebfff640 1 -- 172.21.15.16:0/2558539819 learned_addr learned my addr 172.21.15.16:0/2558539819 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.558+0000 7fedeb7fe640 1 --2- 172.21.15.16:0/2558539819 >> v2:172.21.15.173:3300/0 conn(0x7fedec13e360 0x7fedec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.558+0000 7fedebfff640 1 -- 172.21.15.16:0/2558539819 >> v2:172.21.15.173:3300/0 conn(0x7fedec13e360 msgr2=0x7fedec0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.558+0000 7fedebfff640 1 --2- 172.21.15.16:0/2558539819 >> v2:172.21.15.173:3300/0 conn(0x7fedec13e360 0x7fedec0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.558+0000 7fedebfff640 1 -- 172.21.15.16:0/2558539819 >> v1:172.21.15.173:6789/0 conn(0x7fedec15f9c0 legacy=0x7fedec1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.558+0000 7fedebfff640 1 -- 172.21.15.16:0/2558539819 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec06cd10 con 0x7fedec13df80 2023-11-24T06:28:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.558+0000 7fedeb7fe640 1 --2- 172.21.15.16:0/2558539819 >> v2:172.21.15.173:3300/0 conn(0x7fedec13e360 0x7fedec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.559+0000 7fedebfff640 1 --2- 172.21.15.16:0/2558539819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13df80 0x7fedec15bee0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fedec06ded0 tx=0x7feddc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d45f606c320194c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.559+0000 7fedeaffd640 1 -- 172.21.15.16:0/2558539819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc02ca80 con 0x7fedec13df80 2023-11-24T06:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.559+0000 7fedeaffd640 1 -- 172.21.15.16:0/2558539819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc02cbe0 con 0x7fedec13df80 2023-11-24T06:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.559+0000 7fedeaffd640 1 -- 172.21.15.16:0/2558539819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc0343f0 con 0x7fedec13df80 2023-11-24T06:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.560+0000 7fedf24ee640 1 -- 172.21.15.16:0/2558539819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13df80 msgr2=0x7fedec15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.560+0000 7fedf24ee640 1 --2- 172.21.15.16:0/2558539819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13df80 0x7fedec15bee0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fedec06ded0 tx=0x7feddc02bee0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.560+0000 7fedf24ee640 1 -- 172.21.15.16:0/2558539819 shutdown_connections 2023-11-24T06:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.560+0000 7fedf24ee640 1 --2- 172.21.15.16:0/2558539819 >> v2:172.21.15.173:3300/0 conn(0x7fedec13e360 0x7fedec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.560+0000 7fedf24ee640 1 --2- 172.21.15.16:0/2558539819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13df80 0x7fedec15bee0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.560+0000 7fedf24ee640 1 -- 172.21.15.16:0/2558539819 >> 172.21.15.16:0/2558539819 conn(0x7fedec06a800 msgr2=0x7fedec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.560+0000 7fedf24ee640 1 -- 172.21.15.16:0/2558539819 shutdown_connections 2023-11-24T06:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.560+0000 7fedf24ee640 1 -- 172.21.15.16:0/2558539819 wait complete. 2023-11-24T06:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.562+0000 7fedf24ee640 1 Processor -- start 2023-11-24T06:28:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.562+0000 7fedf24ee640 1 -- start start 2023-11-24T06:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.563+0000 7fedf24ee640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedec13df80 0x7fedec130d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.563+0000 7fedf24ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13e360 0x7fedec0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.563+0000 7fedf24ee640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedec15f9c0 0x7fedec071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.563+0000 7fedf24ee640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedec153c70 con 0x7fedec13e360 2023-11-24T06:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.563+0000 7fedf24ee640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedec132460 con 0x7fedec15f9c0 2023-11-24T06:28:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.563+0000 7fedf24ee640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedec0ae380 con 0x7fedec13df80 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedeb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13e360 0x7fedec0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedeb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13e360 0x7fedec0711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50990/0 (socket says 172.21.15.16:50990) 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedeb7fe640 1 -- 172.21.15.16:0/4236251782 learned_addr learned my addr 172.21.15.16:0/4236251782 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedf0a8b640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedec15f9c0 0x7fedec071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedebfff640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedec13df80 0x7fedec130d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedeb7fe640 1 -- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedec13df80 msgr2=0x7fedec130d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedeb7fe640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedec13df80 0x7fedec130d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedeb7fe640 1 -- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedec15f9c0 msgr2=0x7fedec071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedeb7fe640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedec15f9c0 0x7fedec071700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedeb7fe640 1 -- 172.21.15.16:0/4236251782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc009b30 con 0x7fedec13e360 2023-11-24T06:28:50.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.564+0000 7fedeb7fe640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13e360 0x7fedec0711c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fedd400c820 tx=0x7fedd400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.565+0000 7fede8ff9640 1 -- 172.21.15.16:0/4236251782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4004230 con 0x7fedec13e360 2023-11-24T06:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.565+0000 7fede8ff9640 1 -- 172.21.15.16:0/4236251782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd4004390 con 0x7fedec13e360 2023-11-24T06:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.565+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedec13dd50 con 0x7fedec13e360 2023-11-24T06:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.565+0000 7fede8ff9640 1 -- 172.21.15.16:0/4236251782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4007a70 con 0x7fedec13e360 2023-11-24T06:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.565+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedec131980 con 0x7fedec13e360 2023-11-24T06:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.566+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedec0675d0 con 0x7fedec13e360 2023-11-24T06:28:50.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.567+0000 7fede8ff9640 1 -- 172.21.15.16:0/4236251782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fedd4007bd0 con 0x7fedec13e360 2023-11-24T06:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.568+0000 7fede8ff9640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fedcc06aed0 0x7fedcc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.568+0000 7fede8ff9640 1 -- 172.21.15.16:0/4236251782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedd4089730 con 0x7fedec13e360 2023-11-24T06:28:50.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.568+0000 7fedebfff640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fedcc06aed0 0x7fedcc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:50.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.570+0000 7fedebfff640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fedcc06aed0 0x7fedcc06d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feddc035000 tx=0x7feddc030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:50.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.572+0000 7fede8ff9640 1 -- 172.21.15.16:0/4236251782 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fedd4054670 con 0x7fedec13e360 2023-11-24T06:28:50.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.803+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedec06df00 con 0x7fedec13e360 2023-11-24T06:28:50.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.803+0000 7fede8ff9640 1 -- 172.21.15.16:0/4236251782 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fedd4057c90 con 0x7fedec13e360 2023-11-24T06:28:50.807 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:50.807 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/147489979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":147489979},{"type":"v1","addr":"172.21.15.173:6836","nonce":147489979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1330874839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1330874839},{"type":"v1","addr":"172.21.15.173:6837","nonce":1330874839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2341098684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2341098684},{"type":"v1","addr":"172.21.15.16:6835","nonce":2341098684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:28:20.189904+0000","modified":"2023-11-24T06:28:23.468425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.16:6834/4049479925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4049479925},{"type":"v1","addr":"172.21.15.16:6834","nonce":4049479925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:28:50.807 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2023-11-24T06:28:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fedcc06aed0 msgr2=0x7fedcc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fedcc06aed0 0x7fedcc06d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feddc035000 tx=0x7feddc030000 comp rx=0 tx=0).stop 2023-11-24T06:28:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13e360 msgr2=0x7fedec0711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13e360 0x7fedec0711c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fedd400c820 tx=0x7fedd400cce0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 shutdown_connections 2023-11-24T06:28:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fedcc06aed0 0x7fedcc06d390 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedec15f9c0 0x7fedec071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec13e360 0x7fedec0711c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 --2- 172.21.15.16:0/4236251782 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedec13df80 0x7fedec130d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 >> 172.21.15.16:0/4236251782 conn(0x7fedec06a800 msgr2=0x7fedec15ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.806+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 shutdown_connections 2023-11-24T06:28:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:50.807+0000 7fedf24ee640 1 -- 172.21.15.16:0/4236251782 wait complete. 2023-11-24T06:28:50.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:51.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.041+0000 7f480967e640 1 Processor -- start 2023-11-24T06:28:51.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.041+0000 7f480967e640 1 -- start start 2023-11-24T06:28:51.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f480967e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48040d0100 0x7f48040b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f480967e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f48040d0f40 0x7f48040d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f480967e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48040d0a10 con 0x7f4804160920 2023-11-24T06:28:51.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f480967e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48040cf080 con 0x7f48040d0100 2023-11-24T06:28:51.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f480967e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48040c6be0 con 0x7f48040d0f40 2023-11-24T06:28:51.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f4803fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4804160920 0x7f48040cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33010/0 (socket says 172.21.15.16:33010) 2023-11-24T06:28:51.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f4803fff640 1 -- 172.21.15.16:0/327719783 learned_addr learned my addr 172.21.15.16:0/327719783 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:51.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f47fbfff640 1 --2- 172.21.15.16:0/327719783 >> v2:172.21.15.16:3300/0 conn(0x7f48040d0100 0x7f48040b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f4808e7d640 1 --2- 172.21.15.16:0/327719783 >> v2:172.21.15.173:3300/0 conn(0x7f48040d0f40 0x7f48040d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f48037fe640 1 -- 172.21.15.16:0/327719783 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123597005 0 0) 0x7f48040d0a10 con 0x7f4804160920 2023-11-24T06:28:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.043+0000 7f48037fe640 1 -- 172.21.15.16:0/327719783 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47f0003520 con 0x7f4804160920 2023-11-24T06:28:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.044+0000 7f47fbfff640 1 -- 172.21.15.16:0/327719783 >> v2:172.21.15.173:3300/0 conn(0x7f48040d0f40 msgr2=0x7f48040d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.044+0000 7f47fbfff640 1 --2- 172.21.15.16:0/327719783 >> v2:172.21.15.173:3300/0 conn(0x7f48040d0f40 0x7f48040d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.044+0000 7f47fbfff640 1 -- 172.21.15.16:0/327719783 >> v1:172.21.15.16:6789/0 conn(0x7f4804160920 legacy=0x7f48040cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.044+0000 7f47fbfff640 1 -- 172.21.15.16:0/327719783 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040ce7e0 con 0x7f48040d0100 2023-11-24T06:28:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.045+0000 7f47fbfff640 1 --2- 172.21.15.16:0/327719783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0100 0x7f48040b3f30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f48041620b0 tx=0x7f47ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3490ae1dbaec67e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.045+0000 7f48037fe640 1 -- 172.21.15.16:0/327719783 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec031830 con 0x7f48040d0100 2023-11-24T06:28:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.045+0000 7f48037fe640 1 -- 172.21.15.16:0/327719783 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec031990 con 0x7f48040d0100 2023-11-24T06:28:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.045+0000 7f48037fe640 1 -- 172.21.15.16:0/327719783 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec031d20 con 0x7f48040d0100 2023-11-24T06:28:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.046+0000 7f480967e640 1 -- 172.21.15.16:0/327719783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0100 msgr2=0x7f48040b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.046+0000 7f480967e640 1 --2- 172.21.15.16:0/327719783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0100 0x7f48040b3f30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f48041620b0 tx=0x7f47ec02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.046+0000 7f480967e640 1 -- 172.21.15.16:0/327719783 shutdown_connections 2023-11-24T06:28:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.046+0000 7f480967e640 1 --2- 172.21.15.16:0/327719783 >> v2:172.21.15.173:3300/0 conn(0x7f48040d0f40 0x7f48040d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.046+0000 7f480967e640 1 --2- 172.21.15.16:0/327719783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0100 0x7f48040b3f30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.046+0000 7f480967e640 1 -- 172.21.15.16:0/327719783 >> 172.21.15.16:0/327719783 conn(0x7f48040d3090 msgr2=0x7f48040d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.046+0000 7f480967e640 1 -- 172.21.15.16:0/327719783 shutdown_connections 2023-11-24T06:28:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.046+0000 7f480967e640 1 -- 172.21.15.16:0/327719783 wait complete. 2023-11-24T06:28:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.048+0000 7f480967e640 1 Processor -- start 2023-11-24T06:28:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.048+0000 7f480967e640 1 -- start start 2023-11-24T06:28:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f480967e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040d0100 0x7f48041566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f480967e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0f40 0x7f48040ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f480967e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4804160920 0x7f48040cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f480967e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4804156ea0 con 0x7f48040d0f40 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f480967e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48041570f0 con 0x7f48040d0100 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f480967e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48041573d0 con 0x7f4804160920 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f47fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0f40 0x7f48040ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f47fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0f40 0x7f48040ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51020/0 (socket says 172.21.15.16:51020) 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f47fbfff640 1 -- 172.21.15.16:0/956435032 learned_addr learned my addr 172.21.15.16:0/956435032 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f4808e7d640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4804160920 0x7f48040cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f47fbfff640 1 -- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4804160920 msgr2=0x7f48040cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f47fbfff640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4804160920 0x7f48040cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f4803fff640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040d0100 0x7f48041566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.049+0000 7f47fbfff640 1 -- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040d0100 msgr2=0x7f48041566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f47fbfff640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040d0100 0x7f48041566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f47fbfff640 1 -- 172.21.15.16:0/956435032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec009400 con 0x7f48040d0f40 2023-11-24T06:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f4808e7d640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4804160920 0x7f48040cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f4803fff640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040d0100 0x7f48041566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f47fbfff640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0f40 0x7f48040ca160 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009890 tx=0x7f47ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f48017fa640 1 -- 172.21.15.16:0/956435032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec03b970 con 0x7f48040d0f40 2023-11-24T06:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f48017fa640 1 -- 172.21.15.16:0/956435032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec03bad0 con 0x7f48040d0f40 2023-11-24T06:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f48017fa640 1 -- 172.21.15.16:0/956435032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec03be80 con 0x7f48040d0f40 2023-11-24T06:28:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48041576b0 con 0x7f48040d0f40 2023-11-24T06:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.050+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4804156c30 con 0x7f48040d0f40 2023-11-24T06:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.052+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48040d3b90 con 0x7f48040d0f40 2023-11-24T06:28:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.052+0000 7f48017fa640 1 -- 172.21.15.16:0/956435032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f47ec0453f0 con 0x7f48040d0f40 2023-11-24T06:28:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.053+0000 7f48017fa640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f47e406af20 0x7f47e406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.053+0000 7f48017fa640 1 -- 172.21.15.16:0/956435032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47ec0ab870 con 0x7f48040d0f40 2023-11-24T06:28:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.054+0000 7f4803fff640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f47e406af20 0x7f47e406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.055+0000 7f4803fff640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f47e406af20 0x7f47e406d3e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f47f4005880 tx=0x7f47f4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.058+0000 7f48017fa640 1 -- 172.21.15.16:0/956435032 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48040d3b90 con 0x7f48040d0f40 2023-11-24T06:28:51.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.259+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48040d3b90 con 0x7f48040d0f40 2023-11-24T06:28:51.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.261+0000 7f48017fa640 1 -- 172.21.15.16:0/956435032 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f48040d3b90 con 0x7f48040d0f40 2023-11-24T06:28:51.264 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:51.264 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:21.731503+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:28:18.648898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:28:19.200832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f47e406af20 msgr2=0x7f47e406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f47e406af20 0x7f47e406d3e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f47f4005880 tx=0x7f47f4009040 comp rx=0 tx=0).stop 2023-11-24T06:28:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0f40 msgr2=0x7f48040ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0f40 0x7f48040ca160 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009890 tx=0x7f47ec0046c0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 shutdown_connections 2023-11-24T06:28:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f47e406af20 0x7f47e406d3e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4804160920 0x7f48040cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48040d0f40 0x7f48040ca160 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 --2- 172.21.15.16:0/956435032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48040d0100 0x7f48041566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 >> 172.21.15.16:0/956435032 conn(0x7f48040d3090 msgr2=0x7f480406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.264+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 shutdown_connections 2023-11-24T06:28:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.265+0000 7f480967e640 1 -- 172.21.15.16:0/956435032 wait complete. 2023-11-24T06:28:51.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:28:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.509+0000 7fe3d6373640 1 Processor -- start 2023-11-24T06:28:51.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.509+0000 7fe3d6373640 1 -- start start 2023-11-24T06:28:51.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.511+0000 7fe3d6373640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3d0143c40 0x7fe3d0071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.511+0000 7fe3d6373640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d0070e50 con 0x7fe3d00717d0 2023-11-24T06:28:51.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.511+0000 7fe3d6373640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d0144020 con 0x7fe3d00ccc10 2023-11-24T06:28:51.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.511+0000 7fe3d6373640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d00723c0 con 0x7fe3d0143c40 2023-11-24T06:28:51.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.511+0000 7fe3d5371640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3d0143c40 0x7fe3d0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.511+0000 7fe3d4b70640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe3d00717d0 0x7fe3d00cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33016/0 (socket says 172.21.15.16:33016) 2023-11-24T06:28:51.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.511+0000 7fe3d4b70640 1 -- 172.21.15.16:0/653137686 learned_addr learned my addr 172.21.15.16:0/653137686 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:51.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.512+0000 7fe3d5371640 1 -- 172.21.15.16:0/653137686 >> v1:172.21.15.173:6789/0 conn(0x7fe3d00ccc10 legacy=0x7fe3d006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:51.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.512+0000 7fe3d5371640 1 -- 172.21.15.16:0/653137686 >> v1:172.21.15.16:6789/0 conn(0x7fe3d00717d0 legacy=0x7fe3d00cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.512+0000 7fe3d5371640 1 -- 172.21.15.16:0/653137686 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d0071e00 con 0x7fe3d0143c40 2023-11-24T06:28:51.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.513+0000 7fe3d5371640 1 --2- 172.21.15.16:0/653137686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0143c40 0x7fe3d0071290 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097a0 tx=0x7fe3cc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47ab315551f3f91 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.514+0000 7fe3c7fff640 1 -- 172.21.15.16:0/653137686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc02e070 con 0x7fe3d0143c40 2023-11-24T06:28:51.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.514+0000 7fe3c7fff640 1 -- 172.21.15.16:0/653137686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc0315a0 con 0x7fe3d0143c40 2023-11-24T06:28:51.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.514+0000 7fe3c7fff640 1 -- 172.21.15.16:0/653137686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc031930 con 0x7fe3d0143c40 2023-11-24T06:28:51.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.514+0000 7fe3d6373640 1 -- 172.21.15.16:0/653137686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0143c40 msgr2=0x7fe3d0071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.514+0000 7fe3d6373640 1 --2- 172.21.15.16:0/653137686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0143c40 0x7fe3d0071290 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097a0 tx=0x7fe3cc02b840 comp rx=0 tx=0).stop 2023-11-24T06:28:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.515+0000 7fe3d6373640 1 -- 172.21.15.16:0/653137686 shutdown_connections 2023-11-24T06:28:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.515+0000 7fe3d6373640 1 --2- 172.21.15.16:0/653137686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d0143c40 0x7fe3d0071290 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.515+0000 7fe3d6373640 1 -- 172.21.15.16:0/653137686 >> 172.21.15.16:0/653137686 conn(0x7fe3d006a800 msgr2=0x7fe3d006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.515+0000 7fe3d6373640 1 -- 172.21.15.16:0/653137686 shutdown_connections 2023-11-24T06:28:51.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.515+0000 7fe3d6373640 1 -- 172.21.15.16:0/653137686 wait complete. 2023-11-24T06:28:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.516+0000 7fe3d6373640 1 Processor -- start 2023-11-24T06:28:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.517+0000 7fe3d6373640 1 -- start start 2023-11-24T06:28:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d6373640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3d00717d0 0x7fe3d013e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d6373640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d00ccc10 0x7fe3d006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d6373640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3d0143c40 0x7fe3d006d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d6373640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00cb3e0 con 0x7fe3d00ccc10 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d6373640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d013ac10 con 0x7fe3d00717d0 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d6373640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d00ca7d0 con 0x7fe3d0143c40 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d4b70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d00ccc10 0x7fe3d006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d4b70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d00ccc10 0x7fe3d006d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51042/0 (socket says 172.21.15.16:51042) 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d4b70640 1 -- 172.21.15.16:0/2912180883 learned_addr learned my addr 172.21.15.16:0/2912180883 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d5b72640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3d0143c40 0x7fe3d006d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d4b70640 1 -- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3d0143c40 msgr2=0x7fe3d006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d5371640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3d00717d0 0x7fe3d013e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.518+0000 7fe3d4b70640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3d0143c40 0x7fe3d006d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3d4b70640 1 -- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3d00717d0 msgr2=0x7fe3d013e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3d4b70640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3d00717d0 0x7fe3d013e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3d4b70640 1 -- 172.21.15.16:0/2912180883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc009400 con 0x7fe3d00ccc10 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3d5b72640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3d0143c40 0x7fe3d006d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3d5371640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3d00717d0 0x7fe3d013e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3d4b70640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d00ccc10 0x7fe3d006d160 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c480 tx=0x7fe3c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3c5ffb640 1 -- 172.21.15.16:0/2912180883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0005ce0 con 0x7fe3d00ccc10 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3c5ffb640 1 -- 172.21.15.16:0/2912180883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c0005e40 con 0x7fe3d00ccc10 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3c5ffb640 1 -- 172.21.15.16:0/2912180883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00072f0 con 0x7fe3d00ccc10 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d00caab0 con 0x7fe3d00ccc10 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.519+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d00cb070 con 0x7fe3d00ccc10 2023-11-24T06:28:51.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.521+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d0067450 con 0x7fe3d00ccc10 2023-11-24T06:28:51.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.521+0000 7fe3c5ffb640 1 -- 172.21.15.16:0/2912180883 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe3c0007450 con 0x7fe3d00ccc10 2023-11-24T06:28:51.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.522+0000 7fe3c5ffb640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe3b406af00 0x7fe3b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:51.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.523+0000 7fe3c5ffb640 1 -- 172.21.15.16:0/2912180883 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3c00889d0 con 0x7fe3d00ccc10 2023-11-24T06:28:51.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.523+0000 7fe3d5371640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe3b406af00 0x7fe3b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:51.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.524+0000 7fe3d5371640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe3b406af00 0x7fe3b406d3c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc036000 tx=0x7fe3cc03d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:51.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.527+0000 7fe3c5ffb640 1 -- 172.21.15.16:0/2912180883 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3c0053bd0 con 0x7fe3d00ccc10 2023-11-24T06:28:51.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.762+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe3d00b0820 con 0x7fe3d00ccc10 2023-11-24T06:28:51.765 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-24T06:28:51.763+0000 7f082ae01640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:28:51.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.766+0000 7fe3c5ffb640 1 -- 172.21.15.16:0/2912180883 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe3c0017020 con 0x7fe3d00ccc10 2023-11-24T06:28:51.769 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:28:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.770+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe3b406af00 msgr2=0x7fe3b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.770+0000 7fe3d6373640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe3b406af00 0x7fe3b406d3c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc036000 tx=0x7fe3cc03d040 comp rx=0 tx=0).stop 2023-11-24T06:28:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.770+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d00ccc10 msgr2=0x7fe3d006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.770+0000 7fe3d6373640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d00ccc10 0x7fe3d006d160 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c480 tx=0x7fe3c000c940 comp rx=0 tx=0).stop 2023-11-24T06:28:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.771+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 shutdown_connections 2023-11-24T06:28:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.771+0000 7fe3d6373640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe3b406af00 0x7fe3b406d3c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.771+0000 7fe3d6373640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3d0143c40 0x7fe3d006d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.771+0000 7fe3d6373640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3d00ccc10 0x7fe3d006d160 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.771+0000 7fe3d6373640 1 --2- 172.21.15.16:0/2912180883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3d00717d0 0x7fe3d013e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.771+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 >> 172.21.15.16:0/2912180883 conn(0x7fe3d006a800 msgr2=0x7fe3d00af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.771+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 shutdown_connections 2023-11-24T06:28:51.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:51.771+0000 7fe3d6373640 1 -- 172.21.15.16:0/2912180883 wait complete. 2023-11-24T06:28:51.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:28:51.792 INFO:tasks.ceph.mds.a.smithi016.stderr: -3436> 2023-11-24T06:28:22.244+0000 7fc38754f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:28:51.872 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-24T06:28:52.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.001+0000 7fc62a6ca640 1 Processor -- start 2023-11-24T06:28:52.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.002+0000 7fc62a6ca640 1 -- start start 2023-11-24T06:28:52.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.003+0000 7fc62a6ca640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6240b2e00 0x7fc624070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.003+0000 7fc62a6ca640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6240706a0 con 0x7fc62406cb10 2023-11-24T06:28:52.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.003+0000 7fc62a6ca640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6240708d0 con 0x7fc62406c730 2023-11-24T06:28:52.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.003+0000 7fc62a6ca640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6240b3290 con 0x7fc6240b2e00 2023-11-24T06:28:52.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.004+0000 7fc628ec7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6240b2e00 0x7fc624070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.004+0000 7fc628ec7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6240b2e00 0x7fc624070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51082/0 (socket says 172.21.15.16:51082) 2023-11-24T06:28:52.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.004+0000 7fc628ec7640 1 -- 172.21.15.16:0/2552116704 learned_addr learned my addr 172.21.15.16:0/2552116704 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:52.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.004+0000 7fc628ec7640 1 -- 172.21.15.16:0/2552116704 >> v1:172.21.15.173:6789/0 conn(0x7fc62406c730 legacy=0x7fc624071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:52.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.004+0000 7fc628ec7640 1 -- 172.21.15.16:0/2552116704 >> v1:172.21.15.16:6789/0 conn(0x7fc62406cb10 legacy=0x7fc62406cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.004+0000 7fc628ec7640 1 -- 172.21.15.16:0/2552116704 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc624070bb0 con 0x7fc6240b2e00 2023-11-24T06:28:52.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.005+0000 7fc628ec7640 1 --2- 172.21.15.16:0/2552116704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6240b2e00 0x7fc624070160 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc618009870 tx=0x7fc61802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b85ff47865ec5a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:52.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.006+0000 7fc617fff640 1 -- 172.21.15.16:0/2552116704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618030850 con 0x7fc6240b2e00 2023-11-24T06:28:52.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.006+0000 7fc617fff640 1 -- 172.21.15.16:0/2552116704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6180309b0 con 0x7fc6240b2e00 2023-11-24T06:28:52.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.006+0000 7fc617fff640 1 -- 172.21.15.16:0/2552116704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618030d40 con 0x7fc6240b2e00 2023-11-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.006+0000 7fc62a6ca640 1 -- 172.21.15.16:0/2552116704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6240b2e00 msgr2=0x7fc624070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.006+0000 7fc62a6ca640 1 --2- 172.21.15.16:0/2552116704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6240b2e00 0x7fc624070160 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc618009870 tx=0x7fc61802b990 comp rx=0 tx=0).stop 2023-11-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.006+0000 7fc62a6ca640 1 -- 172.21.15.16:0/2552116704 shutdown_connections 2023-11-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.006+0000 7fc62a6ca640 1 --2- 172.21.15.16:0/2552116704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6240b2e00 0x7fc624070160 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.006+0000 7fc62a6ca640 1 -- 172.21.15.16:0/2552116704 >> 172.21.15.16:0/2552116704 conn(0x7fc62406a7d0 msgr2=0x7fc62406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.007+0000 7fc62a6ca640 1 -- 172.21.15.16:0/2552116704 shutdown_connections 2023-11-24T06:28:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.007+0000 7fc62a6ca640 1 -- 172.21.15.16:0/2552116704 wait complete. 2023-11-24T06:28:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.008+0000 7fc62a6ca640 1 Processor -- start 2023-11-24T06:28:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.009+0000 7fc62a6ca640 1 -- start start 2023-11-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.009+0000 7fc62a6ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc62406c730 0x7fc624137c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.009+0000 7fc62a6ca640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc62406cb10 0x7fc62415f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.009+0000 7fc62a6ca640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6240b2e00 0x7fc624147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc62a6ca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc624138180 con 0x7fc62406c730 2023-11-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc62a6ca640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241f8590 con 0x7fc62406cb10 2023-11-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc62a6ca640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241f8870 con 0x7fc6240b2e00 2023-11-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc6296c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc62406c730 0x7fc624137c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc6296c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc62406c730 0x7fc624137c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51086/0 (socket says 172.21.15.16:51086) 2023-11-24T06:28:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc6296c8640 1 -- 172.21.15.16:0/1823628140 learned_addr learned my addr 172.21.15.16:0/1823628140 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc6296c8640 1 -- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6240b2e00 msgr2=0x7fc624147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc6296c8640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6240b2e00 0x7fc624147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc6296c8640 1 -- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc62406cb10 msgr2=0x7fc62415f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc6296c8640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc62406cb10 0x7fc62415f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.010+0000 7fc6296c8640 1 -- 172.21.15.16:0/1823628140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180094d0 con 0x7fc62406c730 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.011+0000 7fc6296c8640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc62406c730 0x7fc624137c40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c430 tx=0x7fc62000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.011+0000 7fc615ffb640 1 -- 172.21.15.16:0/1823628140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620011790 con 0x7fc62406c730 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.011+0000 7fc615ffb640 1 -- 172.21.15.16:0/1823628140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6200118f0 con 0x7fc62406c730 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.011+0000 7fc615ffb640 1 -- 172.21.15.16:0/1823628140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620011ce0 con 0x7fc62406c730 2023-11-24T06:28:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.011+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6241f8b50 con 0x7fc62406c730 2023-11-24T06:28:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.011+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6241f9110 con 0x7fc62406c730 2023-11-24T06:28:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.012+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6241f9300 con 0x7fc62406c730 2023-11-24T06:28:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.013+0000 7fc615ffb640 1 -- 172.21.15.16:0/1823628140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc62001c460 con 0x7fc62406c730 2023-11-24T06:28:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.014+0000 7fc615ffb640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc60806aed0 0x7fc60806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.015+0000 7fc615ffb640 1 -- 172.21.15.16:0/1823628140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc6200896a0 con 0x7fc62406c730 2023-11-24T06:28:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.015+0000 7fc628ec7640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc60806aed0 0x7fc60806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.016+0000 7fc628ec7640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc60806aed0 0x7fc60806d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc618036000 tx=0x7fc618030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.019+0000 7fc615ffb640 1 -- 172.21.15.16:0/1823628140 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc620054700 con 0x7fc62406c730 2023-11-24T06:28:52.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.251+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc624067450 con 0x7fc62406c730 2023-11-24T06:28:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.770+0000 7fc615ffb640 1 -- 172.21.15.16:0/1823628140 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc620057d20 con 0x7fc62406c730 2023-11-24T06:28:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc60806aed0 msgr2=0x7fc60806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc60806aed0 0x7fc60806d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc618036000 tx=0x7fc618030000 comp rx=0 tx=0).stop 2023-11-24T06:28:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc62406c730 msgr2=0x7fc624137c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc62406c730 0x7fc624137c40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc62000c430 tx=0x7fc62000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 shutdown_connections 2023-11-24T06:28:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc60806aed0 0x7fc60806d390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6240b2e00 0x7fc624147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc62406cb10 0x7fc62415f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 --2- 172.21.15.16:0/1823628140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc62406c730 0x7fc624137c40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 >> 172.21.15.16:0/1823628140 conn(0x7fc62406a7d0 msgr2=0x7fc6240cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 shutdown_connections 2023-11-24T06:28:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:52.774+0000 7fc62a6ca640 1 -- 172.21.15.16:0/1823628140 wait complete. 2023-11-24T06:28:52.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:28:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.012+0000 7f5d00d0f640 1 Processor -- start 2023-11-24T06:28:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.012+0000 7f5d00d0f640 1 -- start start 2023-11-24T06:28:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.013+0000 7f5d00d0f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5cfc15d590 0x7f5cfc0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.013+0000 7f5d00d0f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0d00b0 0x7f5cfc0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.014+0000 7f5d00d0f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cfc0ce6f0 0x7f5cfc0d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.014+0000 7f5d00d0f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc0d0530 con 0x7f5cfc0ce6f0 2023-11-24T06:28:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.014+0000 7f5d00d0f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc0b6750 con 0x7f5cfc15d590 2023-11-24T06:28:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.014+0000 7f5d00d0f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0cf1f0 con 0x7f5cfc0d00b0 2023-11-24T06:28:53.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.014+0000 7f5cfad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cfc0ce6f0 0x7f5cfc0d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.014+0000 7f5cfad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cfc0ce6f0 0x7f5cfc0d0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60388/0 (socket says 172.21.15.16:60388) 2023-11-24T06:28:53.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.014+0000 7f5cfad9d640 1 -- 172.21.15.16:0/260313144 learned_addr learned my addr 172.21.15.16:0/260313144 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:53.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.014+0000 7f5cf3fff640 1 --2- 172.21.15.16:0/260313144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0d00b0 0x7f5cfc0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.014+0000 7f5cfa59c640 1 --2- 172.21.15.16:0/260313144 >> v2:172.21.15.173:3300/0 conn(0x7f5cfc15d590 0x7f5cfc0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.015+0000 7f5cfad9d640 1 -- 172.21.15.16:0/260313144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0d00b0 msgr2=0x7f5cfc0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.015+0000 7f5cfad9d640 1 --2- 172.21.15.16:0/260313144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0d00b0 0x7f5cfc0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.015+0000 7f5cfad9d640 1 -- 172.21.15.16:0/260313144 >> v2:172.21.15.173:3300/0 conn(0x7f5cfc15d590 msgr2=0x7f5cfc0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.015+0000 7f5cfad9d640 1 --2- 172.21.15.16:0/260313144 >> v2:172.21.15.173:3300/0 conn(0x7f5cfc15d590 0x7f5cfc0cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.015+0000 7f5cfad9d640 1 -- 172.21.15.16:0/260313144 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0cef10 con 0x7f5cfc0ce6f0 2023-11-24T06:28:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.016+0000 7f5cfad9d640 1 --2- 172.21.15.16:0/260313144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc0ce6f0 0x7f5cfc0d0de0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc06d4d0 tx=0x7f5cf402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0f3df3951f5585 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.016+0000 7f5cf9d9b640 1 -- 172.21.15.16:0/260313144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf402ca70 con 0x7f5cfc0ce6f0 2023-11-24T06:28:53.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.016+0000 7f5cf9d9b640 1 -- 172.21.15.16:0/260313144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf402cbd0 con 0x7f5cfc0ce6f0 2023-11-24T06:28:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.016+0000 7f5cf9d9b640 1 -- 172.21.15.16:0/260313144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40343f0 con 0x7f5cfc0ce6f0 2023-11-24T06:28:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.017+0000 7f5d00d0f640 1 -- 172.21.15.16:0/260313144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc0ce6f0 msgr2=0x7f5cfc0d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.017+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/260313144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc0ce6f0 0x7f5cfc0d0de0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc06d4d0 tx=0x7f5cf402bed0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.017+0000 7f5d00d0f640 1 -- 172.21.15.16:0/260313144 shutdown_connections 2023-11-24T06:28:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.017+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/260313144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc0ce6f0 0x7f5cfc0d0de0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.017+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/260313144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0d00b0 0x7f5cfc0ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.017+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/260313144 >> v2:172.21.15.173:3300/0 conn(0x7f5cfc15d590 0x7f5cfc0cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.017+0000 7f5d00d0f640 1 -- 172.21.15.16:0/260313144 >> 172.21.15.16:0/260313144 conn(0x7f5cfc09c540 msgr2=0x7f5cfc0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.017+0000 7f5d00d0f640 1 -- 172.21.15.16:0/260313144 shutdown_connections 2023-11-24T06:28:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.017+0000 7f5d00d0f640 1 -- 172.21.15.16:0/260313144 wait complete. 2023-11-24T06:28:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.019+0000 7f5d00d0f640 1 Processor -- start 2023-11-24T06:28:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.019+0000 7f5d00d0f640 1 -- start start 2023-11-24T06:28:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.020+0000 7f5d00d0f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0ce6f0 0x7f5cfc0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.020+0000 7f5d00d0f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cfc0d00b0 0x7f5cfc0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.020+0000 7f5d00d0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc15d590 0x7f5cfc0c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.020+0000 7f5d00d0f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0c7ea0 con 0x7f5cfc15d590 2023-11-24T06:28:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.020+0000 7f5d00d0f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc09e3d0 con 0x7f5cfc0d00b0 2023-11-24T06:28:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.020+0000 7f5d00d0f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc0c7220 con 0x7f5cfc0ce6f0 2023-11-24T06:28:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc15d590 0x7f5cfc0c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc15d590 0x7f5cfc0c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60404/0 (socket says 172.21.15.16:60404) 2023-11-24T06:28:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfad9d640 1 -- 172.21.15.16:0/2718060326 learned_addr learned my addr 172.21.15.16:0/2718060326 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cf3fff640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cfc0d00b0 0x7f5cfc0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfa59c640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0ce6f0 0x7f5cfc0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfad9d640 1 -- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0ce6f0 msgr2=0x7f5cfc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfad9d640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0ce6f0 0x7f5cfc0c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfad9d640 1 -- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cfc0d00b0 msgr2=0x7f5cfc0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfad9d640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cfc0d00b0 0x7f5cfc0cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfad9d640 1 -- 172.21.15.16:0/2718060326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf4009b30 con 0x7f5cfc15d590 2023-11-24T06:28:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.021+0000 7f5cfa59c640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0ce6f0 0x7f5cfc0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.022+0000 7f5cfad9d640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc15d590 0x7f5cfc0c82c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4036040 tx=0x7f5cf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.022+0000 7f5cf2ffd640 1 -- 172.21.15.16:0/2718060326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4009510 con 0x7f5cfc15d590 2023-11-24T06:28:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.022+0000 7f5cf2ffd640 1 -- 172.21.15.16:0/2718060326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf403e450 con 0x7f5cfc15d590 2023-11-24T06:28:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.022+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc0c7500 con 0x7f5cfc15d590 2023-11-24T06:28:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.022+0000 7f5cf2ffd640 1 -- 172.21.15.16:0/2718060326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf403e800 con 0x7f5cfc15d590 2023-11-24T06:28:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.022+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc0c7ac0 con 0x7f5cfc15d590 2023-11-24T06:28:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.023+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc0c7fe0 con 0x7f5cfc15d590 2023-11-24T06:28:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.024+0000 7f5cf2ffd640 1 -- 172.21.15.16:0/2718060326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5cf403e960 con 0x7f5cfc15d590 2023-11-24T06:28:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.026+0000 7f5cf2ffd640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cdc06aed0 0x7f5cdc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.026+0000 7f5cf2ffd640 1 -- 172.21.15.16:0/2718060326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5cf40ac530 con 0x7f5cfc15d590 2023-11-24T06:28:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.026+0000 7f5cfa59c640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cdc06aed0 0x7f5cdc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.027+0000 7f5cfa59c640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cdc06aed0 0x7f5cdc06d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00b970 tx=0x7f5cec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.030+0000 7f5cf2ffd640 1 -- 172.21.15.16:0/2718060326 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5cfc0c7fe0 con 0x7f5cfc15d590 2023-11-24T06:28:53.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:53.202 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:28:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:53.220+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5cfc0c7fe0 con 0x7f5cfc15d590 2023-11-24T06:28:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.178+0000 7f5cf2ffd640 1 -- 172.21.15.16:0/2718060326 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5cfc0c7fe0 con 0x7f5cfc15d590 2023-11-24T06:28:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:28:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cdc06aed0 msgr2=0x7f5cdc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cdc06aed0 0x7f5cdc06d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00b970 tx=0x7f5cec008040 comp rx=0 tx=0).stop 2023-11-24T06:28:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc15d590 msgr2=0x7f5cfc0c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc15d590 0x7f5cfc0c82c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4036040 tx=0x7f5cf4004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 shutdown_connections 2023-11-24T06:28:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cdc06aed0 0x7f5cdc06d390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc15d590 0x7f5cfc0c82c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cfc0d00b0 0x7f5cfc0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 --2- 172.21.15.16:0/2718060326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cfc0ce6f0 0x7f5cfc0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.182+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 >> 172.21.15.16:0/2718060326 conn(0x7f5cfc09c540 msgr2=0x7f5cfc15fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:54.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.183+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 shutdown_connections 2023-11-24T06:28:54.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.184+0000 7f5d00d0f640 1 -- 172.21.15.16:0/2718060326 wait complete. 2023-11-24T06:28:54.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:28:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.419+0000 7f35e9d89640 1 Processor -- start 2023-11-24T06:28:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.420+0000 7f35e9d89640 1 -- start start 2023-11-24T06:28:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.421+0000 7f35e9d89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35e40cf9a0 0x7f35e415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.421+0000 7f35e9d89640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f35e415e1e0 0x7f35e40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e9d89640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e40b1ee0 con 0x7f35e4143ae0 2023-11-24T06:28:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e9d89640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e40b2780 con 0x7f35e40cf9a0 2023-11-24T06:28:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e9d89640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e415e990 con 0x7f35e415e1e0 2023-11-24T06:28:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35e40cf9a0 0x7f35e415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35e40cf9a0 0x7f35e415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60414/0 (socket says 172.21.15.16:60414) 2023-11-24T06:28:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e2ffd640 1 -- 172.21.15.16:0/167950906 learned_addr learned my addr 172.21.15.16:0/167950906 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e3fff640 1 --2- 172.21.15.16:0/167950906 >> v2:172.21.15.173:3300/0 conn(0x7f35e415e1e0 0x7f35e40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e2ffd640 1 -- 172.21.15.16:0/167950906 >> v2:172.21.15.173:3300/0 conn(0x7f35e415e1e0 msgr2=0x7f35e40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e2ffd640 1 --2- 172.21.15.16:0/167950906 >> v2:172.21.15.173:3300/0 conn(0x7f35e415e1e0 0x7f35e40702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e2ffd640 1 -- 172.21.15.16:0/167950906 >> v1:172.21.15.173:6789/0 conn(0x7f35e4143ae0 legacy=0x7f35e40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.422+0000 7f35e2ffd640 1 -- 172.21.15.16:0/167950906 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40b2130 con 0x7f35e40cf9a0 2023-11-24T06:28:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.424+0000 7f35e2ffd640 1 --2- 172.21.15.16:0/167950906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 0x7f35e415dca0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f35e40ae9d0 tx=0x7f35d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bacbe8c65725ebf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.424+0000 7f35e27fc640 1 -- 172.21.15.16:0/167950906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8031870 con 0x7f35e40cf9a0 2023-11-24T06:28:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.424+0000 7f35e27fc640 1 -- 172.21.15.16:0/167950906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d80319d0 con 0x7f35e40cf9a0 2023-11-24T06:28:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.424+0000 7f35e27fc640 1 -- 172.21.15.16:0/167950906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8031d60 con 0x7f35e40cf9a0 2023-11-24T06:28:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.425+0000 7f35e9d89640 1 -- 172.21.15.16:0/167950906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 msgr2=0x7f35e415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.425+0000 7f35e9d89640 1 --2- 172.21.15.16:0/167950906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 0x7f35e415dca0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f35e40ae9d0 tx=0x7f35d802b910 comp rx=0 tx=0).stop 2023-11-24T06:28:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.425+0000 7f35e9d89640 1 -- 172.21.15.16:0/167950906 shutdown_connections 2023-11-24T06:28:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.425+0000 7f35e9d89640 1 --2- 172.21.15.16:0/167950906 >> v2:172.21.15.173:3300/0 conn(0x7f35e415e1e0 0x7f35e40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.425+0000 7f35e9d89640 1 --2- 172.21.15.16:0/167950906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 0x7f35e415dca0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.425+0000 7f35e9d89640 1 -- 172.21.15.16:0/167950906 >> 172.21.15.16:0/167950906 conn(0x7f35e406a800 msgr2=0x7f35e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.425+0000 7f35e9d89640 1 -- 172.21.15.16:0/167950906 shutdown_connections 2023-11-24T06:28:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.425+0000 7f35e9d89640 1 -- 172.21.15.16:0/167950906 wait complete. 2023-11-24T06:28:54.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.428+0000 7f35e9d89640 1 Processor -- start 2023-11-24T06:28:54.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.428+0000 7f35e9d89640 1 -- start start 2023-11-24T06:28:54.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.429+0000 7f35e9d89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 0x7f35e4159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.429+0000 7f35e9d89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4143ae0 0x7f35e415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.429+0000 7f35e9d89640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35e415e1e0 0x7f35e415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.429+0000 7f35e9d89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40c8d40 con 0x7f35e40cf9a0 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.429+0000 7f35e9d89640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40c3ff0 con 0x7f35e415e1e0 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.429+0000 7f35e9d89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40c42d0 con 0x7f35e4143ae0 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.430+0000 7f35e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 0x7f35e4159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.430+0000 7f35e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 0x7f35e4159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60416/0 (socket says 172.21.15.16:60416) 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.430+0000 7f35e37fe640 1 -- 172.21.15.16:0/4144358038 learned_addr learned my addr 172.21.15.16:0/4144358038 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.430+0000 7f35e37fe640 1 -- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4143ae0 msgr2=0x7f35e415b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.430+0000 7f35e37fe640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4143ae0 0x7f35e415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.430+0000 7f35e37fe640 1 -- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35e415e1e0 msgr2=0x7f35e415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.430+0000 7f35e37fe640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35e415e1e0 0x7f35e415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.430+0000 7f35e37fe640 1 -- 172.21.15.16:0/4144358038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d80094d0 con 0x7f35e40cf9a0 2023-11-24T06:28:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.430+0000 7f35e37fe640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 0x7f35e4159bb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c410 tx=0x7f35d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.431+0000 7f35cbfff640 1 -- 172.21.15.16:0/4144358038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d40035f0 con 0x7f35e40cf9a0 2023-11-24T06:28:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.431+0000 7f35cbfff640 1 -- 172.21.15.16:0/4144358038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d4003750 con 0x7f35e40cf9a0 2023-11-24T06:28:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.431+0000 7f35cbfff640 1 -- 172.21.15.16:0/4144358038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4003b40 con 0x7f35e40cf9a0 2023-11-24T06:28:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.431+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e40c45b0 con 0x7f35e40cf9a0 2023-11-24T06:28:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.431+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e40c4b70 con 0x7f35e40cf9a0 2023-11-24T06:28:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.432+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e406d810 con 0x7f35e40cf9a0 2023-11-24T06:28:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.433+0000 7f35cbfff640 1 -- 172.21.15.16:0/4144358038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f35d4003ca0 con 0x7f35e40cf9a0 2023-11-24T06:28:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.434+0000 7f35cbfff640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35d006aed0 0x7f35d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.434+0000 7f35cbfff640 1 -- 172.21.15.16:0/4144358038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f35d4088870 con 0x7f35e40cf9a0 2023-11-24T06:28:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.435+0000 7f35e2ffd640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35d006aed0 0x7f35d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.436+0000 7f35e2ffd640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35d006aed0 0x7f35d006d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f35d8036000 tx=0x7f35d8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.438+0000 7f35cbfff640 1 -- 172.21.15.16:0/4144358038 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35d4053a20 con 0x7f35e40cf9a0 2023-11-24T06:28:54.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:54.640+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f35e40c4d60 con 0x7f35e40cf9a0 2023-11-24T06:28:55.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.188+0000 7f35cbfff640 1 -- 172.21.15.16:0/4144358038 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f35d4057040 con 0x7f35e40cf9a0 2023-11-24T06:28:55.195 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-24T06:28:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.195+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35d006aed0 msgr2=0x7f35d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.195+0000 7f35e9d89640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35d006aed0 0x7f35d006d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f35d8036000 tx=0x7f35d8030000 comp rx=0 tx=0).stop 2023-11-24T06:28:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.195+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 msgr2=0x7f35e4159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.195+0000 7f35e9d89640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 0x7f35e4159bb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c410 tx=0x7f35d400c8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.196+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 shutdown_connections 2023-11-24T06:28:55.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.196+0000 7f35e9d89640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35d006aed0 0x7f35d006d390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.196+0000 7f35e9d89640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35e415e1e0 0x7f35e415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.196+0000 7f35e9d89640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4143ae0 0x7f35e415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.196+0000 7f35e9d89640 1 --2- 172.21.15.16:0/4144358038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e40cf9a0 0x7f35e4159bb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.196+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 >> 172.21.15.16:0/4144358038 conn(0x7f35e406a800 msgr2=0x7f35e406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.196+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 shutdown_connections 2023-11-24T06:28:55.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.196+0000 7f35e9d89640 1 -- 172.21.15.16:0/4144358038 wait complete. 2023-11-24T06:28:55.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:28:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.422+0000 7f4e5759e640 1 Processor -- start 2023-11-24T06:28:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.423+0000 7f4e5759e640 1 -- start start 2023-11-24T06:28:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.424+0000 7f4e5759e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4e5815d1d0 0x7f4e580d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.424+0000 7f4e5759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e580b11d0 0x7f4e580d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.424+0000 7f4e5759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e580d0090 con 0x7f4e580b1f70 2023-11-24T06:28:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.424+0000 7f4e5759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580d02e0 con 0x7f4e580b11d0 2023-11-24T06:28:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.424+0000 7f4e5759e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580d05c0 con 0x7f4e5815d1d0 2023-11-24T06:28:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.424+0000 7f4e56d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e580b11d0 0x7f4e580d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.424+0000 7f4e55d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4e580b1f70 0x7f4e580cdb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58452/0 (socket says 172.21.15.16:58452) 2023-11-24T06:28:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.424+0000 7f4e55d9b640 1 -- 172.21.15.16:0/1841621983 learned_addr learned my addr 172.21.15.16:0/1841621983 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.425+0000 7f4e5659c640 1 --2- 172.21.15.16:0/1841621983 >> v2:172.21.15.173:3300/0 conn(0x7f4e5815d1d0 0x7f4e580d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.425+0000 7f4e56d9d640 1 -- 172.21.15.16:0/1841621983 >> v2:172.21.15.173:3300/0 conn(0x7f4e5815d1d0 msgr2=0x7f4e580d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:55.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.425+0000 7f4e56d9d640 1 --2- 172.21.15.16:0/1841621983 >> v2:172.21.15.173:3300/0 conn(0x7f4e5815d1d0 0x7f4e580d1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.425+0000 7f4e56d9d640 1 -- 172.21.15.16:0/1841621983 >> v1:172.21.15.16:6789/0 conn(0x7f4e580b1f70 legacy=0x7f4e580cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.425+0000 7f4e56d9d640 1 -- 172.21.15.16:0/1841621983 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580d08a0 con 0x7f4e580b11d0 2023-11-24T06:28:55.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.425+0000 7f4e5659c640 1 --2- 172.21.15.16:0/1841621983 >> v2:172.21.15.173:3300/0 conn(0x7f4e5815d1d0 0x7f4e580d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:55.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.426+0000 7f4e56d9d640 1 --2- 172.21.15.16:0/1841621983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b11d0 0x7f4e580d1b70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4e5808b460 tx=0x7f4e5002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5177c997d8cd7bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.426+0000 7f4e5559a640 1 -- 172.21.15.16:0/1841621983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50031830 con 0x7f4e580b11d0 2023-11-24T06:28:55.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.426+0000 7f4e5559a640 1 -- 172.21.15.16:0/1841621983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e50031990 con 0x7f4e580b11d0 2023-11-24T06:28:55.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.426+0000 7f4e5559a640 1 -- 172.21.15.16:0/1841621983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50031d20 con 0x7f4e580b11d0 2023-11-24T06:28:55.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.427+0000 7f4e5759e640 1 -- 172.21.15.16:0/1841621983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b11d0 msgr2=0x7f4e580d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.427+0000 7f4e5759e640 1 --2- 172.21.15.16:0/1841621983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b11d0 0x7f4e580d1b70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4e5808b460 tx=0x7f4e5002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.427+0000 7f4e5759e640 1 -- 172.21.15.16:0/1841621983 shutdown_connections 2023-11-24T06:28:55.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.427+0000 7f4e5759e640 1 --2- 172.21.15.16:0/1841621983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b11d0 0x7f4e580d1b70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.427+0000 7f4e5759e640 1 --2- 172.21.15.16:0/1841621983 >> v2:172.21.15.173:3300/0 conn(0x7f4e5815d1d0 0x7f4e580d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.427+0000 7f4e5759e640 1 -- 172.21.15.16:0/1841621983 >> 172.21.15.16:0/1841621983 conn(0x7f4e580d3050 msgr2=0x7f4e580d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:55.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.427+0000 7f4e5759e640 1 -- 172.21.15.16:0/1841621983 shutdown_connections 2023-11-24T06:28:55.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.427+0000 7f4e5759e640 1 -- 172.21.15.16:0/1841621983 wait complete. 2023-11-24T06:28:55.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.429+0000 7f4e5759e640 1 Processor -- start 2023-11-24T06:28:55.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.429+0000 7f4e5759e640 1 -- start start 2023-11-24T06:28:55.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.430+0000 7f4e5759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e580b11d0 0x7f4e581601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.430+0000 7f4e5759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1f70 0x7f4e580cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.430+0000 7f4e5759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5815d1d0 0x7f4e581f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.430+0000 7f4e5759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580ccfc0 con 0x7f4e580b1f70 2023-11-24T06:28:55.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.430+0000 7f4e5759e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580cc690 con 0x7f4e5815d1d0 2023-11-24T06:28:55.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.430+0000 7f4e5759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580cc970 con 0x7f4e580b11d0 2023-11-24T06:28:55.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.430+0000 7f4e55d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1f70 0x7f4e580cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.430+0000 7f4e55d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1f70 0x7f4e580cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60428/0 (socket says 172.21.15.16:60428) 2023-11-24T06:28:55.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.430+0000 7f4e55d9b640 1 -- 172.21.15.16:0/3300540109 learned_addr learned my addr 172.21.15.16:0/3300540109 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:55.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e55d9b640 1 -- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e580b11d0 msgr2=0x7f4e581601f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e55d9b640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e580b11d0 0x7f4e581601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e55d9b640 1 -- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5815d1d0 msgr2=0x7f4e581f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e55d9b640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5815d1d0 0x7f4e581f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e55d9b640 1 -- 172.21.15.16:0/3300540109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e50009400 con 0x7f4e580b1f70 2023-11-24T06:28:55.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e55d9b640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1f70 0x7f4e580cbc30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000c3c0 tx=0x7f4e4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e4effd640 1 -- 172.21.15.16:0/3300540109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40005ce0 con 0x7f4e580b1f70 2023-11-24T06:28:55.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e4effd640 1 -- 172.21.15.16:0/3300540109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40005e40 con 0x7f4e580b1f70 2023-11-24T06:28:55.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e4effd640 1 -- 172.21.15.16:0/3300540109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e400072f0 con 0x7f4e580b1f70 2023-11-24T06:28:55.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e580ccc50 con 0x7f4e580b1f70 2023-11-24T06:28:55.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.431+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e580cc170 con 0x7f4e580b1f70 2023-11-24T06:28:55.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.432+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e580cd170 con 0x7f4e580b1f70 2023-11-24T06:28:55.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.433+0000 7f4e4effd640 1 -- 172.21.15.16:0/3300540109 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4e400074d0 con 0x7f4e580b1f70 2023-11-24T06:28:55.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.434+0000 7f4e4effd640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4e3806aed0 0x7f4e3806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.434+0000 7f4e4effd640 1 -- 172.21.15.16:0/3300540109 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4e400889b0 con 0x7f4e580b1f70 2023-11-24T06:28:55.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.435+0000 7f4e5659c640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4e3806aed0 0x7f4e3806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.436+0000 7f4e5659c640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4e3806aed0 0x7f4e3806d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e48008ea0 tx=0x7f4e480083d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.439+0000 7f4e4effd640 1 -- 172.21.15.16:0/3300540109 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e580cd170 con 0x7f4e580b1f70 2023-11-24T06:28:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.635+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4e580cc3f0 con 0x7f4e580b1f70 2023-11-24T06:28:55.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.642+0000 7f4e4effd640 1 -- 172.21.15.16:0/3300540109 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4e400170a0 con 0x7f4e580b1f70 2023-11-24T06:28:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4e3806aed0 msgr2=0x7f4e3806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4e3806aed0 0x7f4e3806d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4e48008ea0 tx=0x7f4e480083d0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1f70 msgr2=0x7f4e580cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1f70 0x7f4e580cbc30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000c3c0 tx=0x7f4e4000c880 comp rx=0 tx=0).stop 2023-11-24T06:28:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 shutdown_connections 2023-11-24T06:28:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4e3806aed0 0x7f4e3806d390 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5815d1d0 0x7f4e581f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e580b1f70 0x7f4e580cbc30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 --2- 172.21.15.16:0/3300540109 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e580b11d0 0x7f4e581601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.645+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 >> 172.21.15.16:0/3300540109 conn(0x7f4e580d3050 msgr2=0x7f4e580d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.646+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 shutdown_connections 2023-11-24T06:28:55.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.646+0000 7f4e5759e640 1 -- 172.21.15.16:0/3300540109 wait complete. 2023-11-24T06:28:55.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T06:28:55.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.887+0000 7f11f9b7a640 1 Processor -- start 2023-11-24T06:28:55.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.887+0000 7f11f9b7a640 1 -- start start 2023-11-24T06:28:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.888+0000 7f11f9b7a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f11f4143c40 0x7f11f4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f9b7a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11f40717d0 0x7f11f40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f9b7a640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f4070e50 con 0x7f11f40cca40 2023-11-24T06:28:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f9b7a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f40723c0 con 0x7f11f40717d0 2023-11-24T06:28:55.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f9b7a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f40cce40 con 0x7f11f4143c40 2023-11-24T06:28:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11f40717d0 0x7f11f40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11f40717d0 0x7f11f40cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60438/0 (socket says 172.21.15.16:60438) 2023-11-24T06:28:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f3fff640 1 -- 172.21.15.16:0/530264920 learned_addr learned my addr 172.21.15.16:0/530264920 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f3fff640 1 -- 172.21.15.16:0/530264920 >> v2:172.21.15.173:3300/0 conn(0x7f11f4143c40 msgr2=0x7f11f4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f8b78640 1 --2- 172.21.15.16:0/530264920 >> v2:172.21.15.173:3300/0 conn(0x7f11f4143c40 0x7f11f4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f3fff640 1 --2- 172.21.15.16:0/530264920 >> v2:172.21.15.173:3300/0 conn(0x7f11f4143c40 0x7f11f4071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f3fff640 1 -- 172.21.15.16:0/530264920 >> v1:172.21.15.173:6789/0 conn(0x7f11f40cca40 legacy=0x7f11f415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f3fff640 1 -- 172.21.15.16:0/530264920 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f406e170 con 0x7f11f40717d0 2023-11-24T06:28:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.889+0000 7f11f8b78640 1 --2- 172.21.15.16:0/530264920 >> v2:172.21.15.173:3300/0 conn(0x7f11f4143c40 0x7f11f4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:55.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.890+0000 7f11f3fff640 1 --2- 172.21.15.16:0/530264920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 0x7f11f40cc500 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f11f406e410 tx=0x7f11e802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407c2aea2ddb6e62 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.891+0000 7f11f37fe640 1 -- 172.21.15.16:0/530264920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80317e0 con 0x7f11f40717d0 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.891+0000 7f11f37fe640 1 -- 172.21.15.16:0/530264920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8031940 con 0x7f11f40717d0 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.891+0000 7f11f37fe640 1 -- 172.21.15.16:0/530264920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8031cd0 con 0x7f11f40717d0 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.891+0000 7f11f9b7a640 1 -- 172.21.15.16:0/530264920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 msgr2=0x7f11f40cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.891+0000 7f11f9b7a640 1 --2- 172.21.15.16:0/530264920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 0x7f11f40cc500 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f11f406e410 tx=0x7f11e802b880 comp rx=0 tx=0).stop 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.891+0000 7f11f9b7a640 1 -- 172.21.15.16:0/530264920 shutdown_connections 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.891+0000 7f11f9b7a640 1 --2- 172.21.15.16:0/530264920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 0x7f11f40cc500 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.891+0000 7f11f9b7a640 1 --2- 172.21.15.16:0/530264920 >> v2:172.21.15.173:3300/0 conn(0x7f11f4143c40 0x7f11f4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.891+0000 7f11f9b7a640 1 -- 172.21.15.16:0/530264920 >> 172.21.15.16:0/530264920 conn(0x7f11f406a800 msgr2=0x7f11f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.892+0000 7f11f9b7a640 1 -- 172.21.15.16:0/530264920 shutdown_connections 2023-11-24T06:28:55.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.892+0000 7f11f9b7a640 1 -- 172.21.15.16:0/530264920 wait complete. 2023-11-24T06:28:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.893+0000 7f11f9b7a640 1 Processor -- start 2023-11-24T06:28:55.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.894+0000 7f11f9b7a640 1 -- start start 2023-11-24T06:28:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.894+0000 7f11f9b7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 0x7f11f415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.894+0000 7f11f9b7a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11f40cca40 0x7f11f4138e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.894+0000 7f11f9b7a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f4143c40 0x7f11f4139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.894+0000 7f11f9b7a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4138780 con 0x7f11f40717d0 2023-11-24T06:28:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.894+0000 7f11f9b7a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4142d70 con 0x7f11f4143c40 2023-11-24T06:28:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.894+0000 7f11f9b7a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f413c0e0 con 0x7f11f40cca40 2023-11-24T06:28:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.895+0000 7f11f8b78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 0x7f11f415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.895+0000 7f11f8b78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 0x7f11f415a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60442/0 (socket says 172.21.15.16:60442) 2023-11-24T06:28:55.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.895+0000 7f11f8b78640 1 -- 172.21.15.16:0/3134145973 learned_addr learned my addr 172.21.15.16:0/3134145973 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:55.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.895+0000 7f11f8b78640 1 -- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11f40cca40 msgr2=0x7f11f4138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.895+0000 7f11f8b78640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11f40cca40 0x7f11f4138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.895+0000 7f11f8b78640 1 -- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f4143c40 msgr2=0x7f11f4139390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.895+0000 7f11f9379640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f4143c40 0x7f11f4139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.895+0000 7f11f8b78640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f4143c40 0x7f11f4139390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.895+0000 7f11f8b78640 1 -- 172.21.15.16:0/3134145973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e80094d0 con 0x7f11f40717d0 2023-11-24T06:28:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.896+0000 7f11f8b78640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 0x7f11f415a980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c7f0 tx=0x7f11e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.896+0000 7f11f17fa640 1 -- 172.21.15.16:0/3134145973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4007820 con 0x7f11f40717d0 2023-11-24T06:28:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.896+0000 7f11f17fa640 1 -- 172.21.15.16:0/3134145973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e4007980 con 0x7f11f40717d0 2023-11-24T06:28:55.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.896+0000 7f11f17fa640 1 -- 172.21.15.16:0/3134145973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4007cf0 con 0x7f11f40717d0 2023-11-24T06:28:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.896+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f413c3c0 con 0x7f11f40717d0 2023-11-24T06:28:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.896+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f413c980 con 0x7f11f40717d0 2023-11-24T06:28:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.897+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f4067450 con 0x7f11f40717d0 2023-11-24T06:28:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.898+0000 7f11f17fa640 1 -- 172.21.15.16:0/3134145973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f11e40113f0 con 0x7f11f40717d0 2023-11-24T06:28:55.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.900+0000 7f11f17fa640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f11e006aed0 0x7f11e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:55.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.900+0000 7f11f17fa640 1 -- 172.21.15.16:0/3134145973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f11e4089860 con 0x7f11f40717d0 2023-11-24T06:28:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.901+0000 7f11f3fff640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f11e006aed0 0x7f11e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.902+0000 7f11f3fff640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f11e006aed0 0x7f11e006d390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11e8036000 tx=0x7f11e8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:55.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:55.905+0000 7f11f17fa640 1 -- 172.21.15.16:0/3134145973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11e4054a10 con 0x7f11f40717d0 2023-11-24T06:28:56.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.175+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f11f40cc800 con 0x7f11f40717d0 2023-11-24T06:28:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.205+0000 7f11f17fa640 1 -- 172.21.15.16:0/3134145973 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v131) v1 ==== 181+0+0 (secure 0 0 0) 0x7f11e4058030 con 0x7f11f40717d0 2023-11-24T06:28:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f11e006aed0 msgr2=0x7f11e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f11e006aed0 0x7f11e006d390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11e8036000 tx=0x7f11e8030000 comp rx=0 tx=0).stop 2023-11-24T06:28:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 msgr2=0x7f11f415a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 0x7f11f415a980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c7f0 tx=0x7f11e400ccb0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 shutdown_connections 2023-11-24T06:28:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f11e006aed0 0x7f11e006d390 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11f4143c40 0x7f11f4139390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11f40cca40 0x7f11f4138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 --2- 172.21.15.16:0/3134145973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f40717d0 0x7f11f415a980 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 >> 172.21.15.16:0/3134145973 conn(0x7f11f406a800 msgr2=0x7f11f413f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.208+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 shutdown_connections 2023-11-24T06:28:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.209+0000 7f11f9b7a640 1 -- 172.21.15.16:0/3134145973 wait complete. 2023-11-24T06:28:56.232 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... ok 2023-11-24T06:28:56.232 INFO:tasks.cephfs_test_runner:Starting test: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) Validate cleaning of stale subvolume snapshot metadata. 2023-11-24T06:28:56.232 INFO:tasks.cephfs_test_runner:test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:28:56.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:28:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.447+0000 7f1a6b0bb640 1 Processor -- start 2023-11-24T06:28:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.447+0000 7f1a6b0bb640 1 -- start start 2023-11-24T06:28:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.448+0000 7f1a6b0bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a6415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.448+0000 7f1a6b0bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a640cd530 0x7f1a6413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.448+0000 7f1a6b0bb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1a640cd9b0 0x7f1a6413fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.448+0000 7f1a6b0bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a640cdec0 con 0x7f1a640cd530 2023-11-24T06:28:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a6b0bb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a640ce1a0 con 0x7f1a640cd9b0 2023-11-24T06:28:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a6b0bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a640ce480 con 0x7f1a6413cfa0 2023-11-24T06:28:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a698b8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a640cd530 0x7f1a6413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a698b8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a640cd530 0x7f1a6413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60454/0 (socket says 172.21.15.16:60454) 2023-11-24T06:28:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a698b8640 1 -- 172.21.15.16:0/1891959285 learned_addr learned my addr 172.21.15.16:0/1891959285 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:56.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a6a0b9640 1 --2- 172.21.15.16:0/1891959285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a6415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a6a8ba640 1 --2- 172.21.15.16:0/1891959285 >> v2:172.21.15.173:3300/0 conn(0x7f1a640cd9b0 0x7f1a6413fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a698b8640 1 -- 172.21.15.16:0/1891959285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 msgr2=0x7f1a6415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a698b8640 1 --2- 172.21.15.16:0/1891959285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a6415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a698b8640 1 -- 172.21.15.16:0/1891959285 >> v2:172.21.15.173:3300/0 conn(0x7f1a640cd9b0 msgr2=0x7f1a6413fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a698b8640 1 --2- 172.21.15.16:0/1891959285 >> v2:172.21.15.173:3300/0 conn(0x7f1a640cd9b0 0x7f1a6413fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a698b8640 1 -- 172.21.15.16:0/1891959285 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a640ce760 con 0x7f1a640cd530 2023-11-24T06:28:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.449+0000 7f1a6a0b9640 1 --2- 172.21.15.16:0/1891959285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a6415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.450+0000 7f1a698b8640 1 --2- 172.21.15.16:0/1891959285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 0x7f1a6413f5a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1a6406ce70 tx=0x7f1a5402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3c1abadf10cf17 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.451+0000 7f1a690b7640 1 -- 172.21.15.16:0/1891959285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a540317d0 con 0x7f1a640cd530 2023-11-24T06:28:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.451+0000 7f1a690b7640 1 -- 172.21.15.16:0/1891959285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a54031930 con 0x7f1a640cd530 2023-11-24T06:28:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.451+0000 7f1a690b7640 1 -- 172.21.15.16:0/1891959285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54031cc0 con 0x7f1a640cd530 2023-11-24T06:28:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.452+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/1891959285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 msgr2=0x7f1a6413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.452+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/1891959285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 0x7f1a6413f5a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1a6406ce70 tx=0x7f1a5402b870 comp rx=0 tx=0).stop 2023-11-24T06:28:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.452+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/1891959285 shutdown_connections 2023-11-24T06:28:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.452+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/1891959285 >> v2:172.21.15.173:3300/0 conn(0x7f1a640cd9b0 0x7f1a6413fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.452+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/1891959285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 0x7f1a6413f5a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.452+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/1891959285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a6415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.452+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/1891959285 >> 172.21.15.16:0/1891959285 conn(0x7f1a6406a800 msgr2=0x7f1a6406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.452+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/1891959285 shutdown_connections 2023-11-24T06:28:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.452+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/1891959285 wait complete. 2023-11-24T06:28:56.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.454+0000 7f1a6b0bb640 1 Processor -- start 2023-11-24T06:28:56.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.454+0000 7f1a6b0bb640 1 -- start start 2023-11-24T06:28:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6b0bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 0x7f1a6413bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6b0bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a640cd9b0 0x7f1a6415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6b0bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a641e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6b0bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64154c30 con 0x7f1a640cd530 2023-11-24T06:28:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6b0bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6414fb90 con 0x7f1a640cd9b0 2023-11-24T06:28:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6b0bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6414fe70 con 0x7f1a6413cfa0 2023-11-24T06:28:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6a0b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 0x7f1a6413bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6a0b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 0x7f1a6413bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60456/0 (socket says 172.21.15.16:60456) 2023-11-24T06:28:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6a0b9640 1 -- 172.21.15.16:0/511274112 learned_addr learned my addr 172.21.15.16:0/511274112 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.455+0000 7f1a6a0b9640 1 -- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 msgr2=0x7f1a641e3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a6a8ba640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a641e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a698b8640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a640cd9b0 0x7f1a6415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a6a0b9640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a641e3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a6a0b9640 1 -- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a640cd9b0 msgr2=0x7f1a6415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a6a0b9640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a640cd9b0 0x7f1a6415cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a6a0b9640 1 -- 172.21.15.16:0/511274112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a540094d0 con 0x7f1a640cd530 2023-11-24T06:28:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a6a8ba640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a641e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a698b8640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a640cd9b0 0x7f1a6415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a6a0b9640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 0x7f1a6413bb50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c7f0 tx=0x7f1a6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a5affd640 1 -- 172.21.15.16:0/511274112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60007820 con 0x7f1a640cd530 2023-11-24T06:28:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a5affd640 1 -- 172.21.15.16:0/511274112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60007980 con 0x7f1a640cd530 2023-11-24T06:28:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a5affd640 1 -- 172.21.15.16:0/511274112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60007cf0 con 0x7f1a640cd530 2023-11-24T06:28:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a64150150 con 0x7f1a640cd530 2023-11-24T06:28:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.456+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6413c090 con 0x7f1a640cd530 2023-11-24T06:28:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.458+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a64154ea0 con 0x7f1a640cd530 2023-11-24T06:28:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.459+0000 7f1a5affd640 1 -- 172.21.15.16:0/511274112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1a600113f0 con 0x7f1a640cd530 2023-11-24T06:28:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.460+0000 7f1a5affd640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1a4806aed0 0x7f1a4806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.460+0000 7f1a5affd640 1 -- 172.21.15.16:0/511274112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1a600897c0 con 0x7f1a640cd530 2023-11-24T06:28:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.460+0000 7f1a698b8640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1a4806aed0 0x7f1a4806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.461+0000 7f1a698b8640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1a4806aed0 0x7f1a4806d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a54036000 tx=0x7f1a5403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.464+0000 7f1a5affd640 1 -- 172.21.15.16:0/511274112 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a60054970 con 0x7f1a640cd530 2023-11-24T06:28:56.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.660+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a64067450 con 0x7f1a640cd530 2023-11-24T06:28:56.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.662+0000 7f1a5affd640 1 -- 172.21.15.16:0/511274112 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+12986 (secure 0 0 0) 0x7f1a60057f90 con 0x7f1a640cd530 2023-11-24T06:28:56.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:56.664 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":37,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:28:55.178489+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/4049479925":"2023-11-25T06:28:51.763285+0000","172.21.15.16:6833/4049479925":"2023-11-25T06:28:51.763285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:28:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1a4806aed0 msgr2=0x7f1a4806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1a4806aed0 0x7f1a4806d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a54036000 tx=0x7f1a5403d040 comp rx=0 tx=0).stop 2023-11-24T06:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 msgr2=0x7f1a6413bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 0x7f1a6413bb50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c7f0 tx=0x7f1a6000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 shutdown_connections 2023-11-24T06:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1a4806aed0 0x7f1a4806d390 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6413cfa0 0x7f1a641e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a640cd9b0 0x7f1a6415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 --2- 172.21.15.16:0/511274112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a640cd530 0x7f1a6413bb50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 >> 172.21.15.16:0/511274112 conn(0x7f1a6406a800 msgr2=0x7f1a6415a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 shutdown_connections 2023-11-24T06:28:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.665+0000 7f1a6b0bb640 1 -- 172.21.15.16:0/511274112 wait complete. 2023-11-24T06:28:56.688 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:28:56.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:28:56.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.914+0000 7fb8f27ac640 1 Processor -- start 2023-11-24T06:28:56.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.915+0000 7fb8f27ac640 1 -- start start 2023-11-24T06:28:56.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.916+0000 7fb8f27ac640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8ec143810 0x7fb8ec071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.916+0000 7fb8f27ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8ec15d770 0x7fb8ec15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.916+0000 7fb8f27ac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec15e140 con 0x7fb8ec0ae170 2023-11-24T06:28:56.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.916+0000 7fb8f27ac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec143bf0 con 0x7fb8ec15d770 2023-11-24T06:28:56.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.916+0000 7fb8f27ac640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec06c730 con 0x7fb8ec143810 2023-11-24T06:28:56.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.917+0000 7fb8f1fab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8ec15d770 0x7fb8ec15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.917+0000 7fb8f0fa9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8ec0ae170 0x7fb8ec15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58458/0 (socket says 172.21.15.16:58458) 2023-11-24T06:28:56.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.917+0000 7fb8f0fa9640 1 -- 172.21.15.16:0/445180765 learned_addr learned my addr 172.21.15.16:0/445180765 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.917+0000 7fb8f17aa640 1 --2- 172.21.15.16:0/445180765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8ec143810 0x7fb8ec071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.917+0000 7fb8f1fab640 1 -- 172.21.15.16:0/445180765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8ec143810 msgr2=0x7fb8ec071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.917+0000 7fb8f1fab640 1 --2- 172.21.15.16:0/445180765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8ec143810 0x7fb8ec071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.917+0000 7fb8f1fab640 1 -- 172.21.15.16:0/445180765 >> v1:172.21.15.16:6789/0 conn(0x7fb8ec0ae170 legacy=0x7fb8ec15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.917+0000 7fb8f1fab640 1 -- 172.21.15.16:0/445180765 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0ae5f0 con 0x7fb8ec15d770 2023-11-24T06:28:56.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.917+0000 7fb8e3fff640 1 -- 172.21.15.16:0/445180765 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793571255 0 0) 0x7fb8ec15e140 con 0x7fb8ec0ae170 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.919+0000 7fb8f1fab640 1 --2- 172.21.15.16:0/445180765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec15d770 0x7fb8ec15dc00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009d90 tx=0x7fb8e402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e50c47617e2d519 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.919+0000 7fb8e3fff640 1 -- 172.21.15.16:0/445180765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e402cb40 con 0x7fb8ec15d770 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.919+0000 7fb8e3fff640 1 -- 172.21.15.16:0/445180765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e402cca0 con 0x7fb8ec15d770 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.919+0000 7fb8e3fff640 1 -- 172.21.15.16:0/445180765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4034660 con 0x7fb8ec15d770 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.919+0000 7fb8f27ac640 1 -- 172.21.15.16:0/445180765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec15d770 msgr2=0x7fb8ec15dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.919+0000 7fb8f27ac640 1 --2- 172.21.15.16:0/445180765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec15d770 0x7fb8ec15dc00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009d90 tx=0x7fb8e402bd90 comp rx=0 tx=0).stop 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.920+0000 7fb8f27ac640 1 -- 172.21.15.16:0/445180765 shutdown_connections 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.920+0000 7fb8f27ac640 1 --2- 172.21.15.16:0/445180765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec15d770 0x7fb8ec15dc00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.920+0000 7fb8f27ac640 1 --2- 172.21.15.16:0/445180765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8ec143810 0x7fb8ec071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.920+0000 7fb8f27ac640 1 -- 172.21.15.16:0/445180765 >> 172.21.15.16:0/445180765 conn(0x7fb8ec06a7d0 msgr2=0x7fb8ec06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:56.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.920+0000 7fb8f27ac640 1 -- 172.21.15.16:0/445180765 shutdown_connections 2023-11-24T06:28:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.920+0000 7fb8f27ac640 1 -- 172.21.15.16:0/445180765 wait complete. 2023-11-24T06:28:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.921+0000 7fb8f27ac640 1 Processor -- start 2023-11-24T06:28:56.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.922+0000 7fb8f27ac640 1 -- start start 2023-11-24T06:28:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.922+0000 7fb8f27ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec0ae170 0x7fb8ec0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.922+0000 7fb8f27ac640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8ec143810 0x7fb8ec15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f27ac640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8ec15d770 0x7fb8ec06d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f27ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec15c120 con 0x7fb8ec0ae170 2023-11-24T06:28:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f27ac640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0ccb40 con 0x7fb8ec143810 2023-11-24T06:28:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f27ac640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0cce20 con 0x7fb8ec15d770 2023-11-24T06:28:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f17aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec0ae170 0x7fb8ec0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f17aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec0ae170 0x7fb8ec0cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60486/0 (socket says 172.21.15.16:60486) 2023-11-24T06:28:56.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f17aa640 1 -- 172.21.15.16:0/1123522595 learned_addr learned my addr 172.21.15.16:0/1123522595 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:56.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f17aa640 1 -- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8ec15d770 msgr2=0x7fb8ec06d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f17aa640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8ec15d770 0x7fb8ec06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f17aa640 1 -- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8ec143810 msgr2=0x7fb8ec15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:56.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f17aa640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8ec143810 0x7fb8ec15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:56.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.923+0000 7fb8f17aa640 1 -- 172.21.15.16:0/1123522595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e4009a70 con 0x7fb8ec0ae170 2023-11-24T06:28:56.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.924+0000 7fb8f17aa640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec0ae170 0x7fb8ec0cdbe0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800d800 tx=0x7fb8e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:56.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.924+0000 7fb8e1ffb640 1 -- 172.21.15.16:0/1123522595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e800b6c0 con 0x7fb8ec0ae170 2023-11-24T06:28:56.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.924+0000 7fb8e1ffb640 1 -- 172.21.15.16:0/1123522595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e800b820 con 0x7fb8ec0ae170 2023-11-24T06:28:56.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.924+0000 7fb8e1ffb640 1 -- 172.21.15.16:0/1123522595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e800bbb0 con 0x7fb8ec0ae170 2023-11-24T06:28:56.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.924+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec0cd100 con 0x7fb8ec0ae170 2023-11-24T06:28:56.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.924+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec0cd6c0 con 0x7fb8ec0ae170 2023-11-24T06:28:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.926+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec067450 con 0x7fb8ec0ae170 2023-11-24T06:28:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.926+0000 7fb8e1ffb640 1 -- 172.21.15.16:0/1123522595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb8e800bd10 con 0x7fb8ec0ae170 2023-11-24T06:28:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.927+0000 7fb8e1ffb640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8d006aed0 0x7fb8d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.928+0000 7fb8e1ffb640 1 -- 172.21.15.16:0/1123522595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb8e80899e0 con 0x7fb8ec0ae170 2023-11-24T06:28:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.928+0000 7fb8f0fa9640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8d006aed0 0x7fb8d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.930+0000 7fb8f0fa9640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8d006aed0 0x7fb8d006d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc007fd0 tx=0x7fb8dc007920 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:56.932+0000 7fb8e1ffb640 1 -- 172.21.15.16:0/1123522595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8e8054ca0 con 0x7fb8ec0ae170 2023-11-24T06:28:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.125+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8ec0ad150 con 0x7fb8ec0ae170 2023-11-24T06:28:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.126+0000 7fb8e1ffb640 1 -- 172.21.15.16:0/1123522595 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8e80582c0 con 0x7fb8ec0ae170 2023-11-24T06:28:57.128 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:28:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8d006aed0 msgr2=0x7fb8d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8d006aed0 0x7fb8d006d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc007fd0 tx=0x7fb8dc007920 comp rx=0 tx=0).stop 2023-11-24T06:28:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec0ae170 msgr2=0x7fb8ec0cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec0ae170 0x7fb8ec0cdbe0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800d800 tx=0x7fb8e800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 shutdown_connections 2023-11-24T06:28:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8d006aed0 0x7fb8d006d390 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8ec15d770 0x7fb8ec06d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8ec143810 0x7fb8ec15bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 --2- 172.21.15.16:0/1123522595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8ec0ae170 0x7fb8ec0cdbe0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 >> 172.21.15.16:0/1123522595 conn(0x7fb8ec06a7d0 msgr2=0x7fb8ec070950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 shutdown_connections 2023-11-24T06:28:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.129+0000 7fb8f27ac640 1 -- 172.21.15.16:0/1123522595 wait complete. 2023-11-24T06:28:57.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T06:28:57.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.363+0000 7ffae0a31640 1 Processor -- start 2023-11-24T06:28:57.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.363+0000 7ffae0a31640 1 -- start start 2023-11-24T06:28:57.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.364+0000 7ffae0a31640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffadc0d17e0 0x7ffadc0d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.364+0000 7ffae0a31640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ffadc0b3890 0x7ffadc0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.364+0000 7ffae0a31640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffadc0d2180 con 0x7ffadc15d610 2023-11-24T06:28:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.365+0000 7ffae0a31640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffadc0d1390 con 0x7ffadc0b3890 2023-11-24T06:28:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.365+0000 7ffae0a31640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0b3400 con 0x7ffadc0d17e0 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.365+0000 7ffada59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffadc15d610 0x7ffadc0ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58472/0 (socket says 172.21.15.16:58472) 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.365+0000 7ffada59c640 1 -- 172.21.15.16:0/1279390506 learned_addr learned my addr 172.21.15.16:0/1279390506 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.365+0000 7ffad9d9b640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffadc0d17e0 0x7ffadc0d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.365+0000 7ffadad9d640 1 --2- 172.21.15.16:0/1279390506 >> v2:172.21.15.173:3300/0 conn(0x7ffadc0b3890 0x7ffadc0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.365+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439561370 0 0) 0x7ffadc0d2180 con 0x7ffadc15d610 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.365+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffac8003520 con 0x7ffadc15d610 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 440648410 0 0) 0x7ffac8003520 con 0x7ffadc15d610 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffadc0d17e0 msgr2=0x7ffadc0d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffadc0d17e0 0x7ffadc0d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 >> v2:172.21.15.173:3300/0 conn(0x7ffadc0b3890 msgr2=0x7ffadc0b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 --2- 172.21.15.16:0/1279390506 >> v2:172.21.15.173:3300/0 conn(0x7ffadc0b3890 0x7ffadc0b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc0b27f0 con 0x7ffadc15d610 2023-11-24T06:28:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7ffacc0019f0 con 0x7ffadc15d610 2023-11-24T06:28:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 >> v1:172.21.15.16:6789/0 conn(0x7ffadc15d610 legacy=0x7ffadc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffac8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac80047c0 0x7ffac8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac8005180 0x7ffac800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0d1390 con 0x7ffac8003970 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc0b3400 con 0x7ffac8005180 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac800bf40 con 0x7ffac80047c0 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffad9d9b640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffadc0d17e0 0x7ffadc0d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.366+0000 7ffada59c640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffac8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.367+0000 7ffadad9d640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac8005180 0x7ffac800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.367+0000 7ffada59c640 1 -- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac80047c0 msgr2=0x7ffac8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.367+0000 7ffada59c640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac80047c0 0x7ffac8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.367+0000 7ffada59c640 1 -- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac8005180 msgr2=0x7ffac800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.367+0000 7ffada59c640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac8005180 0x7ffac800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.367+0000 7ffada59c640 1 -- 172.21.15.16:0/1279390506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac8003520 con 0x7ffac8003970 2023-11-24T06:28:57.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.367+0000 7ffadad9d640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac8005180 0x7ffac800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.368+0000 7ffada59c640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffac8003d90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00a790 tx=0x7ffac80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f8463639b4f927 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.368+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc034630 con 0x7ffac8003970 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.368+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc02f070 con 0x7ffac8003970 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.368+0000 7ffad959a640 1 -- 172.21.15.16:0/1279390506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc0349f0 con 0x7ffac8003970 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.369+0000 7ffae0a31640 1 -- 172.21.15.16:0/1279390506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 msgr2=0x7ffac8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.369+0000 7ffae0a31640 1 --2- 172.21.15.16:0/1279390506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffac8003d90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00a790 tx=0x7ffac80057f0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.369+0000 7ffadad9d640 1 -- 172.21.15.16:0/1279390506 reap_dead start 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.369+0000 7ffae0a31640 1 -- 172.21.15.16:0/1279390506 shutdown_connections 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.369+0000 7ffae0a31640 1 -- 172.21.15.16:0/1279390506 >> 172.21.15.16:0/1279390506 conn(0x7ffadc09c540 msgr2=0x7ffadc0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.369+0000 7ffae0a31640 1 -- 172.21.15.16:0/1279390506 shutdown_connections 2023-11-24T06:28:57.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.369+0000 7ffae0a31640 1 -- 172.21.15.16:0/1279390506 wait complete. 2023-11-24T06:28:57.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.371+0000 7ffae0a31640 1 Processor -- start 2023-11-24T06:28:57.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.371+0000 7ffae0a31640 1 -- start start 2023-11-24T06:28:57.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.372+0000 7ffae0a31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffadc0b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.372+0000 7ffae0a31640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffadc0b8df0 0x7ffadc0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.372+0000 7ffae0a31640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffadc15a150 0x7ffadc15fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.372+0000 7ffae0a31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc159fc0 con 0x7ffac8003970 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.372+0000 7ffae0a31640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc15ee40 con 0x7ffadc15a150 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.372+0000 7ffae0a31640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc15f120 con 0x7ffadc0b8df0 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.372+0000 7ffada59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffadc0b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.372+0000 7ffada59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffadc0b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60504/0 (socket says 172.21.15.16:60504) 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.372+0000 7ffada59c640 1 -- 172.21.15.16:0/269090310 learned_addr learned my addr 172.21.15.16:0/269090310 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffada59c640 1 -- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffadc0b8df0 msgr2=0x7ffadc0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffada59c640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffadc0b8df0 0x7ffadc0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffadad9d640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffadc15a150 0x7ffadc15fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffada59c640 1 -- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffadc15a150 msgr2=0x7ffadc15fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffada59c640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffadc15a150 0x7ffadc15fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffada59c640 1 -- 172.21.15.16:0/269090310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc15f400 con 0x7ffac8003970 2023-11-24T06:28:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffada59c640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffadc0b64f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffacc02cad0 tx=0x7ffacc001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffad2ffd640 1 -- 172.21.15.16:0/269090310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc034630 con 0x7ffac8003970 2023-11-24T06:28:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffad2ffd640 1 -- 172.21.15.16:0/269090310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc034790 con 0x7ffac8003970 2023-11-24T06:28:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffad2ffd640 1 -- 172.21.15.16:0/269090310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc034b20 con 0x7ffac8003970 2023-11-24T06:28:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffadc15f6e0 con 0x7ffac8003970 2023-11-24T06:28:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.373+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffadc0b6a30 con 0x7ffac8003970 2023-11-24T06:28:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.375+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffadc0b6c20 con 0x7ffac8003970 2023-11-24T06:28:57.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.376+0000 7ffad2ffd640 1 -- 172.21.15.16:0/269090310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ffacc031030 con 0x7ffac8003970 2023-11-24T06:28:57.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.376+0000 7ffad2ffd640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffabc06af00 0x7ffabc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:57.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.377+0000 7ffad2ffd640 1 -- 172.21.15.16:0/269090310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffacc0abb60 con 0x7ffac8003970 2023-11-24T06:28:57.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.377+0000 7ffad9d9b640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffabc06af00 0x7ffabc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:57.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.378+0000 7ffad9d9b640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffabc06af00 0x7ffabc06d3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffac4006fd0 tx=0x7ffac4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:57.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.381+0000 7ffad2ffd640 1 -- 172.21.15.16:0/269090310 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffacc076d10 con 0x7ffac8003970 2023-11-24T06:28:57.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:57.644+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7ffadc0b6c20 con 0x7ffac8003970 2023-11-24T06:28:58.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.217+0000 7ffad2ffd640 1 -- 172.21.15.16:0/269090310 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v133) v1 ==== 175+0+0 (secure 0 0 0) 0x7ffacc07a330 con 0x7ffac8003970 2023-11-24T06:28:58.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffabc06af00 msgr2=0x7ffabc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffabc06af00 0x7ffabc06d3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffac4006fd0 tx=0x7ffac4008040 comp rx=0 tx=0).stop 2023-11-24T06:28:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 msgr2=0x7ffadc0b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffadc0b64f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffacc02cad0 tx=0x7ffacc001cb0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 shutdown_connections 2023-11-24T06:28:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffabc06af00 0x7ffabc06d3c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffadc15a150 0x7ffadc15fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffadc0b8df0 0x7ffadc0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 --2- 172.21.15.16:0/269090310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac8003970 0x7ffadc0b64f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.221+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 >> 172.21.15.16:0/269090310 conn(0x7ffadc09c540 msgr2=0x7ffadc0b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.222+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 shutdown_connections 2023-11-24T06:28:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.222+0000 7ffae0a31640 1 -- 172.21.15.16:0/269090310 wait complete. 2023-11-24T06:28:58.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:28:58.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.476+0000 7fabd0832640 1 Processor -- start 2023-11-24T06:28:58.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.477+0000 7fabd0832640 1 -- start start 2023-11-24T06:28:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.478+0000 7fabd0832640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabcc13cda0 0x7fabcc15da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.478+0000 7fabd0832640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabcc0ce4f0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.479+0000 7fabd0832640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabcc0700f0 con 0x7fabcc0ce0c0 2023-11-24T06:28:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.479+0000 7fabd0832640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabcc15e3e0 con 0x7fabcc13cda0 2023-11-24T06:28:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.479+0000 7fabcb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabcc13cda0 0x7fabcc15da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.479+0000 7fabc3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fabcc15dfa0 0x7fabcc0cd9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58486/0 (socket says 172.21.15.16:58486) 2023-11-24T06:28:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.479+0000 7fabc3fff640 1 -- 172.21.15.16:0/1281695495 learned_addr learned my addr 172.21.15.16:0/1281695495 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.479+0000 7fabcb7fe640 1 -- 172.21.15.16:0/1281695495 >> v1:172.21.15.173:6789/0 conn(0x7fabcc0ce0c0 legacy=0x7fabcc070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.479+0000 7fabcaffd640 1 -- 172.21.15.16:0/1281695495 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92928726 0 0) 0x7fabcc0ce4f0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.480+0000 7fabcb7fe640 1 -- 172.21.15.16:0/1281695495 >> v1:172.21.15.16:6789/0 conn(0x7fabcc15dfa0 legacy=0x7fabcc0cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.480+0000 7fabcb7fe640 1 -- 172.21.15.16:0/1281695495 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc0ce720 con 0x7fabcc13cda0 2023-11-24T06:28:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.481+0000 7fabcb7fe640 1 --2- 172.21.15.16:0/1281695495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc13cda0 0x7fabcc15da60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0097a0 tx=0x7fabbc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c18eff7e227d9d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.481+0000 7fabcaffd640 1 -- 172.21.15.16:0/1281695495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc02e070 con 0x7fabcc13cda0 2023-11-24T06:28:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.481+0000 7fabcaffd640 1 -- 172.21.15.16:0/1281695495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc0315a0 con 0x7fabcc13cda0 2023-11-24T06:28:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.481+0000 7fabcaffd640 1 -- 172.21.15.16:0/1281695495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc031930 con 0x7fabcc13cda0 2023-11-24T06:28:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.482+0000 7fabd0832640 1 -- 172.21.15.16:0/1281695495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc13cda0 msgr2=0x7fabcc15da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.482+0000 7fabd0832640 1 --2- 172.21.15.16:0/1281695495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc13cda0 0x7fabcc15da60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0097a0 tx=0x7fabbc02b840 comp rx=0 tx=0).stop 2023-11-24T06:28:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.482+0000 7fabd0832640 1 -- 172.21.15.16:0/1281695495 shutdown_connections 2023-11-24T06:28:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.482+0000 7fabd0832640 1 --2- 172.21.15.16:0/1281695495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc13cda0 0x7fabcc15da60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.482+0000 7fabd0832640 1 -- 172.21.15.16:0/1281695495 >> 172.21.15.16:0/1281695495 conn(0x7fabcc06a3e0 msgr2=0x7fabcc06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.482+0000 7fabd0832640 1 -- 172.21.15.16:0/1281695495 shutdown_connections 2023-11-24T06:28:58.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.482+0000 7fabd0832640 1 -- 172.21.15.16:0/1281695495 wait complete. 2023-11-24T06:28:58.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.484+0000 7fabd0832640 1 Processor -- start 2023-11-24T06:28:58.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.484+0000 7fabd0832640 1 -- start start 2023-11-24T06:28:58.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabd0832640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabcc0ce0c0 0x7fabcc155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabd0832640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabcc13cda0 0x7fabcc0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabd0832640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc15dfa0 0x7fabcc142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabd0832640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc140ac0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabd0832640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc142910 con 0x7fabcc0ce0c0 2023-11-24T06:28:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabd0832640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc155590 con 0x7fabcc13cda0 2023-11-24T06:28:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabcbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc15dfa0 0x7fabcc142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabcbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc15dfa0 0x7fabcc142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60518/0 (socket says 172.21.15.16:60518) 2023-11-24T06:28:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabcbfff640 1 -- 172.21.15.16:0/1883177997 learned_addr learned my addr 172.21.15.16:0/1883177997 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.486+0000 7fabcb7fe640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabcc0ce0c0 0x7fabcc155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabcbfff640 1 -- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabcc13cda0 msgr2=0x7fabcc0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabc3fff640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabcc13cda0 0x7fabcc0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabcbfff640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabcc13cda0 0x7fabcc0b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabcbfff640 1 -- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabcc0ce0c0 msgr2=0x7fabcc155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabcbfff640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabcc0ce0c0 0x7fabcc155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabcbfff640 1 -- 172.21.15.16:0/1883177997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc009400 con 0x7fabcc15dfa0 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabcb7fe640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabcc0ce0c0 0x7fabcc155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabc3fff640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabcc13cda0 0x7fabcc0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabcbfff640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc15dfa0 0x7fabcc142da0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b970 tx=0x7fabc400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:58.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabc8ff9640 1 -- 172.21.15.16:0/1883177997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc400cb40 con 0x7fabcc15dfa0 2023-11-24T06:28:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabc8ff9640 1 -- 172.21.15.16:0/1883177997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc400cca0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabcc0b21a0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabc8ff9640 1 -- 172.21.15.16:0/1883177997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4014680 con 0x7fabcc15dfa0 2023-11-24T06:28:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.487+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabcc0b24f0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.489+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabcc1426c0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.490+0000 7fabc8ff9640 1 -- 172.21.15.16:0/1883177997 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fabc401d3b0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.490+0000 7fabc8ff9640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabac06aed0 0x7fabac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.491+0000 7fabc8ff9640 1 -- 172.21.15.16:0/1883177997 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fabc4089300 con 0x7fabcc15dfa0 2023-11-24T06:28:58.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.491+0000 7fabcb7fe640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabac06aed0 0x7fabac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.492+0000 7fabcb7fe640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabac06aed0 0x7fabac06d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fabbc036000 tx=0x7fabbc03d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.495+0000 7fabc8ff9640 1 -- 172.21.15.16:0/1883177997 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fabc40545c0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.689+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fabcc067450 con 0x7fabcc15dfa0 2023-11-24T06:28:58.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.696+0000 7fabc8ff9640 1 -- 172.21.15.16:0/1883177997 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fabc4057be0 con 0x7fabcc15dfa0 2023-11-24T06:28:58.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabac06aed0 msgr2=0x7fabac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabac06aed0 0x7fabac06d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fabbc036000 tx=0x7fabbc03d040 comp rx=0 tx=0).stop 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc15dfa0 msgr2=0x7fabcc142da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc15dfa0 0x7fabcc142da0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fabc400b970 tx=0x7fabc400be30 comp rx=0 tx=0).stop 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 shutdown_connections 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabac06aed0 0x7fabac06d390 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabcc15dfa0 0x7fabcc142da0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabcc13cda0 0x7fabcc0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 --2- 172.21.15.16:0/1883177997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabcc0ce0c0 0x7fabcc155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 >> 172.21.15.16:0/1883177997 conn(0x7fabcc06a3e0 msgr2=0x7fabcc15ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 shutdown_connections 2023-11-24T06:28:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:58.699+0000 7fabd0832640 1 -- 172.21.15.16:0/1883177997 wait complete. 2023-11-24T06:28:58.722 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:28:58.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:28:58.839 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:28:58.982 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:28:58.983 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:28:58.983 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:59.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:28:59.048 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:28:59.048 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:59.108 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:28:59.108 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:28:59.108 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:59.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:28:59.170 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:28:59.170 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:28:59.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:28:59.232 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:28:59.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:28:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.468+0000 7fea52235640 1 Processor -- start 2023-11-24T06:28:59.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.468+0000 7fea52235640 1 -- start start 2023-11-24T06:28:59.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.470+0000 7fea52235640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea4c071170 0x7fea4c168050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.470+0000 7fea52235640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c161970 con 0x7fea4c145e10 2023-11-24T06:28:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.470+0000 7fea52235640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c0ae7c0 con 0x7fea4c1469e0 2023-11-24T06:28:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.470+0000 7fea52235640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c1461f0 con 0x7fea4c071170 2023-11-24T06:28:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.470+0000 7fea51233640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea4c145e10 0x7fea4c0ae0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58500/0 (socket says 172.21.15.16:58500) 2023-11-24T06:28:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.470+0000 7fea51233640 1 -- 172.21.15.16:0/1250161557 learned_addr learned my addr 172.21.15.16:0/1250161557 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:28:59.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.470+0000 7fea51a34640 1 --2- 172.21.15.16:0/1250161557 >> v2:172.21.15.16:3300/0 conn(0x7fea4c071170 0x7fea4c168050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.471+0000 7fea43fff640 1 -- 172.21.15.16:0/1250161557 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501686947 0 0) 0x7fea4c161970 con 0x7fea4c145e10 2023-11-24T06:28:59.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.471+0000 7fea43fff640 1 -- 172.21.15.16:0/1250161557 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea38003520 con 0x7fea4c145e10 2023-11-24T06:28:59.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.471+0000 7fea51a34640 1 -- 172.21.15.16:0/1250161557 >> v1:172.21.15.173:6789/0 conn(0x7fea4c1469e0 legacy=0x7fea4c0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:28:59.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.471+0000 7fea51a34640 1 -- 172.21.15.16:0/1250161557 >> v1:172.21.15.16:6789/0 conn(0x7fea4c145e10 legacy=0x7fea4c0ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.471+0000 7fea51a34640 1 -- 172.21.15.16:0/1250161557 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c160d60 con 0x7fea4c071170 2023-11-24T06:28:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.472+0000 7fea51a34640 1 --2- 172.21.15.16:0/1250161557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c071170 0x7fea4c168050 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fea44009d90 tx=0x7fea4402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e899eef681caf77b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.473+0000 7fea43fff640 1 -- 172.21.15.16:0/1250161557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4402cd00 con 0x7fea4c071170 2023-11-24T06:28:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.473+0000 7fea43fff640 1 -- 172.21.15.16:0/1250161557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4402ce60 con 0x7fea4c071170 2023-11-24T06:28:59.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.473+0000 7fea43fff640 1 -- 172.21.15.16:0/1250161557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44034660 con 0x7fea4c071170 2023-11-24T06:28:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.473+0000 7fea52235640 1 -- 172.21.15.16:0/1250161557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c071170 msgr2=0x7fea4c168050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.473+0000 7fea52235640 1 --2- 172.21.15.16:0/1250161557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c071170 0x7fea4c168050 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fea44009d90 tx=0x7fea4402bf50 comp rx=0 tx=0).stop 2023-11-24T06:28:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.473+0000 7fea52235640 1 -- 172.21.15.16:0/1250161557 shutdown_connections 2023-11-24T06:28:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.473+0000 7fea52235640 1 --2- 172.21.15.16:0/1250161557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c071170 0x7fea4c168050 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.473+0000 7fea52235640 1 -- 172.21.15.16:0/1250161557 >> 172.21.15.16:0/1250161557 conn(0x7fea4c0b0b00 msgr2=0x7fea4c0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:59.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.474+0000 7fea52235640 1 -- 172.21.15.16:0/1250161557 shutdown_connections 2023-11-24T06:28:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.474+0000 7fea52235640 1 -- 172.21.15.16:0/1250161557 wait complete. 2023-11-24T06:28:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.475+0000 7fea52235640 1 Processor -- start 2023-11-24T06:28:59.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.476+0000 7fea52235640 1 -- start start 2023-11-24T06:28:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea52235640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea4c071170 0x7fea4c13b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea52235640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea4c145e10 0x7fea4c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea52235640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c1469e0 0x7fea4c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea52235640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c144410 con 0x7fea4c1469e0 2023-11-24T06:28:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea52235640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c143400 con 0x7fea4c145e10 2023-11-24T06:28:59.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea52235640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c13bbb0 con 0x7fea4c071170 2023-11-24T06:28:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea51a34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c1469e0 0x7fea4c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea51a34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c1469e0 0x7fea4c0b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60542/0 (socket says 172.21.15.16:60542) 2023-11-24T06:28:59.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea51a34640 1 -- 172.21.15.16:0/530172893 learned_addr learned my addr 172.21.15.16:0/530172893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:28:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea51a34640 1 -- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea4c071170 msgr2=0x7fea4c13b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea51a34640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea4c071170 0x7fea4c13b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea51a34640 1 -- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea4c145e10 msgr2=0x7fea4c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.477+0000 7fea50a32640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea4c145e10 0x7fea4c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.478+0000 7fea51a34640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea4c145e10 0x7fea4c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.478+0000 7fea51a34640 1 -- 172.21.15.16:0/530172893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44009a70 con 0x7fea4c1469e0 2023-11-24T06:28:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.478+0000 7fea50a32640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea4c145e10 0x7fea4c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:28:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.478+0000 7fea51a34640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c1469e0 0x7fea4c0b0000 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fea44009eb0 tx=0x7fea44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.478+0000 7fea41ffb640 1 -- 172.21.15.16:0/530172893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4403d600 con 0x7fea4c1469e0 2023-11-24T06:28:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.478+0000 7fea41ffb640 1 -- 172.21.15.16:0/530172893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4403d780 con 0x7fea4c1469e0 2023-11-24T06:28:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.478+0000 7fea41ffb640 1 -- 172.21.15.16:0/530172893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4403db30 con 0x7fea4c1469e0 2023-11-24T06:28:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.478+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c143650 con 0x7fea4c1469e0 2023-11-24T06:28:59.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.478+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c143c10 con 0x7fea4c1469e0 2023-11-24T06:28:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.479+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c067450 con 0x7fea4c1469e0 2023-11-24T06:28:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.480+0000 7fea41ffb640 1 -- 172.21.15.16:0/530172893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fea4403b040 con 0x7fea4c1469e0 2023-11-24T06:28:59.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.481+0000 7fea41ffb640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3006aed0 0x7fea3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:28:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.481+0000 7fea41ffb640 1 -- 172.21.15.16:0/530172893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fea440ab8f0 con 0x7fea4c1469e0 2023-11-24T06:28:59.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.482+0000 7fea51233640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3006aed0 0x7fea3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:28:59.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.483+0000 7fea51233640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3006aed0 0x7fea3006d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fea48001610 tx=0x7fea48007930 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:28:59.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.485+0000 7fea41ffb640 1 -- 172.21.15.16:0/530172893 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea44076cb0 con 0x7fea4c1469e0 2023-11-24T06:28:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.722+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea4c0adc60 con 0x7fea4c1469e0 2023-11-24T06:28:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.723+0000 7fea41ffb640 1 -- 172.21.15.16:0/530172893 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fea4404e090 con 0x7fea4c1469e0 2023-11-24T06:28:59.725 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:28:59.725 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/147489979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":147489979},{"type":"v1","addr":"172.21.15.173:6836","nonce":147489979}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1330874839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1330874839},{"type":"v1","addr":"172.21.15.173:6837","nonce":1330874839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4867,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2341098684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2341098684},{"type":"v1","addr":"172.21.15.16:6835","nonce":2341098684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1219831218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1219831218},{"type":"v1","addr":"172.21.15.16:6834","nonce":1219831218}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2023-11-24T06:28:59.726 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 19 2023-11-24T06:28:59.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3006aed0 msgr2=0x7fea3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3006aed0 0x7fea3006d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fea48001610 tx=0x7fea48007930 comp rx=0 tx=0).stop 2023-11-24T06:28:59.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c1469e0 msgr2=0x7fea4c0b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:28:59.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c1469e0 0x7fea4c0b0000 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fea44009eb0 tx=0x7fea44004a40 comp rx=0 tx=0).stop 2023-11-24T06:28:59.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 shutdown_connections 2023-11-24T06:28:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3006aed0 0x7fea3006d390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c1469e0 0x7fea4c0b0000 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea4c145e10 0x7fea4c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 --2- 172.21.15.16:0/530172893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea4c071170 0x7fea4c13b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:28:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 >> 172.21.15.16:0/530172893 conn(0x7fea4c0b0b00 msgr2=0x7fea4c0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:28:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 shutdown_connections 2023-11-24T06:28:59.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:28:59.726+0000 7fea52235640 1 -- 172.21.15.16:0/530172893 wait complete. 2023-11-24T06:28:59.749 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:28:59.749 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:28:59.749 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:28:59.749 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:59.749 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:28:59.750 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:28:59.750 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:28:59.750 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:59.750 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:28:59.750 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:28:59.750 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:28:59.751 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:59.751 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:28:59.751 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:28:59.751 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:28:59.751 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:28:59.767 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:28:59.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:28:59.768 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:28:59.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:28:59.773 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:28:59.773 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:28:59.777 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:28:59.777 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:28:59.816 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:28:59.817 DEBUG:teuthology.parallel:result is None 2023-11-24T06:28:59.823 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:28:59.823 DEBUG:teuthology.parallel:result is None 2023-11-24T06:28:59.825 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:28:59.826 DEBUG:teuthology.parallel:result is None 2023-11-24T06:28:59.833 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:28:59.833 DEBUG:teuthology.parallel:result is None 2023-11-24T06:28:59.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:28:59.983 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:28:59.987 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-24T06:28:59.996 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2023-11-24T06:28:59.997 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2023-11-24T06:29:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.036+0000 7f80a97fb640 1 Processor -- start 2023-11-24T06:29:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.036+0000 7f80a97fb640 1 -- start start 2023-11-24T06:29:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.037+0000 7f80a97fb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413df80 0x7f80a415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.037+0000 7f80a97fb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80a40cf9a0 0x7f80a40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.037+0000 7f80a97fb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a406c470 con 0x7f80a413e360 2023-11-24T06:29:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.037+0000 7f80a97fb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a406c750 con 0x7f80a40cf9a0 2023-11-24T06:29:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.037+0000 7f80a97fb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a406ca30 con 0x7f80a413df80 2023-11-24T06:29:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.038+0000 7f80a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80a40cf9a0 0x7f80a40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.038+0000 7f80a2ffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413df80 0x7f80a415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.038+0000 7f80a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80a40cf9a0 0x7f80a40cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60588/0 (socket says 172.21.15.16:60588) 2023-11-24T06:29:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.038+0000 7f80a37fe640 1 -- 172.21.15.16:0/1903912127 learned_addr learned my addr 172.21.15.16:0/1903912127 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.038+0000 7f80a37fe640 1 -- 172.21.15.16:0/1903912127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413df80 msgr2=0x7f80a415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.038+0000 7f80a37fe640 1 --2- 172.21.15.16:0/1903912127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413df80 0x7f80a415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.038+0000 7f80a37fe640 1 -- 172.21.15.16:0/1903912127 >> v1:172.21.15.173:6789/0 conn(0x7f80a413e360 legacy=0x7f80a40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.038+0000 7f80a37fe640 1 -- 172.21.15.16:0/1903912127 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a406cd10 con 0x7f80a40cf9a0 2023-11-24T06:29:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.038+0000 7f80a1ffb640 1 -- 172.21.15.16:0/1903912127 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863303482 0 0) 0x7f80a406c470 con 0x7f80a413e360 2023-11-24T06:29:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.039+0000 7f80a37fe640 1 --2- 172.21.15.16:0/1903912127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a40cf9a0 0x7f80a40cd840 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f809c009e50 tx=0x7f809c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36debb07d652df1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:00.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.039+0000 7f80a1ffb640 1 -- 172.21.15.16:0/1903912127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c02caf0 con 0x7f80a40cf9a0 2023-11-24T06:29:00.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a1ffb640 1 -- 172.21.15.16:0/1903912127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c02cc50 con 0x7f80a40cf9a0 2023-11-24T06:29:00.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a1ffb640 1 -- 172.21.15.16:0/1903912127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c0343f0 con 0x7f80a40cf9a0 2023-11-24T06:29:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a97fb640 1 -- 172.21.15.16:0/1903912127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a40cf9a0 msgr2=0x7f80a40cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a97fb640 1 --2- 172.21.15.16:0/1903912127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a40cf9a0 0x7f80a40cd840 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f809c009e50 tx=0x7f809c02bf50 comp rx=0 tx=0).stop 2023-11-24T06:29:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a97fb640 1 -- 172.21.15.16:0/1903912127 shutdown_connections 2023-11-24T06:29:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a97fb640 1 --2- 172.21.15.16:0/1903912127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a40cf9a0 0x7f80a40cd840 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a97fb640 1 --2- 172.21.15.16:0/1903912127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413df80 0x7f80a415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a97fb640 1 -- 172.21.15.16:0/1903912127 >> 172.21.15.16:0/1903912127 conn(0x7f80a406a800 msgr2=0x7f80a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:00.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a97fb640 1 -- 172.21.15.16:0/1903912127 shutdown_connections 2023-11-24T06:29:00.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.040+0000 7f80a97fb640 1 -- 172.21.15.16:0/1903912127 wait complete. 2023-11-24T06:29:00.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.042+0000 7f80a97fb640 1 Processor -- start 2023-11-24T06:29:00.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.042+0000 7f80a97fb640 1 -- start start 2023-11-24T06:29:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.043+0000 7f80a97fb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a40cf9a0 0x7f80a40b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.043+0000 7f80a97fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a413df80 0x7f80a40b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.043+0000 7f80a97fb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413e360 0x7f80a40b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.043+0000 7f80a97fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a4151f70 con 0x7f80a413df80 2023-11-24T06:29:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.043+0000 7f80a97fb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a41f2d20 con 0x7f80a40cf9a0 2023-11-24T06:29:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.043+0000 7f80a97fb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a414fb80 con 0x7f80a413e360 2023-11-24T06:29:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.043+0000 7f80a27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a413df80 0x7f80a40b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a2ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a40cf9a0 0x7f80a40b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a413df80 0x7f80a40b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60592/0 (socket says 172.21.15.16:60592) 2023-11-24T06:29:00.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a27fc640 1 -- 172.21.15.16:0/3429129388 learned_addr learned my addr 172.21.15.16:0/3429129388 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:00.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a37fe640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413e360 0x7f80a40b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a27fc640 1 -- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413e360 msgr2=0x7f80a40b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a27fc640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413e360 0x7f80a40b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a27fc640 1 -- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a40cf9a0 msgr2=0x7f80a40b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a27fc640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a40cf9a0 0x7f80a40b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a27fc640 1 -- 172.21.15.16:0/3429129388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c009b30 con 0x7f80a413df80 2023-11-24T06:29:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.044+0000 7f80a27fc640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a413df80 0x7f80a40b21e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f809800c4b0 tx=0x7f809800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:00.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.045+0000 7f808b7fe640 1 -- 172.21.15.16:0/3429129388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098005480 con 0x7f80a413df80 2023-11-24T06:29:00.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.045+0000 7f808b7fe640 1 -- 172.21.15.16:0/3429129388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80980035f0 con 0x7f80a413df80 2023-11-24T06:29:00.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.045+0000 7f808b7fe640 1 -- 172.21.15.16:0/3429129388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980039e0 con 0x7f80a413df80 2023-11-24T06:29:00.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.045+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a414fe60 con 0x7f80a413df80 2023-11-24T06:29:00.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.045+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a4151b90 con 0x7f80a413df80 2023-11-24T06:29:00.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.046+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4151710 con 0x7f80a413df80 2023-11-24T06:29:00.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.047+0000 7f808b7fe640 1 -- 172.21.15.16:0/3429129388 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8098003b40 con 0x7f80a413df80 2023-11-24T06:29:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.048+0000 7f808b7fe640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f809006aed0 0x7f809006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.048+0000 7f808b7fe640 1 -- 172.21.15.16:0/3429129388 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f80980888e0 con 0x7f80a413df80 2023-11-24T06:29:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.048+0000 7f80a2ffd640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f809006aed0 0x7f809006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:00.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.049+0000 7f80a2ffd640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f809006aed0 0x7f809006d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8094009b40 tx=0x7f8094009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:00.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.052+0000 7f808b7fe640 1 -- 172.21.15.16:0/3429129388 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80a4151710 con 0x7f80a413df80 2023-11-24T06:29:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:00.266+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f80a4151710 con 0x7f80a413df80 2023-11-24T06:29:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.266+0000 7f808b7fe640 1 -- 172.21.15.16:0/3429129388 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7f80a4151710 con 0x7f80a413df80 2023-11-24T06:29:01.268 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-11-24T06:29:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.269+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f809006aed0 msgr2=0x7f809006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.269+0000 7f80a97fb640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f809006aed0 0x7f809006d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8094009b40 tx=0x7f8094009190 comp rx=0 tx=0).stop 2023-11-24T06:29:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.269+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a413df80 msgr2=0x7f80a40b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.269+0000 7f80a97fb640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a413df80 0x7f80a40b21e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f809800c4b0 tx=0x7f809800c970 comp rx=0 tx=0).stop 2023-11-24T06:29:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.269+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 shutdown_connections 2023-11-24T06:29:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.269+0000 7f80a97fb640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f809006aed0 0x7f809006d390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.269+0000 7f80a97fb640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80a413e360 0x7f80a40b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.269+0000 7f80a97fb640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a413df80 0x7f80a40b21e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.269+0000 7f80a97fb640 1 --2- 172.21.15.16:0/3429129388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80a40cf9a0 0x7f80a40b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.270+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 >> 172.21.15.16:0/3429129388 conn(0x7f80a406a800 msgr2=0x7f80a40ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.270+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 shutdown_connections 2023-11-24T06:29:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.270+0000 7f80a97fb640 1 -- 172.21.15.16:0/3429129388 wait complete. 2023-11-24T06:29:01.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:29:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.518+0000 7f0bc69cb640 1 Processor -- start 2023-11-24T06:29:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.518+0000 7f0bc69cb640 1 -- start start 2023-11-24T06:29:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.520+0000 7f0bc69cb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 0x7f0bc00ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.520+0000 7f0bc69cb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0bc00adb00 0x7f0bc006c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.520+0000 7f0bc69cb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc00b3490 con 0x7f0bc0146850 2023-11-24T06:29:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.520+0000 7f0bc69cb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc00b3770 con 0x7f0bc00adb00 2023-11-24T06:29:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.520+0000 7f0bc69cb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc00b3a50 con 0x7f0bc0071ce0 2023-11-24T06:29:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.521+0000 7f0bc51c8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 0x7f0bc00ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.521+0000 7f0bc61ca640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0bc00adb00 0x7f0bc006c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.521+0000 7f0bc61ca640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0bc00adb00 0x7f0bc006c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:42370/0 (socket says 172.21.15.16:42370) 2023-11-24T06:29:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.521+0000 7f0bc61ca640 1 -- 172.21.15.16:0/3432974012 learned_addr learned my addr 172.21.15.16:0/3432974012 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.521+0000 7f0bc61ca640 1 -- 172.21.15.16:0/3432974012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 msgr2=0x7f0bc00ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.521+0000 7f0bc61ca640 1 --2- 172.21.15.16:0/3432974012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 0x7f0bc00ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.521+0000 7f0bc61ca640 1 -- 172.21.15.16:0/3432974012 >> v1:172.21.15.173:6789/0 conn(0x7f0bc0146850 legacy=0x7f0bc00715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.521+0000 7f0bc61ca640 1 -- 172.21.15.16:0/3432974012 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc00b3d30 con 0x7f0bc00adb00 2023-11-24T06:29:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.523+0000 7f0bc61ca640 1 --2- 172.21.15.16:0/3432974012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc00adb00 0x7f0bc006c470 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009e50 tx=0x7f0bb802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=287d20b0bbe67159 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.523+0000 7f0bc49c7640 1 -- 172.21.15.16:0/3432974012 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb802cb00 con 0x7f0bc00adb00 2023-11-24T06:29:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.523+0000 7f0bc49c7640 1 -- 172.21.15.16:0/3432974012 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb802cc60 con 0x7f0bc00adb00 2023-11-24T06:29:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.523+0000 7f0bc49c7640 1 -- 172.21.15.16:0/3432974012 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80343f0 con 0x7f0bc00adb00 2023-11-24T06:29:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.524+0000 7f0bc69cb640 1 -- 172.21.15.16:0/3432974012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc00adb00 msgr2=0x7f0bc006c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.524+0000 7f0bc69cb640 1 --2- 172.21.15.16:0/3432974012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc00adb00 0x7f0bc006c470 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009e50 tx=0x7f0bb802bf60 comp rx=0 tx=0).stop 2023-11-24T06:29:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.524+0000 7f0bc69cb640 1 -- 172.21.15.16:0/3432974012 shutdown_connections 2023-11-24T06:29:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.524+0000 7f0bc69cb640 1 --2- 172.21.15.16:0/3432974012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc00adb00 0x7f0bc006c470 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.524+0000 7f0bc69cb640 1 --2- 172.21.15.16:0/3432974012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 0x7f0bc00ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.524+0000 7f0bc69cb640 1 -- 172.21.15.16:0/3432974012 >> 172.21.15.16:0/3432974012 conn(0x7f0bc006a3e0 msgr2=0x7f0bc006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.524+0000 7f0bc69cb640 1 -- 172.21.15.16:0/3432974012 shutdown_connections 2023-11-24T06:29:01.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.524+0000 7f0bc69cb640 1 -- 172.21.15.16:0/3432974012 wait complete. 2023-11-24T06:29:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.526+0000 7f0bc69cb640 1 Processor -- start 2023-11-24T06:29:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.526+0000 7f0bc69cb640 1 -- start start 2023-11-24T06:29:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc69cb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 0x7f0bc0142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc69cb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc00adb00 0x7f0bc01583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc69cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc0146850 0x7f0bc013f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc69cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc013fcd0 con 0x7f0bc0146850 2023-11-24T06:29:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc69cb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc015bba0 con 0x7f0bc00adb00 2023-11-24T06:29:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc69cb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc015be80 con 0x7f0bc0071ce0 2023-11-24T06:29:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc61ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc0146850 0x7f0bc013f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc61ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc0146850 0x7f0bc013f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60600/0 (socket says 172.21.15.16:60600) 2023-11-24T06:29:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc61ca640 1 -- 172.21.15.16:0/2437820460 learned_addr learned my addr 172.21.15.16:0/2437820460 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc51c8640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc00adb00 0x7f0bc01583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.528+0000 7f0bc59c9640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 0x7f0bc0142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bc61ca640 1 -- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 msgr2=0x7f0bc0142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bc61ca640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 0x7f0bc0142900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bc61ca640 1 -- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc00adb00 msgr2=0x7f0bc01583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bc61ca640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc00adb00 0x7f0bc01583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bc61ca640 1 -- 172.21.15.16:0/2437820460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8009b30 con 0x7f0bc0146850 2023-11-24T06:29:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bc61ca640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc0146850 0x7f0bc013f790 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009e50 tx=0x7f0bb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bb67fc640 1 -- 172.21.15.16:0/2437820460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8009510 con 0x7f0bc0146850 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bb67fc640 1 -- 172.21.15.16:0/2437820460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb803e450 con 0x7f0bc0146850 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bb67fc640 1 -- 172.21.15.16:0/2437820460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb803e800 con 0x7f0bc0146850 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.529+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc00ae100 con 0x7f0bc0146850 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.530+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc015c440 con 0x7f0bc0146850 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.531+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc0067450 con 0x7f0bc0146850 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.531+0000 7f0bb67fc640 1 -- 172.21.15.16:0/2437820460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0bb803e960 con 0x7f0bc0146850 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.532+0000 7f0bb67fc640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba406aed0 0x7f0ba406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.532+0000 7f0bb67fc640 1 -- 172.21.15.16:0/2437820460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0bb80abf70 con 0x7f0bc0146850 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.533+0000 7f0bc59c9640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba406aed0 0x7f0ba406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.534+0000 7f0bc59c9640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba406aed0 0x7f0ba406d390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0bc01591e0 tx=0x7f0bbc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.537+0000 7f0bb67fc640 1 -- 172.21.15.16:0/2437820460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bb80dee90 con 0x7f0bc0146850 2023-11-24T06:29:01.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.796+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0bc015bce0 con 0x7f0bc0146850 2023-11-24T06:29:01.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.797+0000 7f0bb67fc640 1 -- 172.21.15.16:0/2437820460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0bc015bce0 con 0x7f0bc0146850 2023-11-24T06:29:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:01.800 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:29:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-11-24T06:29:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-11-24T06:29:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBl2seAMxAAL8jbBlU0yStjuIivg2ehGQ==", 2023-11-24T06:29:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.800 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBlbyW+GxAAy15inYIiNwnGVHj7Ghs5hg==", 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.801 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl79LhCRAALZnwoYyKju4aeWNS+jF31A==", 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.802 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl/NBDLBAAR4qYgD5THE/yNeZT9XgK5g==", 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.803 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-11-24T06:29:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD4QWBlarihIxAAczpFVQAODZogIKH0TTdMhw==", 2023-11-24T06:29:01.804 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD8QWBlITYFAhAAMlJyCTojY7WZihUegTcBcQ==", 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.805 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-11-24T06:29:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD/QWBlA/N0DRAAThU0hfAL0+MggufKZC4K8g==", 2023-11-24T06:29:01.806 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQACQmBlM7E3JxAAdmfhQHbjK/JGHgeD0urDZw==", 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.807 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-11-24T06:29:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAKQmBlrM97ARAApYaqyASXJqB5ITNLeBec/w==", 2023-11-24T06:29:01.808 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQANQmBl3s5dHBAA3spWOZQiqazGS6isl8ft3Q==", 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAQQmBlKVmeNhAAuV5j9qZ2NNvEePQx77EM6Q==", 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:01.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAUQmBlwADrBRAAvS3M6+H3X+OQD1h6xwhnBQ==", 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:01.811 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA==", 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:29:01.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBlKhfoFxAA8A+MObA/cjawPN7aclqS9Q==", 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:29:01.813 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBliwu9CxAAPcBaB9CWK2QqRD1k2x+4ig==", 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:29:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBl8HWRIhAAqzUXni0Uqgbmi/imvnNIZg==", 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDzQWBlo6bBIRAADWG8Peciy2ws4wHagrfFZw==", 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-11-24T06:29:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlz+KlJhAAXtiyVVA72YrbDKpNSBUnwA==", 2023-11-24T06:29:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:29:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:29:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-11-24T06:29:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlN4GDGhAAF9YIRqu011Xj6oWUv9LYlw==", 2023-11-24T06:29:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:29:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:29:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.818 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:01.819 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-11-24T06:29:01.819 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:29:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba406aed0 msgr2=0x7f0ba406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba406aed0 0x7f0ba406d390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0bc01591e0 tx=0x7f0bbc009040 comp rx=0 tx=0).stop 2023-11-24T06:29:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc0146850 msgr2=0x7f0bc013f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc0146850 0x7f0bc013f790 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009e50 tx=0x7f0bb8004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 shutdown_connections 2023-11-24T06:29:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba406aed0 0x7f0ba406d390 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc0146850 0x7f0bc013f790 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc00adb00 0x7f0bc01583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 --2- 172.21.15.16:0/2437820460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc0071ce0 0x7f0bc0142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:01.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 >> 172.21.15.16:0/2437820460 conn(0x7f0bc006a3e0 msgr2=0x7f0bc00afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:01.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 shutdown_connections 2023-11-24T06:29:01.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:01.800+0000 7f0bc69cb640 1 -- 172.21.15.16:0/2437820460 wait complete. 2023-11-24T06:29:01.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.048+0000 7f25a5aad640 1 Processor -- start 2023-11-24T06:29:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.049+0000 7f25a5aad640 1 -- start start 2023-11-24T06:29:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.050+0000 7f25a5aad640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f25a0160920 0x7f25a00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.050+0000 7f25a5aad640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a00d0a10 con 0x7f25a00cfea0 2023-11-24T06:29:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.050+0000 7f25a5aad640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a0161e30 con 0x7f25a00cf9f0 2023-11-24T06:29:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.050+0000 7f25a5aad640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a00d1f50 con 0x7f25a0160920 2023-11-24T06:29:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.051+0000 7f25a52ac640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25a00cfea0 0x7f25a0161720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58524/0 (socket says 172.21.15.16:58524) 2023-11-24T06:29:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.051+0000 7f25a52ac640 1 -- 172.21.15.16:0/4181566091 learned_addr learned my addr 172.21.15.16:0/4181566091 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.051+0000 7f25a4aab640 1 --2- 172.21.15.16:0/4181566091 >> v2:172.21.15.173:3300/0 conn(0x7f25a0160920 0x7f25a00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.051+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006303435 0 0) 0x7f25a00d0a10 con 0x7f25a00cfea0 2023-11-24T06:29:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.051+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2588003520 con 0x7f25a00cfea0 2023-11-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2806588417 0 0) 0x7f2588003520 con 0x7f25a00cfea0 2023-11-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 >> v2:172.21.15.173:3300/0 conn(0x7f25a0160920 msgr2=0x7f25a00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 --2- 172.21.15.16:0/4181566091 >> v2:172.21.15.173:3300/0 conn(0x7f25a0160920 0x7f25a00b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 >> v1:172.21.15.173:6789/0 conn(0x7f25a00cf9f0 legacy=0x7f25a0161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00cf080 con 0x7f25a00cfea0 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f25980019f0 con 0x7f25a00cfea0 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 >> v1:172.21.15.16:6789/0 conn(0x7f25a00cfea0 legacy=0x7f25a0161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2588003970 0x7f2588003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25880047c0 0x7f2588004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588005180 0x7f258800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00d1f50 con 0x7f25880047c0 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258800c020 con 0x7f2588005180 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258800c1e0 con 0x7f2588003970 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f259ffff640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25880047c0 0x7f2588004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.052+0000 7f25a4aab640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2588003970 0x7f2588003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.053+0000 7f25a52ac640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588005180 0x7f258800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.053+0000 7f259ffff640 1 -- 172.21.15.16:0/4181566091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2588003970 msgr2=0x7f2588003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.053+0000 7f259ffff640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2588003970 0x7f2588003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.053+0000 7f259ffff640 1 -- 172.21.15.16:0/4181566091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588005180 msgr2=0x7f258800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.053+0000 7f259ffff640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588005180 0x7f258800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.053+0000 7f259ffff640 1 -- 172.21.15.16:0/4181566091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588003520 con 0x7f25880047c0 2023-11-24T06:29:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.053+0000 7f25a52ac640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2588005180 0x7f258800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.053+0000 7f25a52ac640 1 -- 172.21.15.16:0/4181566091 reap_dead start 2023-11-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.054+0000 7f259ffff640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25880047c0 0x7f2588004c40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2590001750 tx=0x7f25880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e98809ec220d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.054+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25900052d0 con 0x7f25880047c0 2023-11-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.054+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259002e050 con 0x7f25880047c0 2023-11-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.054+0000 7f259f7fe640 1 -- 172.21.15.16:0/4181566091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590003830 con 0x7f25880047c0 2023-11-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.054+0000 7f25a5aad640 1 -- 172.21.15.16:0/4181566091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25880047c0 msgr2=0x7f2588004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.054+0000 7f25a5aad640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25880047c0 0x7f2588004c40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2590001750 tx=0x7f25880057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.055+0000 7f25a5aad640 1 -- 172.21.15.16:0/4181566091 shutdown_connections 2023-11-24T06:29:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.055+0000 7f25a5aad640 1 --2- 172.21.15.16:0/4181566091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25880047c0 0x7f2588004c40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.055+0000 7f25a5aad640 1 -- 172.21.15.16:0/4181566091 >> 172.21.15.16:0/4181566091 conn(0x7f25a00d3090 msgr2=0x7f25a00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.055+0000 7f25a5aad640 1 -- 172.21.15.16:0/4181566091 shutdown_connections 2023-11-24T06:29:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.055+0000 7f25a5aad640 1 -- 172.21.15.16:0/4181566091 wait complete. 2023-11-24T06:29:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.056+0000 7f25a5aad640 1 Processor -- start 2023-11-24T06:29:02.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.057+0000 7f25a5aad640 1 -- start start 2023-11-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a5aad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25880047c0 0x7f25a0150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a5aad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a00cab90 0x7f25a00b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a5aad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a00cc170 0x7f25a00c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a5aad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00cb010 con 0x7f25a00cc170 2023-11-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a5aad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0160090 con 0x7f25880047c0 2023-11-24T06:29:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a5aad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a00b6190 con 0x7f25a00cab90 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a52ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a00cc170 0x7f25a00c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a52ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a00cc170 0x7f25a00c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60612/0 (socket says 172.21.15.16:60612) 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a52ac640 1 -- 172.21.15.16:0/2439113652 learned_addr learned my addr 172.21.15.16:0/2439113652 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f259ffff640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a00cab90 0x7f25a00b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a4aab640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25880047c0 0x7f25a0150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a52ac640 1 -- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a00cab90 msgr2=0x7f25a00b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a52ac640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a00cab90 0x7f25a00b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a52ac640 1 -- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25880047c0 msgr2=0x7f25a0150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a52ac640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25880047c0 0x7f25a0150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.058+0000 7f25a52ac640 1 -- 172.21.15.16:0/2439113652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a00b6430 con 0x7f25a00cc170 2023-11-24T06:29:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.059+0000 7f25a4aab640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25880047c0 0x7f25a0150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.059+0000 7f25a52ac640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a00cc170 0x7f25a00c8f50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f25a00d3060 tx=0x7f259800df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.059+0000 7f259d7fa640 1 -- 172.21.15.16:0/2439113652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800b860 con 0x7f25a00cc170 2023-11-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.059+0000 7f259d7fa640 1 -- 172.21.15.16:0/2439113652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259800b9c0 con 0x7f25a00cc170 2023-11-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.059+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a00b6710 con 0x7f25a00cc170 2023-11-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.059+0000 7f259d7fa640 1 -- 172.21.15.16:0/2439113652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800bd50 con 0x7f25a00cc170 2023-11-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.059+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a00cbc50 con 0x7f25a00cc170 2023-11-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.061+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a0067450 con 0x7f25a00cc170 2023-11-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.062+0000 7f259d7fa640 1 -- 172.21.15.16:0/2439113652 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2598007110 con 0x7f25a00cc170 2023-11-24T06:29:02.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.063+0000 7f259d7fa640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f258c06af50 0x7f258c06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.063+0000 7f259d7fa640 1 -- 172.21.15.16:0/2439113652 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25980895c0 con 0x7f25a00cc170 2023-11-24T06:29:02.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.063+0000 7f25a4aab640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f258c06af50 0x7f258c06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.064+0000 7f25a4aab640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f258c06af50 0x7f258c06d410 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f25a00b75e0 tx=0x7f2594008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.068+0000 7f259d7fa640 1 -- 172.21.15.16:0/2439113652 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25980549d0 con 0x7f25a00cc170 2023-11-24T06:29:02.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.264+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25a008c7e0 con 0x7f25a00cc170 2023-11-24T06:29:02.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.266+0000 7f259d7fa640 1 -- 172.21.15.16:0/2439113652 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12857 (secure 0 0 0) 0x7f2598057ff0 con 0x7f25a00cc170 2023-11-24T06:29:02.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:02.269 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:01.263088+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.269+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f258c06af50 msgr2=0x7f258c06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.269+0000 7f25a5aad640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f258c06af50 0x7f258c06d410 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f25a00b75e0 tx=0x7f2594008040 comp rx=0 tx=0).stop 2023-11-24T06:29:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.269+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a00cc170 msgr2=0x7f25a00c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.269+0000 7f25a5aad640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a00cc170 0x7f25a00c8f50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f25a00d3060 tx=0x7f259800df60 comp rx=0 tx=0).stop 2023-11-24T06:29:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.270+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 shutdown_connections 2023-11-24T06:29:02.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.270+0000 7f25a5aad640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f258c06af50 0x7f258c06d410 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.270+0000 7f25a5aad640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a00cc170 0x7f25a00c8f50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.270+0000 7f25a5aad640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25a00cab90 0x7f25a00b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.270+0000 7f25a5aad640 1 --2- 172.21.15.16:0/2439113652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25880047c0 0x7f25a0150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.270+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 >> 172.21.15.16:0/2439113652 conn(0x7f25a00d3090 msgr2=0x7f25a015afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.270+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 shutdown_connections 2023-11-24T06:29:02.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.270+0000 7f25a5aad640 1 -- 172.21.15.16:0/2439113652 wait complete. 2023-11-24T06:29:02.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:02.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.517+0000 7fc4ab3e3640 1 Processor -- start 2023-11-24T06:29:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.518+0000 7fc4ab3e3640 1 -- start start 2023-11-24T06:29:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.519+0000 7fc4ab3e3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4a4072230 0x7fc4a4150760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.519+0000 7fc4ab3e3640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc4a40cf290 0x7fc4a415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.519+0000 7fc4ab3e3640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a4071190 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.519+0000 7fc4ab3e3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a4072610 con 0x7fc4a4072230 2023-11-24T06:29:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.519+0000 7fc4ab3e3640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a4071470 con 0x7fc4a40cf290 2023-11-24T06:29:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.520+0000 7fc4a917f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4a4072230 0x7fc4a4150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.520+0000 7fc4a917f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4a4072230 0x7fc4a4150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60620/0 (socket says 172.21.15.16:60620) 2023-11-24T06:29:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.520+0000 7fc4a917f640 1 -- 172.21.15.16:0/2444361878 learned_addr learned my addr 172.21.15.16:0/2444361878 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.520+0000 7fc4a897e640 1 --2- 172.21.15.16:0/2444361878 >> v2:172.21.15.173:3300/0 conn(0x7fc4a40cf290 0x7fc4a415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.520+0000 7fc4a917f640 1 -- 172.21.15.16:0/2444361878 >> v2:172.21.15.173:3300/0 conn(0x7fc4a40cf290 msgr2=0x7fc4a415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.520+0000 7fc4a917f640 1 --2- 172.21.15.16:0/2444361878 >> v2:172.21.15.173:3300/0 conn(0x7fc4a40cf290 0x7fc4a415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.520+0000 7fc4a917f640 1 -- 172.21.15.16:0/2444361878 >> v1:172.21.15.173:6789/0 conn(0x7fc4a40cf6d0 legacy=0x7fc4a40b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.520+0000 7fc4a917f640 1 -- 172.21.15.16:0/2444361878 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a4071750 con 0x7fc4a4072230 2023-11-24T06:29:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.522+0000 7fc4a917f640 1 --2- 172.21.15.16:0/2444361878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a4072230 0x7fc4a4150760 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40b1ca0 tx=0x7fc4a002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ddc6ee10e8f874 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.522+0000 7fc49bfff640 1 -- 172.21.15.16:0/2444361878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a002ca80 con 0x7fc4a4072230 2023-11-24T06:29:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.522+0000 7fc49bfff640 1 -- 172.21.15.16:0/2444361878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a002cbe0 con 0x7fc4a4072230 2023-11-24T06:29:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.522+0000 7fc49bfff640 1 -- 172.21.15.16:0/2444361878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a00343f0 con 0x7fc4a4072230 2023-11-24T06:29:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.522+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/2444361878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a4072230 msgr2=0x7fc4a4150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.523+0000 7fc4ab3e3640 1 --2- 172.21.15.16:0/2444361878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a4072230 0x7fc4a4150760 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40b1ca0 tx=0x7fc4a002bee0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.523+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/2444361878 shutdown_connections 2023-11-24T06:29:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.523+0000 7fc4ab3e3640 1 --2- 172.21.15.16:0/2444361878 >> v2:172.21.15.173:3300/0 conn(0x7fc4a40cf290 0x7fc4a415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.523+0000 7fc4ab3e3640 1 --2- 172.21.15.16:0/2444361878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a4072230 0x7fc4a4150760 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.523+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/2444361878 >> 172.21.15.16:0/2444361878 conn(0x7fc4a406a800 msgr2=0x7fc4a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.523+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/2444361878 shutdown_connections 2023-11-24T06:29:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.523+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/2444361878 wait complete. 2023-11-24T06:29:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.525+0000 7fc4ab3e3640 1 Processor -- start 2023-11-24T06:29:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.525+0000 7fc4ab3e3640 1 -- start start 2023-11-24T06:29:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4ab3e3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a4072230 0x7fc4a41523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4ab3e3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4a40cf290 0x7fc4a41454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4ab3e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a40cf6d0 0x7fc4a41459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4ab3e3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a406ce40 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4ab3e3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a406d370 con 0x7fc4a4072230 2023-11-24T06:29:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4ab3e3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a41512a0 con 0x7fc4a40cf290 2023-11-24T06:29:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4a9980640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a40cf6d0 0x7fc4a41459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4a9980640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a40cf6d0 0x7fc4a41459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60624/0 (socket says 172.21.15.16:60624) 2023-11-24T06:29:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4a9980640 1 -- 172.21.15.16:0/1833552838 learned_addr learned my addr 172.21.15.16:0/1833552838 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4a917f640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a4072230 0x7fc4a41523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.526+0000 7fc4a897e640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4a40cf290 0x7fc4a41454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc4a9980640 1 -- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4a40cf290 msgr2=0x7fc4a41454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc4a9980640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4a40cf290 0x7fc4a41454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc4a9980640 1 -- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a4072230 msgr2=0x7fc4a41523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc4a9980640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a4072230 0x7fc4a41523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc4a9980640 1 -- 172.21.15.16:0/1833552838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a0009b30 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc4a9980640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a40cf6d0 0x7fc4a41459e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00b410 tx=0x7fc49c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc499ffb640 1 -- 172.21.15.16:0/1833552838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c004020 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc499ffb640 1 -- 172.21.15.16:0/1833552838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c004180 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc499ffb640 1 -- 172.21.15.16:0/1833552838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c011630 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.527+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a406d090 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.528+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a4151820 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.529+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a40675d0 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.530+0000 7fc499ffb640 1 -- 172.21.15.16:0/1833552838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc49c011790 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.531+0000 7fc499ffb640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48806aed0 0x7fc48806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.531+0000 7fc499ffb640 1 -- 172.21.15.16:0/1833552838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc49c088ff0 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.532+0000 7fc4a917f640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48806aed0 0x7fc48806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.533+0000 7fc4a917f640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48806aed0 0x7fc48806d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0035000 tx=0x7fc4a0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:02.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.536+0000 7fc499ffb640 1 -- 172.21.15.16:0/1833552838 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc49c0bbe90 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.729+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4a415c850 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.730+0000 7fc499ffb640 1 -- 172.21.15.16:0/1833552838 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc49c0543b0 con 0x7fc4a40cf6d0 2023-11-24T06:29:02.733 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:29:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48806aed0 msgr2=0x7fc48806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48806aed0 0x7fc48806d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0035000 tx=0x7fc4a0030000 comp rx=0 tx=0).stop 2023-11-24T06:29:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a40cf6d0 msgr2=0x7fc4a41459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a40cf6d0 0x7fc4a41459e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00b410 tx=0x7fc49c00b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 shutdown_connections 2023-11-24T06:29:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc48806aed0 0x7fc48806d390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a40cf6d0 0x7fc4a41459e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4a40cf290 0x7fc4a41454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 --2- 172.21.15.16:0/1833552838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a4072230 0x7fc4a41523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 >> 172.21.15.16:0/1833552838 conn(0x7fc4a406a800 msgr2=0x7fc4a4159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 shutdown_connections 2023-11-24T06:29:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.733+0000 7fc4ab3e3640 1 -- 172.21.15.16:0/1833552838 wait complete. 2023-11-24T06:29:02.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:29:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.974+0000 7f0f27fff640 1 Processor -- start 2023-11-24T06:29:02.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.974+0000 7f0f27fff640 1 -- start start 2023-11-24T06:29:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.976+0000 7f0f27fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0f2813cfa0 0x7f0f2815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.976+0000 7f0f27fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f2813fcb0 0x7f0f280b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f27fff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f280cdec0 con 0x7f0f280cd530 2023-11-24T06:29:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f27fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f280ce1a0 con 0x7f0f2813fcb0 2023-11-24T06:29:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f27fff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f280ce480 con 0x7f0f2813cfa0 2023-11-24T06:29:02.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f277fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f2813fcb0 0x7f0f280b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f277fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f2813fcb0 0x7f0f280b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33974/0 (socket says 172.21.15.16:33974) 2023-11-24T06:29:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f277fe640 1 -- 172.21.15.16:0/1185996056 learned_addr learned my addr 172.21.15.16:0/1185996056 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f26ffd640 1 --2- 172.21.15.16:0/1185996056 >> v2:172.21.15.173:3300/0 conn(0x7f0f2813cfa0 0x7f0f2815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f277fe640 1 -- 172.21.15.16:0/1185996056 >> v2:172.21.15.173:3300/0 conn(0x7f0f2813cfa0 msgr2=0x7f0f2815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f277fe640 1 --2- 172.21.15.16:0/1185996056 >> v2:172.21.15.173:3300/0 conn(0x7f0f2813cfa0 0x7f0f2815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f277fe640 1 -- 172.21.15.16:0/1185996056 >> v1:172.21.15.173:6789/0 conn(0x7f0f280cd530 legacy=0x7f0f2813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.977+0000 7f0f277fe640 1 -- 172.21.15.16:0/1185996056 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280ce760 con 0x7f0f2813fcb0 2023-11-24T06:29:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.979+0000 7f0f277fe640 1 --2- 172.21.15.16:0/1185996056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813fcb0 0x7f0f280b2040 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0f2806cf00 tx=0x7f0f2002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b675f279192eca53 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.979+0000 7f0f267fc640 1 -- 172.21.15.16:0/1185996056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2002c9f0 con 0x7f0f2813fcb0 2023-11-24T06:29:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.979+0000 7f0f267fc640 1 -- 172.21.15.16:0/1185996056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2002cb50 con 0x7f0f2813fcb0 2023-11-24T06:29:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.979+0000 7f0f267fc640 1 -- 172.21.15.16:0/1185996056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f200343f0 con 0x7f0f2813fcb0 2023-11-24T06:29:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.980+0000 7f0f27fff640 1 -- 172.21.15.16:0/1185996056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813fcb0 msgr2=0x7f0f280b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.980+0000 7f0f27fff640 1 --2- 172.21.15.16:0/1185996056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813fcb0 0x7f0f280b2040 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0f2806cf00 tx=0x7f0f2002be50 comp rx=0 tx=0).stop 2023-11-24T06:29:02.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.980+0000 7f0f27fff640 1 -- 172.21.15.16:0/1185996056 shutdown_connections 2023-11-24T06:29:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.980+0000 7f0f27fff640 1 --2- 172.21.15.16:0/1185996056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813fcb0 0x7f0f280b2040 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.980+0000 7f0f27fff640 1 --2- 172.21.15.16:0/1185996056 >> v2:172.21.15.173:3300/0 conn(0x7f0f2813cfa0 0x7f0f2815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.980+0000 7f0f27fff640 1 -- 172.21.15.16:0/1185996056 >> 172.21.15.16:0/1185996056 conn(0x7f0f2806a800 msgr2=0x7f0f2806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.980+0000 7f0f27fff640 1 -- 172.21.15.16:0/1185996056 shutdown_connections 2023-11-24T06:29:02.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.980+0000 7f0f27fff640 1 -- 172.21.15.16:0/1185996056 wait complete. 2023-11-24T06:29:02.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.982+0000 7f0f27fff640 1 Processor -- start 2023-11-24T06:29:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.983+0000 7f0f27fff640 1 -- start start 2023-11-24T06:29:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f280cd530 0x7f0f28152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813cfa0 0x7f0f281534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f27fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f2813fcb0 0x7f0f28153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f27fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28136d60 con 0x7f0f2813cfa0 2023-11-24T06:29:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f27fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f281589f0 con 0x7f0f280cd530 2023-11-24T06:29:02.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f27fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f281580c0 con 0x7f0f2813fcb0 2023-11-24T06:29:02.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813cfa0 0x7f0f281534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f1e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813cfa0 0x7f0f281534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33986/0 (socket says 172.21.15.16:33986) 2023-11-24T06:29:02.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f1e7fc640 1 -- 172.21.15.16:0/987553535 learned_addr learned my addr 172.21.15.16:0/987553535 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:02.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.985+0000 7f0f277fe640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f2813fcb0 0x7f0f28153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.986+0000 7f0f26ffd640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f280cd530 0x7f0f28152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.986+0000 7f0f1e7fc640 1 -- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f2813fcb0 msgr2=0x7f0f28153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.986+0000 7f0f1e7fc640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f2813fcb0 0x7f0f28153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.986+0000 7f0f1e7fc640 1 -- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f280cd530 msgr2=0x7f0f28152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:02.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.986+0000 7f0f1e7fc640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f280cd530 0x7f0f28152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:02.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.986+0000 7f0f1e7fc640 1 -- 172.21.15.16:0/987553535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20009b30 con 0x7f0f2813cfa0 2023-11-24T06:29:02.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.986+0000 7f0f1e7fc640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813cfa0 0x7f0f281534e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c430 tx=0x7f0f1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:02.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.986+0000 7f0f1ffff640 1 -- 172.21.15.16:0/987553535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10011790 con 0x7f0f2813cfa0 2023-11-24T06:29:02.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.987+0000 7f0f1ffff640 1 -- 172.21.15.16:0/987553535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f100118f0 con 0x7f0f2813cfa0 2023-11-24T06:29:02.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.987+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f281583a0 con 0x7f0f2813cfa0 2023-11-24T06:29:02.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.987+0000 7f0f1ffff640 1 -- 172.21.15.16:0/987553535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10011ca0 con 0x7f0f2813cfa0 2023-11-24T06:29:02.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.987+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f28154cf0 con 0x7f0f2813cfa0 2023-11-24T06:29:02.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.988+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f28158b60 con 0x7f0f2813cfa0 2023-11-24T06:29:02.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.989+0000 7f0f1ffff640 1 -- 172.21.15.16:0/987553535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0f1001c460 con 0x7f0f2813cfa0 2023-11-24T06:29:02.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.990+0000 7f0f1ffff640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f0806af20 0x7f0f0806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:02.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.990+0000 7f0f1ffff640 1 -- 172.21.15.16:0/987553535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0f10089290 con 0x7f0f2813cfa0 2023-11-24T06:29:02.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.990+0000 7f0f26ffd640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f0806af20 0x7f0f0806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:02.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.992+0000 7f0f26ffd640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f0806af20 0x7f0f0806d3e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:02.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:02.994+0000 7f0f1ffff640 1 -- 172.21.15.16:0/987553535 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f10054620 con 0x7f0f2813cfa0 2023-11-24T06:29:03.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.192+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0f28067450 con 0x7f0f2813cfa0 2023-11-24T06:29:03.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.193+0000 7f0f1ffff640 1 -- 172.21.15.16:0/987553535 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0f1005e020 con 0x7f0f2813cfa0 2023-11-24T06:29:03.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:03.195 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:29:03.195 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:03.196 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-24T06:29:03.196 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-24T06:29:03.196 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:03.196 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:29:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f0806af20 msgr2=0x7f0f0806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f0806af20 0x7f0f0806d3e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18008040 comp rx=0 tx=0).stop 2023-11-24T06:29:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813cfa0 msgr2=0x7f0f281534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813cfa0 0x7f0f281534e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000c430 tx=0x7f0f1000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 shutdown_connections 2023-11-24T06:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f0806af20 0x7f0f0806d3e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f2813fcb0 0x7f0f28153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2813cfa0 0x7f0f281534e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 --2- 172.21.15.16:0/987553535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f280cd530 0x7f0f28152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 >> 172.21.15.16:0/987553535 conn(0x7f0f2806a800 msgr2=0x7f0f280b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 shutdown_connections 2023-11-24T06:29:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.196+0000 7f0f27fff640 1 -- 172.21.15.16:0/987553535 wait complete. 2023-11-24T06:29:03.217 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:29:03.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:29:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.443+0000 7fa15f8bb640 1 Processor -- start 2023-11-24T06:29:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.443+0000 7fa15f8bb640 1 -- start start 2023-11-24T06:29:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.445+0000 7fa15f8bb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa1580cf9a0 0x7fa158166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.445+0000 7fa15f8bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15815c5f0 con 0x7fa15813e360 2023-11-24T06:29:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.445+0000 7fa15f8bb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15806c470 con 0x7fa15813df80 2023-11-24T06:29:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.445+0000 7fa15f8bb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15806c750 con 0x7fa1580cf9a0 2023-11-24T06:29:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.445+0000 7fa15ce56640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa15813e360 0x7fa1580cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48644/0 (socket says 172.21.15.16:48644) 2023-11-24T06:29:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.446+0000 7fa15ce56640 1 -- 172.21.15.16:0/1129239105 learned_addr learned my addr 172.21.15.16:0/1129239105 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.446+0000 7fa15de58640 1 --2- 172.21.15.16:0/1129239105 >> v2:172.21.15.173:3300/0 conn(0x7fa1580cf9a0 0x7fa158166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.446+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220135715 0 0) 0x7fa15815c5f0 con 0x7fa15813e360 2023-11-24T06:29:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.446+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa144003520 con 0x7fa15813e360 2023-11-24T06:29:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.446+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475200045 0 0) 0x7fa15806c470 con 0x7fa15813df80 2023-11-24T06:29:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.446+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa15815c5f0 con 0x7fa15813df80 2023-11-24T06:29:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.446+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4286189849 0 0) 0x7fa144003520 con 0x7fa15813e360 2023-11-24T06:29:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 >> v2:172.21.15.173:3300/0 conn(0x7fa1580cf9a0 msgr2=0x7fa158166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 --2- 172.21.15.16:0/1129239105 >> v2:172.21.15.173:3300/0 conn(0x7fa1580cf9a0 0x7fa158166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 >> v1:172.21.15.173:6789/0 conn(0x7fa15813df80 legacy=0x7fa15815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15806ca30 con 0x7fa15813e360 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fa148001670 con 0x7fa15813e360 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 >> v1:172.21.15.16:6789/0 conn(0x7fa15813e360 legacy=0x7fa1580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa144003b50 0x7fa144003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1440047c0 0x7fa144004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144005180 0x7fa14400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15806c750 con 0x7fa144003b50 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14400c050 con 0x7fa144005180 2023-11-24T06:29:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14400c210 con 0x7fa1440047c0 2023-11-24T06:29:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa15d657640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa144003b50 0x7fa144003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa15de58640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144005180 0x7fa14400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.447+0000 7fa15ce56640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1440047c0 0x7fa144004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.448+0000 7fa15d657640 1 -- 172.21.15.16:0/1129239105 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1440047c0 msgr2=0x7fa144004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.448+0000 7fa15d657640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1440047c0 0x7fa144004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.448+0000 7fa15d657640 1 -- 172.21.15.16:0/1129239105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144005180 msgr2=0x7fa14400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.448+0000 7fa15d657640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144005180 0x7fa14400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.448+0000 7fa15d657640 1 -- 172.21.15.16:0/1129239105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa144003520 con 0x7fa144003b50 2023-11-24T06:29:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.448+0000 7fa15de58640 1 -- 172.21.15.16:0/1129239105 reap_dead start 2023-11-24T06:29:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.448+0000 7fa15ce56640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1440047c0 0x7fa144004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.449+0000 7fa15d657640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa144003b50 0x7fa144003f50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa1540018d0 tx=0x7fa1440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b45520adb3faa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.449+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540052d0 con 0x7fa144003b50 2023-11-24T06:29:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.449+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa154005430 con 0x7fa144003b50 2023-11-24T06:29:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.449+0000 7fa14ffff640 1 -- 172.21.15.16:0/1129239105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154003810 con 0x7fa144003b50 2023-11-24T06:29:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.450+0000 7fa15f8bb640 1 -- 172.21.15.16:0/1129239105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa144003b50 msgr2=0x7fa144003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.450+0000 7fa15f8bb640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa144003b50 0x7fa144003f50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa1540018d0 tx=0x7fa1440057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.450+0000 7fa15f8bb640 1 -- 172.21.15.16:0/1129239105 shutdown_connections 2023-11-24T06:29:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.450+0000 7fa15f8bb640 1 --2- 172.21.15.16:0/1129239105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa144003b50 0x7fa144003f50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.450+0000 7fa15f8bb640 1 -- 172.21.15.16:0/1129239105 >> 172.21.15.16:0/1129239105 conn(0x7fa15806a800 msgr2=0x7fa15806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.450+0000 7fa15f8bb640 1 -- 172.21.15.16:0/1129239105 shutdown_connections 2023-11-24T06:29:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.450+0000 7fa15f8bb640 1 -- 172.21.15.16:0/1129239105 wait complete. 2023-11-24T06:29:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.452+0000 7fa15f8bb640 1 Processor -- start 2023-11-24T06:29:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.452+0000 7fa15f8bb640 1 -- start start 2023-11-24T06:29:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.453+0000 7fa15f8bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144003b50 0x7fa1581510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.453+0000 7fa15f8bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1580705d0 0x7fa15813d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.453+0000 7fa15f8bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1580713a0 0x7fa158071820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.453+0000 7fa15f8bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158070f50 con 0x7fa1580705d0 2023-11-24T06:29:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.453+0000 7fa15f8bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158154350 con 0x7fa144003b50 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.453+0000 7fa15f8bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa158154630 con 0x7fa1580713a0 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.453+0000 7fa15ce56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1580705d0 0x7fa15813d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.453+0000 7fa15ce56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1580705d0 0x7fa15813d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34010/0 (socket says 172.21.15.16:34010) 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15ce56640 1 -- 172.21.15.16:0/504925002 learned_addr learned my addr 172.21.15.16:0/504925002 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15d657640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144003b50 0x7fa1581510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15de58640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1580713a0 0x7fa158071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15ce56640 1 -- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1580713a0 msgr2=0x7fa158071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15ce56640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1580713a0 0x7fa158071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15ce56640 1 -- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144003b50 msgr2=0x7fa1581510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15ce56640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144003b50 0x7fa1581510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15ce56640 1 -- 172.21.15.16:0/504925002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa158154910 con 0x7fa1580705d0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15de58640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1580713a0 0x7fa158071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.454+0000 7fa15ce56640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1580705d0 0x7fa15813d7d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa14800d7a0 tx=0x7fa14800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.455+0000 7fa14dffb640 1 -- 172.21.15.16:0/504925002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1480072d0 con 0x7fa1580705d0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.455+0000 7fa14dffb640 1 -- 172.21.15.16:0/504925002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa148007430 con 0x7fa1580705d0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.455+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa158154bf0 con 0x7fa1580705d0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.455+0000 7fa14dffb640 1 -- 172.21.15.16:0/504925002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1480047e0 con 0x7fa1580705d0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.455+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15813dd10 con 0x7fa1580705d0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.456+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1580675d0 con 0x7fa1580705d0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.457+0000 7fa14dffb640 1 -- 172.21.15.16:0/504925002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa148004940 con 0x7fa1580705d0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.458+0000 7fa14dffb640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa13c06af00 0x7fa13c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.458+0000 7fa14dffb640 1 -- 172.21.15.16:0/504925002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa148088cd0 con 0x7fa1580705d0 2023-11-24T06:29:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.459+0000 7fa15d657640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa13c06af00 0x7fa13c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.460+0000 7fa15d657640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa13c06af00 0x7fa13c06d3c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa154034000 tx=0x7fa154031000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.462+0000 7fa14dffb640 1 -- 172.21.15.16:0/504925002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa148053ed0 con 0x7fa1580705d0 2023-11-24T06:29:03.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:03.653+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa15806df00 con 0x7fa1580705d0 2023-11-24T06:29:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.267+0000 7fa14dffb640 1 -- 172.21.15.16:0/504925002 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa148019020 con 0x7fa1580705d0 2023-11-24T06:29:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-24T06:29:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa13c06af00 msgr2=0x7fa13c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa13c06af00 0x7fa13c06d3c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa154034000 tx=0x7fa154031000 comp rx=0 tx=0).stop 2023-11-24T06:29:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1580705d0 msgr2=0x7fa15813d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1580705d0 0x7fa15813d7d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa14800d7a0 tx=0x7fa14800dc60 comp rx=0 tx=0).stop 2023-11-24T06:29:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 shutdown_connections 2023-11-24T06:29:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa13c06af00 0x7fa13c06d3c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa1580713a0 0x7fa158071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1580705d0 0x7fa15813d7d0 secure :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa14800d7a0 tx=0x7fa14800dc60 comp rx=0 tx=0).stop 2023-11-24T06:29:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 --2- 172.21.15.16:0/504925002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144003b50 0x7fa1581510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.272+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 >> 172.21.15.16:0/504925002 conn(0x7fa15806a800 msgr2=0x7fa1580b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:04.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.273+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 shutdown_connections 2023-11-24T06:29:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.273+0000 7fa15f8bb640 1 -- 172.21.15.16:0/504925002 wait complete. 2023-11-24T06:29:04.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:29:04.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.527+0000 7f727bfff640 1 Processor -- start 2023-11-24T06:29:04.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.527+0000 7f727bfff640 1 -- start start 2023-11-24T06:29:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.529+0000 7f727bfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f727c143c40 0x7f727c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.529+0000 7f727bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f727c0717d0 0x7f727c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.529+0000 7f727bfff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727c070e50 con 0x7f727c0cca40 2023-11-24T06:29:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.529+0000 7f727bfff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727c0723c0 con 0x7f727c0717d0 2023-11-24T06:29:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.529+0000 7f727bfff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727c0cce40 con 0x7f727c143c40 2023-11-24T06:29:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.529+0000 7f7273fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f727c0717d0 0x7f727c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.530+0000 7f7273fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f727c0717d0 0x7f727c0cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34018/0 (socket says 172.21.15.16:34018) 2023-11-24T06:29:04.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.530+0000 7f7273fff640 1 -- 172.21.15.16:0/3240594134 learned_addr learned my addr 172.21.15.16:0/3240594134 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.530+0000 7f7273fff640 1 -- 172.21.15.16:0/3240594134 >> v2:172.21.15.173:3300/0 conn(0x7f727c143c40 msgr2=0x7f727c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.530+0000 7f7273fff640 1 --2- 172.21.15.16:0/3240594134 >> v2:172.21.15.173:3300/0 conn(0x7f727c143c40 0x7f727c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.530+0000 7f7273fff640 1 -- 172.21.15.16:0/3240594134 >> v1:172.21.15.173:6789/0 conn(0x7f727c0cca40 legacy=0x7f727c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.530+0000 7f7273fff640 1 -- 172.21.15.16:0/3240594134 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c06e170 con 0x7f727c0717d0 2023-11-24T06:29:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.531+0000 7f7273fff640 1 --2- 172.21.15.16:0/3240594134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 0x7f727c0cc500 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f727c06e410 tx=0x7f726402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fc9d10a1672a2c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.531+0000 7f727a7fc640 1 -- 172.21.15.16:0/3240594134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72640317e0 con 0x7f727c0717d0 2023-11-24T06:29:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.531+0000 7f727a7fc640 1 -- 172.21.15.16:0/3240594134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7264031940 con 0x7f727c0717d0 2023-11-24T06:29:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.531+0000 7f727a7fc640 1 -- 172.21.15.16:0/3240594134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264031cd0 con 0x7f727c0717d0 2023-11-24T06:29:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.532+0000 7f727bfff640 1 -- 172.21.15.16:0/3240594134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 msgr2=0x7f727c0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.532+0000 7f727bfff640 1 --2- 172.21.15.16:0/3240594134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 0x7f727c0cc500 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f727c06e410 tx=0x7f726402b880 comp rx=0 tx=0).stop 2023-11-24T06:29:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.532+0000 7f727bfff640 1 -- 172.21.15.16:0/3240594134 shutdown_connections 2023-11-24T06:29:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.532+0000 7f727bfff640 1 --2- 172.21.15.16:0/3240594134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 0x7f727c0cc500 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.532+0000 7f727bfff640 1 --2- 172.21.15.16:0/3240594134 >> v2:172.21.15.173:3300/0 conn(0x7f727c143c40 0x7f727c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.532+0000 7f727bfff640 1 -- 172.21.15.16:0/3240594134 >> 172.21.15.16:0/3240594134 conn(0x7f727c06a800 msgr2=0x7f727c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.532+0000 7f727bfff640 1 -- 172.21.15.16:0/3240594134 shutdown_connections 2023-11-24T06:29:04.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.532+0000 7f727bfff640 1 -- 172.21.15.16:0/3240594134 wait complete. 2023-11-24T06:29:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.534+0000 7f727bfff640 1 Processor -- start 2023-11-24T06:29:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.534+0000 7f727bfff640 1 -- start start 2023-11-24T06:29:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.535+0000 7f727bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 0x7f727c131cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.535+0000 7f727bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c0cca40 0x7f727c12f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.535+0000 7f727bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f727c143c40 0x7f727c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.535+0000 7f727bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c12f690 con 0x7f727c0717d0 2023-11-24T06:29:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.535+0000 7f727bfff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c15aec0 con 0x7f727c0cca40 2023-11-24T06:29:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.535+0000 7f727bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727c12f880 con 0x7f727c143c40 2023-11-24T06:29:04.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.535+0000 7f727affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 0x7f727c131cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.535+0000 7f727affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 0x7f727c131cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34034/0 (socket says 172.21.15.16:34034) 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.535+0000 7f727affd640 1 -- 172.21.15.16:0/1763052125 learned_addr learned my addr 172.21.15.16:0/1763052125 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.536+0000 7f7273fff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c0cca40 0x7f727c12f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.536+0000 7f727affd640 1 -- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f727c143c40 msgr2=0x7f727c15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.536+0000 7f727affd640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f727c143c40 0x7f727c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.536+0000 7f727affd640 1 -- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c0cca40 msgr2=0x7f727c12f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.536+0000 7f727affd640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c0cca40 0x7f727c12f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.536+0000 7f727affd640 1 -- 172.21.15.16:0/1763052125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640094d0 con 0x7f727c0717d0 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.536+0000 7f7273fff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c0cca40 0x7f727c12f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.536+0000 7f727affd640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 0x7f727c131cd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b410 tx=0x7f726c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.537+0000 7f7278ff9640 1 -- 172.21.15.16:0/1763052125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c011830 con 0x7f727c0717d0 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.537+0000 7f7278ff9640 1 -- 172.21.15.16:0/1763052125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c0119b0 con 0x7f727c0717d0 2023-11-24T06:29:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.537+0000 7f7278ff9640 1 -- 172.21.15.16:0/1763052125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c011d60 con 0x7f727c0717d0 2023-11-24T06:29:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.537+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727c12fb60 con 0x7f727c0717d0 2023-11-24T06:29:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.537+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c130120 con 0x7f727c0717d0 2023-11-24T06:29:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.538+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727c067450 con 0x7f727c0717d0 2023-11-24T06:29:04.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.539+0000 7f7278ff9640 1 -- 172.21.15.16:0/1763052125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f726c01a400 con 0x7f727c0717d0 2023-11-24T06:29:04.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.540+0000 7f7278ff9640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f725c06aed0 0x7f725c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.540+0000 7f7278ff9640 1 -- 172.21.15.16:0/1763052125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f726c0896f0 con 0x7f727c0717d0 2023-11-24T06:29:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.541+0000 7f7273fff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f725c06aed0 0x7f725c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.542+0000 7f7273fff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f725c06aed0 0x7f725c06d390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7264036000 tx=0x7f726403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.545+0000 7f7278ff9640 1 -- 172.21.15.16:0/1763052125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f726c054890 con 0x7f727c0717d0 2023-11-24T06:29:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:04.741+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f727c067bd0 con 0x7f727c0717d0 2023-11-24T06:29:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.274+0000 7f7278ff9640 1 -- 172.21.15.16:0/1763052125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f726c087e60 con 0x7f727c0717d0 2023-11-24T06:29:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-24T06:29:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.277+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f725c06aed0 msgr2=0x7f725c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.277+0000 7f727bfff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f725c06aed0 0x7f725c06d390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7264036000 tx=0x7f726403d040 comp rx=0 tx=0).stop 2023-11-24T06:29:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.277+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 msgr2=0x7f727c131cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.277+0000 7f727bfff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 0x7f727c131cd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b410 tx=0x7f726c00b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.278+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 shutdown_connections 2023-11-24T06:29:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.278+0000 7f727bfff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f725c06aed0 0x7f725c06d390 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.278+0000 7f727bfff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f727c143c40 0x7f727c15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.278+0000 7f727bfff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c0cca40 0x7f727c12f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.278+0000 7f727bfff640 1 --2- 172.21.15.16:0/1763052125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f727c0717d0 0x7f727c131cd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.278+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 >> 172.21.15.16:0/1763052125 conn(0x7f727c06a800 msgr2=0x7f727c140a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.279+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 shutdown_connections 2023-11-24T06:29:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.279+0000 7f727bfff640 1 -- 172.21.15.16:0/1763052125 wait complete. 2023-11-24T06:29:05.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:29:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.504+0000 7f5928dfb640 1 Processor -- start 2023-11-24T06:29:05.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.505+0000 7f5928dfb640 1 -- start start 2023-11-24T06:29:05.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.506+0000 7f5928dfb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f592415d1d0 0x7f59240d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.506+0000 7f5928dfb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59240b11d0 0x7f59240d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.506+0000 7f5928dfb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59240d0090 con 0x7f59240b1f70 2023-11-24T06:29:05.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.506+0000 7f5928dfb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59240d02e0 con 0x7f592415d1d0 2023-11-24T06:29:05.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.506+0000 7f5928dfb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59240d05c0 con 0x7f59240b11d0 2023-11-24T06:29:05.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.507+0000 7f59237fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f592415d1d0 0x7f59240d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.507+0000 7f5923fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59240b11d0 0x7f59240d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.507+0000 7f59237fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f592415d1d0 0x7f59240d1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:59472/0 (socket says 172.21.15.16:59472) 2023-11-24T06:29:05.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.507+0000 7f59237fe640 1 -- 172.21.15.16:0/3036184445 learned_addr learned my addr 172.21.15.16:0/3036184445 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.508+0000 7f59237fe640 1 -- 172.21.15.16:0/3036184445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59240b11d0 msgr2=0x7f59240d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.508+0000 7f59237fe640 1 --2- 172.21.15.16:0/3036184445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59240b11d0 0x7f59240d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.508+0000 7f59237fe640 1 -- 172.21.15.16:0/3036184445 >> v1:172.21.15.173:6789/0 conn(0x7f59240b1f70 legacy=0x7f59240cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.508+0000 7f59237fe640 1 -- 172.21.15.16:0/3036184445 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59240d08a0 con 0x7f592415d1d0 2023-11-24T06:29:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.508+0000 7f5923fff640 1 --2- 172.21.15.16:0/3036184445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59240b11d0 0x7f59240d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.509+0000 7f59237fe640 1 --2- 172.21.15.16:0/3036184445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f592415d1d0 0x7f59240d1630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5914009870 tx=0x7f591402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3831e95248d3664a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:05.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.510+0000 7f5922ffd640 1 -- 172.21.15.16:0/3036184445 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914030840 con 0x7f592415d1d0 2023-11-24T06:29:05.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.510+0000 7f5922ffd640 1 -- 172.21.15.16:0/3036184445 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59140309a0 con 0x7f592415d1d0 2023-11-24T06:29:05.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.510+0000 7f5922ffd640 1 -- 172.21.15.16:0/3036184445 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914030d30 con 0x7f592415d1d0 2023-11-24T06:29:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.510+0000 7f5928dfb640 1 -- 172.21.15.16:0/3036184445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f592415d1d0 msgr2=0x7f59240d1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.510+0000 7f5928dfb640 1 --2- 172.21.15.16:0/3036184445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f592415d1d0 0x7f59240d1630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5914009870 tx=0x7f591402b980 comp rx=0 tx=0).stop 2023-11-24T06:29:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.511+0000 7f5928dfb640 1 -- 172.21.15.16:0/3036184445 shutdown_connections 2023-11-24T06:29:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.511+0000 7f5928dfb640 1 --2- 172.21.15.16:0/3036184445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59240b11d0 0x7f59240d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.511+0000 7f5928dfb640 1 --2- 172.21.15.16:0/3036184445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f592415d1d0 0x7f59240d1630 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.511+0000 7f5928dfb640 1 -- 172.21.15.16:0/3036184445 >> 172.21.15.16:0/3036184445 conn(0x7f59240d3050 msgr2=0x7f59240d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.511+0000 7f5928dfb640 1 -- 172.21.15.16:0/3036184445 shutdown_connections 2023-11-24T06:29:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.511+0000 7f5928dfb640 1 -- 172.21.15.16:0/3036184445 wait complete. 2023-11-24T06:29:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.513+0000 7f5928dfb640 1 Processor -- start 2023-11-24T06:29:05.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.513+0000 7f5928dfb640 1 -- start start 2023-11-24T06:29:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f5928dfb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59240b11d0 0x7f59240b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f5928dfb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59240b1f70 0x7f5924156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f5928dfb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f592415d1d0 0x7f59240cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f5928dfb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59240cd320 con 0x7f59240b11d0 2023-11-24T06:29:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f5928dfb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59240b2c30 con 0x7f59240b1f70 2023-11-24T06:29:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f5928dfb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59240cd040 con 0x7f592415d1d0 2023-11-24T06:29:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f59237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59240b11d0 0x7f59240b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f59237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59240b11d0 0x7f59240b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34050/0 (socket says 172.21.15.16:34050) 2023-11-24T06:29:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f59237fe640 1 -- 172.21.15.16:0/2736197415 learned_addr learned my addr 172.21.15.16:0/2736197415 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f591bfff640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59240b1f70 0x7f5924156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f59237fe640 1 -- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f592415d1d0 msgr2=0x7f59240cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f59237fe640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f592415d1d0 0x7f59240cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f59237fe640 1 -- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59240b1f70 msgr2=0x7f5924156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.515+0000 7f59237fe640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59240b1f70 0x7f5924156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.516+0000 7f59237fe640 1 -- 172.21.15.16:0/2736197415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c009a70 con 0x7f59240b11d0 2023-11-24T06:29:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.516+0000 7f59237fe640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59240b11d0 0x7f59240b26f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5914000c00 tx=0x7f5914004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.516+0000 7f5920ff9640 1 -- 172.21.15.16:0/2736197415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914039c10 con 0x7f59240b11d0 2023-11-24T06:29:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.516+0000 7f5920ff9640 1 -- 172.21.15.16:0/2736197415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914039d70 con 0x7f59240b11d0 2023-11-24T06:29:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.516+0000 7f5920ff9640 1 -- 172.21.15.16:0/2736197415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914043640 con 0x7f59240b11d0 2023-11-24T06:29:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.516+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59140094d0 con 0x7f59240b11d0 2023-11-24T06:29:05.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.517+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59240ca5a0 con 0x7f59240b11d0 2023-11-24T06:29:05.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.518+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59240ca100 con 0x7f59240b11d0 2023-11-24T06:29:05.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.520+0000 7f5920ff9640 1 -- 172.21.15.16:0/2736197415 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f59140437a0 con 0x7f59240b11d0 2023-11-24T06:29:05.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.520+0000 7f5920ff9640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f590406aed0 0x7f590406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:05.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.521+0000 7f5920ff9640 1 -- 172.21.15.16:0/2736197415 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f591403e070 con 0x7f59240b11d0 2023-11-24T06:29:05.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.521+0000 7f591bfff640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f590406aed0 0x7f590406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:05.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.522+0000 7f591bfff640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f590406aed0 0x7f590406d390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5924157d70 tx=0x7f590c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:05.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.525+0000 7f5920ff9640 1 -- 172.21.15.16:0/2736197415 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5914078170 con 0x7f59240b11d0 2023-11-24T06:29:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:05.752+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5924067450 con 0x7f59240b11d0 2023-11-24T06:29:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.318+0000 7f5920ff9640 1 -- 172.21.15.16:0/2736197415 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f591407b790 con 0x7f59240b11d0 2023-11-24T06:29:06.325 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 6 and data pool 7 2023-11-24T06:29:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.327+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f590406aed0 msgr2=0x7f590406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.327+0000 7f5928dfb640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f590406aed0 0x7f590406d390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5924157d70 tx=0x7f590c009040 comp rx=0 tx=0).stop 2023-11-24T06:29:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.327+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59240b11d0 msgr2=0x7f59240b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.327+0000 7f5928dfb640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59240b11d0 0x7f59240b26f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5914000c00 tx=0x7f5914004730 comp rx=0 tx=0).stop 2023-11-24T06:29:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.327+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 shutdown_connections 2023-11-24T06:29:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.327+0000 7f5928dfb640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f590406aed0 0x7f590406d390 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.328+0000 7f5928dfb640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f592415d1d0 0x7f59240cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.328+0000 7f5928dfb640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59240b1f70 0x7f5924156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.328+0000 7f5928dfb640 1 --2- 172.21.15.16:0/2736197415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59240b11d0 0x7f59240b26f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.328+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 >> 172.21.15.16:0/2736197415 conn(0x7f59240d3050 msgr2=0x7f59240cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:06.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.328+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 shutdown_connections 2023-11-24T06:29:06.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.329+0000 7f5928dfb640 1 -- 172.21.15.16:0/2736197415 wait complete. 2023-11-24T06:29:06.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.538+0000 7f5b44fbc640 1 Processor -- start 2023-11-24T06:29:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.538+0000 7f5b44fbc640 1 -- start start 2023-11-24T06:29:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.540+0000 7f5b44fbc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40162430 0x7f5b400b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.540+0000 7f5b44fbc640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5b400b3650 0x7f5b4006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.540+0000 7f5b44fbc640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b400702e0 con 0x7f5b40162810 2023-11-24T06:29:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.540+0000 7f5b44fbc640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b400705c0 con 0x7f5b400b3650 2023-11-24T06:29:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.540+0000 7f5b44fbc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b400708a0 con 0x7f5b40162430 2023-11-24T06:29:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.541+0000 7f5b3ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5b400b3650 0x7f5b4006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.541+0000 7f5b3f7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40162430 0x7f5b400b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.541+0000 7f5b3ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5b400b3650 0x7f5b4006c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:59496/0 (socket says 172.21.15.16:59496) 2023-11-24T06:29:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.541+0000 7f5b3ffff640 1 -- 172.21.15.16:0/1528886859 learned_addr learned my addr 172.21.15.16:0/1528886859 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:06.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.541+0000 7f5b3f7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40162430 0x7f5b400b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:57168/0 (socket says 172.21.15.16:57168) 2023-11-24T06:29:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.541+0000 7f5b3f7fe640 1 -- 172.21.15.16:0/1528886859 >> v2:172.21.15.173:3300/0 conn(0x7f5b400b3650 msgr2=0x7f5b4006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.541+0000 7f5b3f7fe640 1 --2- 172.21.15.16:0/1528886859 >> v2:172.21.15.173:3300/0 conn(0x7f5b400b3650 0x7f5b4006c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.541+0000 7f5b3f7fe640 1 -- 172.21.15.16:0/1528886859 >> v1:172.21.15.173:6789/0 conn(0x7f5b40162810 legacy=0x7f5b40071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.541+0000 7f5b3f7fe640 1 -- 172.21.15.16:0/1528886859 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b40070b80 con 0x7f5b40162430 2023-11-24T06:29:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.543+0000 7f5b3f7fe640 1 --2- 172.21.15.16:0/1528886859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40162430 0x7f5b400b3110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009d90 tx=0x7f5b3002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c106f9e65dead58 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.543+0000 7f5b3effd640 1 -- 172.21.15.16:0/1528886859 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3002cc50 con 0x7f5b40162430 2023-11-24T06:29:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.543+0000 7f5b3effd640 1 -- 172.21.15.16:0/1528886859 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3002cdb0 con 0x7f5b40162430 2023-11-24T06:29:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.543+0000 7f5b3effd640 1 -- 172.21.15.16:0/1528886859 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b30034660 con 0x7f5b40162430 2023-11-24T06:29:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.544+0000 7f5b44fbc640 1 -- 172.21.15.16:0/1528886859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40162430 msgr2=0x7f5b400b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.544+0000 7f5b44fbc640 1 --2- 172.21.15.16:0/1528886859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40162430 0x7f5b400b3110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009d90 tx=0x7f5b3002bea0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.544+0000 7f5b44fbc640 1 -- 172.21.15.16:0/1528886859 shutdown_connections 2023-11-24T06:29:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.544+0000 7f5b44fbc640 1 --2- 172.21.15.16:0/1528886859 >> v2:172.21.15.173:3300/0 conn(0x7f5b400b3650 0x7f5b4006c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.544+0000 7f5b44fbc640 1 --2- 172.21.15.16:0/1528886859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b40162430 0x7f5b400b3110 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.544+0000 7f5b44fbc640 1 -- 172.21.15.16:0/1528886859 >> 172.21.15.16:0/1528886859 conn(0x7f5b4006a800 msgr2=0x7f5b4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.544+0000 7f5b44fbc640 1 -- 172.21.15.16:0/1528886859 shutdown_connections 2023-11-24T06:29:06.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.544+0000 7f5b44fbc640 1 -- 172.21.15.16:0/1528886859 wait complete. 2023-11-24T06:29:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.546+0000 7f5b44fbc640 1 Processor -- start 2023-11-24T06:29:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.546+0000 7f5b44fbc640 1 -- start start 2023-11-24T06:29:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.547+0000 7f5b44fbc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b400b3650 0x7f5b40153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.547+0000 7f5b44fbc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40162430 0x7f5b40164190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.547+0000 7f5b44fbc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b401646d0 0x7f5b40158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.547+0000 7f5b44fbc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b401ec780 con 0x7f5b40162430 2023-11-24T06:29:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.547+0000 7f5b44fbc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b40164b60 con 0x7f5b400b3650 2023-11-24T06:29:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.547+0000 7f5b44fbc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b40159a50 con 0x7f5b401646d0 2023-11-24T06:29:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.547+0000 7f5b36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40162430 0x7f5b40164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b36ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40162430 0x7f5b40164190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34058/0 (socket says 172.21.15.16:34058) 2023-11-24T06:29:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b36ffd640 1 -- 172.21.15.16:0/4067332828 learned_addr learned my addr 172.21.15.16:0/4067332828 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b3f7fe640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b400b3650 0x7f5b40153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b3ffff640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b401646d0 0x7f5b40158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b36ffd640 1 -- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b401646d0 msgr2=0x7f5b40158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b36ffd640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b401646d0 0x7f5b40158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b36ffd640 1 -- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b400b3650 msgr2=0x7f5b40153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b36ffd640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b400b3650 0x7f5b40153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b36ffd640 1 -- 172.21.15.16:0/4067332828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b30009a70 con 0x7f5b40162430 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b3ffff640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b401646d0 0x7f5b40158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.548+0000 7f5b36ffd640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40162430 0x7f5b40164190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c3c0 tx=0x7f5b2800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.549+0000 7f5b3cff9640 1 -- 172.21.15.16:0/4067332828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b280052d0 con 0x7f5b40162430 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.549+0000 7f5b3cff9640 1 -- 172.21.15.16:0/4067332828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b28005450 con 0x7f5b40162430 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.549+0000 7f5b3cff9640 1 -- 172.21.15.16:0/4067332828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28007260 con 0x7f5b40162430 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.549+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b40159d30 con 0x7f5b40162430 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.549+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b401ec3a0 con 0x7f5b40162430 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.550+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b401ec8c0 con 0x7f5b40162430 2023-11-24T06:29:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.551+0000 7f5b3cff9640 1 -- 172.21.15.16:0/4067332828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5b280073c0 con 0x7f5b40162430 2023-11-24T06:29:06.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.552+0000 7f5b3cff9640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b2006aed0 0x7f5b2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.552+0000 7f5b3cff9640 1 -- 172.21.15.16:0/4067332828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b28088ac0 con 0x7f5b40162430 2023-11-24T06:29:06.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.552+0000 7f5b3f7fe640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b2006aed0 0x7f5b2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.554+0000 7f5b3f7fe640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b2006aed0 0x7f5b2006d390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5b30036000 tx=0x7f5b30030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.556+0000 7f5b3cff9640 1 -- 172.21.15.16:0/4067332828 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b401ec8c0 con 0x7f5b40162430 2023-11-24T06:29:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.753+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b401ec8c0 con 0x7f5b40162430 2023-11-24T06:29:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.755+0000 7f5b3cff9640 1 -- 172.21.15.16:0/4067332828 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16526 (secure 0 0 0) 0x7f5b401ec8c0 con 0x7f5b40162430 2023-11-24T06:29:06.759 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:06.761 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:05.753567+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:03.705566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:04.794925+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b2006aed0 msgr2=0x7f5b2006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b2006aed0 0x7f5b2006d390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5b30036000 tx=0x7f5b30030000 comp rx=0 tx=0).stop 2023-11-24T06:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40162430 msgr2=0x7f5b40164190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40162430 0x7f5b40164190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800c3c0 tx=0x7f5b2800c880 comp rx=0 tx=0).stop 2023-11-24T06:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 shutdown_connections 2023-11-24T06:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b2006aed0 0x7f5b2006d390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b401646d0 0x7f5b40158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40162430 0x7f5b40164190 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 --2- 172.21.15.16:0/4067332828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b400b3650 0x7f5b40153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 >> 172.21.15.16:0/4067332828 conn(0x7f5b4006a800 msgr2=0x7f5b400cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 shutdown_connections 2023-11-24T06:29:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.758+0000 7f5b44fbc640 1 -- 172.21.15.16:0/4067332828 wait complete. 2023-11-24T06:29:06.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.988+0000 7f5cdedf4640 1 Processor -- start 2023-11-24T06:29:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.988+0000 7f5cdedf4640 1 -- start start 2023-11-24T06:29:06.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.989+0000 7f5cdedf4640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5cd815d590 0x7f5cd80d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.989+0000 7f5cdedf4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd80c6750 0x7f5cd8099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.989+0000 7f5cdedf4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd80c7110 con 0x7f5cd8099fa0 2023-11-24T06:29:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.989+0000 7f5cdedf4640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd809a430 con 0x7f5cd815d590 2023-11-24T06:29:06.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.989+0000 7f5cdedf4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd80c6bb0 con 0x7f5cd80c6750 2023-11-24T06:29:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.990+0000 7f5cde5f3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5cd8099fa0 0x7f5cd806bb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48660/0 (socket says 172.21.15.16:48660) 2023-11-24T06:29:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.990+0000 7f5cde5f3640 1 -- 172.21.15.16:0/3677073875 learned_addr learned my addr 172.21.15.16:0/3677073875 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.991+0000 7f5cdddf2640 1 --2- 172.21.15.16:0/3677073875 >> v2:172.21.15.173:3300/0 conn(0x7f5cd815d590 0x7f5cd80d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.991+0000 7f5cdd5f1640 1 --2- 172.21.15.16:0/3677073875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd80c6750 0x7f5cd8099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:06.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.991+0000 7f5cdddf2640 1 -- 172.21.15.16:0/3677073875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd80c6750 msgr2=0x7f5cd8099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.991+0000 7f5cdddf2640 1 --2- 172.21.15.16:0/3677073875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd80c6750 0x7f5cd8099a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.991+0000 7f5cdcdf0640 1 -- 172.21.15.16:0/3677073875 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661374056 0 0) 0x7f5cd80c7110 con 0x7f5cd8099fa0 2023-11-24T06:29:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.991+0000 7f5cdddf2640 1 -- 172.21.15.16:0/3677073875 >> v1:172.21.15.16:6789/0 conn(0x7f5cd8099fa0 legacy=0x7f5cd806bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.991+0000 7f5cdddf2640 1 -- 172.21.15.16:0/3677073875 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80c6e30 con 0x7f5cd815d590 2023-11-24T06:29:06.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.993+0000 7f5cdddf2640 1 --2- 172.21.15.16:0/3677073875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd815d590 0x7f5cd80d19a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40097a0 tx=0x7f5cd402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b87a5aa1eff73b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.995+0000 7f5cdcdf0640 1 -- 172.21.15.16:0/3677073875 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd402d040 con 0x7f5cd815d590 2023-11-24T06:29:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.995+0000 7f5cdcdf0640 1 -- 172.21.15.16:0/3677073875 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd4031810 con 0x7f5cd815d590 2023-11-24T06:29:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.995+0000 7f5cdcdf0640 1 -- 172.21.15.16:0/3677073875 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4031ba0 con 0x7f5cd815d590 2023-11-24T06:29:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.995+0000 7f5cdedf4640 1 -- 172.21.15.16:0/3677073875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd815d590 msgr2=0x7f5cd80d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.995+0000 7f5cdedf4640 1 --2- 172.21.15.16:0/3677073875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd815d590 0x7f5cd80d19a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40097a0 tx=0x7f5cd402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.996+0000 7f5cdedf4640 1 -- 172.21.15.16:0/3677073875 shutdown_connections 2023-11-24T06:29:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.996+0000 7f5cdedf4640 1 --2- 172.21.15.16:0/3677073875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd80c6750 0x7f5cd8099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.996+0000 7f5cdedf4640 1 --2- 172.21.15.16:0/3677073875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd815d590 0x7f5cd80d19a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.996+0000 7f5cdedf4640 1 -- 172.21.15.16:0/3677073875 >> 172.21.15.16:0/3677073875 conn(0x7f5cd80d0e80 msgr2=0x7f5cd80d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:07.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.996+0000 7f5cdedf4640 1 -- 172.21.15.16:0/3677073875 shutdown_connections 2023-11-24T06:29:07.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.996+0000 7f5cdedf4640 1 -- 172.21.15.16:0/3677073875 wait complete. 2023-11-24T06:29:07.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.997+0000 7f5cdedf4640 1 Processor -- start 2023-11-24T06:29:07.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.997+0000 7f5cdedf4640 1 -- start start 2023-11-24T06:29:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.998+0000 7f5cdedf4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd8099fa0 0x7f5cd814f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.998+0000 7f5cdedf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd80c6750 0x7f5cd80cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.998+0000 7f5cdedf4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd815d590 0x7f5cd80b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.998+0000 7f5cdedf4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd814f6a0 con 0x7f5cd80c6750 2023-11-24T06:29:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.998+0000 7f5cdedf4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd80ce310 con 0x7f5cd8099fa0 2023-11-24T06:29:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.998+0000 7f5cdedf4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd80ce560 con 0x7f5cd815d590 2023-11-24T06:29:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdd5f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd80c6750 0x7f5cd80cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdd5f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd80c6750 0x7f5cd80cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34060/0 (socket says 172.21.15.16:34060) 2023-11-24T06:29:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdd5f1640 1 -- 172.21.15.16:0/1691139869 learned_addr learned my addr 172.21.15.16:0/1691139869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdddf2640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd8099fa0 0x7f5cd814f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cde5f3640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd815d590 0x7f5cd80b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdd5f1640 1 -- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd815d590 msgr2=0x7f5cd80b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdd5f1640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd815d590 0x7f5cd80b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdd5f1640 1 -- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd8099fa0 msgr2=0x7f5cd814f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdd5f1640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd8099fa0 0x7f5cd814f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdd5f1640 1 -- 172.21.15.16:0/1691139869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd4009400 con 0x7f5cd80c6750 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:06.999+0000 7f5cdddf2640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd8099fa0 0x7f5cd814f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.000+0000 7f5cdd5f1640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd80c6750 0x7f5cd80cf320 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800e9b0 tx=0x7f5cc800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.000+0000 7f5cce7fc640 1 -- 172.21.15.16:0/1691139869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8009670 con 0x7f5cd80c6750 2023-11-24T06:29:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.000+0000 7f5cce7fc640 1 -- 172.21.15.16:0/1691139869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc800c9a0 con 0x7f5cd80c6750 2023-11-24T06:29:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.000+0000 7f5cce7fc640 1 -- 172.21.15.16:0/1691139869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc800cd50 con 0x7f5cd80c6750 2023-11-24T06:29:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.000+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd80ce840 con 0x7f5cd80c6750 2023-11-24T06:29:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.000+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd80cee00 con 0x7f5cd80c6750 2023-11-24T06:29:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.001+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd8067b60 con 0x7f5cd80c6750 2023-11-24T06:29:07.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.002+0000 7f5cce7fc640 1 -- 172.21.15.16:0/1691139869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5cc8013470 con 0x7f5cd80c6750 2023-11-24T06:29:07.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.003+0000 7f5cce7fc640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cbc06af00 0x7f5cbc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.003+0000 7f5cce7fc640 1 -- 172.21.15.16:0/1691139869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cc8089a10 con 0x7f5cd80c6750 2023-11-24T06:29:07.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.004+0000 7f5cdddf2640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cbc06af00 0x7f5cbc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.005+0000 7f5cdddf2640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cbc06af00 0x7f5cbc06d3c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4036000 tx=0x7f5cd403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.008+0000 7f5cce7fc640 1 -- 172.21.15.16:0/1691139869 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5cc8054940 con 0x7f5cd80c6750 2023-11-24T06:29:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.193+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cd815ace0 con 0x7f5cd80c6750 2023-11-24T06:29:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.196+0000 7f5cce7fc640 1 -- 172.21.15.16:0/1691139869 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16526 (secure 0 0 0) 0x7f5cc8017090 con 0x7f5cd80c6750 2023-11-24T06:29:07.199 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:07.200 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:05.753567+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:03.705566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:04.794925+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.198+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cbc06af00 msgr2=0x7f5cbc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.198+0000 7f5cdedf4640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cbc06af00 0x7f5cbc06d3c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4036000 tx=0x7f5cd403c040 comp rx=0 tx=0).stop 2023-11-24T06:29:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd80c6750 msgr2=0x7f5cd80cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd80c6750 0x7f5cd80cf320 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800e9b0 tx=0x7f5cc800ee70 comp rx=0 tx=0).stop 2023-11-24T06:29:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 shutdown_connections 2023-11-24T06:29:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5cbc06af00 0x7f5cbc06d3c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cd815d590 0x7f5cd80b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd80c6750 0x7f5cd80cf320 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 --2- 172.21.15.16:0/1691139869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cd8099fa0 0x7f5cd814f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 >> 172.21.15.16:0/1691139869 conn(0x7f5cd80d0e80 msgr2=0x7f5cd806e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 shutdown_connections 2023-11-24T06:29:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.199+0000 7f5cdedf4640 1 -- 172.21.15.16:0/1691139869 wait complete. 2023-11-24T06:29:07.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:29:07.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.450+0000 7fc5f6a10640 1 Processor -- start 2023-11-24T06:29:07.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.450+0000 7fc5f6a10640 1 -- start start 2023-11-24T06:29:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.452+0000 7fc5f6a10640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5f00adf20 0x7fc5f0070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.452+0000 7fc5f6a10640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f015f4e0 con 0x7fc5f015bf10 2023-11-24T06:29:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.452+0000 7fc5f6a10640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f0071130 con 0x7fc5f00ccb50 2023-11-24T06:29:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.452+0000 7fc5f6a10640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f015f970 con 0x7fc5f00adf20 2023-11-24T06:29:07.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.452+0000 7fc5f520d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc5f015bf10 0x7fc5f00cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48670/0 (socket says 172.21.15.16:48670) 2023-11-24T06:29:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.452+0000 7fc5f520d640 1 -- 172.21.15.16:0/3286863977 learned_addr learned my addr 172.21.15.16:0/3286863977 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.452+0000 7fc5f5a0e640 1 --2- 172.21.15.16:0/3286863977 >> v2:172.21.15.16:3300/0 conn(0x7fc5f00adf20 0x7fc5f0070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.453+0000 7fc5f4a0c640 1 -- 172.21.15.16:0/3286863977 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615708640 0 0) 0x7fc5f015f4e0 con 0x7fc5f015bf10 2023-11-24T06:29:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.453+0000 7fc5f4a0c640 1 -- 172.21.15.16:0/3286863977 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5e0003520 con 0x7fc5f015bf10 2023-11-24T06:29:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.453+0000 7fc5f5a0e640 1 -- 172.21.15.16:0/3286863977 >> v1:172.21.15.173:6789/0 conn(0x7fc5f00ccb50 legacy=0x7fc5f0143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.453+0000 7fc5f5a0e640 1 -- 172.21.15.16:0/3286863977 >> v1:172.21.15.16:6789/0 conn(0x7fc5f015bf10 legacy=0x7fc5f00cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.453+0000 7fc5f5a0e640 1 -- 172.21.15.16:0/3286863977 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f015c390 con 0x7fc5f00adf20 2023-11-24T06:29:07.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.454+0000 7fc5f5a0e640 1 --2- 172.21.15.16:0/3286863977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f00adf20 0x7fc5f0070a30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009d90 tx=0x7fc5e402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d0f809e7582d51 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.454+0000 7fc5f4a0c640 1 -- 172.21.15.16:0/3286863977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e402cc60 con 0x7fc5f00adf20 2023-11-24T06:29:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.454+0000 7fc5f4a0c640 1 -- 172.21.15.16:0/3286863977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e402cdc0 con 0x7fc5f00adf20 2023-11-24T06:29:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.454+0000 7fc5f4a0c640 1 -- 172.21.15.16:0/3286863977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4034640 con 0x7fc5f00adf20 2023-11-24T06:29:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.455+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3286863977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f00adf20 msgr2=0x7fc5f0070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.455+0000 7fc5f6a10640 1 --2- 172.21.15.16:0/3286863977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f00adf20 0x7fc5f0070a30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009d90 tx=0x7fc5e402beb0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.455+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3286863977 shutdown_connections 2023-11-24T06:29:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.455+0000 7fc5f6a10640 1 --2- 172.21.15.16:0/3286863977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f00adf20 0x7fc5f0070a30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.455+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3286863977 >> 172.21.15.16:0/3286863977 conn(0x7fc5f006a7d0 msgr2=0x7fc5f006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.455+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3286863977 shutdown_connections 2023-11-24T06:29:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.455+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3286863977 wait complete. 2023-11-24T06:29:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.457+0000 7fc5f6a10640 1 Processor -- start 2023-11-24T06:29:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.457+0000 7fc5f6a10640 1 -- start start 2023-11-24T06:29:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.458+0000 7fc5f6a10640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5f00adf20 0x7fc5f015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.458+0000 7fc5f6a10640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5f00ccb50 0x7fc5f00b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.458+0000 7fc5f6a10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f015bf10 0x7fc5f00b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.458+0000 7fc5f6a10640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0141ef0 con 0x7fc5f015bf10 2023-11-24T06:29:07.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.458+0000 7fc5f6a10640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0140fb0 con 0x7fc5f00adf20 2023-11-24T06:29:07.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.458+0000 7fc5f6a10640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0141270 con 0x7fc5f00ccb50 2023-11-24T06:29:07.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.458+0000 7fc5f620f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f015bf10 0x7fc5f00b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f620f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f015bf10 0x7fc5f00b3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34086/0 (socket says 172.21.15.16:34086) 2023-11-24T06:29:07.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f620f640 1 -- 172.21.15.16:0/3831877818 learned_addr learned my addr 172.21.15.16:0/3831877818 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:07.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f5a0e640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5f00adf20 0x7fc5f015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f520d640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5f00ccb50 0x7fc5f00b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f620f640 1 -- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5f00ccb50 msgr2=0x7fc5f00b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f620f640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5f00ccb50 0x7fc5f00b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f620f640 1 -- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5f00adf20 msgr2=0x7fc5f015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f620f640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5f00adf20 0x7fc5f015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f620f640 1 -- 172.21.15.16:0/3831877818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e4009a70 con 0x7fc5f015bf10 2023-11-24T06:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f5a0e640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5f00adf20 0x7fc5f015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5f620f640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f015bf10 0x7fc5f00b3420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800c390 tx=0x7fc5e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.459+0000 7fc5ee7fc640 1 -- 172.21.15.16:0/3831877818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80052d0 con 0x7fc5f015bf10 2023-11-24T06:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.460+0000 7fc5ee7fc640 1 -- 172.21.15.16:0/3831877818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8005430 con 0x7fc5f015bf10 2023-11-24T06:29:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.460+0000 7fc5ee7fc640 1 -- 172.21.15.16:0/3831877818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8003810 con 0x7fc5f015bf10 2023-11-24T06:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.460+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f0141550 con 0x7fc5f015bf10 2023-11-24T06:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.460+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f0141b10 con 0x7fc5f015bf10 2023-11-24T06:29:07.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.461+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0141cd0 con 0x7fc5f015bf10 2023-11-24T06:29:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.462+0000 7fc5ee7fc640 1 -- 172.21.15.16:0/3831877818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc5e8003970 con 0x7fc5f015bf10 2023-11-24T06:29:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.463+0000 7fc5ee7fc640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc5d406af00 0x7fc5d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.463+0000 7fc5ee7fc640 1 -- 172.21.15.16:0/3831877818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5e8088c00 con 0x7fc5f015bf10 2023-11-24T06:29:07.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.464+0000 7fc5f5a0e640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc5d406af00 0x7fc5d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:07.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.465+0000 7fc5f5a0e640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc5d406af00 0x7fc5d406d3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4036000 tx=0x7fc5e4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:07.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.467+0000 7fc5ee7fc640 1 -- 172.21.15.16:0/3831877818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5e8053bc0 con 0x7fc5f015bf10 2023-11-24T06:29:07.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:07.702+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc5f0142030 con 0x7fc5f015bf10 2023-11-24T06:29:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.336+0000 7fc5ee7fc640 1 -- 172.21.15.16:0/3831877818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc5f0142030 con 0x7fc5f015bf10 2023-11-24T06:29:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc5d406af00 msgr2=0x7fc5d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc5d406af00 0x7fc5d406d3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4036000 tx=0x7fc5e4030000 comp rx=0 tx=0).stop 2023-11-24T06:29:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f015bf10 msgr2=0x7fc5f00b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f015bf10 0x7fc5f00b3420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800c390 tx=0x7fc5e800c850 comp rx=0 tx=0).stop 2023-11-24T06:29:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 shutdown_connections 2023-11-24T06:29:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc5d406af00 0x7fc5d406d3c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5f015bf10 0x7fc5f00b3420 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5f00ccb50 0x7fc5f00b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 --2- 172.21.15.16:0/3831877818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5f00adf20 0x7fc5f015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 >> 172.21.15.16:0/3831877818 conn(0x7fc5f006a7d0 msgr2=0x7fc5f00cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.340+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 shutdown_connections 2023-11-24T06:29:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.341+0000 7fc5f6a10640 1 -- 172.21.15.16:0/3831877818 wait complete. 2023-11-24T06:29:08.368 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:29:08.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.588+0000 7fed5c3dc640 1 Processor -- start 2023-11-24T06:29:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.588+0000 7fed5c3dc640 1 -- start start 2023-11-24T06:29:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.589+0000 7fed5c3dc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413df80 0x7fed5415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.590+0000 7fed5c3dc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5406c470 con 0x7fed540cf9a0 2023-11-24T06:29:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.590+0000 7fed5c3dc640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5406c750 con 0x7fed5413e360 2023-11-24T06:29:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.590+0000 7fed5c3dc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5406ca30 con 0x7fed5413df80 2023-11-24T06:29:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.590+0000 7fed5a979640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fed540cf9a0 0x7fed540cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48680/0 (socket says 172.21.15.16:48680) 2023-11-24T06:29:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.590+0000 7fed5a979640 1 -- 172.21.15.16:0/760297589 learned_addr learned my addr 172.21.15.16:0/760297589 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.590+0000 7fed5a178640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413df80 0x7fed5415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.590+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604730425 0 0) 0x7fed5406c470 con 0x7fed540cf9a0 2023-11-24T06:29:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.590+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed40003520 con 0x7fed540cf9a0 2023-11-24T06:29:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1013810082 0 0) 0x7fed40003520 con 0x7fed540cf9a0 2023-11-24T06:29:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413df80 msgr2=0x7fed5415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413df80 0x7fed5415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 >> v1:172.21.15.173:6789/0 conn(0x7fed5413e360 legacy=0x7fed540cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5406cd10 con 0x7fed540cf9a0 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fed4c0019f0 con 0x7fed540cf9a0 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 >> v1:172.21.15.16:6789/0 conn(0x7fed540cf9a0 legacy=0x7fed540cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413df80 0x7fed400039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed40003f20 0x7fed400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed40004d00 0x7fed40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5406ca30 con 0x7fed40004d00 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4000ba60 con 0x7fed40003f20 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4000bc20 con 0x7fed5413df80 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.591+0000 7fed5a979640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed40004d00 0x7fed40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.592+0000 7fed5a178640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413df80 0x7fed400039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.592+0000 7fed59977640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed40003f20 0x7fed400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.592+0000 7fed5a979640 1 -- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413df80 msgr2=0x7fed400039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.592+0000 7fed5a979640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413df80 0x7fed400039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.592+0000 7fed5a979640 1 -- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed40003f20 msgr2=0x7fed400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.592+0000 7fed5a979640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed40003f20 0x7fed400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.592+0000 7fed5a979640 1 -- 172.21.15.16:0/760297589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed40003520 con 0x7fed40004d00 2023-11-24T06:29:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.593+0000 7fed5a979640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed40004d00 0x7fed40005180 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00a830 tx=0x7fed400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de28851773d4f3a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.593+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0072d0 con 0x7fed40004d00 2023-11-24T06:29:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.593+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c02f070 con 0x7fed40004d00 2023-11-24T06:29:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.593+0000 7fed59176640 1 -- 172.21.15.16:0/760297589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c007730 con 0x7fed40004d00 2023-11-24T06:29:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.594+0000 7fed5c3dc640 1 -- 172.21.15.16:0/760297589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed40004d00 msgr2=0x7fed40005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.594+0000 7fed5c3dc640 1 --2- 172.21.15.16:0/760297589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed40004d00 0x7fed40005180 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00a830 tx=0x7fed400057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.594+0000 7fed5a979640 1 -- 172.21.15.16:0/760297589 reap_dead start 2023-11-24T06:29:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.594+0000 7fed5c3dc640 1 -- 172.21.15.16:0/760297589 shutdown_connections 2023-11-24T06:29:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.594+0000 7fed5c3dc640 1 -- 172.21.15.16:0/760297589 >> 172.21.15.16:0/760297589 conn(0x7fed5406a800 msgr2=0x7fed5406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.594+0000 7fed5c3dc640 1 -- 172.21.15.16:0/760297589 shutdown_connections 2023-11-24T06:29:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.594+0000 7fed5c3dc640 1 -- 172.21.15.16:0/760297589 wait complete. 2023-11-24T06:29:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.596+0000 7fed5c3dc640 1 Processor -- start 2023-11-24T06:29:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.596+0000 7fed5c3dc640 1 -- start start 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.597+0000 7fed5c3dc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed40004d00 0x7fed54159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.597+0000 7fed5c3dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540c4010 0x7fed540c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.597+0000 7fed5c3dc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413da80 0x7fed5415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.597+0000 7fed5c3dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5415a120 con 0x7fed540c4010 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.597+0000 7fed5c3dc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed54158b40 con 0x7fed40004d00 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.597+0000 7fed5c3dc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed54158e20 con 0x7fed5413da80 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.597+0000 7fed59977640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540c4010 0x7fed540c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.597+0000 7fed59977640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540c4010 0x7fed540c47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34104/0 (socket says 172.21.15.16:34104) 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.597+0000 7fed59977640 1 -- 172.21.15.16:0/4128376983 learned_addr learned my addr 172.21.15.16:0/4128376983 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed5a979640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413da80 0x7fed5415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed59977640 1 -- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413da80 msgr2=0x7fed5415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed59977640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413da80 0x7fed5415d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed59977640 1 -- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed40004d00 msgr2=0x7fed54159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed59977640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed40004d00 0x7fed54159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed59977640 1 -- 172.21.15.16:0/4128376983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54159100 con 0x7fed540c4010 2023-11-24T06:29:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed59977640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540c4010 0x7fed540c47c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fed4400e4d0 tx=0x7fed4400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed4affd640 1 -- 172.21.15.16:0/4128376983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed440052e0 con 0x7fed540c4010 2023-11-24T06:29:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed4affd640 1 -- 172.21.15.16:0/4128376983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed44005440 con 0x7fed540c4010 2023-11-24T06:29:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed4affd640 1 -- 172.21.15.16:0/4128376983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44003810 con 0x7fed540c4010 2023-11-24T06:29:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed541593e0 con 0x7fed540c4010 2023-11-24T06:29:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.598+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed540c4d00 con 0x7fed540c4010 2023-11-24T06:29:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.600+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed540c4e90 con 0x7fed540c4010 2023-11-24T06:29:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.601+0000 7fed4affd640 1 -- 172.21.15.16:0/4128376983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fed44003970 con 0x7fed540c4010 2023-11-24T06:29:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.602+0000 7fed4affd640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed3806af20 0x7fed3806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.602+0000 7fed4affd640 1 -- 172.21.15.16:0/4128376983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed44089430 con 0x7fed540c4010 2023-11-24T06:29:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.602+0000 7fed5a178640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed3806af20 0x7fed3806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.604+0000 7fed5a178640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed3806af20 0x7fed3806d3e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fed50009910 tx=0x7fed50008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.606+0000 7fed4affd640 1 -- 172.21.15.16:0/4128376983 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed44054470 con 0x7fed540c4010 2023-11-24T06:29:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:08.835+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fed540675d0 con 0x7fed540c4010 2023-11-24T06:29:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.340+0000 7fed4affd640 1 -- 172.21.15.16:0/4128376983 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7fed44057a90 con 0x7fed540c4010 2023-11-24T06:29:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed3806af20 msgr2=0x7fed3806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed3806af20 0x7fed3806d3e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fed50009910 tx=0x7fed50008040 comp rx=0 tx=0).stop 2023-11-24T06:29:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540c4010 msgr2=0x7fed540c47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540c4010 0x7fed540c47c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fed4400e4d0 tx=0x7fed4400e990 comp rx=0 tx=0).stop 2023-11-24T06:29:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 shutdown_connections 2023-11-24T06:29:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed3806af20 0x7fed3806d3e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed5413da80 0x7fed5415d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed540c4010 0x7fed540c47c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 --2- 172.21.15.16:0/4128376983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed40004d00 0x7fed54159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 >> 172.21.15.16:0/4128376983 conn(0x7fed5406a800 msgr2=0x7fed540afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 shutdown_connections 2023-11-24T06:29:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.344+0000 7fed5c3dc640 1 -- 172.21.15.16:0/4128376983 wait complete. 2023-11-24T06:29:09.369 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:29:09.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:09.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.589+0000 7ffa76248640 1 Processor -- start 2023-11-24T06:29:09.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.589+0000 7ffa76248640 1 -- start start 2023-11-24T06:29:09.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.591+0000 7ffa76248640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ffa7013df80 0x7ffa7015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.591+0000 7ffa76248640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa7013e360 0x7ffa700cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.591+0000 7ffa76248640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7006c470 con 0x7ffa7015f9c0 2023-11-24T06:29:09.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.591+0000 7ffa76248640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7006c750 con 0x7ffa7013e360 2023-11-24T06:29:09.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.591+0000 7ffa76248640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7006ca30 con 0x7ffa7013df80 2023-11-24T06:29:09.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.591+0000 7ffa66ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa7013e360 0x7ffa700cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.591+0000 7ffa66ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa7013e360 0x7ffa700cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34108/0 (socket says 172.21.15.16:34108) 2023-11-24T06:29:09.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.591+0000 7ffa66ffd640 1 -- 172.21.15.16:0/4173698095 learned_addr learned my addr 172.21.15.16:0/4173698095 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:09.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.592+0000 7ffa6f7fe640 1 --2- 172.21.15.16:0/4173698095 >> v2:172.21.15.173:3300/0 conn(0x7ffa7013df80 0x7ffa7015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.592+0000 7ffa66ffd640 1 -- 172.21.15.16:0/4173698095 >> v2:172.21.15.173:3300/0 conn(0x7ffa7013df80 msgr2=0x7ffa7015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:09.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.592+0000 7ffa66ffd640 1 --2- 172.21.15.16:0/4173698095 >> v2:172.21.15.173:3300/0 conn(0x7ffa7013df80 0x7ffa7015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.592+0000 7ffa66ffd640 1 -- 172.21.15.16:0/4173698095 >> v1:172.21.15.173:6789/0 conn(0x7ffa7015f9c0 legacy=0x7ffa701408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:09.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.592+0000 7ffa66ffd640 1 -- 172.21.15.16:0/4173698095 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7006cd10 con 0x7ffa7013e360 2023-11-24T06:29:09.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.593+0000 7ffa66ffd640 1 --2- 172.21.15.16:0/4173698095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 0x7ffa700cf290 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffa70072770 tx=0x7ffa5802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4dc4c7ce300a35 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.593+0000 7ffa6effd640 1 -- 172.21.15.16:0/4173698095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58031870 con 0x7ffa7013e360 2023-11-24T06:29:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.593+0000 7ffa6effd640 1 -- 172.21.15.16:0/4173698095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa580319d0 con 0x7ffa7013e360 2023-11-24T06:29:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.594+0000 7ffa6effd640 1 -- 172.21.15.16:0/4173698095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58031d60 con 0x7ffa7013e360 2023-11-24T06:29:09.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.594+0000 7ffa76248640 1 -- 172.21.15.16:0/4173698095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 msgr2=0x7ffa700cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.594+0000 7ffa76248640 1 --2- 172.21.15.16:0/4173698095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 0x7ffa700cf290 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffa70072770 tx=0x7ffa5802b910 comp rx=0 tx=0).stop 2023-11-24T06:29:09.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.594+0000 7ffa76248640 1 -- 172.21.15.16:0/4173698095 shutdown_connections 2023-11-24T06:29:09.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.594+0000 7ffa76248640 1 --2- 172.21.15.16:0/4173698095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 0x7ffa700cf290 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.594+0000 7ffa76248640 1 --2- 172.21.15.16:0/4173698095 >> v2:172.21.15.173:3300/0 conn(0x7ffa7013df80 0x7ffa7015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.594+0000 7ffa76248640 1 -- 172.21.15.16:0/4173698095 >> 172.21.15.16:0/4173698095 conn(0x7ffa7006a800 msgr2=0x7ffa7006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:09.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.594+0000 7ffa76248640 1 -- 172.21.15.16:0/4173698095 shutdown_connections 2023-11-24T06:29:09.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.594+0000 7ffa76248640 1 -- 172.21.15.16:0/4173698095 wait complete. 2023-11-24T06:29:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.596+0000 7ffa76248640 1 Processor -- start 2023-11-24T06:29:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.596+0000 7ffa76248640 1 -- start start 2023-11-24T06:29:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.597+0000 7ffa76248640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa7013df80 0x7ffa7013c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.597+0000 7ffa76248640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 0x7ffa700b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.597+0000 7ffa76248640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa7015f9c0 0x7ffa70154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.597+0000 7ffa76248640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b2f40 con 0x7ffa7013e360 2023-11-24T06:29:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.597+0000 7ffa76248640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b2330 con 0x7ffa7015f9c0 2023-11-24T06:29:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.597+0000 7ffa76248640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b2610 con 0x7ffa7013df80 2023-11-24T06:29:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.597+0000 7ffa66ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 0x7ffa700b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.597+0000 7ffa66ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 0x7ffa700b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34120/0 (socket says 172.21.15.16:34120) 2023-11-24T06:29:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.597+0000 7ffa66ffd640 1 -- 172.21.15.16:0/2439998301 learned_addr learned my addr 172.21.15.16:0/2439998301 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa6ffff640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa7015f9c0 0x7ffa70154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa6f7fe640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa7013df80 0x7ffa7013c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa66ffd640 1 -- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa7013df80 msgr2=0x7ffa7013c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa66ffd640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa7013df80 0x7ffa7013c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa66ffd640 1 -- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa7015f9c0 msgr2=0x7ffa70154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa66ffd640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa7015f9c0 0x7ffa70154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa66ffd640 1 -- 172.21.15.16:0/2439998301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580094d0 con 0x7ffa7013e360 2023-11-24T06:29:09.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa66ffd640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 0x7ffa700b0240 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffa58000c00 tx=0x7ffa580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:09.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa6cff9640 1 -- 172.21.15.16:0/2439998301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5803abe0 con 0x7ffa7013e360 2023-11-24T06:29:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa6cff9640 1 -- 172.21.15.16:0/2439998301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5803ad40 con 0x7ffa7013e360 2023-11-24T06:29:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa6cff9640 1 -- 172.21.15.16:0/2439998301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58044610 con 0x7ffa7013e360 2023-11-24T06:29:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.598+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa700b28f0 con 0x7ffa7013e360 2023-11-24T06:29:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.599+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7014e690 con 0x7ffa7013e360 2023-11-24T06:29:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.600+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa700675d0 con 0x7ffa7013e360 2023-11-24T06:29:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.601+0000 7ffa6cff9640 1 -- 172.21.15.16:0/2439998301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ffa5804d450 con 0x7ffa7013e360 2023-11-24T06:29:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.602+0000 7ffa6cff9640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa5006af00 0x7ffa5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.602+0000 7ffa6cff9640 1 -- 172.21.15.16:0/2439998301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa5802d040 con 0x7ffa7013e360 2023-11-24T06:29:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.602+0000 7ffa6f7fe640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa5006af00 0x7ffa5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.604+0000 7ffa6f7fe640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa5006af00 0x7ffa5006d3c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffa600099c0 tx=0x7ffa60003990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.606+0000 7ffa6cff9640 1 -- 172.21.15.16:0/2439998301 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa58076d00 con 0x7ffa7013e360 2023-11-24T06:29:09.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.835+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa7006df00 con 0x7ffa7013e360 2023-11-24T06:29:09.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.836+0000 7ffa6cff9640 1 -- 172.21.15.16:0/2439998301 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7ffa58035090 con 0x7ffa7013e360 2023-11-24T06:29:09.840 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:09.840 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110540698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110540698},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110540698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1298790428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1298790428},{"type":"v1","addr":"172.21.15.16:6835","nonce":1298790428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2876598187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2876598187},{"type":"v1","addr":"172.21.15.173:6836","nonce":2876598187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:05.754412+0000","modified":"2023-11-24T06:29:08.837901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/2818758442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2818758442},{"type":"v1","addr":"172.21.15.16:6833","nonce":2818758442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:29:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2023-11-24T06:29:09.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa5006af00 msgr2=0x7ffa5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa5006af00 0x7ffa5006d3c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffa600099c0 tx=0x7ffa60003990 comp rx=0 tx=0).stop 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 msgr2=0x7ffa700b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 0x7ffa700b0240 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ffa58000c00 tx=0x7ffa580046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 shutdown_connections 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa5006af00 0x7ffa5006d3c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa7015f9c0 0x7ffa70154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa7013e360 0x7ffa700b0240 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 --2- 172.21.15.16:0/2439998301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa7013df80 0x7ffa7013c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 >> 172.21.15.16:0/2439998301 conn(0x7ffa7006a800 msgr2=0x7ffa7015d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 shutdown_connections 2023-11-24T06:29:09.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:09.839+0000 7ffa76248640 1 -- 172.21.15.16:0/2439998301 wait complete. 2023-11-24T06:29:09.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.086+0000 7f68382d3640 1 Processor -- start 2023-11-24T06:29:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.087+0000 7f68382d3640 1 -- start start 2023-11-24T06:29:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.088+0000 7f68382d3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6830157c30 0x7f68300cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.088+0000 7f68382d3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68301618f0 0x7f6830070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.088+0000 7f68382d3640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6830071030 0x7f683006dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.088+0000 7f68382d3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683015ea30 con 0x7f6830157c30 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.088+0000 7f68382d3640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683006d860 con 0x7f6830071030 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.088+0000 7f68382d3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6830071460 con 0x7f68301618f0 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.089+0000 7f683606f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6830157c30 0x7f68300cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.089+0000 7f683606f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6830157c30 0x7f68300cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34134/0 (socket says 172.21.15.16:34134) 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.089+0000 7f683606f640 1 -- 172.21.15.16:0/2481717726 learned_addr learned my addr 172.21.15.16:0/2481717726 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.089+0000 7f683606f640 1 -- 172.21.15.16:0/2481717726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68301618f0 msgr2=0x7f6830070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.089+0000 7f683606f640 1 --2- 172.21.15.16:0/2481717726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68301618f0 0x7f6830070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.089+0000 7f683606f640 1 -- 172.21.15.16:0/2481717726 >> v2:172.21.15.173:3300/0 conn(0x7f6830071030 msgr2=0x7f683006dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.089+0000 7f683606f640 1 --2- 172.21.15.16:0/2481717726 >> v2:172.21.15.173:3300/0 conn(0x7f6830071030 0x7f683006dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.089+0000 7f683606f640 1 -- 172.21.15.16:0/2481717726 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683015deb0 con 0x7f6830157c30 2023-11-24T06:29:10.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.090+0000 7f683606f640 1 --2- 172.21.15.16:0/2481717726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830157c30 0x7f68300cd600 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f683006b630 tx=0x7f682c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abe4cd51c55e07d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:10.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.091+0000 7f683506d640 1 -- 172.21.15.16:0/2481717726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c031870 con 0x7f6830157c30 2023-11-24T06:29:10.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.091+0000 7f683506d640 1 -- 172.21.15.16:0/2481717726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c0319d0 con 0x7f6830157c30 2023-11-24T06:29:10.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.091+0000 7f683506d640 1 -- 172.21.15.16:0/2481717726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c031d60 con 0x7f6830157c30 2023-11-24T06:29:10.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.091+0000 7f68382d3640 1 -- 172.21.15.16:0/2481717726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830157c30 msgr2=0x7f68300cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.091+0000 7f68382d3640 1 --2- 172.21.15.16:0/2481717726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830157c30 0x7f68300cd600 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f683006b630 tx=0x7f682c02b910 comp rx=0 tx=0).stop 2023-11-24T06:29:10.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.092+0000 7f68382d3640 1 -- 172.21.15.16:0/2481717726 shutdown_connections 2023-11-24T06:29:10.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.092+0000 7f68382d3640 1 --2- 172.21.15.16:0/2481717726 >> v2:172.21.15.173:3300/0 conn(0x7f6830071030 0x7f683006dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.092+0000 7f68382d3640 1 --2- 172.21.15.16:0/2481717726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68301618f0 0x7f6830070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.092+0000 7f68382d3640 1 --2- 172.21.15.16:0/2481717726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830157c30 0x7f68300cd600 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.092+0000 7f68382d3640 1 -- 172.21.15.16:0/2481717726 >> 172.21.15.16:0/2481717726 conn(0x7f68300cf700 msgr2=0x7f68300cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.092+0000 7f68382d3640 1 -- 172.21.15.16:0/2481717726 shutdown_connections 2023-11-24T06:29:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.092+0000 7f68382d3640 1 -- 172.21.15.16:0/2481717726 wait complete. 2023-11-24T06:29:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.094+0000 7f68382d3640 1 Processor -- start 2023-11-24T06:29:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.094+0000 7f68382d3640 1 -- start start 2023-11-24T06:29:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f68382d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830071030 0x7f68301f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f68382d3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6830157c30 0x7f68301f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f68382d3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68301618f0 0x7f68300b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f68382d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683015ca00 con 0x7f6830071030 2023-11-24T06:29:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f68382d3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301f8160 con 0x7f6830157c30 2023-11-24T06:29:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f68382d3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683015bdf0 con 0x7f68301618f0 2023-11-24T06:29:10.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f683606f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830071030 0x7f68301f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f683606f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830071030 0x7f68301f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34136/0 (socket says 172.21.15.16:34136) 2023-11-24T06:29:10.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f683606f640 1 -- 172.21.15.16:0/3569371093 learned_addr learned my addr 172.21.15.16:0/3569371093 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f683606f640 1 -- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68301618f0 msgr2=0x7f68300b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f683606f640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68301618f0 0x7f68300b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f683606f640 1 -- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6830157c30 msgr2=0x7f68301f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f683606f640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6830157c30 0x7f68301f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.095+0000 7f683606f640 1 -- 172.21.15.16:0/3569371093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c0094d0 con 0x7f6830071030 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.096+0000 7f683606f640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830071030 0x7f68301f7c20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f682c000c00 tx=0x7f682c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.096+0000 7f6826ffd640 1 -- 172.21.15.16:0/3569371093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c031840 con 0x7f6830071030 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.096+0000 7f6826ffd640 1 -- 172.21.15.16:0/3569371093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c0443f0 con 0x7f6830071030 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.096+0000 7f6826ffd640 1 -- 172.21.15.16:0/3569371093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c0447a0 con 0x7f6830071030 2023-11-24T06:29:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.096+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683015c0d0 con 0x7f6830071030 2023-11-24T06:29:10.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.096+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683015c690 con 0x7f6830071030 2023-11-24T06:29:10.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.097+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683006ce10 con 0x7f6830071030 2023-11-24T06:29:10.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.098+0000 7f6826ffd640 1 -- 172.21.15.16:0/3569371093 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f682c04d450 con 0x7f6830071030 2023-11-24T06:29:10.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.099+0000 7f6826ffd640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f681406aed0 0x7f681406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.099+0000 7f6826ffd640 1 -- 172.21.15.16:0/3569371093 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f682c02d040 con 0x7f6830071030 2023-11-24T06:29:10.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.100+0000 7f683586e640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f681406aed0 0x7f681406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.101+0000 7f683586e640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f681406aed0 0x7f681406d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f683014c0b0 tx=0x7f6820006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:10.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.104+0000 7f6826ffd640 1 -- 172.21.15.16:0/3569371093 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f682c076e60 con 0x7f6830071030 2023-11-24T06:29:10.252 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-11-24T06:29:10.249+0000 7fd76da7a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.301+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f683006d3d0 con 0x7f6830071030 2023-11-24T06:29:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.304+0000 7f6826ffd640 1 -- 172.21.15.16:0/3569371093 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16513 (secure 0 0 0) 0x7f682c07a480 con 0x7f6830071030 2023-11-24T06:29:10.308 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:10.308 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:07.314174+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:03.705566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:04.794925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f681406aed0 msgr2=0x7f681406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f681406aed0 0x7f681406d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f683014c0b0 tx=0x7f6820006c90 comp rx=0 tx=0).stop 2023-11-24T06:29:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830071030 msgr2=0x7f68301f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830071030 0x7f68301f7c20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f682c000c00 tx=0x7f682c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 shutdown_connections 2023-11-24T06:29:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f681406aed0 0x7f681406d390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68301618f0 0x7f68300b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6830157c30 0x7f68301f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 --2- 172.21.15.16:0/3569371093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6830071030 0x7f68301f7c20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.307+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 >> 172.21.15.16:0/3569371093 conn(0x7f68300cf700 msgr2=0x7f68300b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:10.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.308+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 shutdown_connections 2023-11-24T06:29:10.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.308+0000 7f68382d3640 1 -- 172.21.15.16:0/3569371093 wait complete. 2023-11-24T06:29:10.329 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:29:10.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.553+0000 7fe1dd431640 1 Processor -- start 2023-11-24T06:29:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.553+0000 7fe1dd431640 1 -- start start 2023-11-24T06:29:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.555+0000 7fe1dd431640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1d813df80 0x7fe1d815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.555+0000 7fe1dd431640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d813e360 0x7fe1d80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.555+0000 7fe1dd431640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d806c470 con 0x7fe1d815f9c0 2023-11-24T06:29:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.555+0000 7fe1dd431640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d806c750 con 0x7fe1d813df80 2023-11-24T06:29:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.555+0000 7fe1dd431640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d806ca30 con 0x7fe1d813e360 2023-11-24T06:29:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.556+0000 7fe1d6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1d813df80 0x7fe1d815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.556+0000 7fe1d6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1d813df80 0x7fe1d815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34146/0 (socket says 172.21.15.16:34146) 2023-11-24T06:29:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.556+0000 7fe1d6ffd640 1 -- 172.21.15.16:0/1133940497 learned_addr learned my addr 172.21.15.16:0/1133940497 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.556+0000 7fe1d67fc640 1 --2- 172.21.15.16:0/1133940497 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d813e360 0x7fe1d80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.556+0000 7fe1d6ffd640 1 -- 172.21.15.16:0/1133940497 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d813e360 msgr2=0x7fe1d80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.556+0000 7fe1d6ffd640 1 --2- 172.21.15.16:0/1133940497 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d813e360 0x7fe1d80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.556+0000 7fe1d6ffd640 1 -- 172.21.15.16:0/1133940497 >> v1:172.21.15.173:6789/0 conn(0x7fe1d815f9c0 legacy=0x7fe1d81408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.556+0000 7fe1d6ffd640 1 -- 172.21.15.16:0/1133940497 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d806cd10 con 0x7fe1d813df80 2023-11-24T06:29:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.556+0000 7fe1d67fc640 1 --2- 172.21.15.16:0/1133940497 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d813e360 0x7fe1d80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.558+0000 7fe1d6ffd640 1 --2- 172.21.15.16:0/1133940497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813df80 0x7fe1d815bee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009870 tx=0x7fe1c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3456d02d0d9fecfe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.558+0000 7fe1d5ffb640 1 -- 172.21.15.16:0/1133940497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8030840 con 0x7fe1d813df80 2023-11-24T06:29:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.558+0000 7fe1d5ffb640 1 -- 172.21.15.16:0/1133940497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c80309a0 con 0x7fe1d813df80 2023-11-24T06:29:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.558+0000 7fe1d5ffb640 1 -- 172.21.15.16:0/1133940497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8030d30 con 0x7fe1d813df80 2023-11-24T06:29:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.558+0000 7fe1dd431640 1 -- 172.21.15.16:0/1133940497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813df80 msgr2=0x7fe1d815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.558+0000 7fe1dd431640 1 --2- 172.21.15.16:0/1133940497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813df80 0x7fe1d815bee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009870 tx=0x7fe1c802b980 comp rx=0 tx=0).stop 2023-11-24T06:29:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.559+0000 7fe1dd431640 1 -- 172.21.15.16:0/1133940497 shutdown_connections 2023-11-24T06:29:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.559+0000 7fe1dd431640 1 --2- 172.21.15.16:0/1133940497 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d813e360 0x7fe1d80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.559+0000 7fe1dd431640 1 --2- 172.21.15.16:0/1133940497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813df80 0x7fe1d815bee0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.559+0000 7fe1dd431640 1 -- 172.21.15.16:0/1133940497 >> 172.21.15.16:0/1133940497 conn(0x7fe1d806a800 msgr2=0x7fe1d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.559+0000 7fe1dd431640 1 -- 172.21.15.16:0/1133940497 shutdown_connections 2023-11-24T06:29:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.559+0000 7fe1dd431640 1 -- 172.21.15.16:0/1133940497 wait complete. 2023-11-24T06:29:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.560+0000 7fe1dd431640 1 Processor -- start 2023-11-24T06:29:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.561+0000 7fe1dd431640 1 -- start start 2023-11-24T06:29:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1dd431640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d813df80 0x7fe1d815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1dd431640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813e360 0x7fe1d81f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1dd431640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d815f9c0 0x7fe1d81f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1dd431640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d8152640 con 0x7fe1d813e360 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1dd431640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d8152920 con 0x7fe1d813df80 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1dd431640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d8152c00 con 0x7fe1d815f9c0 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813e360 0x7fe1d81f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813e360 0x7fe1d81f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34148/0 (socket says 172.21.15.16:34148) 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d67fc640 1 -- 172.21.15.16:0/684757683 learned_addr learned my addr 172.21.15.16:0/684757683 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d77fe640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d815f9c0 0x7fe1d81f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d6ffd640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d813df80 0x7fe1d815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d67fc640 1 -- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d815f9c0 msgr2=0x7fe1d81f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d67fc640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d815f9c0 0x7fe1d81f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d67fc640 1 -- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d813df80 msgr2=0x7fe1d815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d67fc640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d813df80 0x7fe1d815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d67fc640 1 -- 172.21.15.16:0/684757683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80094d0 con 0x7fe1d813e360 2023-11-24T06:29:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.562+0000 7fe1d77fe640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d815f9c0 0x7fe1d81f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.563+0000 7fe1d6ffd640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d813df80 0x7fe1d815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.563+0000 7fe1d67fc640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813e360 0x7fe1d81f59a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00e730 tx=0x7fe1cc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.563+0000 7fe1bf7fe640 1 -- 172.21.15.16:0/684757683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc004230 con 0x7fe1d813e360 2023-11-24T06:29:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.563+0000 7fe1bf7fe640 1 -- 172.21.15.16:0/684757683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc004390 con 0x7fe1d813e360 2023-11-24T06:29:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.563+0000 7fe1bf7fe640 1 -- 172.21.15.16:0/684757683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00c8f0 con 0x7fe1d813e360 2023-11-24T06:29:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.563+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d8152ee0 con 0x7fe1d813e360 2023-11-24T06:29:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.563+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d815e050 con 0x7fe1d813e360 2023-11-24T06:29:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.564+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d80675d0 con 0x7fe1d813e360 2023-11-24T06:29:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.565+0000 7fe1bf7fe640 1 -- 172.21.15.16:0/684757683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe1cc00ca50 con 0x7fe1d813e360 2023-11-24T06:29:10.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.566+0000 7fe1bf7fe640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe1c406aed0 0x7fe1c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.567+0000 7fe1bf7fe640 1 -- 172.21.15.16:0/684757683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1cc0893c0 con 0x7fe1d813e360 2023-11-24T06:29:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.567+0000 7fe1d6ffd640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe1c406aed0 0x7fe1c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:10.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.568+0000 7fe1d6ffd640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe1c406aed0 0x7fe1c406d390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8036000 tx=0x7fe1c803b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:10.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.570+0000 7fe1bf7fe640 1 -- 172.21.15.16:0/684757683 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1cc054300 con 0x7fe1d813e360 2023-11-24T06:29:10.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.760+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1d80ce850 con 0x7fe1c406aed0 2023-11-24T06:29:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.785+0000 7fe1bf7fe640 1 -- 172.21.15.16:0/684757683 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286591 (secure 0 0 0) 0x7fe1c8030be0 con 0x7fe1c406aed0 2023-11-24T06:29:10.796 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:10.797 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe1c406aed0 msgr2=0x7fe1c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe1c406aed0 0x7fe1c406d390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8036000 tx=0x7fe1c803b040 comp rx=0 tx=0).stop 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813e360 msgr2=0x7fe1d81f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813e360 0x7fe1d81f59a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00e730 tx=0x7fe1cc00ebf0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 shutdown_connections 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe1c406aed0 0x7fe1c406d390 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d815f9c0 0x7fe1d81f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1d813e360 0x7fe1d81f59a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 --2- 172.21.15.16:0/684757683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d813df80 0x7fe1d815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 >> 172.21.15.16:0/684757683 conn(0x7fe1d806a800 msgr2=0x7fe1d80af400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:10.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 shutdown_connections 2023-11-24T06:29:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:10.792+0000 7fe1dd431640 1 -- 172.21.15.16:0/684757683 wait complete. 2023-11-24T06:29:10.817 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-24T06:29:10.148016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238272,"kb_used_data":5424,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736448,"statfs":{"total":773094113280,"available":772850122752,"internally_reserved":0,"allocated":5554176,"data_stored":2936524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.876398"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334821+0000","last_change":"2023-11-24T06:29:06.328584+0000","last_active":"2023-11-24T06:29:07.334821+0000","last_peered":"2023-11-24T06:29:07.334821+0000","last_clean":"2023-11-24T06:29:07.334821+0000","last_became_active":"2023-11-24T06:29:06.326783+0000","last_became_peered":"2023-11-24T06:29:06.326783+0000","last_unstale":"2023-11-24T06:29:07.334821+0000","last_undegraded":"2023-11-24T06:29:07.334821+0000","last_fullsized":"2023-11-24T06:29:07.334821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:05.666642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.971199+0000","last_change":"2023-11-24T06:29:06.353972+0000","last_active":"2023-11-24T06:29:07.971199+0000","last_peered":"2023-11-24T06:29:07.971199+0000","last_clean":"2023-11-24T06:29:07.971199+0000","last_became_active":"2023-11-24T06:29:06.353580+0000","last_became_peered":"2023-11-24T06:29:06.353580+0000","last_unstale":"2023-11-24T06:29:07.971199+0000","last_undegraded":"2023-11-24T06:29:07.971199+0000","last_fullsized":"2023-11-24T06:29:07.971199+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:01.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174115+0000","last_change":"2023-11-24T06:29:06.334126+0000","last_active":"2023-11-24T06:29:09.174115+0000","last_peered":"2023-11-24T06:29:09.174115+0000","last_clean":"2023-11-24T06:29:09.174115+0000","last_became_active":"2023-11-24T06:29:06.332002+0000","last_became_peered":"2023-11-24T06:29:06.332002+0000","last_unstale":"2023-11-24T06:29:09.174115+0000","last_undegraded":"2023-11-24T06:29:09.174115+0000","last_fullsized":"2023-11-24T06:29:09.174115+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:29.339213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693838+0000","last_change":"2023-11-24T06:29:06.342691+0000","last_active":"2023-11-24T06:29:08.693838+0000","last_peered":"2023-11-24T06:29:08.693838+0000","last_clean":"2023-11-24T06:29:08.693838+0000","last_became_active":"2023-11-24T06:29:06.342434+0000","last_became_peered":"2023-11-24T06:29:06.342434+0000","last_unstale":"2023-11-24T06:29:08.693838+0000","last_undegraded":"2023-11-24T06:29:08.693838+0000","last_fullsized":"2023-11-24T06:29:08.693838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:35.027087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970607+0000","last_change":"2023-11-24T06:29:06.341116+0000","last_active":"2023-11-24T06:29:07.970607+0000","last_peered":"2023-11-24T06:29:07.970607+0000","last_clean":"2023-11-24T06:29:07.970607+0000","last_became_active":"2023-11-24T06:29:06.340830+0000","last_became_peered":"2023-11-24T06:29:06.340830+0000","last_unstale":"2023-11-24T06:29:07.970607+0000","last_undegraded":"2023-11-24T06:29:07.970607+0000","last_fullsized":"2023-11-24T06:29:07.970607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:36.894234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331931+0000","last_change":"2023-11-24T06:29:06.337450+0000","last_active":"2023-11-24T06:29:07.331931+0000","last_peered":"2023-11-24T06:29:07.331931+0000","last_clean":"2023-11-24T06:29:07.331931+0000","last_became_active":"2023-11-24T06:29:06.337169+0000","last_became_peered":"2023-11-24T06:29:06.337169+0000","last_unstale":"2023-11-24T06:29:07.331931+0000","last_undegraded":"2023-11-24T06:29:07.331931+0000","last_fullsized":"2023-11-24T06:29:07.331931+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:13:07.856195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334104+0000","last_change":"2023-11-24T06:29:06.314221+0000","last_active":"2023-11-24T06:29:07.334104+0000","last_peered":"2023-11-24T06:29:07.334104+0000","last_clean":"2023-11-24T06:29:07.334104+0000","last_became_active":"2023-11-24T06:29:06.313621+0000","last_became_peered":"2023-11-24T06:29:06.313621+0000","last_unstale":"2023-11-24T06:29:07.334104+0000","last_undegraded":"2023-11-24T06:29:07.334104+0000","last_fullsized":"2023-11-24T06:29:07.334104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:32.445582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693288+0000","last_change":"2023-11-24T06:29:06.341882+0000","last_active":"2023-11-24T06:29:08.693288+0000","last_peered":"2023-11-24T06:29:08.693288+0000","last_clean":"2023-11-24T06:29:08.693288+0000","last_became_active":"2023-11-24T06:29:06.341588+0000","last_became_peered":"2023-11-24T06:29:06.341588+0000","last_unstale":"2023-11-24T06:29:08.693288+0000","last_undegraded":"2023-11-24T06:29:08.693288+0000","last_fullsized":"2023-11-24T06:29:08.693288+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:54.640138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694245+0000","last_change":"2023-11-24T06:29:06.347482+0000","last_active":"2023-11-24T06:29:08.694245+0000","last_peered":"2023-11-24T06:29:08.694245+0000","last_clean":"2023-11-24T06:29:08.694245+0000","last_became_active":"2023-11-24T06:29:06.347069+0000","last_became_peered":"2023-11-24T06:29:06.347069+0000","last_unstale":"2023-11-24T06:29:08.694245+0000","last_undegraded":"2023-11-24T06:29:08.694245+0000","last_fullsized":"2023-11-24T06:29:08.694245+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:01.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693809+0000","last_change":"2023-11-24T06:29:06.347193+0000","last_active":"2023-11-24T06:29:08.693809+0000","last_peered":"2023-11-24T06:29:08.693809+0000","last_clean":"2023-11-24T06:29:08.693809+0000","last_became_active":"2023-11-24T06:29:06.346918+0000","last_became_peered":"2023-11-24T06:29:06.346918+0000","last_unstale":"2023-11-24T06:29:08.693809+0000","last_undegraded":"2023-11-24T06:29:08.693809+0000","last_fullsized":"2023-11-24T06:29:08.693809+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:12.374074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694016+0000","last_change":"2023-11-24T06:29:06.343158+0000","last_active":"2023-11-24T06:29:08.694016+0000","last_peered":"2023-11-24T06:29:08.694016+0000","last_clean":"2023-11-24T06:29:08.694016+0000","last_became_active":"2023-11-24T06:29:06.342895+0000","last_became_peered":"2023-11-24T06:29:06.342895+0000","last_unstale":"2023-11-24T06:29:08.694016+0000","last_undegraded":"2023-11-24T06:29:08.694016+0000","last_fullsized":"2023-11-24T06:29:08.694016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:00.100140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970561+0000","last_change":"2023-11-24T06:29:06.326238+0000","last_active":"2023-11-24T06:29:07.970561+0000","last_peered":"2023-11-24T06:29:07.970561+0000","last_clean":"2023-11-24T06:29:07.970561+0000","last_became_active":"2023-11-24T06:29:06.325845+0000","last_became_peered":"2023-11-24T06:29:06.325845+0000","last_unstale":"2023-11-24T06:29:07.970561+0000","last_undegraded":"2023-11-24T06:29:07.970561+0000","last_fullsized":"2023-11-24T06:29:07.970561+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:45.404029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328231+0000","last_change":"2023-11-24T06:29:06.334566+0000","last_active":"2023-11-24T06:29:07.328231+0000","last_peered":"2023-11-24T06:29:07.328231+0000","last_clean":"2023-11-24T06:29:07.328231+0000","last_became_active":"2023-11-24T06:29:06.334212+0000","last_became_peered":"2023-11-24T06:29:06.334212+0000","last_unstale":"2023-11-24T06:29:07.328231+0000","last_undegraded":"2023-11-24T06:29:07.328231+0000","last_fullsized":"2023-11-24T06:29:07.328231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:44:07.134660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693063+0000","last_change":"2023-11-24T06:29:06.347734+0000","last_active":"2023-11-24T06:29:08.693063+0000","last_peered":"2023-11-24T06:29:08.693063+0000","last_clean":"2023-11-24T06:29:08.693063+0000","last_became_active":"2023-11-24T06:29:06.346876+0000","last_became_peered":"2023-11-24T06:29:06.346876+0000","last_unstale":"2023-11-24T06:29:08.693063+0000","last_undegraded":"2023-11-24T06:29:08.693063+0000","last_fullsized":"2023-11-24T06:29:08.693063+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:51.205136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331877+0000","last_change":"2023-11-24T06:29:06.327701+0000","last_active":"2023-11-24T06:29:07.331877+0000","last_peered":"2023-11-24T06:29:07.331877+0000","last_clean":"2023-11-24T06:29:07.331877+0000","last_became_active":"2023-11-24T06:29:06.327345+0000","last_became_peered":"2023-11-24T06:29:06.327345+0000","last_unstale":"2023-11-24T06:29:07.331877+0000","last_undegraded":"2023-11-24T06:29:07.331877+0000","last_fullsized":"2023-11-24T06:29:07.331877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:31.638204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367613+0000","last_change":"2023-11-24T06:29:06.344131+0000","last_active":"2023-11-24T06:29:07.367613+0000","last_peered":"2023-11-24T06:29:07.367613+0000","last_clean":"2023-11-24T06:29:07.367613+0000","last_became_active":"2023-11-24T06:29:06.343794+0000","last_became_peered":"2023-11-24T06:29:06.343794+0000","last_unstale":"2023-11-24T06:29:07.367613+0000","last_undegraded":"2023-11-24T06:29:07.367613+0000","last_fullsized":"2023-11-24T06:29:07.367613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:34.946415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174363+0000","last_change":"2023-11-24T06:29:06.331570+0000","last_active":"2023-11-24T06:29:09.174363+0000","last_peered":"2023-11-24T06:29:09.174363+0000","last_clean":"2023-11-24T06:29:09.174363+0000","last_became_active":"2023-11-24T06:29:06.329451+0000","last_became_peered":"2023-11-24T06:29:06.329451+0000","last_unstale":"2023-11-24T06:29:09.174363+0000","last_undegraded":"2023-11-24T06:29:09.174363+0000","last_fullsized":"2023-11-24T06:29:09.174363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:06.918223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174389+0000","last_change":"2023-11-24T06:29:06.355860+0000","last_active":"2023-11-24T06:29:09.174389+0000","last_peered":"2023-11-24T06:29:09.174389+0000","last_clean":"2023-11-24T06:29:09.174389+0000","last_became_active":"2023-11-24T06:29:06.355323+0000","last_became_peered":"2023-11-24T06:29:06.355323+0000","last_unstale":"2023-11-24T06:29:09.174389+0000","last_undegraded":"2023-11-24T06:29:09.174389+0000","last_fullsized":"2023-11-24T06:29:09.174389+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:46.035866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368585+0000","last_change":"2023-11-24T06:29:06.350773+0000","last_active":"2023-11-24T06:29:07.368585+0000","last_peered":"2023-11-24T06:29:07.368585+0000","last_clean":"2023-11-24T06:29:07.368585+0000","last_became_active":"2023-11-24T06:29:06.350451+0000","last_became_peered":"2023-11-24T06:29:06.350451+0000","last_unstale":"2023-11-24T06:29:07.368585+0000","last_undegraded":"2023-11-24T06:29:07.368585+0000","last_fullsized":"2023-11-24T06:29:07.368585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:29.060969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328650+0000","last_change":"2023-11-24T06:29:06.341930+0000","last_active":"2023-11-24T06:29:07.328650+0000","last_peered":"2023-11-24T06:29:07.328650+0000","last_clean":"2023-11-24T06:29:07.328650+0000","last_became_active":"2023-11-24T06:29:06.341172+0000","last_became_peered":"2023-11-24T06:29:06.341172+0000","last_unstale":"2023-11-24T06:29:07.328650+0000","last_undegraded":"2023-11-24T06:29:07.328650+0000","last_fullsized":"2023-11-24T06:29:07.328650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:37.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331956+0000","last_change":"2023-11-24T06:29:06.335775+0000","last_active":"2023-11-24T06:29:07.331956+0000","last_peered":"2023-11-24T06:29:07.331956+0000","last_clean":"2023-11-24T06:29:07.331956+0000","last_became_active":"2023-11-24T06:29:06.335340+0000","last_became_peered":"2023-11-24T06:29:06.335340+0000","last_unstale":"2023-11-24T06:29:07.331956+0000","last_undegraded":"2023-11-24T06:29:07.331956+0000","last_fullsized":"2023-11-24T06:29:07.331956+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:58.147360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693280+0000","last_change":"2023-11-24T06:29:06.342019+0000","last_active":"2023-11-24T06:29:08.693280+0000","last_peered":"2023-11-24T06:29:08.693280+0000","last_clean":"2023-11-24T06:29:08.693280+0000","last_became_active":"2023-11-24T06:29:06.341628+0000","last_became_peered":"2023-11-24T06:29:06.341628+0000","last_unstale":"2023-11-24T06:29:08.693280+0000","last_undegraded":"2023-11-24T06:29:08.693280+0000","last_fullsized":"2023-11-24T06:29:08.693280+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:05.535656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367668+0000","last_change":"2023-11-24T06:29:06.330209+0000","last_active":"2023-11-24T06:29:07.367668+0000","last_peered":"2023-11-24T06:29:07.367668+0000","last_clean":"2023-11-24T06:29:07.367668+0000","last_became_active":"2023-11-24T06:29:06.329923+0000","last_became_peered":"2023-11-24T06:29:06.329923+0000","last_unstale":"2023-11-24T06:29:07.367668+0000","last_undegraded":"2023-11-24T06:29:07.367668+0000","last_fullsized":"2023-11-24T06:29:07.367668+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:02.271987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367844+0000","last_change":"2023-11-24T06:29:06.326109+0000","last_active":"2023-11-24T06:29:07.367844+0000","last_peered":"2023-11-24T06:29:07.367844+0000","last_clean":"2023-11-24T06:29:07.367844+0000","last_became_active":"2023-11-24T06:29:06.325819+0000","last_became_peered":"2023-11-24T06:29:06.325819+0000","last_unstale":"2023-11-24T06:29:07.367844+0000","last_undegraded":"2023-11-24T06:29:07.367844+0000","last_fullsized":"2023-11-24T06:29:07.367844+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:29:34.079427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368178+0000","last_change":"2023-11-24T06:29:06.332397+0000","last_active":"2023-11-24T06:29:07.368178+0000","last_peered":"2023-11-24T06:29:07.368178+0000","last_clean":"2023-11-24T06:29:07.368178+0000","last_became_active":"2023-11-24T06:29:06.331595+0000","last_became_peered":"2023-11-24T06:29:06.331595+0000","last_unstale":"2023-11-24T06:29:07.368178+0000","last_undegraded":"2023-11-24T06:29:07.368178+0000","last_fullsized":"2023-11-24T06:29:07.368178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:06.453332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328696+0000","last_change":"2023-11-24T06:29:06.336701+0000","last_active":"2023-11-24T06:29:07.328696+0000","last_peered":"2023-11-24T06:29:07.328696+0000","last_clean":"2023-11-24T06:29:07.328696+0000","last_became_active":"2023-11-24T06:29:06.334281+0000","last_became_peered":"2023-11-24T06:29:06.334281+0000","last_unstale":"2023-11-24T06:29:07.328696+0000","last_undegraded":"2023-11-24T06:29:07.328696+0000","last_fullsized":"2023-11-24T06:29:07.328696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:06:21.778193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174342+0000","last_change":"2023-11-24T06:29:06.323064+0000","last_active":"2023-11-24T06:29:09.174342+0000","last_peered":"2023-11-24T06:29:09.174342+0000","last_clean":"2023-11-24T06:29:09.174342+0000","last_became_active":"2023-11-24T06:29:06.322653+0000","last_became_peered":"2023-11-24T06:29:06.322653+0000","last_unstale":"2023-11-24T06:29:09.174342+0000","last_undegraded":"2023-11-24T06:29:09.174342+0000","last_fullsized":"2023-11-24T06:29:09.174342+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:51.205136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970531+0000","last_change":"2023-11-24T06:29:06.326138+0000","last_active":"2023-11-24T06:29:07.970531+0000","last_peered":"2023-11-24T06:29:07.970531+0000","last_clean":"2023-11-24T06:29:07.970531+0000","last_became_active":"2023-11-24T06:29:06.325450+0000","last_became_peered":"2023-11-24T06:29:06.325450+0000","last_unstale":"2023-11-24T06:29:07.970531+0000","last_undegraded":"2023-11-24T06:29:07.970531+0000","last_fullsized":"2023-11-24T06:29:07.970531+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:00.100140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367562+0000","last_change":"2023-11-24T06:29:06.333238+0000","last_active":"2023-11-24T06:29:07.367562+0000","last_peered":"2023-11-24T06:29:07.367562+0000","last_clean":"2023-11-24T06:29:07.367562+0000","last_became_active":"2023-11-24T06:29:06.332927+0000","last_became_peered":"2023-11-24T06:29:06.332927+0000","last_unstale":"2023-11-24T06:29:07.367562+0000","last_undegraded":"2023-11-24T06:29:07.367562+0000","last_fullsized":"2023-11-24T06:29:07.367562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:14.728462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.173972+0000","last_change":"2023-11-24T06:29:06.327451+0000","last_active":"2023-11-24T06:29:09.173972+0000","last_peered":"2023-11-24T06:29:09.173972+0000","last_clean":"2023-11-24T06:29:09.173972+0000","last_became_active":"2023-11-24T06:29:06.327166+0000","last_became_peered":"2023-11-24T06:29:06.327166+0000","last_unstale":"2023-11-24T06:29:09.173972+0000","last_undegraded":"2023-11-24T06:29:09.173972+0000","last_fullsized":"2023-11-24T06:29:09.173972+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:29.060969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331798+0000","last_change":"2023-11-24T06:29:06.327708+0000","last_active":"2023-11-24T06:29:07.331798+0000","last_peered":"2023-11-24T06:29:07.331798+0000","last_clean":"2023-11-24T06:29:07.331798+0000","last_became_active":"2023-11-24T06:29:06.327332+0000","last_became_peered":"2023-11-24T06:29:06.327332+0000","last_unstale":"2023-11-24T06:29:07.331798+0000","last_undegraded":"2023-11-24T06:29:07.331798+0000","last_fullsized":"2023-11-24T06:29:07.331798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:06.670418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328039+0000","last_change":"2023-11-24T06:29:06.327041+0000","last_active":"2023-11-24T06:29:07.328039+0000","last_peered":"2023-11-24T06:29:07.328039+0000","last_clean":"2023-11-24T06:29:07.328039+0000","last_became_active":"2023-11-24T06:29:06.326686+0000","last_became_peered":"2023-11-24T06:29:06.326686+0000","last_unstale":"2023-11-24T06:29:07.328039+0000","last_undegraded":"2023-11-24T06:29:07.328039+0000","last_fullsized":"2023-11-24T06:29:07.328039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:38.297508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847556+0000","last_change":"2023-11-24T06:29:06.328709+0000","last_active":"2023-11-24T06:29:08.847556+0000","last_peered":"2023-11-24T06:29:08.847556+0000","last_clean":"2023-11-24T06:29:08.847556+0000","last_became_active":"2023-11-24T06:29:06.328417+0000","last_became_peered":"2023-11-24T06:29:06.328417+0000","last_unstale":"2023-11-24T06:29:08.847556+0000","last_undegraded":"2023-11-24T06:29:08.847556+0000","last_fullsized":"2023-11-24T06:29:08.847556+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:56.358878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332926+0000","last_change":"2023-11-24T06:29:05.315430+0000","last_active":"2023-11-24T06:29:07.332926+0000","last_peered":"2023-11-24T06:29:07.332926+0000","last_clean":"2023-11-24T06:29:07.332926+0000","last_became_active":"2023-11-24T06:29:05.314673+0000","last_became_peered":"2023-11-24T06:29:05.314673+0000","last_unstale":"2023-11-24T06:29:07.332926+0000","last_undegraded":"2023-11-24T06:29:07.332926+0000","last_fullsized":"2023-11-24T06:29:07.332926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:14.294532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368144+0000","last_change":"2023-11-24T06:29:06.330031+0000","last_active":"2023-11-24T06:29:07.368144+0000","last_peered":"2023-11-24T06:29:07.368144+0000","last_clean":"2023-11-24T06:29:07.368144+0000","last_became_active":"2023-11-24T06:29:06.328959+0000","last_became_peered":"2023-11-24T06:29:06.328959+0000","last_unstale":"2023-11-24T06:29:07.368144+0000","last_undegraded":"2023-11-24T06:29:07.368144+0000","last_fullsized":"2023-11-24T06:29:07.368144+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:36.840838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174866+0000","last_change":"2023-11-24T06:29:05.316968+0000","last_active":"2023-11-24T06:29:09.174866+0000","last_peered":"2023-11-24T06:29:09.174866+0000","last_clean":"2023-11-24T06:29:09.174866+0000","last_became_active":"2023-11-24T06:29:05.316628+0000","last_became_peered":"2023-11-24T06:29:05.316628+0000","last_unstale":"2023-11-24T06:29:09.174866+0000","last_undegraded":"2023-11-24T06:29:09.174866+0000","last_fullsized":"2023-11-24T06:29:09.174866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:05.439986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970743+0000","last_change":"2023-11-24T06:29:06.331094+0000","last_active":"2023-11-24T06:29:07.970743+0000","last_peered":"2023-11-24T06:29:07.970743+0000","last_clean":"2023-11-24T06:29:07.970743+0000","last_became_active":"2023-11-24T06:29:06.329737+0000","last_became_peered":"2023-11-24T06:29:06.329737+0000","last_unstale":"2023-11-24T06:29:07.970743+0000","last_undegraded":"2023-11-24T06:29:07.970743+0000","last_fullsized":"2023-11-24T06:29:07.970743+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:32.445582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847491+0000","last_change":"2023-11-24T06:29:05.300374+0000","last_active":"2023-11-24T06:29:08.847491+0000","last_peered":"2023-11-24T06:29:08.847491+0000","last_clean":"2023-11-24T06:29:08.847491+0000","last_became_active":"2023-11-24T06:29:05.298984+0000","last_became_peered":"2023-11-24T06:29:05.298984+0000","last_unstale":"2023-11-24T06:29:08.847491+0000","last_undegraded":"2023-11-24T06:29:08.847491+0000","last_fullsized":"2023-11-24T06:29:08.847491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:13.715116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847401+0000","last_change":"2023-11-24T06:29:06.327156+0000","last_active":"2023-11-24T06:29:08.847401+0000","last_peered":"2023-11-24T06:29:08.847401+0000","last_clean":"2023-11-24T06:29:08.847401+0000","last_became_active":"2023-11-24T06:29:06.326863+0000","last_became_peered":"2023-11-24T06:29:06.326863+0000","last_unstale":"2023-11-24T06:29:08.847401+0000","last_undegraded":"2023-11-24T06:29:08.847401+0000","last_fullsized":"2023-11-24T06:29:08.847401+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:06.453332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334422+0000","last_change":"2023-11-24T06:29:05.305604+0000","last_active":"2023-11-24T06:29:07.334422+0000","last_peered":"2023-11-24T06:29:07.334422+0000","last_clean":"2023-11-24T06:29:07.334422+0000","last_became_active":"2023-11-24T06:29:05.305231+0000","last_became_peered":"2023-11-24T06:29:05.305231+0000","last_unstale":"2023-11-24T06:29:07.334422+0000","last_undegraded":"2023-11-24T06:29:07.334422+0000","last_fullsized":"2023-11-24T06:29:07.334422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:45.022521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.173989+0000","last_change":"2023-11-24T06:29:06.333673+0000","last_active":"2023-11-24T06:29:09.173989+0000","last_peered":"2023-11-24T06:29:09.173989+0000","last_clean":"2023-11-24T06:29:09.173989+0000","last_became_active":"2023-11-24T06:29:06.332688+0000","last_became_peered":"2023-11-24T06:29:06.332688+0000","last_unstale":"2023-11-24T06:29:09.173989+0000","last_undegraded":"2023-11-24T06:29:09.173989+0000","last_fullsized":"2023-11-24T06:29:09.173989+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:38.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332166+0000","last_change":"2023-11-24T06:29:05.302707+0000","last_active":"2023-11-24T06:29:07.332166+0000","last_peered":"2023-11-24T06:29:07.332166+0000","last_clean":"2023-11-24T06:29:07.332166+0000","last_became_active":"2023-11-24T06:29:05.302478+0000","last_became_peered":"2023-11-24T06:29:05.302478+0000","last_unstale":"2023-11-24T06:29:07.332166+0000","last_undegraded":"2023-11-24T06:29:07.332166+0000","last_fullsized":"2023-11-24T06:29:07.332166+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:23:29.372693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694261+0000","last_change":"2023-11-24T06:29:06.340275+0000","last_active":"2023-11-24T06:29:08.694261+0000","last_peered":"2023-11-24T06:29:08.694261+0000","last_clean":"2023-11-24T06:29:08.694261+0000","last_became_active":"2023-11-24T06:29:06.339516+0000","last_became_peered":"2023-11-24T06:29:06.339516+0000","last_unstale":"2023-11-24T06:29:08.694261+0000","last_undegraded":"2023-11-24T06:29:08.694261+0000","last_fullsized":"2023-11-24T06:29:08.694261+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:08.411981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174481+0000","last_change":"2023-11-24T06:29:05.312126+0000","last_active":"2023-11-24T06:29:09.174481+0000","last_peered":"2023-11-24T06:29:09.174481+0000","last_clean":"2023-11-24T06:29:09.174481+0000","last_became_active":"2023-11-24T06:29:05.311140+0000","last_became_peered":"2023-11-24T06:29:05.311140+0000","last_unstale":"2023-11-24T06:29:09.174481+0000","last_undegraded":"2023-11-24T06:29:09.174481+0000","last_fullsized":"2023-11-24T06:29:09.174481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:08.253511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693247+0000","last_change":"2023-11-24T06:29:06.338882+0000","last_active":"2023-11-24T06:29:08.693247+0000","last_peered":"2023-11-24T06:29:08.693247+0000","last_clean":"2023-11-24T06:29:08.693247+0000","last_became_active":"2023-11-24T06:29:06.338613+0000","last_became_peered":"2023-11-24T06:29:06.338613+0000","last_unstale":"2023-11-24T06:29:08.693247+0000","last_undegraded":"2023-11-24T06:29:08.693247+0000","last_fullsized":"2023-11-24T06:29:08.693247+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:46.035866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970140+0000","last_change":"2023-11-24T06:29:05.295971+0000","last_active":"2023-11-24T06:29:07.970140+0000","last_peered":"2023-11-24T06:29:07.970140+0000","last_clean":"2023-11-24T06:29:07.970140+0000","last_became_active":"2023-11-24T06:29:05.295555+0000","last_became_peered":"2023-11-24T06:29:05.295555+0000","last_unstale":"2023-11-24T06:29:07.970140+0000","last_undegraded":"2023-11-24T06:29:07.970140+0000","last_fullsized":"2023-11-24T06:29:07.970140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:28.047624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367349+0000","last_change":"2023-11-24T06:29:06.333751+0000","last_active":"2023-11-24T06:29:07.367349+0000","last_peered":"2023-11-24T06:29:07.367349+0000","last_clean":"2023-11-24T06:29:07.367349+0000","last_became_active":"2023-11-24T06:29:06.332467+0000","last_became_peered":"2023-11-24T06:29:06.332467+0000","last_unstale":"2023-11-24T06:29:07.367349+0000","last_undegraded":"2023-11-24T06:29:07.367349+0000","last_fullsized":"2023-11-24T06:29:07.367349+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:09.266856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693538+0000","last_change":"2023-11-24T06:29:05.302909+0000","last_active":"2023-11-24T06:29:08.693538+0000","last_peered":"2023-11-24T06:29:08.693538+0000","last_clean":"2023-11-24T06:29:08.693538+0000","last_became_active":"2023-11-24T06:29:05.301890+0000","last_became_peered":"2023-11-24T06:29:05.301890+0000","last_unstale":"2023-11-24T06:29:08.693538+0000","last_undegraded":"2023-11-24T06:29:08.693538+0000","last_fullsized":"2023-11-24T06:29:08.693538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:24.625401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174764+0000","last_change":"2023-11-24T06:29:06.356308+0000","last_active":"2023-11-24T06:29:09.174764+0000","last_peered":"2023-11-24T06:29:09.174764+0000","last_clean":"2023-11-24T06:29:09.174764+0000","last_became_active":"2023-11-24T06:29:06.355808+0000","last_became_peered":"2023-11-24T06:29:06.355808+0000","last_unstale":"2023-11-24T06:29:09.174764+0000","last_undegraded":"2023-11-24T06:29:09.174764+0000","last_fullsized":"2023-11-24T06:29:09.174764+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:54.640138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174603+0000","last_change":"2023-11-24T06:29:05.309896+0000","last_active":"2023-11-24T06:29:09.174603+0000","last_peered":"2023-11-24T06:29:09.174603+0000","last_clean":"2023-11-24T06:29:09.174603+0000","last_became_active":"2023-11-24T06:29:05.306490+0000","last_became_peered":"2023-11-24T06:29:05.306490+0000","last_unstale":"2023-11-24T06:29:09.174603+0000","last_undegraded":"2023-11-24T06:29:09.174603+0000","last_fullsized":"2023-11-24T06:29:09.174603+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:57:31.088381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334968+0000","last_change":"2023-11-24T06:29:06.320137+0000","last_active":"2023-11-24T06:29:07.334968+0000","last_peered":"2023-11-24T06:29:07.334968+0000","last_clean":"2023-11-24T06:29:07.334968+0000","last_became_active":"2023-11-24T06:29:06.319811+0000","last_became_peered":"2023-11-24T06:29:06.319811+0000","last_unstale":"2023-11-24T06:29:07.334968+0000","last_undegraded":"2023-11-24T06:29:07.334968+0000","last_fullsized":"2023-11-24T06:29:07.334968+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:28.783174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693911+0000","last_change":"2023-11-24T06:29:05.304961+0000","last_active":"2023-11-24T06:29:08.693911+0000","last_peered":"2023-11-24T06:29:08.693911+0000","last_clean":"2023-11-24T06:29:08.693911+0000","last_became_active":"2023-11-24T06:29:05.304645+0000","last_became_peered":"2023-11-24T06:29:05.304645+0000","last_unstale":"2023-11-24T06:29:08.693911+0000","last_undegraded":"2023-11-24T06:29:08.693911+0000","last_fullsized":"2023-11-24T06:29:08.693911+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:37.004764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332604+0000","last_change":"2023-11-24T06:29:06.329494+0000","last_active":"2023-11-24T06:29:07.332604+0000","last_peered":"2023-11-24T06:29:07.332604+0000","last_clean":"2023-11-24T06:29:07.332604+0000","last_became_active":"2023-11-24T06:29:06.329174+0000","last_became_peered":"2023-11-24T06:29:06.329174+0000","last_unstale":"2023-11-24T06:29:07.332604+0000","last_undegraded":"2023-11-24T06:29:07.332604+0000","last_fullsized":"2023-11-24T06:29:07.332604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:22.265792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334869+0000","last_change":"2023-11-24T06:29:05.316665+0000","last_active":"2023-11-24T06:29:07.334869+0000","last_peered":"2023-11-24T06:29:07.334869+0000","last_clean":"2023-11-24T06:29:07.334869+0000","last_became_active":"2023-11-24T06:29:05.316403+0000","last_became_peered":"2023-11-24T06:29:05.316403+0000","last_unstale":"2023-11-24T06:29:07.334869+0000","last_undegraded":"2023-11-24T06:29:07.334869+0000","last_fullsized":"2023-11-24T06:29:07.334869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:59.086794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334595+0000","last_change":"2023-11-24T06:29:06.327449+0000","last_active":"2023-11-24T06:29:07.334595+0000","last_peered":"2023-11-24T06:29:07.334595+0000","last_clean":"2023-11-24T06:29:07.334595+0000","last_became_active":"2023-11-24T06:29:06.327156+0000","last_became_peered":"2023-11-24T06:29:06.327156+0000","last_unstale":"2023-11-24T06:29:07.334595+0000","last_undegraded":"2023-11-24T06:29:07.334595+0000","last_fullsized":"2023-11-24T06:29:07.334595+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:08.411981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328753+0000","last_change":"2023-11-24T06:29:05.302526+0000","last_active":"2023-11-24T06:29:07.328753+0000","last_peered":"2023-11-24T06:29:07.328753+0000","last_clean":"2023-11-24T06:29:07.328753+0000","last_became_active":"2023-11-24T06:29:05.302257+0000","last_became_peered":"2023-11-24T06:29:05.302257+0000","last_unstale":"2023-11-24T06:29:07.328753+0000","last_undegraded":"2023-11-24T06:29:07.328753+0000","last_fullsized":"2023-11-24T06:29:07.328753+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:44.641158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368307+0000","last_change":"2023-11-24T06:29:06.349088+0000","last_active":"2023-11-24T06:29:07.368307+0000","last_peered":"2023-11-24T06:29:07.368307+0000","last_clean":"2023-11-24T06:29:07.368307+0000","last_became_active":"2023-11-24T06:29:06.348771+0000","last_became_peered":"2023-11-24T06:29:06.348771+0000","last_unstale":"2023-11-24T06:29:07.368307+0000","last_undegraded":"2023-11-24T06:29:07.368307+0000","last_fullsized":"2023-11-24T06:29:07.368307+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:56.358878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332293+0000","last_change":"2023-11-24T06:29:05.299387+0000","last_active":"2023-11-24T06:29:07.332293+0000","last_peered":"2023-11-24T06:29:07.332293+0000","last_clean":"2023-11-24T06:29:07.332293+0000","last_became_active":"2023-11-24T06:29:05.299110+0000","last_became_peered":"2023-11-24T06:29:05.299110+0000","last_unstale":"2023-11-24T06:29:07.332293+0000","last_undegraded":"2023-11-24T06:29:07.332293+0000","last_fullsized":"2023-11-24T06:29:07.332293+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:08.652639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694058+0000","last_change":"2023-11-24T06:29:06.345421+0000","last_active":"2023-11-24T06:29:08.694058+0000","last_peered":"2023-11-24T06:29:08.694058+0000","last_clean":"2023-11-24T06:29:08.694058+0000","last_became_active":"2023-11-24T06:29:06.345062+0000","last_became_peered":"2023-11-24T06:29:06.345062+0000","last_unstale":"2023-11-24T06:29:08.694058+0000","last_undegraded":"2023-11-24T06:29:08.694058+0000","last_fullsized":"2023-11-24T06:29:08.694058+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:45.404029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334123+0000","last_change":"2023-11-24T06:29:05.301715+0000","last_active":"2023-11-24T06:29:07.334123+0000","last_peered":"2023-11-24T06:29:07.334123+0000","last_clean":"2023-11-24T06:29:07.334123+0000","last_became_active":"2023-11-24T06:29:05.301351+0000","last_became_peered":"2023-11-24T06:29:05.301351+0000","last_unstale":"2023-11-24T06:29:07.334123+0000","last_undegraded":"2023-11-24T06:29:07.334123+0000","last_fullsized":"2023-11-24T06:29:07.334123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.191791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331824+0000","last_change":"2023-11-24T06:29:06.332462+0000","last_active":"2023-11-24T06:29:07.331824+0000","last_peered":"2023-11-24T06:29:07.331824+0000","last_clean":"2023-11-24T06:29:07.331824+0000","last_became_active":"2023-11-24T06:29:06.331894+0000","last_became_peered":"2023-11-24T06:29:06.331894+0000","last_unstale":"2023-11-24T06:29:07.331824+0000","last_undegraded":"2023-11-24T06:29:07.331824+0000","last_fullsized":"2023-11-24T06:29:07.331824+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:07.056016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331928+0000","last_change":"2023-11-24T06:29:05.302681+0000","last_active":"2023-11-24T06:29:07.331928+0000","last_peered":"2023-11-24T06:29:07.331928+0000","last_clean":"2023-11-24T06:29:07.331928+0000","last_became_active":"2023-11-24T06:29:05.302383+0000","last_became_peered":"2023-11-24T06:29:05.302383+0000","last_unstale":"2023-11-24T06:29:07.331928+0000","last_undegraded":"2023-11-24T06:29:07.331928+0000","last_fullsized":"2023-11-24T06:29:07.331928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:44.390683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334184+0000","last_change":"2023-11-24T06:29:06.330350+0000","last_active":"2023-11-24T06:29:07.334184+0000","last_peered":"2023-11-24T06:29:07.334184+0000","last_clean":"2023-11-24T06:29:07.334184+0000","last_became_active":"2023-11-24T06:29:06.329973+0000","last_became_peered":"2023-11-24T06:29:06.329973+0000","last_unstale":"2023-11-24T06:29:07.334184+0000","last_undegraded":"2023-11-24T06:29:07.334184+0000","last_fullsized":"2023-11-24T06:29:07.334184+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:01.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693567+0000","last_change":"2023-11-24T06:29:05.313668+0000","last_active":"2023-11-24T06:29:08.693567+0000","last_peered":"2023-11-24T06:29:08.693567+0000","last_clean":"2023-11-24T06:29:08.693567+0000","last_became_active":"2023-11-24T06:29:05.313336+0000","last_became_peered":"2023-11-24T06:29:05.313336+0000","last_unstale":"2023-11-24T06:29:08.693567+0000","last_undegraded":"2023-11-24T06:29:08.693567+0000","last_fullsized":"2023-11-24T06:29:08.693567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:19.332329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328737+0000","last_change":"2023-11-24T06:29:06.323089+0000","last_active":"2023-11-24T06:29:07.328737+0000","last_peered":"2023-11-24T06:29:07.328737+0000","last_clean":"2023-11-24T06:29:07.328737+0000","last_became_active":"2023-11-24T06:29:06.322785+0000","last_became_peered":"2023-11-24T06:29:06.322785+0000","last_unstale":"2023-11-24T06:29:07.328737+0000","last_undegraded":"2023-11-24T06:29:07.328737+0000","last_fullsized":"2023-11-24T06:29:07.328737+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:36.785281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328588+0000","last_change":"2023-11-24T06:29:05.301582+0000","last_active":"2023-11-24T06:29:07.328588+0000","last_peered":"2023-11-24T06:29:07.328588+0000","last_clean":"2023-11-24T06:29:07.328588+0000","last_became_active":"2023-11-24T06:29:05.300677+0000","last_became_peered":"2023-11-24T06:29:05.300677+0000","last_unstale":"2023-11-24T06:29:07.328588+0000","last_undegraded":"2023-11-24T06:29:07.328588+0000","last_fullsized":"2023-11-24T06:29:07.328588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:20.950862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694398+0000","last_change":"2023-11-24T06:29:06.350894+0000","last_active":"2023-11-24T06:29:08.694398+0000","last_peered":"2023-11-24T06:29:08.694398+0000","last_clean":"2023-11-24T06:29:08.694398+0000","last_became_active":"2023-11-24T06:29:06.350625+0000","last_became_peered":"2023-11-24T06:29:06.350625+0000","last_unstale":"2023-11-24T06:29:08.694398+0000","last_undegraded":"2023-11-24T06:29:08.694398+0000","last_fullsized":"2023-11-24T06:29:08.694398+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:15.307878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174994+0000","last_change":"2023-11-24T06:29:05.311136+0000","last_active":"2023-11-24T06:29:09.174994+0000","last_peered":"2023-11-24T06:29:09.174994+0000","last_clean":"2023-11-24T06:29:09.174994+0000","last_became_active":"2023-11-24T06:29:05.310389+0000","last_became_peered":"2023-11-24T06:29:05.310389+0000","last_unstale":"2023-11-24T06:29:09.174994+0000","last_undegraded":"2023-11-24T06:29:09.174994+0000","last_fullsized":"2023-11-24T06:29:09.174994+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:55.345533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368349+0000","last_change":"2023-11-24T06:29:06.335401+0000","last_active":"2023-11-24T06:29:07.368349+0000","last_peered":"2023-11-24T06:29:07.368349+0000","last_clean":"2023-11-24T06:29:07.368349+0000","last_became_active":"2023-11-24T06:29:06.334627+0000","last_became_peered":"2023-11-24T06:29:06.334627+0000","last_unstale":"2023-11-24T06:29:07.368349+0000","last_undegraded":"2023-11-24T06:29:07.368349+0000","last_fullsized":"2023-11-24T06:29:07.368349+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:57:32.101727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970709+0000","last_change":"2023-11-24T06:29:05.306030+0000","last_active":"2023-11-24T06:29:07.970709+0000","last_peered":"2023-11-24T06:29:07.970709+0000","last_clean":"2023-11-24T06:29:07.970709+0000","last_became_active":"2023-11-24T06:29:05.304779+0000","last_became_peered":"2023-11-24T06:29:05.304779+0000","last_unstale":"2023-11-24T06:29:07.970709+0000","last_undegraded":"2023-11-24T06:29:07.970709+0000","last_fullsized":"2023-11-24T06:29:07.970709+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:28.325867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368441+0000","last_change":"2023-11-24T06:29:06.334209+0000","last_active":"2023-11-24T06:29:07.368441+0000","last_peered":"2023-11-24T06:29:07.368441+0000","last_clean":"2023-11-24T06:29:07.368441+0000","last_became_active":"2023-11-24T06:29:06.332842+0000","last_became_peered":"2023-11-24T06:29:06.332842+0000","last_unstale":"2023-11-24T06:29:07.368441+0000","last_undegraded":"2023-11-24T06:29:07.368441+0000","last_fullsized":"2023-11-24T06:29:07.368441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:13.529104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332416+0000","last_change":"2023-11-24T06:29:05.313374+0000","last_active":"2023-11-24T06:29:07.332416+0000","last_peered":"2023-11-24T06:29:07.332416+0000","last_clean":"2023-11-24T06:29:07.332416+0000","last_became_active":"2023-11-24T06:29:05.313043+0000","last_became_peered":"2023-11-24T06:29:05.313043+0000","last_unstale":"2023-11-24T06:29:07.332416+0000","last_undegraded":"2023-11-24T06:29:07.332416+0000","last_fullsized":"2023-11-24T06:29:07.332416+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:04.653296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332114+0000","last_change":"2023-11-24T06:29:06.335382+0000","last_active":"2023-11-24T06:29:07.332114+0000","last_peered":"2023-11-24T06:29:07.332114+0000","last_clean":"2023-11-24T06:29:07.332114+0000","last_became_active":"2023-11-24T06:29:06.334885+0000","last_became_peered":"2023-11-24T06:29:06.334885+0000","last_unstale":"2023-11-24T06:29:07.332114+0000","last_undegraded":"2023-11-24T06:29:07.332114+0000","last_fullsized":"2023-11-24T06:29:07.332114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:45:58.472377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332551+0000","last_change":"2023-11-24T06:29:05.306482+0000","last_active":"2023-11-24T06:29:07.332551+0000","last_peered":"2023-11-24T06:29:07.332551+0000","last_clean":"2023-11-24T06:29:07.332551+0000","last_became_active":"2023-11-24T06:29:05.306168+0000","last_became_peered":"2023-11-24T06:29:05.306168+0000","last_unstale":"2023-11-24T06:29:07.332551+0000","last_undegraded":"2023-11-24T06:29:07.332551+0000","last_fullsized":"2023-11-24T06:29:07.332551+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:37.230777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847159+0000","last_change":"2023-11-24T06:29:06.320098+0000","last_active":"2023-11-24T06:29:08.847159+0000","last_peered":"2023-11-24T06:29:08.847159+0000","last_clean":"2023-11-24T06:29:08.847159+0000","last_became_active":"2023-11-24T06:29:06.319791+0000","last_became_peered":"2023-11-24T06:29:06.319791+0000","last_unstale":"2023-11-24T06:29:08.847159+0000","last_undegraded":"2023-11-24T06:29:08.847159+0000","last_fullsized":"2023-11-24T06:29:08.847159+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:36.840838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970324+0000","last_change":"2023-11-24T06:29:05.314367+0000","last_active":"2023-11-24T06:29:07.970324+0000","last_peered":"2023-11-24T06:29:07.970324+0000","last_clean":"2023-11-24T06:29:07.970324+0000","last_became_active":"2023-11-24T06:29:05.314058+0000","last_became_peered":"2023-11-24T06:29:05.314058+0000","last_unstale":"2023-11-24T06:29:07.970324+0000","last_undegraded":"2023-11-24T06:29:07.970324+0000","last_fullsized":"2023-11-24T06:29:07.970324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:12.515758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.327989+0000","last_change":"2023-11-24T06:29:06.331684+0000","last_active":"2023-11-24T06:29:07.327989+0000","last_peered":"2023-11-24T06:29:07.327989+0000","last_clean":"2023-11-24T06:29:07.327989+0000","last_became_active":"2023-11-24T06:29:06.330068+0000","last_became_peered":"2023-11-24T06:29:06.330068+0000","last_unstale":"2023-11-24T06:29:07.327989+0000","last_undegraded":"2023-11-24T06:29:07.327989+0000","last_fullsized":"2023-11-24T06:29:07.327989+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:59.832018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693124+0000","last_change":"2023-11-24T06:29:05.305941+0000","last_active":"2023-11-24T06:29:08.693124+0000","last_peered":"2023-11-24T06:29:08.693124+0000","last_clean":"2023-11-24T06:29:08.693124+0000","last_became_active":"2023-11-24T06:29:05.304622+0000","last_became_peered":"2023-11-24T06:29:05.304622+0000","last_unstale":"2023-11-24T06:29:08.693124+0000","last_undegraded":"2023-11-24T06:29:08.693124+0000","last_fullsized":"2023-11-24T06:29:08.693124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:05.439986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693193+0000","last_change":"2023-11-24T06:29:06.343114+0000","last_active":"2023-11-24T06:29:08.693193+0000","last_peered":"2023-11-24T06:29:08.693193+0000","last_clean":"2023-11-24T06:29:08.693193+0000","last_became_active":"2023-11-24T06:29:06.342829+0000","last_became_peered":"2023-11-24T06:29:06.342829+0000","last_unstale":"2023-11-24T06:29:08.693193+0000","last_undegraded":"2023-11-24T06:29:08.693193+0000","last_fullsized":"2023-11-24T06:29:08.693193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:06.918223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.333844+0000","last_change":"2023-11-24T06:29:05.295441+0000","last_active":"2023-11-24T06:29:07.333844+0000","last_peered":"2023-11-24T06:29:07.333844+0000","last_clean":"2023-11-24T06:29:07.333844+0000","last_became_active":"2023-11-24T06:29:05.294655+0000","last_became_peered":"2023-11-24T06:29:05.294655+0000","last_unstale":"2023-11-24T06:29:07.333844+0000","last_undegraded":"2023-11-24T06:29:07.333844+0000","last_fullsized":"2023-11-24T06:29:07.333844+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:37.004764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368407+0000","last_change":"2023-11-24T06:29:06.355651+0000","last_active":"2023-11-24T06:29:07.368407+0000","last_peered":"2023-11-24T06:29:07.368407+0000","last_clean":"2023-11-24T06:29:07.368407+0000","last_became_active":"2023-11-24T06:29:06.355372+0000","last_became_peered":"2023-11-24T06:29:06.355372+0000","last_unstale":"2023-11-24T06:29:07.368407+0000","last_undegraded":"2023-11-24T06:29:07.368407+0000","last_fullsized":"2023-11-24T06:29:07.368407+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:16.218640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332590+0000","last_change":"2023-11-24T06:29:05.315575+0000","last_active":"2023-11-24T06:29:07.332590+0000","last_peered":"2023-11-24T06:29:07.332590+0000","last_clean":"2023-11-24T06:29:07.332590+0000","last_became_active":"2023-11-24T06:29:05.315255+0000","last_became_peered":"2023-11-24T06:29:05.315255+0000","last_unstale":"2023-11-24T06:29:07.332590+0000","last_undegraded":"2023-11-24T06:29:07.332590+0000","last_fullsized":"2023-11-24T06:29:07.332590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:27.769829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694196+0000","last_change":"2023-11-24T06:29:06.339764+0000","last_active":"2023-11-24T06:29:08.694196+0000","last_peered":"2023-11-24T06:29:08.694196+0000","last_clean":"2023-11-24T06:29:08.694196+0000","last_became_active":"2023-11-24T06:29:06.339233+0000","last_became_peered":"2023-11-24T06:29:06.339233+0000","last_unstale":"2023-11-24T06:29:08.694196+0000","last_undegraded":"2023-11-24T06:29:08.694196+0000","last_fullsized":"2023-11-24T06:29:08.694196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:32.445582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334896+0000","last_change":"2023-11-24T06:29:05.315202+0000","last_active":"2023-11-24T06:29:07.334896+0000","last_peered":"2023-11-24T06:29:07.334896+0000","last_clean":"2023-11-24T06:29:07.334896+0000","last_became_active":"2023-11-24T06:29:05.314920+0000","last_became_peered":"2023-11-24T06:29:05.314920+0000","last_unstale":"2023-11-24T06:29:07.334896+0000","last_undegraded":"2023-11-24T06:29:07.334896+0000","last_fullsized":"2023-11-24T06:29:07.334896+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:44.390683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334622+0000","last_change":"2023-11-24T06:29:06.330176+0000","last_active":"2023-11-24T06:29:07.334622+0000","last_peered":"2023-11-24T06:29:07.334622+0000","last_clean":"2023-11-24T06:29:07.334622+0000","last_became_active":"2023-11-24T06:29:06.329800+0000","last_became_peered":"2023-11-24T06:29:06.329800+0000","last_unstale":"2023-11-24T06:29:07.334622+0000","last_undegraded":"2023-11-24T06:29:07.334622+0000","last_fullsized":"2023-11-24T06:29:07.334622+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:15.307878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847086+0000","last_change":"2023-11-24T06:29:05.306907+0000","last_active":"2023-11-24T06:29:08.847086+0000","last_peered":"2023-11-24T06:29:08.847086+0000","last_clean":"2023-11-24T06:29:08.847086+0000","last_became_active":"2023-11-24T06:29:05.305842+0000","last_became_peered":"2023-11-24T06:29:05.305842+0000","last_unstale":"2023-11-24T06:29:08.847086+0000","last_undegraded":"2023-11-24T06:29:08.847086+0000","last_fullsized":"2023-11-24T06:29:08.847086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:24.625401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332126+0000","last_change":"2023-11-24T06:29:06.329730+0000","last_active":"2023-11-24T06:29:07.332126+0000","last_peered":"2023-11-24T06:29:07.332126+0000","last_clean":"2023-11-24T06:29:07.332126+0000","last_became_active":"2023-11-24T06:29:06.329406+0000","last_became_peered":"2023-11-24T06:29:06.329406+0000","last_unstale":"2023-11-24T06:29:07.332126+0000","last_undegraded":"2023-11-24T06:29:07.332126+0000","last_fullsized":"2023-11-24T06:29:07.332126+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:44:18.807302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334457+0000","last_change":"2023-11-24T06:29:05.310752+0000","last_active":"2023-11-24T06:29:07.334457+0000","last_peered":"2023-11-24T06:29:07.334457+0000","last_clean":"2023-11-24T06:29:07.334457+0000","last_became_active":"2023-11-24T06:29:05.310481+0000","last_became_peered":"2023-11-24T06:29:05.310481+0000","last_unstale":"2023-11-24T06:29:07.334457+0000","last_undegraded":"2023-11-24T06:29:07.334457+0000","last_fullsized":"2023-11-24T06:29:07.334457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:35.880888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332436+0000","last_change":"2023-11-24T06:29:06.330845+0000","last_active":"2023-11-24T06:29:07.332436+0000","last_peered":"2023-11-24T06:29:07.332436+0000","last_clean":"2023-11-24T06:29:07.332436+0000","last_became_active":"2023-11-24T06:29:06.330530+0000","last_became_peered":"2023-11-24T06:29:06.330530+0000","last_unstale":"2023-11-24T06:29:07.332436+0000","last_undegraded":"2023-11-24T06:29:07.332436+0000","last_fullsized":"2023-11-24T06:29:07.332436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:06.408910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174304+0000","last_change":"2023-11-24T06:29:05.306621+0000","last_active":"2023-11-24T06:29:09.174304+0000","last_peered":"2023-11-24T06:29:09.174304+0000","last_clean":"2023-11-24T06:29:09.174304+0000","last_became_active":"2023-11-24T06:29:05.306338+0000","last_became_peered":"2023-11-24T06:29:05.306338+0000","last_unstale":"2023-11-24T06:29:09.174304+0000","last_undegraded":"2023-11-24T06:29:09.174304+0000","last_fullsized":"2023-11-24T06:29:09.174304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:29:33.066082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970295+0000","last_change":"2023-11-24T06:29:06.312498+0000","last_active":"2023-11-24T06:29:07.970295+0000","last_peered":"2023-11-24T06:29:07.970295+0000","last_clean":"2023-11-24T06:29:07.970295+0000","last_became_active":"2023-11-24T06:29:06.311781+0000","last_became_peered":"2023-11-24T06:29:06.311781+0000","last_unstale":"2023-11-24T06:29:07.970295+0000","last_undegraded":"2023-11-24T06:29:07.970295+0000","last_fullsized":"2023-11-24T06:29:07.970295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:35.027087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.846993+0000","last_change":"2023-11-24T06:29:05.310496+0000","last_active":"2023-11-24T06:29:08.846993+0000","last_peered":"2023-11-24T06:29:08.846993+0000","last_clean":"2023-11-24T06:29:08.846993+0000","last_became_active":"2023-11-24T06:29:05.310192+0000","last_became_peered":"2023-11-24T06:29:05.310192+0000","last_unstale":"2023-11-24T06:29:08.846993+0000","last_undegraded":"2023-11-24T06:29:08.846993+0000","last_fullsized":"2023-11-24T06:29:08.846993+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:19.332329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.173921+0000","last_change":"2023-11-24T06:29:06.328679+0000","last_active":"2023-11-24T06:29:09.173921+0000","last_peered":"2023-11-24T06:29:09.173921+0000","last_clean":"2023-11-24T06:29:09.173921+0000","last_became_active":"2023-11-24T06:29:06.328284+0000","last_became_peered":"2023-11-24T06:29:06.328284+0000","last_unstale":"2023-11-24T06:29:09.173921+0000","last_undegraded":"2023-11-24T06:29:09.173921+0000","last_fullsized":"2023-11-24T06:29:09.173921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:13.529104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693864+0000","last_change":"2023-11-24T06:29:05.302224+0000","last_active":"2023-11-24T06:29:08.693864+0000","last_peered":"2023-11-24T06:29:08.693864+0000","last_clean":"2023-11-24T06:29:08.693864+0000","last_became_active":"2023-11-24T06:29:05.301928+0000","last_became_peered":"2023-11-24T06:29:05.301928+0000","last_unstale":"2023-11-24T06:29:08.693864+0000","last_undegraded":"2023-11-24T06:29:08.693864+0000","last_fullsized":"2023-11-24T06:29:08.693864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:08.253511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694088+0000","last_change":"2023-11-24T06:29:06.349020+0000","last_active":"2023-11-24T06:29:08.694088+0000","last_peered":"2023-11-24T06:29:08.694088+0000","last_clean":"2023-11-24T06:29:08.694088+0000","last_became_active":"2023-11-24T06:29:06.346943+0000","last_became_peered":"2023-11-24T06:29:06.346943+0000","last_unstale":"2023-11-24T06:29:08.694088+0000","last_undegraded":"2023-11-24T06:29:08.694088+0000","last_fullsized":"2023-11-24T06:29:08.694088+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:36.894234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.971257+0000","last_change":"2023-11-24T06:29:05.311828+0000","last_active":"2023-11-24T06:29:07.971257+0000","last_peered":"2023-11-24T06:29:07.971257+0000","last_clean":"2023-11-24T06:29:07.971257+0000","last_became_active":"2023-11-24T06:29:05.311541+0000","last_became_peered":"2023-11-24T06:29:05.311541+0000","last_unstale":"2023-11-24T06:29:07.971257+0000","last_undegraded":"2023-11-24T06:29:07.971257+0000","last_fullsized":"2023-11-24T06:29:07.971257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.191791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:29:06.302049+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:29:06.302049+0000","last_peered":"2023-11-24T06:29:06.302049+0000","last_clean":"2023-11-24T06:29:06.302049+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:29:06.302049+0000","last_undegraded":"2023-11-24T06:29:06.302049+0000","last_fullsized":"2023-11-24T06:29:06.302049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1138688,"data_stored":810904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.29,"15min":1.3280000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5780000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.181,"5min":1.2789999999999999,"15min":1.2949999999999999},"min":{"1min":0.75700000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.76,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.78900000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.65100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.218,"5min":1.359,"15min":1.383},"min":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.585,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.052}]},{"osd":2,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.2649999999999999,"15min":1.2849999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6479999999999999,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1919999999999999,"15min":1.1990000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.754,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.84799999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.365,"15min":1.403},"min":{"1min":0.80900000000000005,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.637,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2050000000000001,"15min":1.2030000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.3779999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.542,"15min":1.585},"min":{"1min":0.72099999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7170000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.5569999999999999,"15min":1.585},"min":{"1min":1.0589999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.962,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":1.5740000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.395,"15min":1.411},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.921,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.7170000000000001,"15min":1.758},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.845,"5min":6.7359999999999998,"15min":6.7359999999999998},"last":1.331}]},{"osd":6,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.825,"15min":1.8779999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.093,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.141},{"interface":"front","average":{"1min":1.377,"5min":1.659,"15min":1.706},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.881,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":1.516}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":221118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0940000000000001,"15min":1.1000000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1100000000000001,"15min":1.111},"min":{"1min":0.77800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.8009999999999999,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.623}]},{"osd":1,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.208,"15min":1.2090000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.768,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.401},{"interface":"front","average":{"1min":1.218,"5min":1.2110000000000001,"15min":1.21},"min":{"1min":0.81699999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5649999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4650000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.157,"15min":1.163},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5660000000000001,"5min":3.597,"15min":3.597},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.296,"15min":1.3149999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6319999999999999,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.5649999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3169999999999999,"15min":1.319},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.706,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.232},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.3149999999999999,"15min":1.331},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.6259999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.52}]},{"osd":4,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.425,"15min":1.4470000000000001},"min":{"1min":0.93300000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.8089999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.375,"5min":1.3979999999999999,"15min":1.4019999999999999},"min":{"1min":0.90300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.9219999999999999,"5min":3.04,"15min":3.04},"last":1.3520000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3720000000000001,"15min":1.377},"min":{"1min":0.875,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.76,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.593,"15min":1.615},"min":{"1min":1.03,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.762,"5min":3.774,"15min":3.774},"last":1.0289999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5569999999999999,"15min":1.5629999999999999},"min":{"1min":1.204,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.016,"5min":3.403,"15min":3.403},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.5069999999999999,"15min":1.5149999999999999},"min":{"1min":1.0189999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.7230000000000001,"5min":3.649,"15min":3.649},"last":1.7030000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":220516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.1699999999999999,"15min":1.1879999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.885,"5min":1.903,"15min":1.903},"last":1.016},{"interface":"front","average":{"1min":1.161,"5min":1.2929999999999999,"15min":1.3149999999999999},"min":{"1min":0.67400000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.135}]},{"osd":1,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1259999999999999,"15min":1.1339999999999999},"min":{"1min":0.72499999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.494},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.101,"15min":1.099},"min":{"1min":0.74399999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.4039999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.208,"15min":1.2050000000000001},"min":{"1min":0.88400000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.992,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.294,"15min":1.292},"min":{"1min":0.87,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.776,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":2.024}]},{"osd":3,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.363,"15min":1.3839999999999999},"min":{"1min":0.85199999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.536,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.379,"15min":1.389},"min":{"1min":0.90700000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7549999999999999,"5min":1.994,"15min":1.994},"last":2.0870000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3180000000000001,"15min":1.3340000000000001},"min":{"1min":0.626,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.673,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.266},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.52,"15min":1.542},"min":{"1min":1.077,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.823,"5min":3.085,"15min":3.085},"last":1.5589999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.4490000000000001,"15min":1.4379999999999999},"min":{"1min":1.173,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.8959999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.4530000000000001,"15min":1.4610000000000001},"min":{"1min":0.96099999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.8999999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.95}]},{"osd":7,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.6100000000000001,"15min":1.623},"min":{"1min":1.2410000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8380000000000001,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.6950000000000001,"15min":1.7170000000000001},"min":{"1min":1.02,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.157,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":2.202}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":215758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.083,"15min":1.0760000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.147},{"interface":"front","average":{"1min":1.024,"5min":1.1040000000000001,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4510000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.1339999999999999,"15min":1.109},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3129999999999999,"15min":1.323},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.845,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.292,"15min":1.306},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.772,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.329,"15min":1.337},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.8320000000000001,"5min":2.254,"15min":2.254},"last":1.4159999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3260000000000001,"15min":1.3440000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.048,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.242},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.383,"15min":1.3740000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.94199999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.5129999999999999,"15min":1.5249999999999999},"min":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0830000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.534},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.5600000000000001,"15min":1.5740000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.1970000000000001,"5min":3.52,"15min":3.52},"last":1.6000000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.4870000000000001,"15min":1.486},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.5760000000000001,"15min":1.585},"min":{"1min":0.71699999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.3359999999999999,"5min":2.452,"15min":2.452},"last":1.641}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.657,"15min":1.677},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.1379999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.504},{"interface":"front","average":{"1min":1.6659999999999999,"5min":1.6220000000000001,"15min":1.615},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.2669999999999999,"5min":2.363,"15min":2.363},"last":1.5580000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.40200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.90700000000000003,"5min":1.355,"15min":1.355},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.41199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4690000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1160000000000001},"min":{"1min":0.51000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.4950000000000001,"5min":1.786,"15min":1.786},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.499,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4710000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4450000000000001,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.0299999999999998,"5min":3.081,"15min":3.081},"last":1.044},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3300000000000001,"15min":1.3380000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0910000000000002,"5min":2.758,"15min":2.758},"last":0.94199999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.377,"15min":1.4039999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8200000000000001,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.111},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.722,"15min":1.782},"min":{"1min":0.88600000000000001,"5min":0.745,"15min":0.745},"max":{"1min":2.149,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3819999999999999,"15min":1.409},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.6919999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.032},{"interface":"front","average":{"1min":1.474,"5min":1.5349999999999999,"15min":1.5449999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.1030000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.125}]},{"osd":7,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.5960000000000001,"15min":1.6160000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.1829999999999998,"5min":4.641,"15min":4.641},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.55,"15min":1.5760000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.9750000000000001,"5min":3.798,"15min":3.798},"last":0.95499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":222080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2330000000000001,"5min":1.427,"15min":1.427},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.432,"5min":0.31,"15min":0.31},"max":{"1min":1.161,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.94199999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.96299999999999997},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2410000000000001,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0449999999999999,"15min":1.0509999999999999},"min":{"1min":0.56899999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.512,"5min":1.55,"15min":1.55},"last":1.046}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.101,"15min":1.1100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6850000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.103,"15min":1.119},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.597,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.6990000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.492,"15min":1.52},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.79,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.4690000000000001,"15min":1.49},"min":{"1min":0.94999999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6899999999999999,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.4430000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4590000000000001,"15min":1.466},"min":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.911,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.409},{"interface":"front","average":{"1min":1.385,"5min":1.49,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.9410000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.48}]},{"osd":6,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4279999999999999,"15min":1.4379999999999999},"min":{"1min":0.78200000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.6899999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.579},{"interface":"front","average":{"1min":1.419,"5min":1.5229999999999999,"15min":1.5409999999999999},"min":{"1min":0.85199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.0720000000000001,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.524}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.5369999999999999,"15min":1.5569999999999999},"min":{"1min":0.83099999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.974,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.341},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.5309999999999999,"15min":1.5389999999999999},"min":{"1min":0.90300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1699999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.613}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":220516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0289999999999999,"15min":1.0409999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5840000000000001,"5min":2.593,"15min":2.593},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.026,"15min":1.034},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.135}]},{"osd":2,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.871,"15min":0.875},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.2869999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.022,"5min":1.01,"15min":1.008},"min":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.556,"5min":1.829,"15min":1.829},"last":1.206}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1259999999999999,"15min":1.1279999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.708,"5min":1.766,"15min":1.766},"last":1.355},{"interface":"front","average":{"1min":1.016,"5min":1.0740000000000001,"15min":1.0840000000000001},"min":{"1min":0.57099999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3740000000000001,"5min":1.798,"15min":1.798},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.258,"15min":1.2569999999999999},"min":{"1min":0.82599999999999996,"5min":0.627,"15min":0.627},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.347,"5min":1.4370000000000001,"15min":1.452},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":1.7529999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.167}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3069999999999999,"15min":1.306},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.228},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3919999999999999,"15min":1.3999999999999999},"min":{"1min":0.93899999999999995,"5min":0.877,"15min":0.877},"max":{"1min":1.796,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.508}]},{"osd":6,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.399,"15min":1.395},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.46},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4530000000000001,"15min":1.456},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.0910000000000002,"5min":2.371,"15min":2.371},"last":1.6499999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.45},"min":{"1min":0.88500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.417},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.524,"15min":1.5289999999999999},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":804548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.48999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.607,"5min":1.722,"15min":1.722},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.5,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.999,"15min":0.998},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.119,"15min":1.137},"min":{"1min":0.63400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.538,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1259999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.93100000000000005,"15min":0.94899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5229999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.085,"15min":1.0900000000000001},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.639,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.25}]},{"osd":4,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2090000000000001,"15min":1.2190000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6659999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3100000000000001,"15min":1.3080000000000001},"min":{"1min":0.91400000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.986,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.399}]},{"osd":5,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.321,"15min":1.333},"min":{"1min":0.71799999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.704,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.2509999999999999,"15min":1.2709999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6000000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.865}]},{"osd":6,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4710000000000001,"15min":1.4870000000000001},"min":{"1min":0.95499999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7230000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.345},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.492,"15min":1.518},"min":{"1min":0.91300000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.8069999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.462}]},{"osd":7,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.446,"15min":1.456},"min":{"1min":0.86499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9059999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.399,"15min":1.419},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.853,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.8120000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:10.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:11.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.089+0000 7feeda133640 1 Processor -- start 2023-11-24T06:29:11.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.090+0000 7feeda133640 1 -- start start 2023-11-24T06:29:11.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.091+0000 7feeda133640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feed415d240 0x7feed40b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.092+0000 7feeda133640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed40b1d20 con 0x7feed40af4c0 2023-11-24T06:29:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.092+0000 7feeda133640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed40b38d0 con 0x7feed406d7d0 2023-11-24T06:29:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.092+0000 7feeda133640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed40b1fc0 con 0x7feed415d240 2023-11-24T06:29:11.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.092+0000 7feed3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feed40af4c0 0x7feed415d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48692/0 (socket says 172.21.15.16:48692) 2023-11-24T06:29:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.092+0000 7feed3fff640 1 -- 172.21.15.16:0/2116763657 learned_addr learned my addr 172.21.15.16:0/2116763657 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.092+0000 7feed37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feed415d240 0x7feed40b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.092+0000 7feed37fe640 1 -- 172.21.15.16:0/2116763657 >> v1:172.21.15.173:6789/0 conn(0x7feed406d7d0 legacy=0x7feed40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.092+0000 7feed37fe640 1 -- 172.21.15.16:0/2116763657 >> v1:172.21.15.16:6789/0 conn(0x7feed40af4c0 legacy=0x7feed415d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.093+0000 7feed37fe640 1 -- 172.21.15.16:0/2116763657 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed40b22a0 con 0x7feed415d240 2023-11-24T06:29:11.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.092+0000 7feed27fc640 1 -- 172.21.15.16:0/2116763657 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102302936 0 0) 0x7feed40b1d20 con 0x7feed40af4c0 2023-11-24T06:29:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.094+0000 7feed37fe640 1 --2- 172.21.15.16:0/2116763657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed415d240 0x7feed40b2f60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feec4009d90 tx=0x7feec402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e7f0c6ef432053 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.094+0000 7feed27fc640 1 -- 172.21.15.16:0/2116763657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec402cbd0 con 0x7feed415d240 2023-11-24T06:29:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.094+0000 7feed27fc640 1 -- 172.21.15.16:0/2116763657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec402cd30 con 0x7feed415d240 2023-11-24T06:29:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.094+0000 7feed27fc640 1 -- 172.21.15.16:0/2116763657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec402cea0 con 0x7feed415d240 2023-11-24T06:29:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.095+0000 7feeda133640 1 -- 172.21.15.16:0/2116763657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed415d240 msgr2=0x7feed40b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.095+0000 7feeda133640 1 --2- 172.21.15.16:0/2116763657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed415d240 0x7feed40b2f60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feec4009d90 tx=0x7feec402be20 comp rx=0 tx=0).stop 2023-11-24T06:29:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.095+0000 7feeda133640 1 -- 172.21.15.16:0/2116763657 shutdown_connections 2023-11-24T06:29:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.095+0000 7feeda133640 1 --2- 172.21.15.16:0/2116763657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed415d240 0x7feed40b2f60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.095+0000 7feeda133640 1 -- 172.21.15.16:0/2116763657 >> 172.21.15.16:0/2116763657 conn(0x7feed406b4f0 msgr2=0x7feed406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.095+0000 7feeda133640 1 -- 172.21.15.16:0/2116763657 shutdown_connections 2023-11-24T06:29:11.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.095+0000 7feeda133640 1 -- 172.21.15.16:0/2116763657 wait complete. 2023-11-24T06:29:11.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.097+0000 7feeda133640 1 Processor -- start 2023-11-24T06:29:11.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.097+0000 7feeda133640 1 -- start start 2023-11-24T06:29:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feeda133640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feed406d7d0 0x7feed415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feeda133640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed40af4c0 0x7feed40b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feeda133640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed415d240 0x7feed414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feeda133640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed4157300 con 0x7feed40af4c0 2023-11-24T06:29:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feeda133640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed415c880 con 0x7feed415d240 2023-11-24T06:29:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feeda133640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed40c5440 con 0x7feed406d7d0 2023-11-24T06:29:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feed2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed40af4c0 0x7feed40b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feed2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed40af4c0 0x7feed40b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34154/0 (socket says 172.21.15.16:34154) 2023-11-24T06:29:11.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feed2ffd640 1 -- 172.21.15.16:0/649863950 learned_addr learned my addr 172.21.15.16:0/649863950 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.098+0000 7feed3fff640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed415d240 0x7feed414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feed2ffd640 1 -- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feed406d7d0 msgr2=0x7feed415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feed2ffd640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feed406d7d0 0x7feed415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feed2ffd640 1 -- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed415d240 msgr2=0x7feed414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feed2ffd640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed415d240 0x7feed414a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feed2ffd640 1 -- 172.21.15.16:0/649863950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4009a70 con 0x7feed40af4c0 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feed2ffd640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed40af4c0 0x7feed40b0d70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c430 tx=0x7feebc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feecbfff640 1 -- 172.21.15.16:0/649863950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc011790 con 0x7feed40af4c0 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feecbfff640 1 -- 172.21.15.16:0/649863950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc0118f0 con 0x7feed40af4c0 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feecbfff640 1 -- 172.21.15.16:0/649863950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc011ca0 con 0x7feed40af4c0 2023-11-24T06:29:11.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed40c5720 con 0x7feed40af4c0 2023-11-24T06:29:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.099+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed40c5ce0 con 0x7feed40af4c0 2023-11-24T06:29:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.101+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed40675d0 con 0x7feed40af4c0 2023-11-24T06:29:11.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.101+0000 7feecbfff640 1 -- 172.21.15.16:0/649863950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7feebc01c460 con 0x7feed40af4c0 2023-11-24T06:29:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.102+0000 7feecbfff640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feeb406aed0 0x7feeb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.102+0000 7feecbfff640 1 -- 172.21.15.16:0/649863950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feebc0894d0 con 0x7feed40af4c0 2023-11-24T06:29:11.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.103+0000 7feed37fe640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feeb406aed0 0x7feeb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.104+0000 7feed37fe640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feeb406aed0 0x7feeb406d390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feec4036000 tx=0x7feec402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.107+0000 7feecbfff640 1 -- 172.21.15.16:0/649863950 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feebc0545a0 con 0x7feed40af4c0 2023-11-24T06:29:11.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.296+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feed40b27c0 con 0x7feeb406aed0 2023-11-24T06:29:11.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.320+0000 7feecbfff640 1 -- 172.21.15.16:0/649863950 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286591 (secure 0 0 0) 0x7feec402c9d0 con 0x7feeb406aed0 2023-11-24T06:29:11.327 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:11.329 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:29:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feeb406aed0 msgr2=0x7feeb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feeb406aed0 0x7feeb406d390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feec4036000 tx=0x7feec402e000 comp rx=0 tx=0).stop 2023-11-24T06:29:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed40af4c0 msgr2=0x7feed40b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed40af4c0 0x7feed40b0d70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c430 tx=0x7feebc00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 shutdown_connections 2023-11-24T06:29:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7feeb406aed0 0x7feeb406d390 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed415d240 0x7feed414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feed40af4c0 0x7feed40b0d70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 --2- 172.21.15.16:0/649863950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feed406d7d0 0x7feed415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.326+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 >> 172.21.15.16:0/649863950 conn(0x7feed406b4f0 msgr2=0x7feed415d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.327+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 shutdown_connections 2023-11-24T06:29:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.327+0000 7feeda133640 1 -- 172.21.15.16:0/649863950 wait complete. 2023-11-24T06:29:11.352 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-11-24T06:29:10.148016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238272,"kb_used_data":5424,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736448,"statfs":{"total":773094113280,"available":772850122752,"internally_reserved":0,"allocated":5554176,"data_stored":2936524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.876398"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334821+0000","last_change":"2023-11-24T06:29:06.328584+0000","last_active":"2023-11-24T06:29:07.334821+0000","last_peered":"2023-11-24T06:29:07.334821+0000","last_clean":"2023-11-24T06:29:07.334821+0000","last_became_active":"2023-11-24T06:29:06.326783+0000","last_became_peered":"2023-11-24T06:29:06.326783+0000","last_unstale":"2023-11-24T06:29:07.334821+0000","last_undegraded":"2023-11-24T06:29:07.334821+0000","last_fullsized":"2023-11-24T06:29:07.334821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:05.666642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.971199+0000","last_change":"2023-11-24T06:29:06.353972+0000","last_active":"2023-11-24T06:29:07.971199+0000","last_peered":"2023-11-24T06:29:07.971199+0000","last_clean":"2023-11-24T06:29:07.971199+0000","last_became_active":"2023-11-24T06:29:06.353580+0000","last_became_peered":"2023-11-24T06:29:06.353580+0000","last_unstale":"2023-11-24T06:29:07.971199+0000","last_undegraded":"2023-11-24T06:29:07.971199+0000","last_fullsized":"2023-11-24T06:29:07.971199+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:01.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174115+0000","last_change":"2023-11-24T06:29:06.334126+0000","last_active":"2023-11-24T06:29:09.174115+0000","last_peered":"2023-11-24T06:29:09.174115+0000","last_clean":"2023-11-24T06:29:09.174115+0000","last_became_active":"2023-11-24T06:29:06.332002+0000","last_became_peered":"2023-11-24T06:29:06.332002+0000","last_unstale":"2023-11-24T06:29:09.174115+0000","last_undegraded":"2023-11-24T06:29:09.174115+0000","last_fullsized":"2023-11-24T06:29:09.174115+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:29.339213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693838+0000","last_change":"2023-11-24T06:29:06.342691+0000","last_active":"2023-11-24T06:29:08.693838+0000","last_peered":"2023-11-24T06:29:08.693838+0000","last_clean":"2023-11-24T06:29:08.693838+0000","last_became_active":"2023-11-24T06:29:06.342434+0000","last_became_peered":"2023-11-24T06:29:06.342434+0000","last_unstale":"2023-11-24T06:29:08.693838+0000","last_undegraded":"2023-11-24T06:29:08.693838+0000","last_fullsized":"2023-11-24T06:29:08.693838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:35.027087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970607+0000","last_change":"2023-11-24T06:29:06.341116+0000","last_active":"2023-11-24T06:29:07.970607+0000","last_peered":"2023-11-24T06:29:07.970607+0000","last_clean":"2023-11-24T06:29:07.970607+0000","last_became_active":"2023-11-24T06:29:06.340830+0000","last_became_peered":"2023-11-24T06:29:06.340830+0000","last_unstale":"2023-11-24T06:29:07.970607+0000","last_undegraded":"2023-11-24T06:29:07.970607+0000","last_fullsized":"2023-11-24T06:29:07.970607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:36.894234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331931+0000","last_change":"2023-11-24T06:29:06.337450+0000","last_active":"2023-11-24T06:29:07.331931+0000","last_peered":"2023-11-24T06:29:07.331931+0000","last_clean":"2023-11-24T06:29:07.331931+0000","last_became_active":"2023-11-24T06:29:06.337169+0000","last_became_peered":"2023-11-24T06:29:06.337169+0000","last_unstale":"2023-11-24T06:29:07.331931+0000","last_undegraded":"2023-11-24T06:29:07.331931+0000","last_fullsized":"2023-11-24T06:29:07.331931+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:13:07.856195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334104+0000","last_change":"2023-11-24T06:29:06.314221+0000","last_active":"2023-11-24T06:29:07.334104+0000","last_peered":"2023-11-24T06:29:07.334104+0000","last_clean":"2023-11-24T06:29:07.334104+0000","last_became_active":"2023-11-24T06:29:06.313621+0000","last_became_peered":"2023-11-24T06:29:06.313621+0000","last_unstale":"2023-11-24T06:29:07.334104+0000","last_undegraded":"2023-11-24T06:29:07.334104+0000","last_fullsized":"2023-11-24T06:29:07.334104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:32.445582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693288+0000","last_change":"2023-11-24T06:29:06.341882+0000","last_active":"2023-11-24T06:29:08.693288+0000","last_peered":"2023-11-24T06:29:08.693288+0000","last_clean":"2023-11-24T06:29:08.693288+0000","last_became_active":"2023-11-24T06:29:06.341588+0000","last_became_peered":"2023-11-24T06:29:06.341588+0000","last_unstale":"2023-11-24T06:29:08.693288+0000","last_undegraded":"2023-11-24T06:29:08.693288+0000","last_fullsized":"2023-11-24T06:29:08.693288+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:54.640138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694245+0000","last_change":"2023-11-24T06:29:06.347482+0000","last_active":"2023-11-24T06:29:08.694245+0000","last_peered":"2023-11-24T06:29:08.694245+0000","last_clean":"2023-11-24T06:29:08.694245+0000","last_became_active":"2023-11-24T06:29:06.347069+0000","last_became_peered":"2023-11-24T06:29:06.347069+0000","last_unstale":"2023-11-24T06:29:08.694245+0000","last_undegraded":"2023-11-24T06:29:08.694245+0000","last_fullsized":"2023-11-24T06:29:08.694245+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:01.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693809+0000","last_change":"2023-11-24T06:29:06.347193+0000","last_active":"2023-11-24T06:29:08.693809+0000","last_peered":"2023-11-24T06:29:08.693809+0000","last_clean":"2023-11-24T06:29:08.693809+0000","last_became_active":"2023-11-24T06:29:06.346918+0000","last_became_peered":"2023-11-24T06:29:06.346918+0000","last_unstale":"2023-11-24T06:29:08.693809+0000","last_undegraded":"2023-11-24T06:29:08.693809+0000","last_fullsized":"2023-11-24T06:29:08.693809+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:12.374074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694016+0000","last_change":"2023-11-24T06:29:06.343158+0000","last_active":"2023-11-24T06:29:08.694016+0000","last_peered":"2023-11-24T06:29:08.694016+0000","last_clean":"2023-11-24T06:29:08.694016+0000","last_became_active":"2023-11-24T06:29:06.342895+0000","last_became_peered":"2023-11-24T06:29:06.342895+0000","last_unstale":"2023-11-24T06:29:08.694016+0000","last_undegraded":"2023-11-24T06:29:08.694016+0000","last_fullsized":"2023-11-24T06:29:08.694016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:00.100140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970561+0000","last_change":"2023-11-24T06:29:06.326238+0000","last_active":"2023-11-24T06:29:07.970561+0000","last_peered":"2023-11-24T06:29:07.970561+0000","last_clean":"2023-11-24T06:29:07.970561+0000","last_became_active":"2023-11-24T06:29:06.325845+0000","last_became_peered":"2023-11-24T06:29:06.325845+0000","last_unstale":"2023-11-24T06:29:07.970561+0000","last_undegraded":"2023-11-24T06:29:07.970561+0000","last_fullsized":"2023-11-24T06:29:07.970561+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:45.404029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328231+0000","last_change":"2023-11-24T06:29:06.334566+0000","last_active":"2023-11-24T06:29:07.328231+0000","last_peered":"2023-11-24T06:29:07.328231+0000","last_clean":"2023-11-24T06:29:07.328231+0000","last_became_active":"2023-11-24T06:29:06.334212+0000","last_became_peered":"2023-11-24T06:29:06.334212+0000","last_unstale":"2023-11-24T06:29:07.328231+0000","last_undegraded":"2023-11-24T06:29:07.328231+0000","last_fullsized":"2023-11-24T06:29:07.328231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:44:07.134660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693063+0000","last_change":"2023-11-24T06:29:06.347734+0000","last_active":"2023-11-24T06:29:08.693063+0000","last_peered":"2023-11-24T06:29:08.693063+0000","last_clean":"2023-11-24T06:29:08.693063+0000","last_became_active":"2023-11-24T06:29:06.346876+0000","last_became_peered":"2023-11-24T06:29:06.346876+0000","last_unstale":"2023-11-24T06:29:08.693063+0000","last_undegraded":"2023-11-24T06:29:08.693063+0000","last_fullsized":"2023-11-24T06:29:08.693063+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:51.205136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331877+0000","last_change":"2023-11-24T06:29:06.327701+0000","last_active":"2023-11-24T06:29:07.331877+0000","last_peered":"2023-11-24T06:29:07.331877+0000","last_clean":"2023-11-24T06:29:07.331877+0000","last_became_active":"2023-11-24T06:29:06.327345+0000","last_became_peered":"2023-11-24T06:29:06.327345+0000","last_unstale":"2023-11-24T06:29:07.331877+0000","last_undegraded":"2023-11-24T06:29:07.331877+0000","last_fullsized":"2023-11-24T06:29:07.331877+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:17:31.638204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367613+0000","last_change":"2023-11-24T06:29:06.344131+0000","last_active":"2023-11-24T06:29:07.367613+0000","last_peered":"2023-11-24T06:29:07.367613+0000","last_clean":"2023-11-24T06:29:07.367613+0000","last_became_active":"2023-11-24T06:29:06.343794+0000","last_became_peered":"2023-11-24T06:29:06.343794+0000","last_unstale":"2023-11-24T06:29:07.367613+0000","last_undegraded":"2023-11-24T06:29:07.367613+0000","last_fullsized":"2023-11-24T06:29:07.367613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:34.946415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174363+0000","last_change":"2023-11-24T06:29:06.331570+0000","last_active":"2023-11-24T06:29:09.174363+0000","last_peered":"2023-11-24T06:29:09.174363+0000","last_clean":"2023-11-24T06:29:09.174363+0000","last_became_active":"2023-11-24T06:29:06.329451+0000","last_became_peered":"2023-11-24T06:29:06.329451+0000","last_unstale":"2023-11-24T06:29:09.174363+0000","last_undegraded":"2023-11-24T06:29:09.174363+0000","last_fullsized":"2023-11-24T06:29:09.174363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:06.918223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174389+0000","last_change":"2023-11-24T06:29:06.355860+0000","last_active":"2023-11-24T06:29:09.174389+0000","last_peered":"2023-11-24T06:29:09.174389+0000","last_clean":"2023-11-24T06:29:09.174389+0000","last_became_active":"2023-11-24T06:29:06.355323+0000","last_became_peered":"2023-11-24T06:29:06.355323+0000","last_unstale":"2023-11-24T06:29:09.174389+0000","last_undegraded":"2023-11-24T06:29:09.174389+0000","last_fullsized":"2023-11-24T06:29:09.174389+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:46.035866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368585+0000","last_change":"2023-11-24T06:29:06.350773+0000","last_active":"2023-11-24T06:29:07.368585+0000","last_peered":"2023-11-24T06:29:07.368585+0000","last_clean":"2023-11-24T06:29:07.368585+0000","last_became_active":"2023-11-24T06:29:06.350451+0000","last_became_peered":"2023-11-24T06:29:06.350451+0000","last_unstale":"2023-11-24T06:29:07.368585+0000","last_undegraded":"2023-11-24T06:29:07.368585+0000","last_fullsized":"2023-11-24T06:29:07.368585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:29.060969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328650+0000","last_change":"2023-11-24T06:29:06.341930+0000","last_active":"2023-11-24T06:29:07.328650+0000","last_peered":"2023-11-24T06:29:07.328650+0000","last_clean":"2023-11-24T06:29:07.328650+0000","last_became_active":"2023-11-24T06:29:06.341172+0000","last_became_peered":"2023-11-24T06:29:06.341172+0000","last_unstale":"2023-11-24T06:29:07.328650+0000","last_undegraded":"2023-11-24T06:29:07.328650+0000","last_fullsized":"2023-11-24T06:29:07.328650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:04:37.612976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331956+0000","last_change":"2023-11-24T06:29:06.335775+0000","last_active":"2023-11-24T06:29:07.331956+0000","last_peered":"2023-11-24T06:29:07.331956+0000","last_clean":"2023-11-24T06:29:07.331956+0000","last_became_active":"2023-11-24T06:29:06.335340+0000","last_became_peered":"2023-11-24T06:29:06.335340+0000","last_unstale":"2023-11-24T06:29:07.331956+0000","last_undegraded":"2023-11-24T06:29:07.331956+0000","last_fullsized":"2023-11-24T06:29:07.331956+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:58.147360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693280+0000","last_change":"2023-11-24T06:29:06.342019+0000","last_active":"2023-11-24T06:29:08.693280+0000","last_peered":"2023-11-24T06:29:08.693280+0000","last_clean":"2023-11-24T06:29:08.693280+0000","last_became_active":"2023-11-24T06:29:06.341628+0000","last_became_peered":"2023-11-24T06:29:06.341628+0000","last_unstale":"2023-11-24T06:29:08.693280+0000","last_undegraded":"2023-11-24T06:29:08.693280+0000","last_fullsized":"2023-11-24T06:29:08.693280+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:05.535656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367668+0000","last_change":"2023-11-24T06:29:06.330209+0000","last_active":"2023-11-24T06:29:07.367668+0000","last_peered":"2023-11-24T06:29:07.367668+0000","last_clean":"2023-11-24T06:29:07.367668+0000","last_became_active":"2023-11-24T06:29:06.329923+0000","last_became_peered":"2023-11-24T06:29:06.329923+0000","last_unstale":"2023-11-24T06:29:07.367668+0000","last_undegraded":"2023-11-24T06:29:07.367668+0000","last_fullsized":"2023-11-24T06:29:07.367668+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:02.271987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367844+0000","last_change":"2023-11-24T06:29:06.326109+0000","last_active":"2023-11-24T06:29:07.367844+0000","last_peered":"2023-11-24T06:29:07.367844+0000","last_clean":"2023-11-24T06:29:07.367844+0000","last_became_active":"2023-11-24T06:29:06.325819+0000","last_became_peered":"2023-11-24T06:29:06.325819+0000","last_unstale":"2023-11-24T06:29:07.367844+0000","last_undegraded":"2023-11-24T06:29:07.367844+0000","last_fullsized":"2023-11-24T06:29:07.367844+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:29:34.079427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368178+0000","last_change":"2023-11-24T06:29:06.332397+0000","last_active":"2023-11-24T06:29:07.368178+0000","last_peered":"2023-11-24T06:29:07.368178+0000","last_clean":"2023-11-24T06:29:07.368178+0000","last_became_active":"2023-11-24T06:29:06.331595+0000","last_became_peered":"2023-11-24T06:29:06.331595+0000","last_unstale":"2023-11-24T06:29:07.368178+0000","last_undegraded":"2023-11-24T06:29:07.368178+0000","last_fullsized":"2023-11-24T06:29:07.368178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:06.453332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328696+0000","last_change":"2023-11-24T06:29:06.336701+0000","last_active":"2023-11-24T06:29:07.328696+0000","last_peered":"2023-11-24T06:29:07.328696+0000","last_clean":"2023-11-24T06:29:07.328696+0000","last_became_active":"2023-11-24T06:29:06.334281+0000","last_became_peered":"2023-11-24T06:29:06.334281+0000","last_unstale":"2023-11-24T06:29:07.328696+0000","last_undegraded":"2023-11-24T06:29:07.328696+0000","last_fullsized":"2023-11-24T06:29:07.328696+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:06:21.778193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174342+0000","last_change":"2023-11-24T06:29:06.323064+0000","last_active":"2023-11-24T06:29:09.174342+0000","last_peered":"2023-11-24T06:29:09.174342+0000","last_clean":"2023-11-24T06:29:09.174342+0000","last_became_active":"2023-11-24T06:29:06.322653+0000","last_became_peered":"2023-11-24T06:29:06.322653+0000","last_unstale":"2023-11-24T06:29:09.174342+0000","last_undegraded":"2023-11-24T06:29:09.174342+0000","last_fullsized":"2023-11-24T06:29:09.174342+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:51.205136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970531+0000","last_change":"2023-11-24T06:29:06.326138+0000","last_active":"2023-11-24T06:29:07.970531+0000","last_peered":"2023-11-24T06:29:07.970531+0000","last_clean":"2023-11-24T06:29:07.970531+0000","last_became_active":"2023-11-24T06:29:06.325450+0000","last_became_peered":"2023-11-24T06:29:06.325450+0000","last_unstale":"2023-11-24T06:29:07.970531+0000","last_undegraded":"2023-11-24T06:29:07.970531+0000","last_fullsized":"2023-11-24T06:29:07.970531+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:00.100140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367562+0000","last_change":"2023-11-24T06:29:06.333238+0000","last_active":"2023-11-24T06:29:07.367562+0000","last_peered":"2023-11-24T06:29:07.367562+0000","last_clean":"2023-11-24T06:29:07.367562+0000","last_became_active":"2023-11-24T06:29:06.332927+0000","last_became_peered":"2023-11-24T06:29:06.332927+0000","last_unstale":"2023-11-24T06:29:07.367562+0000","last_undegraded":"2023-11-24T06:29:07.367562+0000","last_fullsized":"2023-11-24T06:29:07.367562+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:14.728462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.173972+0000","last_change":"2023-11-24T06:29:06.327451+0000","last_active":"2023-11-24T06:29:09.173972+0000","last_peered":"2023-11-24T06:29:09.173972+0000","last_clean":"2023-11-24T06:29:09.173972+0000","last_became_active":"2023-11-24T06:29:06.327166+0000","last_became_peered":"2023-11-24T06:29:06.327166+0000","last_unstale":"2023-11-24T06:29:09.173972+0000","last_undegraded":"2023-11-24T06:29:09.173972+0000","last_fullsized":"2023-11-24T06:29:09.173972+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:29.060969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331798+0000","last_change":"2023-11-24T06:29:06.327708+0000","last_active":"2023-11-24T06:29:07.331798+0000","last_peered":"2023-11-24T06:29:07.331798+0000","last_clean":"2023-11-24T06:29:07.331798+0000","last_became_active":"2023-11-24T06:29:06.327332+0000","last_became_peered":"2023-11-24T06:29:06.327332+0000","last_unstale":"2023-11-24T06:29:07.331798+0000","last_undegraded":"2023-11-24T06:29:07.331798+0000","last_fullsized":"2023-11-24T06:29:07.331798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:06.670418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328039+0000","last_change":"2023-11-24T06:29:06.327041+0000","last_active":"2023-11-24T06:29:07.328039+0000","last_peered":"2023-11-24T06:29:07.328039+0000","last_clean":"2023-11-24T06:29:07.328039+0000","last_became_active":"2023-11-24T06:29:06.326686+0000","last_became_peered":"2023-11-24T06:29:06.326686+0000","last_unstale":"2023-11-24T06:29:07.328039+0000","last_undegraded":"2023-11-24T06:29:07.328039+0000","last_fullsized":"2023-11-24T06:29:07.328039+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:38.297508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847556+0000","last_change":"2023-11-24T06:29:06.328709+0000","last_active":"2023-11-24T06:29:08.847556+0000","last_peered":"2023-11-24T06:29:08.847556+0000","last_clean":"2023-11-24T06:29:08.847556+0000","last_became_active":"2023-11-24T06:29:06.328417+0000","last_became_peered":"2023-11-24T06:29:06.328417+0000","last_unstale":"2023-11-24T06:29:08.847556+0000","last_undegraded":"2023-11-24T06:29:08.847556+0000","last_fullsized":"2023-11-24T06:29:08.847556+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:56.358878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332926+0000","last_change":"2023-11-24T06:29:05.315430+0000","last_active":"2023-11-24T06:29:07.332926+0000","last_peered":"2023-11-24T06:29:07.332926+0000","last_clean":"2023-11-24T06:29:07.332926+0000","last_became_active":"2023-11-24T06:29:05.314673+0000","last_became_peered":"2023-11-24T06:29:05.314673+0000","last_unstale":"2023-11-24T06:29:07.332926+0000","last_undegraded":"2023-11-24T06:29:07.332926+0000","last_fullsized":"2023-11-24T06:29:07.332926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:14.294532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368144+0000","last_change":"2023-11-24T06:29:06.330031+0000","last_active":"2023-11-24T06:29:07.368144+0000","last_peered":"2023-11-24T06:29:07.368144+0000","last_clean":"2023-11-24T06:29:07.368144+0000","last_became_active":"2023-11-24T06:29:06.328959+0000","last_became_peered":"2023-11-24T06:29:06.328959+0000","last_unstale":"2023-11-24T06:29:07.368144+0000","last_undegraded":"2023-11-24T06:29:07.368144+0000","last_fullsized":"2023-11-24T06:29:07.368144+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:36.840838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174866+0000","last_change":"2023-11-24T06:29:05.316968+0000","last_active":"2023-11-24T06:29:09.174866+0000","last_peered":"2023-11-24T06:29:09.174866+0000","last_clean":"2023-11-24T06:29:09.174866+0000","last_became_active":"2023-11-24T06:29:05.316628+0000","last_became_peered":"2023-11-24T06:29:05.316628+0000","last_unstale":"2023-11-24T06:29:09.174866+0000","last_undegraded":"2023-11-24T06:29:09.174866+0000","last_fullsized":"2023-11-24T06:29:09.174866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:05.439986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970743+0000","last_change":"2023-11-24T06:29:06.331094+0000","last_active":"2023-11-24T06:29:07.970743+0000","last_peered":"2023-11-24T06:29:07.970743+0000","last_clean":"2023-11-24T06:29:07.970743+0000","last_became_active":"2023-11-24T06:29:06.329737+0000","last_became_peered":"2023-11-24T06:29:06.329737+0000","last_unstale":"2023-11-24T06:29:07.970743+0000","last_undegraded":"2023-11-24T06:29:07.970743+0000","last_fullsized":"2023-11-24T06:29:07.970743+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:32.445582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847491+0000","last_change":"2023-11-24T06:29:05.300374+0000","last_active":"2023-11-24T06:29:08.847491+0000","last_peered":"2023-11-24T06:29:08.847491+0000","last_clean":"2023-11-24T06:29:08.847491+0000","last_became_active":"2023-11-24T06:29:05.298984+0000","last_became_peered":"2023-11-24T06:29:05.298984+0000","last_unstale":"2023-11-24T06:29:08.847491+0000","last_undegraded":"2023-11-24T06:29:08.847491+0000","last_fullsized":"2023-11-24T06:29:08.847491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:13.715116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847401+0000","last_change":"2023-11-24T06:29:06.327156+0000","last_active":"2023-11-24T06:29:08.847401+0000","last_peered":"2023-11-24T06:29:08.847401+0000","last_clean":"2023-11-24T06:29:08.847401+0000","last_became_active":"2023-11-24T06:29:06.326863+0000","last_became_peered":"2023-11-24T06:29:06.326863+0000","last_unstale":"2023-11-24T06:29:08.847401+0000","last_undegraded":"2023-11-24T06:29:08.847401+0000","last_fullsized":"2023-11-24T06:29:08.847401+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:06.453332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334422+0000","last_change":"2023-11-24T06:29:05.305604+0000","last_active":"2023-11-24T06:29:07.334422+0000","last_peered":"2023-11-24T06:29:07.334422+0000","last_clean":"2023-11-24T06:29:07.334422+0000","last_became_active":"2023-11-24T06:29:05.305231+0000","last_became_peered":"2023-11-24T06:29:05.305231+0000","last_unstale":"2023-11-24T06:29:07.334422+0000","last_undegraded":"2023-11-24T06:29:07.334422+0000","last_fullsized":"2023-11-24T06:29:07.334422+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:45.022521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.173989+0000","last_change":"2023-11-24T06:29:06.333673+0000","last_active":"2023-11-24T06:29:09.173989+0000","last_peered":"2023-11-24T06:29:09.173989+0000","last_clean":"2023-11-24T06:29:09.173989+0000","last_became_active":"2023-11-24T06:29:06.332688+0000","last_became_peered":"2023-11-24T06:29:06.332688+0000","last_unstale":"2023-11-24T06:29:09.173989+0000","last_undegraded":"2023-11-24T06:29:09.173989+0000","last_fullsized":"2023-11-24T06:29:09.173989+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:38.018110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332166+0000","last_change":"2023-11-24T06:29:05.302707+0000","last_active":"2023-11-24T06:29:07.332166+0000","last_peered":"2023-11-24T06:29:07.332166+0000","last_clean":"2023-11-24T06:29:07.332166+0000","last_became_active":"2023-11-24T06:29:05.302478+0000","last_became_peered":"2023-11-24T06:29:05.302478+0000","last_unstale":"2023-11-24T06:29:07.332166+0000","last_undegraded":"2023-11-24T06:29:07.332166+0000","last_fullsized":"2023-11-24T06:29:07.332166+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:23:29.372693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694261+0000","last_change":"2023-11-24T06:29:06.340275+0000","last_active":"2023-11-24T06:29:08.694261+0000","last_peered":"2023-11-24T06:29:08.694261+0000","last_clean":"2023-11-24T06:29:08.694261+0000","last_became_active":"2023-11-24T06:29:06.339516+0000","last_became_peered":"2023-11-24T06:29:06.339516+0000","last_unstale":"2023-11-24T06:29:08.694261+0000","last_undegraded":"2023-11-24T06:29:08.694261+0000","last_fullsized":"2023-11-24T06:29:08.694261+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:08.411981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174481+0000","last_change":"2023-11-24T06:29:05.312126+0000","last_active":"2023-11-24T06:29:09.174481+0000","last_peered":"2023-11-24T06:29:09.174481+0000","last_clean":"2023-11-24T06:29:09.174481+0000","last_became_active":"2023-11-24T06:29:05.311140+0000","last_became_peered":"2023-11-24T06:29:05.311140+0000","last_unstale":"2023-11-24T06:29:09.174481+0000","last_undegraded":"2023-11-24T06:29:09.174481+0000","last_fullsized":"2023-11-24T06:29:09.174481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:08.253511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693247+0000","last_change":"2023-11-24T06:29:06.338882+0000","last_active":"2023-11-24T06:29:08.693247+0000","last_peered":"2023-11-24T06:29:08.693247+0000","last_clean":"2023-11-24T06:29:08.693247+0000","last_became_active":"2023-11-24T06:29:06.338613+0000","last_became_peered":"2023-11-24T06:29:06.338613+0000","last_unstale":"2023-11-24T06:29:08.693247+0000","last_undegraded":"2023-11-24T06:29:08.693247+0000","last_fullsized":"2023-11-24T06:29:08.693247+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:46.035866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970140+0000","last_change":"2023-11-24T06:29:05.295971+0000","last_active":"2023-11-24T06:29:07.970140+0000","last_peered":"2023-11-24T06:29:07.970140+0000","last_clean":"2023-11-24T06:29:07.970140+0000","last_became_active":"2023-11-24T06:29:05.295555+0000","last_became_peered":"2023-11-24T06:29:05.295555+0000","last_unstale":"2023-11-24T06:29:07.970140+0000","last_undegraded":"2023-11-24T06:29:07.970140+0000","last_fullsized":"2023-11-24T06:29:07.970140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:28.047624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.367349+0000","last_change":"2023-11-24T06:29:06.333751+0000","last_active":"2023-11-24T06:29:07.367349+0000","last_peered":"2023-11-24T06:29:07.367349+0000","last_clean":"2023-11-24T06:29:07.367349+0000","last_became_active":"2023-11-24T06:29:06.332467+0000","last_became_peered":"2023-11-24T06:29:06.332467+0000","last_unstale":"2023-11-24T06:29:07.367349+0000","last_undegraded":"2023-11-24T06:29:07.367349+0000","last_fullsized":"2023-11-24T06:29:07.367349+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:09.266856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693538+0000","last_change":"2023-11-24T06:29:05.302909+0000","last_active":"2023-11-24T06:29:08.693538+0000","last_peered":"2023-11-24T06:29:08.693538+0000","last_clean":"2023-11-24T06:29:08.693538+0000","last_became_active":"2023-11-24T06:29:05.301890+0000","last_became_peered":"2023-11-24T06:29:05.301890+0000","last_unstale":"2023-11-24T06:29:08.693538+0000","last_undegraded":"2023-11-24T06:29:08.693538+0000","last_fullsized":"2023-11-24T06:29:08.693538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:24.625401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174764+0000","last_change":"2023-11-24T06:29:06.356308+0000","last_active":"2023-11-24T06:29:09.174764+0000","last_peered":"2023-11-24T06:29:09.174764+0000","last_clean":"2023-11-24T06:29:09.174764+0000","last_became_active":"2023-11-24T06:29:06.355808+0000","last_became_peered":"2023-11-24T06:29:06.355808+0000","last_unstale":"2023-11-24T06:29:09.174764+0000","last_undegraded":"2023-11-24T06:29:09.174764+0000","last_fullsized":"2023-11-24T06:29:09.174764+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:58:54.640138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174603+0000","last_change":"2023-11-24T06:29:05.309896+0000","last_active":"2023-11-24T06:29:09.174603+0000","last_peered":"2023-11-24T06:29:09.174603+0000","last_clean":"2023-11-24T06:29:09.174603+0000","last_became_active":"2023-11-24T06:29:05.306490+0000","last_became_peered":"2023-11-24T06:29:05.306490+0000","last_unstale":"2023-11-24T06:29:09.174603+0000","last_undegraded":"2023-11-24T06:29:09.174603+0000","last_fullsized":"2023-11-24T06:29:09.174603+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:57:31.088381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334968+0000","last_change":"2023-11-24T06:29:06.320137+0000","last_active":"2023-11-24T06:29:07.334968+0000","last_peered":"2023-11-24T06:29:07.334968+0000","last_clean":"2023-11-24T06:29:07.334968+0000","last_became_active":"2023-11-24T06:29:06.319811+0000","last_became_peered":"2023-11-24T06:29:06.319811+0000","last_unstale":"2023-11-24T06:29:07.334968+0000","last_undegraded":"2023-11-24T06:29:07.334968+0000","last_fullsized":"2023-11-24T06:29:07.334968+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:28.783174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693911+0000","last_change":"2023-11-24T06:29:05.304961+0000","last_active":"2023-11-24T06:29:08.693911+0000","last_peered":"2023-11-24T06:29:08.693911+0000","last_clean":"2023-11-24T06:29:08.693911+0000","last_became_active":"2023-11-24T06:29:05.304645+0000","last_became_peered":"2023-11-24T06:29:05.304645+0000","last_unstale":"2023-11-24T06:29:08.693911+0000","last_undegraded":"2023-11-24T06:29:08.693911+0000","last_fullsized":"2023-11-24T06:29:08.693911+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:37.004764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332604+0000","last_change":"2023-11-24T06:29:06.329494+0000","last_active":"2023-11-24T06:29:07.332604+0000","last_peered":"2023-11-24T06:29:07.332604+0000","last_clean":"2023-11-24T06:29:07.332604+0000","last_became_active":"2023-11-24T06:29:06.329174+0000","last_became_peered":"2023-11-24T06:29:06.329174+0000","last_unstale":"2023-11-24T06:29:07.332604+0000","last_undegraded":"2023-11-24T06:29:07.332604+0000","last_fullsized":"2023-11-24T06:29:07.332604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:22.265792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334869+0000","last_change":"2023-11-24T06:29:05.316665+0000","last_active":"2023-11-24T06:29:07.334869+0000","last_peered":"2023-11-24T06:29:07.334869+0000","last_clean":"2023-11-24T06:29:07.334869+0000","last_became_active":"2023-11-24T06:29:05.316403+0000","last_became_peered":"2023-11-24T06:29:05.316403+0000","last_unstale":"2023-11-24T06:29:07.334869+0000","last_undegraded":"2023-11-24T06:29:07.334869+0000","last_fullsized":"2023-11-24T06:29:07.334869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:59.086794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334595+0000","last_change":"2023-11-24T06:29:06.327449+0000","last_active":"2023-11-24T06:29:07.334595+0000","last_peered":"2023-11-24T06:29:07.334595+0000","last_clean":"2023-11-24T06:29:07.334595+0000","last_became_active":"2023-11-24T06:29:06.327156+0000","last_became_peered":"2023-11-24T06:29:06.327156+0000","last_unstale":"2023-11-24T06:29:07.334595+0000","last_undegraded":"2023-11-24T06:29:07.334595+0000","last_fullsized":"2023-11-24T06:29:07.334595+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:08.411981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328753+0000","last_change":"2023-11-24T06:29:05.302526+0000","last_active":"2023-11-24T06:29:07.328753+0000","last_peered":"2023-11-24T06:29:07.328753+0000","last_clean":"2023-11-24T06:29:07.328753+0000","last_became_active":"2023-11-24T06:29:05.302257+0000","last_became_peered":"2023-11-24T06:29:05.302257+0000","last_unstale":"2023-11-24T06:29:07.328753+0000","last_undegraded":"2023-11-24T06:29:07.328753+0000","last_fullsized":"2023-11-24T06:29:07.328753+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:44.641158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368307+0000","last_change":"2023-11-24T06:29:06.349088+0000","last_active":"2023-11-24T06:29:07.368307+0000","last_peered":"2023-11-24T06:29:07.368307+0000","last_clean":"2023-11-24T06:29:07.368307+0000","last_became_active":"2023-11-24T06:29:06.348771+0000","last_became_peered":"2023-11-24T06:29:06.348771+0000","last_unstale":"2023-11-24T06:29:07.368307+0000","last_undegraded":"2023-11-24T06:29:07.368307+0000","last_fullsized":"2023-11-24T06:29:07.368307+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:56.358878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332293+0000","last_change":"2023-11-24T06:29:05.299387+0000","last_active":"2023-11-24T06:29:07.332293+0000","last_peered":"2023-11-24T06:29:07.332293+0000","last_clean":"2023-11-24T06:29:07.332293+0000","last_became_active":"2023-11-24T06:29:05.299110+0000","last_became_peered":"2023-11-24T06:29:05.299110+0000","last_unstale":"2023-11-24T06:29:07.332293+0000","last_undegraded":"2023-11-24T06:29:07.332293+0000","last_fullsized":"2023-11-24T06:29:07.332293+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:08.652639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694058+0000","last_change":"2023-11-24T06:29:06.345421+0000","last_active":"2023-11-24T06:29:08.694058+0000","last_peered":"2023-11-24T06:29:08.694058+0000","last_clean":"2023-11-24T06:29:08.694058+0000","last_became_active":"2023-11-24T06:29:06.345062+0000","last_became_peered":"2023-11-24T06:29:06.345062+0000","last_unstale":"2023-11-24T06:29:08.694058+0000","last_undegraded":"2023-11-24T06:29:08.694058+0000","last_fullsized":"2023-11-24T06:29:08.694058+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:45.404029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334123+0000","last_change":"2023-11-24T06:29:05.301715+0000","last_active":"2023-11-24T06:29:07.334123+0000","last_peered":"2023-11-24T06:29:07.334123+0000","last_clean":"2023-11-24T06:29:07.334123+0000","last_became_active":"2023-11-24T06:29:05.301351+0000","last_became_peered":"2023-11-24T06:29:05.301351+0000","last_unstale":"2023-11-24T06:29:07.334123+0000","last_undegraded":"2023-11-24T06:29:07.334123+0000","last_fullsized":"2023-11-24T06:29:07.334123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.191791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331824+0000","last_change":"2023-11-24T06:29:06.332462+0000","last_active":"2023-11-24T06:29:07.331824+0000","last_peered":"2023-11-24T06:29:07.331824+0000","last_clean":"2023-11-24T06:29:07.331824+0000","last_became_active":"2023-11-24T06:29:06.331894+0000","last_became_peered":"2023-11-24T06:29:06.331894+0000","last_unstale":"2023-11-24T06:29:07.331824+0000","last_undegraded":"2023-11-24T06:29:07.331824+0000","last_fullsized":"2023-11-24T06:29:07.331824+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:07.056016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.331928+0000","last_change":"2023-11-24T06:29:05.302681+0000","last_active":"2023-11-24T06:29:07.331928+0000","last_peered":"2023-11-24T06:29:07.331928+0000","last_clean":"2023-11-24T06:29:07.331928+0000","last_became_active":"2023-11-24T06:29:05.302383+0000","last_became_peered":"2023-11-24T06:29:05.302383+0000","last_unstale":"2023-11-24T06:29:07.331928+0000","last_undegraded":"2023-11-24T06:29:07.331928+0000","last_fullsized":"2023-11-24T06:29:07.331928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:44.390683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334184+0000","last_change":"2023-11-24T06:29:06.330350+0000","last_active":"2023-11-24T06:29:07.334184+0000","last_peered":"2023-11-24T06:29:07.334184+0000","last_clean":"2023-11-24T06:29:07.334184+0000","last_became_active":"2023-11-24T06:29:06.329973+0000","last_became_peered":"2023-11-24T06:29:06.329973+0000","last_unstale":"2023-11-24T06:29:07.334184+0000","last_undegraded":"2023-11-24T06:29:07.334184+0000","last_fullsized":"2023-11-24T06:29:07.334184+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:01.914708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693567+0000","last_change":"2023-11-24T06:29:05.313668+0000","last_active":"2023-11-24T06:29:08.693567+0000","last_peered":"2023-11-24T06:29:08.693567+0000","last_clean":"2023-11-24T06:29:08.693567+0000","last_became_active":"2023-11-24T06:29:05.313336+0000","last_became_peered":"2023-11-24T06:29:05.313336+0000","last_unstale":"2023-11-24T06:29:08.693567+0000","last_undegraded":"2023-11-24T06:29:08.693567+0000","last_fullsized":"2023-11-24T06:29:08.693567+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:19.332329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328737+0000","last_change":"2023-11-24T06:29:06.323089+0000","last_active":"2023-11-24T06:29:07.328737+0000","last_peered":"2023-11-24T06:29:07.328737+0000","last_clean":"2023-11-24T06:29:07.328737+0000","last_became_active":"2023-11-24T06:29:06.322785+0000","last_became_peered":"2023-11-24T06:29:06.322785+0000","last_unstale":"2023-11-24T06:29:07.328737+0000","last_undegraded":"2023-11-24T06:29:07.328737+0000","last_fullsized":"2023-11-24T06:29:07.328737+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:05:36.785281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.328588+0000","last_change":"2023-11-24T06:29:05.301582+0000","last_active":"2023-11-24T06:29:07.328588+0000","last_peered":"2023-11-24T06:29:07.328588+0000","last_clean":"2023-11-24T06:29:07.328588+0000","last_became_active":"2023-11-24T06:29:05.300677+0000","last_became_peered":"2023-11-24T06:29:05.300677+0000","last_unstale":"2023-11-24T06:29:07.328588+0000","last_undegraded":"2023-11-24T06:29:07.328588+0000","last_fullsized":"2023-11-24T06:29:07.328588+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:46:20.950862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694398+0000","last_change":"2023-11-24T06:29:06.350894+0000","last_active":"2023-11-24T06:29:08.694398+0000","last_peered":"2023-11-24T06:29:08.694398+0000","last_clean":"2023-11-24T06:29:08.694398+0000","last_became_active":"2023-11-24T06:29:06.350625+0000","last_became_peered":"2023-11-24T06:29:06.350625+0000","last_unstale":"2023-11-24T06:29:08.694398+0000","last_undegraded":"2023-11-24T06:29:08.694398+0000","last_fullsized":"2023-11-24T06:29:08.694398+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:15.307878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174994+0000","last_change":"2023-11-24T06:29:05.311136+0000","last_active":"2023-11-24T06:29:09.174994+0000","last_peered":"2023-11-24T06:29:09.174994+0000","last_clean":"2023-11-24T06:29:09.174994+0000","last_became_active":"2023-11-24T06:29:05.310389+0000","last_became_peered":"2023-11-24T06:29:05.310389+0000","last_unstale":"2023-11-24T06:29:09.174994+0000","last_undegraded":"2023-11-24T06:29:09.174994+0000","last_fullsized":"2023-11-24T06:29:09.174994+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:55.345533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368349+0000","last_change":"2023-11-24T06:29:06.335401+0000","last_active":"2023-11-24T06:29:07.368349+0000","last_peered":"2023-11-24T06:29:07.368349+0000","last_clean":"2023-11-24T06:29:07.368349+0000","last_became_active":"2023-11-24T06:29:06.334627+0000","last_became_peered":"2023-11-24T06:29:06.334627+0000","last_unstale":"2023-11-24T06:29:07.368349+0000","last_undegraded":"2023-11-24T06:29:07.368349+0000","last_fullsized":"2023-11-24T06:29:07.368349+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:57:32.101727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970709+0000","last_change":"2023-11-24T06:29:05.306030+0000","last_active":"2023-11-24T06:29:07.970709+0000","last_peered":"2023-11-24T06:29:07.970709+0000","last_clean":"2023-11-24T06:29:07.970709+0000","last_became_active":"2023-11-24T06:29:05.304779+0000","last_became_peered":"2023-11-24T06:29:05.304779+0000","last_unstale":"2023-11-24T06:29:07.970709+0000","last_undegraded":"2023-11-24T06:29:07.970709+0000","last_fullsized":"2023-11-24T06:29:07.970709+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:28.325867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368441+0000","last_change":"2023-11-24T06:29:06.334209+0000","last_active":"2023-11-24T06:29:07.368441+0000","last_peered":"2023-11-24T06:29:07.368441+0000","last_clean":"2023-11-24T06:29:07.368441+0000","last_became_active":"2023-11-24T06:29:06.332842+0000","last_became_peered":"2023-11-24T06:29:06.332842+0000","last_unstale":"2023-11-24T06:29:07.368441+0000","last_undegraded":"2023-11-24T06:29:07.368441+0000","last_fullsized":"2023-11-24T06:29:07.368441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:13.529104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332416+0000","last_change":"2023-11-24T06:29:05.313374+0000","last_active":"2023-11-24T06:29:07.332416+0000","last_peered":"2023-11-24T06:29:07.332416+0000","last_clean":"2023-11-24T06:29:07.332416+0000","last_became_active":"2023-11-24T06:29:05.313043+0000","last_became_peered":"2023-11-24T06:29:05.313043+0000","last_unstale":"2023-11-24T06:29:07.332416+0000","last_undegraded":"2023-11-24T06:29:07.332416+0000","last_fullsized":"2023-11-24T06:29:07.332416+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:04.653296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332114+0000","last_change":"2023-11-24T06:29:06.335382+0000","last_active":"2023-11-24T06:29:07.332114+0000","last_peered":"2023-11-24T06:29:07.332114+0000","last_clean":"2023-11-24T06:29:07.332114+0000","last_became_active":"2023-11-24T06:29:06.334885+0000","last_became_peered":"2023-11-24T06:29:06.334885+0000","last_unstale":"2023-11-24T06:29:07.332114+0000","last_undegraded":"2023-11-24T06:29:07.332114+0000","last_fullsized":"2023-11-24T06:29:07.332114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:45:58.472377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332551+0000","last_change":"2023-11-24T06:29:05.306482+0000","last_active":"2023-11-24T06:29:07.332551+0000","last_peered":"2023-11-24T06:29:07.332551+0000","last_clean":"2023-11-24T06:29:07.332551+0000","last_became_active":"2023-11-24T06:29:05.306168+0000","last_became_peered":"2023-11-24T06:29:05.306168+0000","last_unstale":"2023-11-24T06:29:07.332551+0000","last_undegraded":"2023-11-24T06:29:07.332551+0000","last_fullsized":"2023-11-24T06:29:07.332551+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:37.230777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847159+0000","last_change":"2023-11-24T06:29:06.320098+0000","last_active":"2023-11-24T06:29:08.847159+0000","last_peered":"2023-11-24T06:29:08.847159+0000","last_clean":"2023-11-24T06:29:08.847159+0000","last_became_active":"2023-11-24T06:29:06.319791+0000","last_became_peered":"2023-11-24T06:29:06.319791+0000","last_unstale":"2023-11-24T06:29:08.847159+0000","last_undegraded":"2023-11-24T06:29:08.847159+0000","last_fullsized":"2023-11-24T06:29:08.847159+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:36.840838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970324+0000","last_change":"2023-11-24T06:29:05.314367+0000","last_active":"2023-11-24T06:29:07.970324+0000","last_peered":"2023-11-24T06:29:07.970324+0000","last_clean":"2023-11-24T06:29:07.970324+0000","last_became_active":"2023-11-24T06:29:05.314058+0000","last_became_peered":"2023-11-24T06:29:05.314058+0000","last_unstale":"2023-11-24T06:29:07.970324+0000","last_undegraded":"2023-11-24T06:29:07.970324+0000","last_fullsized":"2023-11-24T06:29:07.970324+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:12.515758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.327989+0000","last_change":"2023-11-24T06:29:06.331684+0000","last_active":"2023-11-24T06:29:07.327989+0000","last_peered":"2023-11-24T06:29:07.327989+0000","last_clean":"2023-11-24T06:29:07.327989+0000","last_became_active":"2023-11-24T06:29:06.330068+0000","last_became_peered":"2023-11-24T06:29:06.330068+0000","last_unstale":"2023-11-24T06:29:07.327989+0000","last_undegraded":"2023-11-24T06:29:07.327989+0000","last_fullsized":"2023-11-24T06:29:07.327989+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:15:59.832018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693124+0000","last_change":"2023-11-24T06:29:05.305941+0000","last_active":"2023-11-24T06:29:08.693124+0000","last_peered":"2023-11-24T06:29:08.693124+0000","last_clean":"2023-11-24T06:29:08.693124+0000","last_became_active":"2023-11-24T06:29:05.304622+0000","last_became_peered":"2023-11-24T06:29:05.304622+0000","last_unstale":"2023-11-24T06:29:08.693124+0000","last_undegraded":"2023-11-24T06:29:08.693124+0000","last_fullsized":"2023-11-24T06:29:08.693124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:05.439986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693193+0000","last_change":"2023-11-24T06:29:06.343114+0000","last_active":"2023-11-24T06:29:08.693193+0000","last_peered":"2023-11-24T06:29:08.693193+0000","last_clean":"2023-11-24T06:29:08.693193+0000","last_became_active":"2023-11-24T06:29:06.342829+0000","last_became_peered":"2023-11-24T06:29:06.342829+0000","last_unstale":"2023-11-24T06:29:08.693193+0000","last_undegraded":"2023-11-24T06:29:08.693193+0000","last_fullsized":"2023-11-24T06:29:08.693193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:06.918223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.333844+0000","last_change":"2023-11-24T06:29:05.295441+0000","last_active":"2023-11-24T06:29:07.333844+0000","last_peered":"2023-11-24T06:29:07.333844+0000","last_clean":"2023-11-24T06:29:07.333844+0000","last_became_active":"2023-11-24T06:29:05.294655+0000","last_became_peered":"2023-11-24T06:29:05.294655+0000","last_unstale":"2023-11-24T06:29:07.333844+0000","last_undegraded":"2023-11-24T06:29:07.333844+0000","last_fullsized":"2023-11-24T06:29:07.333844+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:06:37.004764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.368407+0000","last_change":"2023-11-24T06:29:06.355651+0000","last_active":"2023-11-24T06:29:07.368407+0000","last_peered":"2023-11-24T06:29:07.368407+0000","last_clean":"2023-11-24T06:29:07.368407+0000","last_became_active":"2023-11-24T06:29:06.355372+0000","last_became_peered":"2023-11-24T06:29:06.355372+0000","last_unstale":"2023-11-24T06:29:07.368407+0000","last_undegraded":"2023-11-24T06:29:07.368407+0000","last_fullsized":"2023-11-24T06:29:07.368407+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:16.218640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332590+0000","last_change":"2023-11-24T06:29:05.315575+0000","last_active":"2023-11-24T06:29:07.332590+0000","last_peered":"2023-11-24T06:29:07.332590+0000","last_clean":"2023-11-24T06:29:07.332590+0000","last_became_active":"2023-11-24T06:29:05.315255+0000","last_became_peered":"2023-11-24T06:29:05.315255+0000","last_unstale":"2023-11-24T06:29:07.332590+0000","last_undegraded":"2023-11-24T06:29:07.332590+0000","last_fullsized":"2023-11-24T06:29:07.332590+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:27.769829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694196+0000","last_change":"2023-11-24T06:29:06.339764+0000","last_active":"2023-11-24T06:29:08.694196+0000","last_peered":"2023-11-24T06:29:08.694196+0000","last_clean":"2023-11-24T06:29:08.694196+0000","last_became_active":"2023-11-24T06:29:06.339233+0000","last_became_peered":"2023-11-24T06:29:06.339233+0000","last_unstale":"2023-11-24T06:29:08.694196+0000","last_undegraded":"2023-11-24T06:29:08.694196+0000","last_fullsized":"2023-11-24T06:29:08.694196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:32.445582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334896+0000","last_change":"2023-11-24T06:29:05.315202+0000","last_active":"2023-11-24T06:29:07.334896+0000","last_peered":"2023-11-24T06:29:07.334896+0000","last_clean":"2023-11-24T06:29:07.334896+0000","last_became_active":"2023-11-24T06:29:05.314920+0000","last_became_peered":"2023-11-24T06:29:05.314920+0000","last_unstale":"2023-11-24T06:29:07.334896+0000","last_undegraded":"2023-11-24T06:29:07.334896+0000","last_fullsized":"2023-11-24T06:29:07.334896+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:44.390683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334622+0000","last_change":"2023-11-24T06:29:06.330176+0000","last_active":"2023-11-24T06:29:07.334622+0000","last_peered":"2023-11-24T06:29:07.334622+0000","last_clean":"2023-11-24T06:29:07.334622+0000","last_became_active":"2023-11-24T06:29:06.329800+0000","last_became_peered":"2023-11-24T06:29:06.329800+0000","last_unstale":"2023-11-24T06:29:07.334622+0000","last_undegraded":"2023-11-24T06:29:07.334622+0000","last_fullsized":"2023-11-24T06:29:07.334622+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:15.307878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.847086+0000","last_change":"2023-11-24T06:29:05.306907+0000","last_active":"2023-11-24T06:29:08.847086+0000","last_peered":"2023-11-24T06:29:08.847086+0000","last_clean":"2023-11-24T06:29:08.847086+0000","last_became_active":"2023-11-24T06:29:05.305842+0000","last_became_peered":"2023-11-24T06:29:05.305842+0000","last_unstale":"2023-11-24T06:29:08.847086+0000","last_undegraded":"2023-11-24T06:29:08.847086+0000","last_fullsized":"2023-11-24T06:29:08.847086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:05:24.625401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332126+0000","last_change":"2023-11-24T06:29:06.329730+0000","last_active":"2023-11-24T06:29:07.332126+0000","last_peered":"2023-11-24T06:29:07.332126+0000","last_clean":"2023-11-24T06:29:07.332126+0000","last_became_active":"2023-11-24T06:29:06.329406+0000","last_became_peered":"2023-11-24T06:29:06.329406+0000","last_unstale":"2023-11-24T06:29:07.332126+0000","last_undegraded":"2023-11-24T06:29:07.332126+0000","last_fullsized":"2023-11-24T06:29:07.332126+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:44:18.807302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.334457+0000","last_change":"2023-11-24T06:29:05.310752+0000","last_active":"2023-11-24T06:29:07.334457+0000","last_peered":"2023-11-24T06:29:07.334457+0000","last_clean":"2023-11-24T06:29:07.334457+0000","last_became_active":"2023-11-24T06:29:05.310481+0000","last_became_peered":"2023-11-24T06:29:05.310481+0000","last_unstale":"2023-11-24T06:29:07.334457+0000","last_undegraded":"2023-11-24T06:29:07.334457+0000","last_fullsized":"2023-11-24T06:29:07.334457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:35.880888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.332436+0000","last_change":"2023-11-24T06:29:06.330845+0000","last_active":"2023-11-24T06:29:07.332436+0000","last_peered":"2023-11-24T06:29:07.332436+0000","last_clean":"2023-11-24T06:29:07.332436+0000","last_became_active":"2023-11-24T06:29:06.330530+0000","last_became_peered":"2023-11-24T06:29:06.330530+0000","last_unstale":"2023-11-24T06:29:07.332436+0000","last_undegraded":"2023-11-24T06:29:07.332436+0000","last_fullsized":"2023-11-24T06:29:07.332436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:06.408910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.174304+0000","last_change":"2023-11-24T06:29:05.306621+0000","last_active":"2023-11-24T06:29:09.174304+0000","last_peered":"2023-11-24T06:29:09.174304+0000","last_clean":"2023-11-24T06:29:09.174304+0000","last_became_active":"2023-11-24T06:29:05.306338+0000","last_became_peered":"2023-11-24T06:29:05.306338+0000","last_unstale":"2023-11-24T06:29:09.174304+0000","last_undegraded":"2023-11-24T06:29:09.174304+0000","last_fullsized":"2023-11-24T06:29:09.174304+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:29:33.066082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.970295+0000","last_change":"2023-11-24T06:29:06.312498+0000","last_active":"2023-11-24T06:29:07.970295+0000","last_peered":"2023-11-24T06:29:07.970295+0000","last_clean":"2023-11-24T06:29:07.970295+0000","last_became_active":"2023-11-24T06:29:06.311781+0000","last_became_peered":"2023-11-24T06:29:06.311781+0000","last_unstale":"2023-11-24T06:29:07.970295+0000","last_undegraded":"2023-11-24T06:29:07.970295+0000","last_fullsized":"2023-11-24T06:29:07.970295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:35.027087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.846993+0000","last_change":"2023-11-24T06:29:05.310496+0000","last_active":"2023-11-24T06:29:08.846993+0000","last_peered":"2023-11-24T06:29:08.846993+0000","last_clean":"2023-11-24T06:29:08.846993+0000","last_became_active":"2023-11-24T06:29:05.310192+0000","last_became_peered":"2023-11-24T06:29:05.310192+0000","last_unstale":"2023-11-24T06:29:08.846993+0000","last_undegraded":"2023-11-24T06:29:08.846993+0000","last_fullsized":"2023-11-24T06:29:08.846993+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:19.332329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:09.173921+0000","last_change":"2023-11-24T06:29:06.328679+0000","last_active":"2023-11-24T06:29:09.173921+0000","last_peered":"2023-11-24T06:29:09.173921+0000","last_clean":"2023-11-24T06:29:09.173921+0000","last_became_active":"2023-11-24T06:29:06.328284+0000","last_became_peered":"2023-11-24T06:29:06.328284+0000","last_unstale":"2023-11-24T06:29:09.173921+0000","last_undegraded":"2023-11-24T06:29:09.173921+0000","last_fullsized":"2023-11-24T06:29:09.173921+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:13.529104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.693864+0000","last_change":"2023-11-24T06:29:05.302224+0000","last_active":"2023-11-24T06:29:08.693864+0000","last_peered":"2023-11-24T06:29:08.693864+0000","last_clean":"2023-11-24T06:29:08.693864+0000","last_became_active":"2023-11-24T06:29:05.301928+0000","last_became_peered":"2023-11-24T06:29:05.301928+0000","last_unstale":"2023-11-24T06:29:08.693864+0000","last_undegraded":"2023-11-24T06:29:08.693864+0000","last_fullsized":"2023-11-24T06:29:08.693864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:08.253511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:08.694088+0000","last_change":"2023-11-24T06:29:06.349020+0000","last_active":"2023-11-24T06:29:08.694088+0000","last_peered":"2023-11-24T06:29:08.694088+0000","last_clean":"2023-11-24T06:29:08.694088+0000","last_became_active":"2023-11-24T06:29:06.346943+0000","last_became_peered":"2023-11-24T06:29:06.346943+0000","last_unstale":"2023-11-24T06:29:08.694088+0000","last_undegraded":"2023-11-24T06:29:08.694088+0000","last_fullsized":"2023-11-24T06:29:08.694088+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:05.267534+0000","last_clean_scrub_stamp":"2023-11-24T06:29:05.267534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:36.894234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T06:29:07.971257+0000","last_change":"2023-11-24T06:29:05.311828+0000","last_active":"2023-11-24T06:29:07.971257+0000","last_peered":"2023-11-24T06:29:07.971257+0000","last_clean":"2023-11-24T06:29:07.971257+0000","last_became_active":"2023-11-24T06:29:05.311541+0000","last_became_peered":"2023-11-24T06:29:05.311541+0000","last_unstale":"2023-11-24T06:29:07.971257+0000","last_undegraded":"2023-11-24T06:29:07.971257+0000","last_fullsized":"2023-11-24T06:29:07.971257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:04.254188+0000","last_clean_scrub_stamp":"2023-11-24T06:29:04.254188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:00:50.191791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:29:06.302049+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:29:06.302049+0000","last_peered":"2023-11-24T06:29:06.302049+0000","last_clean":"2023-11-24T06:29:06.302049+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:29:06.302049+0000","last_undegraded":"2023-11-24T06:29:06.302049+0000","last_fullsized":"2023-11-24T06:29:06.302049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1138688,"data_stored":810904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.29,"15min":1.3280000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5780000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.181,"5min":1.2789999999999999,"15min":1.2949999999999999},"min":{"1min":0.75700000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.76,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.78900000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.65100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.218,"5min":1.359,"15min":1.383},"min":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.585,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.052}]},{"osd":2,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.2649999999999999,"15min":1.2849999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6479999999999999,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1919999999999999,"15min":1.1990000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.754,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.84799999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.365,"15min":1.403},"min":{"1min":0.80900000000000005,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.637,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2050000000000001,"15min":1.2030000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.3779999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.542,"15min":1.585},"min":{"1min":0.72099999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7170000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.5569999999999999,"15min":1.585},"min":{"1min":1.0589999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.962,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":1.5740000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.395,"15min":1.411},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.921,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.7170000000000001,"15min":1.758},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.845,"5min":6.7359999999999998,"15min":6.7359999999999998},"last":1.331}]},{"osd":6,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.825,"15min":1.8779999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.093,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.141},{"interface":"front","average":{"1min":1.377,"5min":1.659,"15min":1.706},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.881,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":1.516}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":221118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0940000000000001,"15min":1.1000000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1100000000000001,"15min":1.111},"min":{"1min":0.77800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.8009999999999999,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.623}]},{"osd":1,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.208,"15min":1.2090000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.768,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.401},{"interface":"front","average":{"1min":1.218,"5min":1.2110000000000001,"15min":1.21},"min":{"1min":0.81699999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5649999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4650000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.157,"15min":1.163},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5660000000000001,"5min":3.597,"15min":3.597},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.296,"15min":1.3149999999999999},"min":{"1min":0.77500000000000002,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6319999999999999,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.5649999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3169999999999999,"15min":1.319},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.706,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.232},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.3149999999999999,"15min":1.331},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.6259999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.52}]},{"osd":4,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.425,"15min":1.4470000000000001},"min":{"1min":0.93300000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.8089999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.375,"5min":1.3979999999999999,"15min":1.4019999999999999},"min":{"1min":0.90300000000000002,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.9219999999999999,"5min":3.04,"15min":3.04},"last":1.3520000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3720000000000001,"15min":1.377},"min":{"1min":0.875,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.76,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.593,"15min":1.615},"min":{"1min":1.03,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.762,"5min":3.774,"15min":3.774},"last":1.0289999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5569999999999999,"15min":1.5629999999999999},"min":{"1min":1.204,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.016,"5min":3.403,"15min":3.403},"last":1.7430000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.5069999999999999,"15min":1.5149999999999999},"min":{"1min":1.0189999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.7230000000000001,"5min":3.649,"15min":3.649},"last":1.7030000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":220516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.1699999999999999,"15min":1.1879999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.885,"5min":1.903,"15min":1.903},"last":1.016},{"interface":"front","average":{"1min":1.161,"5min":1.2929999999999999,"15min":1.3149999999999999},"min":{"1min":0.67400000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.135}]},{"osd":1,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1259999999999999,"15min":1.1339999999999999},"min":{"1min":0.72499999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.494},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.101,"15min":1.099},"min":{"1min":0.74399999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.4039999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.208,"15min":1.2050000000000001},"min":{"1min":0.88400000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.992,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.294,"15min":1.292},"min":{"1min":0.87,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.776,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":2.024}]},{"osd":3,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.363,"15min":1.3839999999999999},"min":{"1min":0.85199999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.536,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.379,"15min":1.389},"min":{"1min":0.90700000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7549999999999999,"5min":1.994,"15min":1.994},"last":2.0870000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3180000000000001,"15min":1.3340000000000001},"min":{"1min":0.626,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.673,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.266},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.52,"15min":1.542},"min":{"1min":1.077,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.823,"5min":3.085,"15min":3.085},"last":1.5589999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.4490000000000001,"15min":1.4379999999999999},"min":{"1min":1.173,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.8959999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.4530000000000001,"15min":1.4610000000000001},"min":{"1min":0.96099999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.8999999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.95}]},{"osd":7,"last update":"Fri Nov 24 06:28:50 2023","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.6100000000000001,"15min":1.623},"min":{"1min":1.2410000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8380000000000001,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.8149999999999999},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.6950000000000001,"15min":1.7170000000000001},"min":{"1min":1.02,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.157,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":2.202}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":540672,"data_stored":215758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.083,"15min":1.0760000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.147},{"interface":"front","average":{"1min":1.024,"5min":1.1040000000000001,"15min":1.117},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4510000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.1339999999999999,"15min":1.109},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3129999999999999,"15min":1.323},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.845,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.292,"15min":1.306},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.772,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.329,"15min":1.337},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.8320000000000001,"5min":2.254,"15min":2.254},"last":1.4159999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.3260000000000001,"15min":1.3440000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.048,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.242},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.383,"15min":1.3740000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.94199999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.5129999999999999,"15min":1.5249999999999999},"min":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.0830000000000002,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.534},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.5600000000000001,"15min":1.5740000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.1970000000000001,"5min":3.52,"15min":3.52},"last":1.6000000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.4870000000000001,"15min":1.486},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.5760000000000001,"15min":1.585},"min":{"1min":0.71699999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.3359999999999999,"5min":2.452,"15min":2.452},"last":1.641}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.657,"15min":1.677},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.1379999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.504},{"interface":"front","average":{"1min":1.6659999999999999,"5min":1.6220000000000001,"15min":1.615},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.2669999999999999,"5min":2.363,"15min":2.363},"last":1.5580000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.40200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.90700000000000003,"5min":1.355,"15min":1.355},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.41199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4690000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1160000000000001},"min":{"1min":0.51000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.4950000000000001,"5min":1.786,"15min":1.786},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.499,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4710000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4450000000000001,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.0299999999999998,"5min":3.081,"15min":3.081},"last":1.044},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3300000000000001,"15min":1.3380000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0910000000000002,"5min":2.758,"15min":2.758},"last":0.94199999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.377,"15min":1.4039999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8200000000000001,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.111},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.722,"15min":1.782},"min":{"1min":0.88600000000000001,"5min":0.745,"15min":0.745},"max":{"1min":2.149,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":0.90000000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3819999999999999,"15min":1.409},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.6919999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.032},{"interface":"front","average":{"1min":1.474,"5min":1.5349999999999999,"15min":1.5449999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.1030000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.125}]},{"osd":7,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.5960000000000001,"15min":1.6160000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.1829999999999998,"5min":4.641,"15min":4.641},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.55,"15min":1.5760000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.9750000000000001,"5min":3.798,"15min":3.798},"last":0.95499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":222080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2330000000000001,"5min":1.427,"15min":1.427},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.432,"5min":0.31,"15min":0.31},"max":{"1min":1.161,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.94199999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.96299999999999997},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2410000000000001,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0449999999999999,"15min":1.0509999999999999},"min":{"1min":0.56899999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.512,"5min":1.55,"15min":1.55},"last":1.046}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.101,"15min":1.1100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6850000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.103,"15min":1.119},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.597,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.6990000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.492,"15min":1.52},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.79,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.4690000000000001,"15min":1.49},"min":{"1min":0.94999999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6899999999999999,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.4430000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4590000000000001,"15min":1.466},"min":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.911,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.409},{"interface":"front","average":{"1min":1.385,"5min":1.49,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.9410000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.48}]},{"osd":6,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4279999999999999,"15min":1.4379999999999999},"min":{"1min":0.78200000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.6899999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.579},{"interface":"front","average":{"1min":1.419,"5min":1.5229999999999999,"15min":1.5409999999999999},"min":{"1min":0.85199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.0720000000000001,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.524}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.5369999999999999,"15min":1.5569999999999999},"min":{"1min":0.83099999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.974,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.341},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.5309999999999999,"15min":1.5389999999999999},"min":{"1min":0.90300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1699999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.613}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":220516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":1.0289999999999999,"15min":1.0409999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5840000000000001,"5min":2.593,"15min":2.593},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.026,"15min":1.034},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.135}]},{"osd":2,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.871,"15min":0.875},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.2869999999999999,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.022,"5min":1.01,"15min":1.008},"min":{"1min":0.54900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.556,"5min":1.829,"15min":1.829},"last":1.206}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1259999999999999,"15min":1.1279999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.708,"5min":1.766,"15min":1.766},"last":1.355},{"interface":"front","average":{"1min":1.016,"5min":1.0740000000000001,"15min":1.0840000000000001},"min":{"1min":0.57099999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3740000000000001,"5min":1.798,"15min":1.798},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.258,"15min":1.2569999999999999},"min":{"1min":0.82599999999999996,"5min":0.627,"15min":0.627},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.347,"5min":1.4370000000000001,"15min":1.452},"min":{"1min":1.032,"5min":1.032,"15min":1.032},"max":{"1min":1.7529999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.167}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3069999999999999,"15min":1.306},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":1.228},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3919999999999999,"15min":1.3999999999999999},"min":{"1min":0.93899999999999995,"5min":0.877,"15min":0.877},"max":{"1min":1.796,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.508}]},{"osd":6,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.399,"15min":1.395},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.46},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4530000000000001,"15min":1.456},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.0910000000000002,"5min":2.371,"15min":2.371},"last":1.6499999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4490000000000001,"15min":1.45},"min":{"1min":0.88500000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.417},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.524,"15min":1.5289999999999999},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.3759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":804548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.48999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.607,"5min":1.722,"15min":1.722},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.5,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.999,"15min":0.998},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.119,"15min":1.137},"min":{"1min":0.63400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.538,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1259999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.93100000000000005,"15min":0.94899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5229999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.085,"15min":1.0900000000000001},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.639,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.25}]},{"osd":4,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2090000000000001,"15min":1.2190000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6659999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3100000000000001,"15min":1.3080000000000001},"min":{"1min":0.91400000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.986,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.399}]},{"osd":5,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.321,"15min":1.333},"min":{"1min":0.71799999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.704,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.2509999999999999,"15min":1.2709999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6000000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.865}]},{"osd":6,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4710000000000001,"15min":1.4870000000000001},"min":{"1min":0.95499999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7230000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.345},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.492,"15min":1.518},"min":{"1min":0.91300000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.8069999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.462}]},{"osd":7,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.446,"15min":1.456},"min":{"1min":0.86499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9059999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.399,"15min":1.419},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.853,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.8120000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:11.358 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:29:11.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:29:11.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.635+0000 7f493f59e640 1 Processor -- start 2023-11-24T06:29:11.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.635+0000 7f493f59e640 1 -- start start 2023-11-24T06:29:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.637+0000 7f493f59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400b47d0 0x7f49400ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.637+0000 7f493f59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49400ceb80 con 0x7f49400b43f0 2023-11-24T06:29:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.637+0000 7f493f59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494015bac0 con 0x7f49400adfc0 2023-11-24T06:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.637+0000 7f493f59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494015bda0 con 0x7f49400b47d0 2023-11-24T06:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.637+0000 7f493e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f49400b43f0 0x7f49400ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48694/0 (socket says 172.21.15.16:48694) 2023-11-24T06:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.637+0000 7f493e59c640 1 -- 172.21.15.16:0/4219221440 learned_addr learned my addr 172.21.15.16:0/4219221440 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.638+0000 7f493ed9d640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400b47d0 0x7f49400ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.638+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547542509 0 0) 0x7f49400ceb80 con 0x7f49400b43f0 2023-11-24T06:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.638+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f492c003520 con 0x7f49400b43f0 2023-11-24T06:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.638+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4090531820 0 0) 0x7f492c003520 con 0x7f49400b43f0 2023-11-24T06:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.638+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400b47d0 msgr2=0x7f49400ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.638+0000 7f493dd9b640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400b47d0 0x7f49400ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.638+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 >> v1:172.21.15.173:6789/0 conn(0x7f49400adfc0 legacy=0x7f49400cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.638+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494015c080 con 0x7f49400b43f0 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.638+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f4930001a10 con 0x7f49400b43f0 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 >> v1:172.21.15.16:6789/0 conn(0x7f49400b43f0 legacy=0x7f49400ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493dd9b640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400b47d0 0x7f492c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493dd9b640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f492c003f70 0x7f492c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493dd9b640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f492c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494015bda0 con 0x7f492c004d00 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c00bac0 con 0x7f492c003f70 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c00bc80 con 0x7f49400b47d0 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493ed9d640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f492c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493e59c640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400b47d0 0x7f492c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f4935d9b640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f492c003f70 0x7f492c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493ed9d640 1 -- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400b47d0 msgr2=0x7f492c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493ed9d640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400b47d0 0x7f492c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493ed9d640 1 -- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f492c003f70 msgr2=0x7f492c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493ed9d640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f492c003f70 0x7f492c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493ed9d640 1 -- 172.21.15.16:0/4219221440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c003520 con 0x7f492c004d00 2023-11-24T06:29:11.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.639+0000 7f493e59c640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49400b47d0 0x7f492c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.641+0000 7f493ed9d640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f492c005180 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4938009a90 tx=0x7f492c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b22b34ff9ebab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.641+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493802c690 con 0x7f492c004d00 2023-11-24T06:29:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.641+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493802c7f0 con 0x7f492c004d00 2023-11-24T06:29:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.641+0000 7f493dd9b640 1 -- 172.21.15.16:0/4219221440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493802cba0 con 0x7f492c004d00 2023-11-24T06:29:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.641+0000 7f493f59e640 1 -- 172.21.15.16:0/4219221440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 msgr2=0x7f492c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.641+0000 7f493f59e640 1 --2- 172.21.15.16:0/4219221440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f492c005180 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4938009a90 tx=0x7f492c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.641+0000 7f493ed9d640 1 -- 172.21.15.16:0/4219221440 reap_dead start 2023-11-24T06:29:11.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.642+0000 7f493f59e640 1 -- 172.21.15.16:0/4219221440 shutdown_connections 2023-11-24T06:29:11.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.642+0000 7f493f59e640 1 -- 172.21.15.16:0/4219221440 >> 172.21.15.16:0/4219221440 conn(0x7f49400b0b70 msgr2=0x7f49400b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:11.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.642+0000 7f493f59e640 1 -- 172.21.15.16:0/4219221440 shutdown_connections 2023-11-24T06:29:11.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.642+0000 7f493f59e640 1 -- 172.21.15.16:0/4219221440 wait complete. 2023-11-24T06:29:11.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.644+0000 7f493f59e640 1 Processor -- start 2023-11-24T06:29:11.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.644+0000 7f493f59e640 1 -- start start 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f49401ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493f59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49401569c0 0x7f4940156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493f59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49400aba70 0x7f49400abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493f59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4940157380 con 0x7f492c004d00 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493f59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400ac730 con 0x7f49400aba70 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493f59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400aca10 con 0x7f49401569c0 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f49401ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f49401ef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34168/0 (socket says 172.21.15.16:34168) 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493e59c640 1 -- 172.21.15.16:0/2827325631 learned_addr learned my addr 172.21.15.16:0/2827325631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:11.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493e59c640 1 -- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49401569c0 msgr2=0x7f4940156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493e59c640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49401569c0 0x7f4940156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493e59c640 1 -- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49400aba70 msgr2=0x7f49400abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493e59c640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49400aba70 0x7f49400abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.645+0000 7f493e59c640 1 -- 172.21.15.16:0/2827325631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49400accf0 con 0x7f492c004d00 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.646+0000 7f493e59c640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f49401ef380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f49300021d0 tx=0x7f493000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.646+0000 7f49377fe640 1 -- 172.21.15.16:0/2827325631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930004d60 con 0x7f492c004d00 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.646+0000 7f49377fe640 1 -- 172.21.15.16:0/2827325631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49300070d0 con 0x7f492c004d00 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.646+0000 7f49377fe640 1 -- 172.21.15.16:0/2827325631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930007480 con 0x7f492c004d00 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.646+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49400acfd0 con 0x7f492c004d00 2023-11-24T06:29:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.646+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49400ac430 con 0x7f492c004d00 2023-11-24T06:29:11.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.647+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4940067450 con 0x7f492c004d00 2023-11-24T06:29:11.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.649+0000 7f49377fe640 1 -- 172.21.15.16:0/2827325631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f49300075e0 con 0x7f492c004d00 2023-11-24T06:29:11.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.649+0000 7f49377fe640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492006af00 0x7f492006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:11.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.650+0000 7f49377fe640 1 -- 172.21.15.16:0/2827325631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4930089820 con 0x7f492c004d00 2023-11-24T06:29:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.650+0000 7f4935d9b640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492006af00 0x7f492006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.651+0000 7f4935d9b640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492006af00 0x7f492006d3c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4928000f60 tx=0x7f4928009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.654+0000 7f49377fe640 1 -- 172.21.15.16:0/2827325631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4930054890 con 0x7f492c004d00 2023-11-24T06:29:11.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.914+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4940070bb0 con 0x7f492c004d00 2023-11-24T06:29:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.919+0000 7f49377fe640 1 -- 172.21.15.16:0/2827325631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7f49300190e0 con 0x7f492c004d00 2023-11-24T06:29:11.923 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-24T06:29:11.923 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA== 2023-11-24T06:29:11.923 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2023-11-24T06:29:11.923 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2023-11-24T06:29:11.923 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:29:11.923 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-11-24T06:29:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492006af00 msgr2=0x7f492006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492006af00 0x7f492006d3c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4928000f60 tx=0x7f4928009040 comp rx=0 tx=0).stop 2023-11-24T06:29:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 msgr2=0x7f49401ef380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f49401ef380 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f49300021d0 tx=0x7f493000b560 comp rx=0 tx=0).stop 2023-11-24T06:29:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 shutdown_connections 2023-11-24T06:29:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f492006af00 0x7f492006d3c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49400aba70 0x7f49400abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49401569c0 0x7f4940156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 --2- 172.21.15.16:0/2827325631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492c004d00 0x7f49401ef380 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 >> 172.21.15.16:0/2827325631 conn(0x7f49400b0b70 msgr2=0x7f49400cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.922+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 shutdown_connections 2023-11-24T06:29:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:11.923+0000 7f493f59e640 1 -- 172.21.15.16:0/2827325631 wait complete. 2023-11-24T06:29:11.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.158+0000 7f7dc97fb640 1 Processor -- start 2023-11-24T06:29:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.159+0000 7f7dc97fb640 1 -- start start 2023-11-24T06:29:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.160+0000 7f7dc97fb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc4152110 0x7f7dc40d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.160+0000 7f7dc97fb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7dc406e750 0x7f7dc406eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.160+0000 7f7dc97fb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7dc415ee50 0x7f7dc415f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.160+0000 7f7dc97fb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc40cfa90 con 0x7f7dc406e750 2023-11-24T06:29:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.160+0000 7f7dc97fb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc406f0d0 con 0x7f7dc415ee50 2023-11-24T06:29:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.160+0000 7f7dc97fb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc415fbc0 con 0x7f7dc4152110 2023-11-24T06:29:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.161+0000 7f7dc27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7dc406e750 0x7f7dc406eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.161+0000 7f7dc27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7dc406e750 0x7f7dc406eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34172/0 (socket says 172.21.15.16:34172) 2023-11-24T06:29:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.161+0000 7f7dc27fc640 1 -- 172.21.15.16:0/2967453458 learned_addr learned my addr 172.21.15.16:0/2967453458 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.161+0000 7f7dc27fc640 1 -- 172.21.15.16:0/2967453458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc4152110 msgr2=0x7f7dc40d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.161+0000 7f7dc27fc640 1 --2- 172.21.15.16:0/2967453458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc4152110 0x7f7dc40d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.161+0000 7f7dc27fc640 1 -- 172.21.15.16:0/2967453458 >> v2:172.21.15.173:3300/0 conn(0x7f7dc415ee50 msgr2=0x7f7dc415f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.161+0000 7f7dc27fc640 1 --2- 172.21.15.16:0/2967453458 >> v2:172.21.15.173:3300/0 conn(0x7f7dc415ee50 0x7f7dc415f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.161+0000 7f7dc27fc640 1 -- 172.21.15.16:0/2967453458 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc40cfca0 con 0x7f7dc406e750 2023-11-24T06:29:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.162+0000 7f7dc27fc640 1 --2- 172.21.15.16:0/2967453458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc406e750 0x7f7dc406eb90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40b5bc0 tx=0x7f7dac02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d673922b2148177 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.163+0000 7f7dc1ffb640 1 -- 172.21.15.16:0/2967453458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac031860 con 0x7f7dc406e750 2023-11-24T06:29:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.163+0000 7f7dc1ffb640 1 -- 172.21.15.16:0/2967453458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac0319c0 con 0x7f7dc406e750 2023-11-24T06:29:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.163+0000 7f7dc1ffb640 1 -- 172.21.15.16:0/2967453458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac031d50 con 0x7f7dc406e750 2023-11-24T06:29:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.163+0000 7f7dc97fb640 1 -- 172.21.15.16:0/2967453458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc406e750 msgr2=0x7f7dc406eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.163+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/2967453458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc406e750 0x7f7dc406eb90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40b5bc0 tx=0x7f7dac02b900 comp rx=0 tx=0).stop 2023-11-24T06:29:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.164+0000 7f7dc97fb640 1 -- 172.21.15.16:0/2967453458 shutdown_connections 2023-11-24T06:29:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.164+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/2967453458 >> v2:172.21.15.173:3300/0 conn(0x7f7dc415ee50 0x7f7dc415f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.164+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/2967453458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc406e750 0x7f7dc406eb90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.164+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/2967453458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc4152110 0x7f7dc40d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.164+0000 7f7dc97fb640 1 -- 172.21.15.16:0/2967453458 >> 172.21.15.16:0/2967453458 conn(0x7f7dc40668a0 msgr2=0x7f7dc40b6570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.164+0000 7f7dc97fb640 1 -- 172.21.15.16:0/2967453458 shutdown_connections 2023-11-24T06:29:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.164+0000 7f7dc97fb640 1 -- 172.21.15.16:0/2967453458 wait complete. 2023-11-24T06:29:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.165+0000 7f7dc97fb640 1 Processor -- start 2023-11-24T06:29:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.165+0000 7f7dc97fb640 1 -- start start 2023-11-24T06:29:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.166+0000 7f7dc97fb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dc406e750 0x7f7dc4149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.166+0000 7f7dc97fb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc4152110 0x7f7dc40afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.166+0000 7f7dc97fb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc415ee50 0x7f7dc40b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.166+0000 7f7dc97fb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40b0f10 con 0x7f7dc415ee50 2023-11-24T06:29:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.166+0000 7f7dc97fb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc414b020 con 0x7f7dc406e750 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.166+0000 7f7dc97fb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc414b300 con 0x7f7dc4152110 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.167+0000 7f7dc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc415ee50 0x7f7dc40b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.167+0000 7f7dc37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc415ee50 0x7f7dc40b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34184/0 (socket says 172.21.15.16:34184) 2023-11-24T06:29:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.167+0000 7f7dc37fe640 1 -- 172.21.15.16:0/1400977509 learned_addr learned my addr 172.21.15.16:0/1400977509 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.167+0000 7f7dc37fe640 1 -- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc4152110 msgr2=0x7f7dc40afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.167+0000 7f7dc37fe640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc4152110 0x7f7dc40afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.167+0000 7f7dc37fe640 1 -- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dc406e750 msgr2=0x7f7dc4149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.167+0000 7f7dc37fe640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dc406e750 0x7f7dc4149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.167+0000 7f7dc37fe640 1 -- 172.21.15.16:0/1400977509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac0094d0 con 0x7f7dc415ee50 2023-11-24T06:29:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.168+0000 7f7dc37fe640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc415ee50 0x7f7dc40b1760 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009fd0 tx=0x7f7dbc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.168+0000 7f7dbb7fe640 1 -- 172.21.15.16:0/1400977509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011830 con 0x7f7dc415ee50 2023-11-24T06:29:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.168+0000 7f7dbb7fe640 1 -- 172.21.15.16:0/1400977509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc011990 con 0x7f7dc415ee50 2023-11-24T06:29:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.168+0000 7f7dbb7fe640 1 -- 172.21.15.16:0/1400977509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011d20 con 0x7f7dc415ee50 2023-11-24T06:29:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.168+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc414b5e0 con 0x7f7dc415ee50 2023-11-24T06:29:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.168+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc41576b0 con 0x7f7dc415ee50 2023-11-24T06:29:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.169+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc41578a0 con 0x7f7dc415ee50 2023-11-24T06:29:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.170+0000 7f7dbb7fe640 1 -- 172.21.15.16:0/1400977509 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f7dbc01b460 con 0x7f7dc415ee50 2023-11-24T06:29:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.171+0000 7f7dbb7fe640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7da406aed0 0x7f7da406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.171+0000 7f7dbb7fe640 1 -- 172.21.15.16:0/1400977509 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dbc0894d0 con 0x7f7dc415ee50 2023-11-24T06:29:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.172+0000 7f7dc2ffd640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7da406aed0 0x7f7da406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.173+0000 7f7dc2ffd640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7da406aed0 0x7f7da406d390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40b13c0 tx=0x7f7db4009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:12.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.176+0000 7f7dbb7fe640 1 -- 172.21.15.16:0/1400977509 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7dbc057fb0 con 0x7f7dc415ee50 2023-11-24T06:29:12.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.403+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dc4067640 con 0x7f7dc415ee50 2023-11-24T06:29:12.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.404+0000 7f7dbb7fe640 1 -- 172.21.15.16:0/1400977509 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f7dbc057b40 con 0x7f7dc415ee50 2023-11-24T06:29:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:12.409 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110540698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110540698},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110540698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1298790428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1298790428},{"type":"v1","addr":"172.21.15.16:6835","nonce":1298790428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2876598187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2876598187},{"type":"v1","addr":"172.21.15.173:6836","nonce":2876598187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:05.754412+0000","modified":"2023-11-24T06:29:08.837901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/2818758442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2818758442},{"type":"v1","addr":"172.21.15.16:6833","nonce":2818758442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:29:12.409 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2023-11-24T06:29:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7da406aed0 msgr2=0x7f7da406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7da406aed0 0x7f7da406d390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40b13c0 tx=0x7f7db4009250 comp rx=0 tx=0).stop 2023-11-24T06:29:12.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc415ee50 msgr2=0x7f7dc40b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:12.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc415ee50 0x7f7dc40b1760 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009fd0 tx=0x7f7dbc00b8e0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 shutdown_connections 2023-11-24T06:29:12.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7da406aed0 0x7f7da406d390 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc415ee50 0x7f7dc40b1760 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc4152110 0x7f7dc40afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 --2- 172.21.15.16:0/1400977509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dc406e750 0x7f7dc4149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.407+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 >> 172.21.15.16:0/1400977509 conn(0x7f7dc40668a0 msgr2=0x7f7dc414c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.408+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 shutdown_connections 2023-11-24T06:29:12.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:12.408+0000 7f7dc97fb640 1 -- 172.21.15.16:0/1400977509 wait complete. 2023-11-24T06:29:12.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:29:05.754412+0000', 'modified': '2023-11-24T06:29:08.837901+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5727': {'gid': 5727, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6833/2818758442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 2818758442}, {'type': 'v1', 'addr': '172.21.15.16:6833', 'nonce': 2818758442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:29:12.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:29:12.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:29:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.066+0000 7f4412327640 1 Processor -- start 2023-11-24T06:29:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.066+0000 7f4412327640 1 -- start start 2023-11-24T06:29:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.067+0000 7f4412327640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f440c144410 0x7f440c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.067+0000 7f4412327640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f440c0b2600 0x7f440c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.067+0000 7f4412327640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0acda0 con 0x7f440c144870 2023-11-24T06:29:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.067+0000 7f4412327640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c0af740 con 0x7f440c144410 2023-11-24T06:29:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.067+0000 7f4412327640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c0ad5b0 con 0x7f440c0b2600 2023-11-24T06:29:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.068+0000 7f4411325640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f440c144410 0x7f440c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.068+0000 7f4411325640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f440c144410 0x7f440c06a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49542/0 (socket says 172.21.15.16:49542) 2023-11-24T06:29:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.068+0000 7f4411325640 1 -- 172.21.15.16:0/3224386242 learned_addr learned my addr 172.21.15.16:0/3224386242 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.068+0000 7f4411325640 1 -- 172.21.15.16:0/3224386242 >> v2:172.21.15.173:3300/0 conn(0x7f440c0b2600 msgr2=0x7f440c06a920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:29:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.068+0000 7f4411325640 1 --2- 172.21.15.16:0/3224386242 >> v2:172.21.15.173:3300/0 conn(0x7f440c0b2600 0x7f440c06a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.068+0000 7f4411325640 1 -- 172.21.15.16:0/3224386242 >> v1:172.21.15.173:6789/0 conn(0x7f440c144870 legacy=0x7f440c071160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:29:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.068+0000 7f4411325640 1 -- 172.21.15.16:0/3224386242 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0b3240 con 0x7f440c144410 2023-11-24T06:29:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.069+0000 7f4411325640 1 --2- 172.21.15.16:0/3224386242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144410 0x7f440c06a3e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f440c067820 tx=0x7f440802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5af0acf1737d50 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4403fff640 1 -- 172.21.15.16:0/3224386242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408031890 con 0x7f440c144410 2023-11-24T06:29:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4403fff640 1 -- 172.21.15.16:0/3224386242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44080319f0 con 0x7f440c144410 2023-11-24T06:29:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4403fff640 1 -- 172.21.15.16:0/3224386242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408031d80 con 0x7f440c144410 2023-11-24T06:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4412327640 1 -- 172.21.15.16:0/3224386242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144410 msgr2=0x7f440c06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4412327640 1 --2- 172.21.15.16:0/3224386242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144410 0x7f440c06a3e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f440c067820 tx=0x7f440802b930 comp rx=0 tx=0).stop 2023-11-24T06:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4412327640 1 -- 172.21.15.16:0/3224386242 shutdown_connections 2023-11-24T06:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4412327640 1 --2- 172.21.15.16:0/3224386242 >> v2:172.21.15.173:3300/0 conn(0x7f440c0b2600 0x7f440c06a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4412327640 1 --2- 172.21.15.16:0/3224386242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144410 0x7f440c06a3e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4412327640 1 -- 172.21.15.16:0/3224386242 >> 172.21.15.16:0/3224386242 conn(0x7f440c0b19c0 msgr2=0x7f440c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.070+0000 7f4412327640 1 -- 172.21.15.16:0/3224386242 shutdown_connections 2023-11-24T06:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.071+0000 7f4412327640 1 -- 172.21.15.16:0/3224386242 wait complete. 2023-11-24T06:29:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.072+0000 7f4412327640 1 Processor -- start 2023-11-24T06:29:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.072+0000 7f4412327640 1 -- start start 2023-11-24T06:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.073+0000 7f4412327640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f440c0b2600 0x7f440c15c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.073+0000 7f4412327640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144870 0x7f440c13d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.073+0000 7f4412327640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f440c15dcb0 0x7f440c139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.073+0000 7f4412327640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c15d820 con 0x7f440c144870 2023-11-24T06:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.073+0000 7f4412327640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c13d9b0 con 0x7f440c0b2600 2023-11-24T06:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.073+0000 7f4412327640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c15cc10 con 0x7f440c15dcb0 2023-11-24T06:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.074+0000 7f4410b24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144870 0x7f440c13d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.074+0000 7f4410b24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144870 0x7f440c13d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49548/0 (socket says 172.21.15.16:49548) 2023-11-24T06:29:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.074+0000 7f4410b24640 1 -- 172.21.15.16:0/4152293963 learned_addr learned my addr 172.21.15.16:0/4152293963 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.074+0000 7f4411325640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f440c0b2600 0x7f440c15c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.074+0000 7f4410b24640 1 -- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f440c15dcb0 msgr2=0x7f440c139610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.074+0000 7f4410b24640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f440c15dcb0 0x7f440c139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.074+0000 7f4410b24640 1 -- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f440c0b2600 msgr2=0x7f440c15c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.074+0000 7f4410b24640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f440c0b2600 0x7f440c15c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.074+0000 7f4410b24640 1 -- 172.21.15.16:0/4152293963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44080094d0 con 0x7f440c144870 2023-11-24T06:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.075+0000 7f4410b24640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144870 0x7f440c13d470 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00b450 tx=0x7f43fc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.075+0000 7f4401ffb640 1 -- 172.21.15.16:0/4152293963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc011890 con 0x7f440c144870 2023-11-24T06:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.075+0000 7f4401ffb640 1 -- 172.21.15.16:0/4152293963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc011a10 con 0x7f440c144870 2023-11-24T06:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.075+0000 7f4401ffb640 1 -- 172.21.15.16:0/4152293963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc011dc0 con 0x7f440c144870 2023-11-24T06:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.075+0000 7f4412327640 1 -- 172.21.15.16:0/4152293963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c15cef0 con 0x7f440c144870 2023-11-24T06:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.075+0000 7f4412327640 1 -- 172.21.15.16:0/4152293963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c15d4b0 con 0x7f440c144870 2023-11-24T06:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.077+0000 7f4401ffb640 1 -- 172.21.15.16:0/4152293963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f43fc01a400 con 0x7f440c144870 2023-11-24T06:29:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.078+0000 7f4401ffb640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f43f006af20 0x7f43f006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.078+0000 7f4401ffb640 1 -- 172.21.15.16:0/4152293963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43fc089b50 con 0x7f440c144870 2023-11-24T06:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.079+0000 7f4411325640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f43f006af20 0x7f43f006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.079+0000 7f44161fc740 1 Processor -- start 2023-11-24T06:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.079+0000 7f44161fc740 1 -- start start 2023-11-24T06:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f44161fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e50070 0x55f5d3e50450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f44161fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e4f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f44161fc740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4f7b0 0x55f5d3e93730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f44161fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3dae880 con 0x55f5d3e50070 2023-11-24T06:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f44161fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3dbdae0 con 0x55f5d3e4ad70 2023-11-24T06:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f44161fc740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3dbb1d0 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f4411325640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f43f006af20 0x7f43f006d3e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4408036000 tx=0x7f440803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f4411325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e50070 0x55f5d3e50450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f4411325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e50070 0x55f5d3e50450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49560/0 (socket says 172.21.15.16:49560) 2023-11-24T06:29:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.080+0000 7f4411325640 1 -- 172.21.15.16:0/3185223628 learned_addr learned my addr 172.21.15.16:0/3185223628 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.081+0000 7f4411325640 1 -- 172.21.15.16:0/3185223628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4f7b0 msgr2=0x55f5d3e93730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.081+0000 7f4411325640 1 --2- 172.21.15.16:0/3185223628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4f7b0 0x55f5d3e93730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.081+0000 7f4411325640 1 -- 172.21.15.16:0/3185223628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e4ad70 msgr2=0x55f5d3e4f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.081+0000 7f4411325640 1 --2- 172.21.15.16:0/3185223628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e4f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.081+0000 7f4411325640 1 -- 172.21.15.16:0/3185223628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d3d7e860 con 0x55f5d3e50070 2023-11-24T06:29:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.081+0000 7f4411325640 1 --2- 172.21.15.16:0/3185223628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e50070 0x55f5d3e50450 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f440803a420 tx=0x7f4408030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1bd7c1ff7761cf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.081+0000 7f43eeffd640 1 -- 172.21.15.16:0/3185223628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440803ab50 con 0x55f5d3e50070 2023-11-24T06:29:13.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.081+0000 7f43eeffd640 1 -- 172.21.15.16:0/3185223628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440804a070 con 0x55f5d3e50070 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.081+0000 7f43eeffd640 1 -- 172.21.15.16:0/3185223628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440804f430 con 0x55f5d3e50070 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.082+0000 7f44161fc740 1 -- 172.21.15.16:0/3185223628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e50070 msgr2=0x55f5d3e50450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.082+0000 7f44161fc740 1 --2- 172.21.15.16:0/3185223628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e50070 0x55f5d3e50450 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f440803a420 tx=0x7f4408030000 comp rx=0 tx=0).stop 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.082+0000 7f44161fc740 1 -- 172.21.15.16:0/3185223628 shutdown_connections 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.082+0000 7f44161fc740 1 --2- 172.21.15.16:0/3185223628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4f7b0 0x55f5d3e93730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.082+0000 7f44161fc740 1 --2- 172.21.15.16:0/3185223628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e4f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.082+0000 7f44161fc740 1 --2- 172.21.15.16:0/3185223628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e50070 0x55f5d3e50450 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.082+0000 7f44161fc740 1 -- 172.21.15.16:0/3185223628 >> 172.21.15.16:0/3185223628 conn(0x55f5d3e11c70 msgr2=0x55f5d3e95b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.082+0000 7f44161fc740 1 -- 172.21.15.16:0/3185223628 shutdown_connections 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.082+0000 7f44161fc740 1 -- 172.21.15.16:0/3185223628 wait complete. 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.083+0000 7f44161fc740 1 -- start start 2023-11-24T06:29:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.083+0000 7f44161fc740 1 Processor -- start 2023-11-24T06:29:13.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.084+0000 7f44161fc740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4ad70 0x55f5d3e8d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.084+0000 7f44161fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e74780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.084+0000 7f44161fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e50070 0x55f5d3e98510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.084+0000 7f44161fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3db6ac0 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.084+0000 7f44161fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3d532f0 con 0x55f5d3e50070 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.084+0000 7f44161fc740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3d70c70 con 0x55f5d3e4ad70 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f44161fc740 10 client.0 _ll_register_callbacks cb 0x55f5d3d753b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f4410b24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e74780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f4410b24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e74780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49564/0 (socket says 172.21.15.16:49564) 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f4410b24640 1 -- 172.21.15.16:0/3550499330 learned_addr learned my addr 172.21.15.16:0/3550499330 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f4410b24640 1 -- 172.21.15.16:0/3550499330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4ad70 msgr2=0x55f5d3e8d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f4410b24640 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4ad70 0x55f5d3e8d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f4410b24640 1 -- 172.21.15.16:0/3550499330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e50070 msgr2=0x55f5d3e98510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f4410b24640 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e50070 0x55f5d3e98510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f4410b24640 1 -- 172.21.15.16:0/3550499330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c0b3240 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f4410b24640 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e74780 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f43fc060620 tx=0x7f43fc058900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3425e8b6790ba2c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.085+0000 7f43e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f43e7fff640 1 -- 172.21.15.16:0/3550499330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0608c0 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f43e7fff640 1 -- 172.21.15.16:0/3550499330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43fc059af0 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f44161fc740 1 -- 172.21.15.16:0/3550499330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5d3dafaa0 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f43e7fff640 1 -- 172.21.15.16:0/3550499330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43fc059cb0 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f44161fc740 10 client.5859 fetch_fsmap learned FSMap version 25 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f44161fc740 1 -- 172.21.15.16:0/3550499330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f5d3db5140 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f43e7fff640 1 -- 172.21.15.16:0/3550499330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc075080 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f43e7fff640 1 -- 172.21.15.16:0/3550499330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f43fc0545b0 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f44161fc740 10 client.5859 fetch_fsmap finished waiting for FSMap version 25 2023-11-24T06:29:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f44161fc740 10 client.5859 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:29:13.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f44161fc740 20 client.5859 populate_metadata read hostname 'smithi016' 2023-11-24T06:29:13.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f44161fc740 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] conn(0x55f5d3e9ae30 0x55f5d3e9c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f44161fc740 4 client.5859 mds_command: new command op to 5727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:29:13.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.086+0000 7f44161fc740 1 -- 172.21.15.16:0/3550499330 --> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5d3daf7c0 con 0x55f5d3e9ae30 2023-11-24T06:29:13.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.087+0000 7f4411325640 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] conn(0x55f5d3e9ae30 0x55f5d3e9c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.087+0000 7f4411325640 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] conn(0x55f5d3e9ae30 0x55f5d3e9c6b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d928c8dc2496547b server_cookie=ac2e622981a65a4d in_seq=0 out_seq=0 2023-11-24T06:29:13.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.087+0000 7f43e7fff640 10 client.5859 ms_handle_connect on v2:172.21.15.16:6832/2818758442 2023-11-24T06:29:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f43e7fff640 1 -- 172.21.15.16:0/3550499330 <== mds.0 v2:172.21.15.16:6832/2818758442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5d3daf7c0 con 0x55f5d3e9ae30 2023-11-24T06:29:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f43e7fff640 10 client.5859 handle_command_reply: tid=0 2023-11-24T06:29:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f44161fc740 1 client.5859 shutdown 2023-11-24T06:29:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f44161fc740 1 -- 172.21.15.16:0/3550499330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4f7b0 msgr2=0x55f5d3e74780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f44161fc740 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e74780 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f43fc060620 tx=0x7f43fc058900 comp rx=0 tx=0).stop 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f44161fc740 1 -- 172.21.15.16:0/3550499330 shutdown_connections 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f44161fc740 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] conn(0x55f5d3e9ae30 0x55f5d3e9c6b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f44161fc740 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e50070 0x55f5d3e98510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f44161fc740 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e74780 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f44161fc740 1 --2- 172.21.15.16:0/3550499330 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4ad70 0x55f5d3e8d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f43e7fff640 0 client.5859 ms_handle_reset on v2:172.21.15.16:6832/2818758442 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.088+0000 7f44161fc740 1 -- 172.21.15.16:0/3550499330 >> 172.21.15.16:0/3550499330 conn(0x55f5d3e11c70 msgr2=0x55f5d3e95b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.089+0000 7f44161fc740 1 -- 172.21.15.16:0/3550499330 shutdown_connections 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.089+0000 7f44161fc740 1 -- 172.21.15.16:0/3550499330 wait complete. 2023-11-24T06:29:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.089+0000 7f44161fc740 20 client.5859 trim_cache size 0 max 16384 2023-11-24T06:29:13.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.106+0000 7f44161fc740 1 Processor -- start 2023-11-24T06:29:13.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.106+0000 7f44161fc740 1 -- start start 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.107+0000 7f44161fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e7e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.107+0000 7f44161fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.107+0000 7f44161fc740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e50070 0x55f5d3e95b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.107+0000 7f44161fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3b19f20 con 0x55f5d3e4ad70 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.107+0000 7f44161fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3e91d50 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.107+0000 7f44161fc740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3d6ad90 con 0x55f5d3e50070 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4411325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e7e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4411325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e7e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49576/0 (socket says 172.21.15.16:49576) 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4411325640 1 -- 172.21.15.16:0/2379872418 learned_addr learned my addr 172.21.15.16:0/2379872418 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4410b24640 1 --2- 172.21.15.16:0/2379872418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4411b26640 1 --2- 172.21.15.16:0/2379872418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e50070 0x55f5d3e95b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4411325640 1 -- 172.21.15.16:0/2379872418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e50070 msgr2=0x55f5d3e95b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4411325640 1 --2- 172.21.15.16:0/2379872418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e50070 0x55f5d3e95b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4411325640 1 -- 172.21.15.16:0/2379872418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e4f7b0 msgr2=0x55f5d3e981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4411325640 1 --2- 172.21.15.16:0/2379872418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.108+0000 7f4411325640 1 -- 172.21.15.16:0/2379872418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d3d52b80 con 0x55f5d3e4ad70 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f4411325640 1 --2- 172.21.15.16:0/2379872418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e7e940 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f440802bec0 tx=0x7f4408009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baab81d7b9207b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f43ef7fe640 1 -- 172.21.15.16:0/2379872418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440803bd80 con 0x55f5d3e4ad70 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f43ef7fe640 1 -- 172.21.15.16:0/2379872418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440803ab50 con 0x55f5d3e4ad70 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f43ef7fe640 1 -- 172.21.15.16:0/2379872418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440803bee0 con 0x55f5d3e4ad70 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f44161fc740 1 -- 172.21.15.16:0/2379872418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 msgr2=0x55f5d3e7e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f44161fc740 1 --2- 172.21.15.16:0/2379872418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e7e940 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f440802bec0 tx=0x7f4408009240 comp rx=0 tx=0).stop 2023-11-24T06:29:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f44161fc740 1 -- 172.21.15.16:0/2379872418 shutdown_connections 2023-11-24T06:29:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f44161fc740 1 --2- 172.21.15.16:0/2379872418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e50070 0x55f5d3e95b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f44161fc740 1 --2- 172.21.15.16:0/2379872418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e4f7b0 0x55f5d3e981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f44161fc740 1 --2- 172.21.15.16:0/2379872418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e7e940 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.109+0000 7f44161fc740 1 -- 172.21.15.16:0/2379872418 >> 172.21.15.16:0/2379872418 conn(0x55f5d3e11c70 msgr2=0x55f5d3e4ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.110+0000 7f44161fc740 1 -- 172.21.15.16:0/2379872418 shutdown_connections 2023-11-24T06:29:13.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.110+0000 7f44161fc740 1 -- 172.21.15.16:0/2379872418 wait complete. 2023-11-24T06:29:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.110+0000 7f44161fc740 1 -- start start 2023-11-24T06:29:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.110+0000 7f44161fc740 1 Processor -- start 2023-11-24T06:29:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.111+0000 7f44161fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.111+0000 7f44161fc740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4f7b0 0x55f5d3e8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.111+0000 7f44161fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e50070 0x55f5d3e90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.111+0000 7f44161fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3eb8d30 con 0x55f5d3e4ad70 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.111+0000 7f44161fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3e728a0 con 0x55f5d3e50070 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.111+0000 7f44161fc740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d3e981e0 con 0x55f5d3e4f7b0 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.111+0000 7f44161fc740 10 client.0 _ll_register_callbacks cb 0x7f43fc054b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f4411325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e8cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f4411325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e8cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49588/0 (socket says 172.21.15.16:49588) 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f4411325640 1 -- 172.21.15.16:0/1829404735 learned_addr learned my addr 172.21.15.16:0/1829404735 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f4411325640 1 -- 172.21.15.16:0/1829404735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4f7b0 msgr2=0x55f5d3e8d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f4411325640 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4f7b0 0x55f5d3e8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f4411325640 1 -- 172.21.15.16:0/1829404735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e50070 msgr2=0x55f5d3e90cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f4411325640 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e50070 0x55f5d3e90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f4411325640 1 -- 172.21.15.16:0/1829404735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4408053040 con 0x55f5d3e4ad70 2023-11-24T06:29:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f4411325640 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e8cbe0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4408048040 tx=0x7f4408004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67e3547b5c2bbbb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.112+0000 7f43e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:29:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f43e7fff640 1 -- 172.21.15.16:0/1829404735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440803acc0 con 0x55f5d3e4ad70 2023-11-24T06:29:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f44161fc740 1 -- 172.21.15.16:0/1829404735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f440804a070 con 0x55f5d3e4ad70 2023-11-24T06:29:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f43e7fff640 1 -- 172.21.15.16:0/1829404735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f440803ae20 con 0x55f5d3e4ad70 2023-11-24T06:29:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f43e7fff640 1 -- 172.21.15.16:0/1829404735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4408004030 con 0x55f5d3e4ad70 2023-11-24T06:29:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f44161fc740 10 client.5865 fetch_fsmap learned FSMap version 25 2023-11-24T06:29:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f44161fc740 1 -- 172.21.15.16:0/1829404735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f5d3d78170 con 0x55f5d3e4ad70 2023-11-24T06:29:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f43e7fff640 1 -- 172.21.15.16:0/1829404735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440804a070 con 0x55f5d3e4ad70 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f43e7fff640 1 -- 172.21.15.16:0/1829404735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f440805e020 con 0x55f5d3e4ad70 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f44161fc740 10 client.5865 fetch_fsmap finished waiting for FSMap version 25 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f44161fc740 10 client.5865 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f44161fc740 20 client.5865 populate_metadata read hostname 'smithi016' 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.113+0000 7f44161fc740 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] conn(0x55f5d3e911f0 0x55f5d3e915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.114+0000 7f44161fc740 4 client.5865 mds_command: new command op to 5727 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.114+0000 7f44161fc740 1 -- 172.21.15.16:0/1829404735 --> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f5d3ab76c0 con 0x55f5d3e911f0 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.114+0000 7f4410b24640 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] conn(0x55f5d3e911f0 0x55f5d3e915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.114+0000 7f4410b24640 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] conn(0x55f5d3e911f0 0x55f5d3e915d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f6d26da9092e08 server_cookie=9de827027113fbbd in_seq=0 out_seq=0 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.114+0000 7f43e7fff640 10 client.5865 ms_handle_connect on v2:172.21.15.16:6832/2818758442 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f43e7fff640 1 -- 172.21.15.16:0/1829404735 <== mds.0 v2:172.21.15.16:6832/2818758442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f5d3ab76c0 con 0x55f5d3e911f0 2023-11-24T06:29:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f43e7fff640 10 client.5865 handle_command_reply: tid=0 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 client.5865 shutdown 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 -- 172.21.15.16:0/1829404735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 msgr2=0x55f5d3e8cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e8cbe0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4408048040 tx=0x7f4408004730 comp rx=0 tx=0).stop 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 -- 172.21.15.16:0/1829404735 shutdown_connections 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.16:6832/2818758442,v1:172.21.15.16:6833/2818758442] conn(0x55f5d3e911f0 0x55f5d3e915d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f5d3e50070 0x55f5d3e90cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55f5d3e4f7b0 0x55f5d3e8d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 --2- 172.21.15.16:0/1829404735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5d3e4ad70 0x55f5d3e8cbe0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f43e7fff640 0 client.5865 ms_handle_reset on v2:172.21.15.16:6832/2818758442 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 -- 172.21.15.16:0/1829404735 >> 172.21.15.16:0/1829404735 conn(0x55f5d3e11c70 msgr2=0x55f5d3e49dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 -- 172.21.15.16:0/1829404735 shutdown_connections 2023-11-24T06:29:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 1 -- 172.21.15.16:0/1829404735 wait complete. 2023-11-24T06:29:13.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.115+0000 7f44161fc740 20 client.5865 trim_cache size 0 max 16384 2023-11-24T06:29:13.123 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:29:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6dc1668b-58b0-47dd-b8dc-4164e5a228ab", 2023-11-24T06:29:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-24T06:29:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5727, 2023-11-24T06:29:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-24T06:29:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-24T06:29:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-24T06:29:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.7990501019999998, 2023-11-24T06:29:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2023-11-24T06:29:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 46, 2023-11-24T06:29:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 46, 2023-11-24T06:29:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 13.130607469999999 2023-11-24T06:29:13.126 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 -- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f43f006af20 msgr2=0x7f43f006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f43f006af20 0x7f43f006d3e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4408036000 tx=0x7f440803c040 comp rx=0 tx=0).stop 2023-11-24T06:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 -- 172.21.15.16:0/4152293963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144870 msgr2=0x7f440c13d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144870 0x7f440c13d470 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00b450 tx=0x7f43fc00b910 comp rx=0 tx=0).stop 2023-11-24T06:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 -- 172.21.15.16:0/4152293963 shutdown_connections 2023-11-24T06:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f43f006af20 0x7f43f006d3e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f440c15dcb0 0x7f440c139610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f440c144870 0x7f440c13d470 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 --2- 172.21.15.16:0/4152293963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f440c0b2600 0x7f440c15c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 -- 172.21.15.16:0/4152293963 >> 172.21.15.16:0/4152293963 conn(0x7f440c0b19c0 msgr2=0x7f440c0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 -- 172.21.15.16:0/4152293963 shutdown_connections 2023-11-24T06:29:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.116+0000 7f4412327640 1 -- 172.21.15.16:0/4152293963 wait complete. 2023-11-24T06:29:13.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:29:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.349+0000 7fde2e921640 1 Processor -- start 2023-11-24T06:29:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.349+0000 7fde2e921640 1 -- start start 2023-11-24T06:29:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.351+0000 7fde2e921640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde2813cfa0 0x7fde2815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.351+0000 7fde2e921640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde280cdec0 con 0x7fde280cd530 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.351+0000 7fde2e921640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde280ce1a0 con 0x7fde2813fcb0 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.351+0000 7fde2e921640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde280ce480 con 0x7fde2813cfa0 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.351+0000 7fde2d91f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde2813cfa0 0x7fde2815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.351+0000 7fde2d91f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde2813cfa0 0x7fde2815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49598/0 (socket says 172.21.15.16:49598) 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.351+0000 7fde2d91f640 1 -- 172.21.15.16:0/476684390 learned_addr learned my addr 172.21.15.16:0/476684390 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.352+0000 7fde2d91f640 1 -- 172.21.15.16:0/476684390 >> v1:172.21.15.173:6789/0 conn(0x7fde2813fcb0 legacy=0x7fde280b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.352+0000 7fde2d91f640 1 -- 172.21.15.16:0/476684390 >> v1:172.21.15.16:6789/0 conn(0x7fde280cd530 legacy=0x7fde2813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.352+0000 7fde2d91f640 1 -- 172.21.15.16:0/476684390 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280ce760 con 0x7fde2813cfa0 2023-11-24T06:29:13.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.352+0000 7fde2c91d640 1 -- 172.21.15.16:0/476684390 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043906661 0 0) 0x7fde280cdec0 con 0x7fde280cd530 2023-11-24T06:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.353+0000 7fde2d91f640 1 --2- 172.21.15.16:0/476684390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 0x7fde2815afb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3c105dbbbe49c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.353+0000 7fde2c91d640 1 -- 172.21.15.16:0/476684390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24031870 con 0x7fde2813cfa0 2023-11-24T06:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.353+0000 7fde2c91d640 1 -- 172.21.15.16:0/476684390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde240319d0 con 0x7fde2813cfa0 2023-11-24T06:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.353+0000 7fde2c91d640 1 -- 172.21.15.16:0/476684390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24031d60 con 0x7fde2813cfa0 2023-11-24T06:29:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.354+0000 7fde2e921640 1 -- 172.21.15.16:0/476684390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 msgr2=0x7fde2815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.354+0000 7fde2e921640 1 --2- 172.21.15.16:0/476684390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 0x7fde2815afb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2402b910 comp rx=0 tx=0).stop 2023-11-24T06:29:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.354+0000 7fde2e921640 1 -- 172.21.15.16:0/476684390 shutdown_connections 2023-11-24T06:29:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.354+0000 7fde2e921640 1 --2- 172.21.15.16:0/476684390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 0x7fde2815afb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.354+0000 7fde2e921640 1 -- 172.21.15.16:0/476684390 >> 172.21.15.16:0/476684390 conn(0x7fde2806a800 msgr2=0x7fde2806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.354+0000 7fde2e921640 1 -- 172.21.15.16:0/476684390 shutdown_connections 2023-11-24T06:29:13.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.354+0000 7fde2e921640 1 -- 172.21.15.16:0/476684390 wait complete. 2023-11-24T06:29:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.356+0000 7fde2e921640 1 Processor -- start 2023-11-24T06:29:13.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.357+0000 7fde2e921640 1 -- start start 2023-11-24T06:29:13.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.358+0000 7fde2e921640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde280cd530 0x7fde2815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.358+0000 7fde2e921640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 0x7fde2815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.358+0000 7fde2e921640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde2813fcb0 0x7fde2813b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.358+0000 7fde2e921640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2815d5d0 con 0x7fde2813cfa0 2023-11-24T06:29:13.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.358+0000 7fde2e921640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2813bab0 con 0x7fde280cd530 2023-11-24T06:29:13.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.358+0000 7fde2e921640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2815bbd0 con 0x7fde2813fcb0 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.358+0000 7fde2d11e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 0x7fde2815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.358+0000 7fde2d11e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 0x7fde2815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49600/0 (socket says 172.21.15.16:49600) 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.358+0000 7fde2d11e640 1 -- 172.21.15.16:0/1770550692 learned_addr learned my addr 172.21.15.16:0/1770550692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2d91f640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde280cd530 0x7fde2815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2d11e640 1 -- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde2813fcb0 msgr2=0x7fde2813b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2e120640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde2813fcb0 0x7fde2813b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2d11e640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde2813fcb0 0x7fde2813b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2d11e640 1 -- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde280cd530 msgr2=0x7fde2815b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2d11e640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde280cd530 0x7fde2815b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2d11e640 1 -- 172.21.15.16:0/1770550692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde240094d0 con 0x7fde2813cfa0 2023-11-24T06:29:13.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2d91f640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde280cd530 0x7fde2815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:13.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2e120640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde2813fcb0 0x7fde2813b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:13.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.359+0000 7fde2d11e640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 0x7fde2815d090 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c4b0 tx=0x7fde1800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.362+0000 7fde1e7fc640 1 -- 172.21.15.16:0/1770550692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde18005480 con 0x7fde2813cfa0 2023-11-24T06:29:13.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.362+0000 7fde1e7fc640 1 -- 172.21.15.16:0/1770550692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde180035f0 con 0x7fde2813cfa0 2023-11-24T06:29:13.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.362+0000 7fde1e7fc640 1 -- 172.21.15.16:0/1770550692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde180039e0 con 0x7fde2813cfa0 2023-11-24T06:29:13.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.362+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2815beb0 con 0x7fde2813cfa0 2023-11-24T06:29:13.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.362+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2815c470 con 0x7fde2813cfa0 2023-11-24T06:29:13.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.363+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde28070da0 con 0x7fde2813cfa0 2023-11-24T06:29:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.364+0000 7fde1e7fc640 1 -- 172.21.15.16:0/1770550692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fde18003b40 con 0x7fde2813cfa0 2023-11-24T06:29:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.365+0000 7fde1e7fc640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde0c06aed0 0x7fde0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.365+0000 7fde1e7fc640 1 -- 172.21.15.16:0/1770550692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde18088d00 con 0x7fde2813cfa0 2023-11-24T06:29:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.366+0000 7fde2d91f640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde0c06aed0 0x7fde0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.367+0000 7fde2d91f640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde0c06aed0 0x7fde0c06d390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fde24036000 tx=0x7fde2403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.369+0000 7fde1e7fc640 1 -- 172.21.15.16:0/1770550692 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde18053cc0 con 0x7fde2813cfa0 2023-11-24T06:29:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.602+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fde28067450 con 0x7fde2813cfa0 2023-11-24T06:29:13.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.602+0000 7fde1e7fc640 1 -- 172.21.15.16:0/1770550692 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fde18017070 con 0x7fde2813cfa0 2023-11-24T06:29:13.606 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:29:13.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde0c06aed0 msgr2=0x7fde0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde0c06aed0 0x7fde0c06d390 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fde24036000 tx=0x7fde2403c040 comp rx=0 tx=0).stop 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 msgr2=0x7fde2815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:13.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 0x7fde2815d090 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fde1800c4b0 tx=0x7fde1800c970 comp rx=0 tx=0).stop 2023-11-24T06:29:13.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 shutdown_connections 2023-11-24T06:29:13.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fde0c06aed0 0x7fde0c06d390 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde2813fcb0 0x7fde2813b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde2813cfa0 0x7fde2815d090 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 --2- 172.21.15.16:0/1770550692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde280cd530 0x7fde2815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 >> 172.21.15.16:0/1770550692 conn(0x7fde2806a800 msgr2=0x7fde28071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 shutdown_connections 2023-11-24T06:29:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:13.605+0000 7fde2e921640 1 -- 172.21.15.16:0/1770550692 wait complete. 2023-11-24T06:29:13.628 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:29:13.628 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:29:13.628 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-24T06:29:13.628 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:29:13.628 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:13.629 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:29:13.629 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:29:13.629 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:29:13.629 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:29:13.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:13.630 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-24T06:29:13.685 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:13.686 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42190sec preferred_lft 42190sec 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:13.687 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8cfd:d2ff:fee1:5b18/64 scope link 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8c89:e0ff:fefc:a3fb/64 scope link 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether ea:e3:8b:37:17:ee brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e8e3:8bff:fe37:17ee/64 scope link 2023-11-24T06:29:13.688 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:13.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:13.689 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:29:13.789 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-24T06:29:13.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:13.794 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-24T06:29:13.856 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-24T06:29:13.856 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:29:13.856 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:29:13.856 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:29:13.856 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:29:13.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:13.857 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:29:13.857 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:29:13.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:29:13.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:29:13.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:29:13.857 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:29:14.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:14.053 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-24T06:29:14.107 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:29:14.108 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:29:14.108 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:29:14.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:14.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-24T06:29:14.204 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:29:14.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:29:14.205 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:14.263 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:29:14.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:29:14.263 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:14.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:29:14.424 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:29:14.485 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:29:14.486 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:14.487 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:14.488 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:14.488 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:14.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:14.489 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:29:14.550 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:29:14.551 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:29:14.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:29:14.645 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:29:14.693 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:29:14.694 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:14.695 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:14.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:14.696 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:29:14.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-24T06:29:14.755+0000 7fcccd3ff480 -1 init, newargv = 0x55ffbdcfe960 newargc=15 2023-11-24T06:29:14.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[104605]: starting ceph client 2023-11-24T06:29:14.768 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[104605]: starting fuse 2023-11-24T06:29:15.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:29:15.957 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:29:16.016 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:29:16.017 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:29:16.018 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:29:16.019 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:16.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:16.021 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:29:16.079 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-24T06:29:16.079 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:29:16.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:29:16.079 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-24T06:29:16.079 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-24T06:29:16.079 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-24T06:29:16.080 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-24T06:29:16.080 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:29:16.080 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-24T06:29:16.080 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:29:16.080 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-24T06:29:16.080 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:29:16.080 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:29:16.080 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-24T06:29:16.081 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:29:16.082 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:29:16.082 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-24T06:29:16.082 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-24T06:29:16.271 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.104605.asok 2023-11-24T06:29:16.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:29:16.289 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.104605.asok 2023-11-24T06:29:16.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:16.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.104605.asok status 2023-11-24T06:29:16.540 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:29:16.540 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "104605", 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5880, 2023-11-24T06:29:16.541 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5880 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 958027091 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:16.542 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 958027091 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5880 192.168.0.1:0/958027091", 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/958027091", 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 25, 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 46, 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-24T06:29:16.543 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-24T06:29:16.544 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:29:16.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:16.557 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-24T06:29:16.557 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:16.613 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-24T06:29:16.613 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:16.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:16.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:16.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:29:16.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.932+0000 7f477b59e640 1 Processor -- start 2023-11-24T06:29:16.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.933+0000 7f477b59e640 1 -- start start 2023-11-24T06:29:16.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.934+0000 7f477b59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f477c15d440 0x7f477c0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.934+0000 7f477b59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f477c06c220 0x7f477c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.934+0000 7f477b59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c0cfd00 con 0x7f477c06b700 2023-11-24T06:29:16.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.934+0000 7f477b59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c0cfaf0 con 0x7f477c06c220 2023-11-24T06:29:16.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.934+0000 7f477b59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c0cffe0 con 0x7f477c15d440 2023-11-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.935+0000 7f477ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f477c06c220 0x7f477c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.935+0000 7f477ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f477c06c220 0x7f477c0cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49616/0 (socket says 172.21.15.16:49616) 2023-11-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.935+0000 7f477ad9d640 1 -- 172.21.15.16:0/1185291904 learned_addr learned my addr 172.21.15.16:0/1185291904 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.935+0000 7f477a59c640 1 --2- 172.21.15.16:0/1185291904 >> v2:172.21.15.173:3300/0 conn(0x7f477c15d440 0x7f477c0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.935+0000 7f477ad9d640 1 -- 172.21.15.16:0/1185291904 >> v2:172.21.15.173:3300/0 conn(0x7f477c15d440 msgr2=0x7f477c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.935+0000 7f477ad9d640 1 --2- 172.21.15.16:0/1185291904 >> v2:172.21.15.173:3300/0 conn(0x7f477c15d440 0x7f477c0d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.935+0000 7f477ad9d640 1 -- 172.21.15.16:0/1185291904 >> v1:172.21.15.173:6789/0 conn(0x7f477c06b700 legacy=0x7f477c06bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.935+0000 7f477ad9d640 1 -- 172.21.15.16:0/1185291904 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0d02c0 con 0x7f477c06c220 2023-11-24T06:29:16.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.935+0000 7f477a59c640 1 --2- 172.21.15.16:0/1185291904 >> v2:172.21.15.173:3300/0 conn(0x7f477c15d440 0x7f477c0d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:16.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.936+0000 7f477ad9d640 1 --2- 172.21.15.16:0/1185291904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06c220 0x7f477c0cb950 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f477c0ce8e0 tx=0x7f477402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cc33e78b98387e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.937+0000 7f4779d9b640 1 -- 172.21.15.16:0/1185291904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477402ca80 con 0x7f477c06c220 2023-11-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.937+0000 7f4779d9b640 1 -- 172.21.15.16:0/1185291904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477402cbe0 con 0x7f477c06c220 2023-11-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.937+0000 7f4779d9b640 1 -- 172.21.15.16:0/1185291904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740343f0 con 0x7f477c06c220 2023-11-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.937+0000 7f477b59e640 1 -- 172.21.15.16:0/1185291904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06c220 msgr2=0x7f477c0cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.937+0000 7f477b59e640 1 --2- 172.21.15.16:0/1185291904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06c220 0x7f477c0cb950 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f477c0ce8e0 tx=0x7f477402bee0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.937+0000 7f477b59e640 1 -- 172.21.15.16:0/1185291904 shutdown_connections 2023-11-24T06:29:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.937+0000 7f477b59e640 1 --2- 172.21.15.16:0/1185291904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06c220 0x7f477c0cb950 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.937+0000 7f477b59e640 1 --2- 172.21.15.16:0/1185291904 >> v2:172.21.15.173:3300/0 conn(0x7f477c15d440 0x7f477c0d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.937+0000 7f477b59e640 1 -- 172.21.15.16:0/1185291904 >> 172.21.15.16:0/1185291904 conn(0x7f477c0d0de0 msgr2=0x7f477c0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.938+0000 7f477b59e640 1 -- 172.21.15.16:0/1185291904 shutdown_connections 2023-11-24T06:29:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.938+0000 7f477b59e640 1 -- 172.21.15.16:0/1185291904 wait complete. 2023-11-24T06:29:16.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.940+0000 7f477b59e640 1 Processor -- start 2023-11-24T06:29:16.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.940+0000 7f477b59e640 1 -- start start 2023-11-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06b700 0x7f477c157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477b59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f477c06c220 0x7f477c1f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f477c15d440 0x7f477c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0c5e10 con 0x7f477c06b700 2023-11-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477b59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c1f6d40 con 0x7f477c06c220 2023-11-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c1f7020 con 0x7f477c15d440 2023-11-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06b700 0x7f477c157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06b700 0x7f477c157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49630/0 (socket says 172.21.15.16:49630) 2023-11-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477a59c640 1 -- 172.21.15.16:0/43268132 learned_addr learned my addr 172.21.15.16:0/43268132 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477a59c640 1 -- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f477c15d440 msgr2=0x7f477c06e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f4773fff640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f477c06c220 0x7f477c1f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477a59c640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f477c15d440 0x7f477c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.941+0000 7f477a59c640 1 -- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f477c06c220 msgr2=0x7f477c1f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.942+0000 7f477a59c640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f477c06c220 0x7f477c1f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.942+0000 7f477a59c640 1 -- 172.21.15.16:0/43268132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4774009b30 con 0x7f477c06b700 2023-11-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.942+0000 7f4773fff640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f477c06c220 0x7f477c1f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:16.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.942+0000 7f477a59c640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06b700 0x7f477c157100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c7f0 tx=0x7f476c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.942+0000 7f4772ffd640 1 -- 172.21.15.16:0/43268132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c007820 con 0x7f477c06b700 2023-11-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.942+0000 7f4772ffd640 1 -- 172.21.15.16:0/43268132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c007980 con 0x7f477c06b700 2023-11-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.942+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c0c6060 con 0x7f477c06b700 2023-11-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.942+0000 7f4772ffd640 1 -- 172.21.15.16:0/43268132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c007cf0 con 0x7f477c06b700 2023-11-24T06:29:16.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.942+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c15abe0 con 0x7f477c06b700 2023-11-24T06:29:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.944+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c067450 con 0x7f477c06b700 2023-11-24T06:29:16.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.945+0000 7f4772ffd640 1 -- 172.21.15.16:0/43268132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f476c0113f0 con 0x7f477c06b700 2023-11-24T06:29:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.946+0000 7f4772ffd640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f475c06af20 0x7f475c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:16.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.946+0000 7f4772ffd640 1 -- 172.21.15.16:0/43268132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f476c089a50 con 0x7f477c06b700 2023-11-24T06:29:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.946+0000 7f4773fff640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f475c06af20 0x7f475c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:16.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.947+0000 7f4773fff640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f475c06af20 0x7f475c06d3e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4764007fd0 tx=0x7f4764009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:16.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:16.950+0000 7f4772ffd640 1 -- 172.21.15.16:0/43268132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f476c054a10 con 0x7f477c06b700 2023-11-24T06:29:17.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.185+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f477c06b560 con 0x7f477c06b700 2023-11-24T06:29:17.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.186+0000 7f4772ffd640 1 -- 172.21.15.16:0/43268132 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f477c06b560 con 0x7f477c06b700 2023-11-24T06:29:17.190 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:29:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.188+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f475c06af20 msgr2=0x7f475c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.188+0000 7f477b59e640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f475c06af20 0x7f475c06d3e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4764007fd0 tx=0x7f4764009040 comp rx=0 tx=0).stop 2023-11-24T06:29:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.188+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06b700 msgr2=0x7f477c157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.188+0000 7f477b59e640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06b700 0x7f477c157100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f476c00c7f0 tx=0x7f476c00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.189+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 shutdown_connections 2023-11-24T06:29:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.189+0000 7f477b59e640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f475c06af20 0x7f475c06d3e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.189+0000 7f477b59e640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f477c15d440 0x7f477c06e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.189+0000 7f477b59e640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f477c06c220 0x7f477c1f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.189+0000 7f477b59e640 1 --2- 172.21.15.16:0/43268132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f477c06b700 0x7f477c157100 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.189+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 >> 172.21.15.16:0/43268132 conn(0x7f477c0d0de0 msgr2=0x7f477c06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.189+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 shutdown_connections 2023-11-24T06:29:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:17.189+0000 7f477b59e640 1 -- 172.21.15.16:0/43268132 wait complete. 2023-11-24T06:29:17.210 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:29:17.210 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:29:17.210 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2023-11-24T06:29:17.211 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:29:17.211 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:17.211 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:29:17.211 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:29:17.211 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:29:17.211 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:29:17.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:17.211 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2023-11-24T06:29:17.273 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:17.274 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42246sec preferred_lft 42246sec 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:17.275 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::c4e1:59ff:feae:2f56/64 scope link 2023-11-24T06:29:17.276 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:17.276 INFO:teuthology.orchestra.run.smithi173.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:29:17.276 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ea:3b:9f:a2:9b:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:17.276 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::e83b:9fff:fea2:9ba6/64 scope link 2023-11-24T06:29:17.276 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:17.276 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:29:17.276 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:17.276 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::64de:a3ff:fee0:a827/64 scope link 2023-11-24T06:29:17.276 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:29:17.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:17.277 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:29:17.368 INFO:teuthology.orchestra.run.smithi173.stdout:1 2023-11-24T06:29:17.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:17.373 DEBUG:teuthology.orchestra.run.smithi173:> route 2023-11-24T06:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2023-11-24T06:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:29:17.427 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:29:17.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:17.428 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:29:17.428 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:29:17.428 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:29:17.428 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:29:17.428 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:29:17.428 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:29:17.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:17.609 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2023-11-24T06:29:17.660 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:29:17.661 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:29:17.662 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:29:17.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:17.662 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2023-11-24T06:29:17.756 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:29:17.756 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:29:17.756 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:17.812 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:29:17.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:29:17.813 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:17.865 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:29:17.960 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:29:18.017 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:29:18.018 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:29:18.019 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:29:18.019 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:29:18.019 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:29:18.019 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:29:18.019 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:29:18.019 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:29:18.019 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:29:18.019 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:29:18.019 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:18.020 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:18.021 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:18.021 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:18.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.022 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:29:18.071 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:29:18.071 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:29:18.077 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:29:18.152 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:29:18.194 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:29:18.194 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:29:18.194 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:29:18.194 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:29:18.194 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:29:18.194 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:29:18.194 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:29:18.195 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:29:18.196 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:29:18.197 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:29:18.197 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:18.197 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:29:18.197 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:18.197 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:29:18.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.198 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:29:18.266 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:2023-11-24T06:29:18.262+0000 7f91b3c44480 -1 init, newargv = 0x5621e6118950 newargc=15 2023-11-24T06:29:18.266 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[96197]: starting ceph client 2023-11-24T06:29:18.277 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[96197]: starting fuse 2023-11-24T06:29:18.286 INFO:teuthology.orchestra.run.smithi173.stdout:39 2023-11-24T06:29:18.287 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-11-24T06:29:18.287 DEBUG:teuthology.orchestra.run.smithi173:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:29:18.287 DEBUG:teuthology.orchestra.run.smithi173:> import glob 2023-11-24T06:29:18.288 DEBUG:teuthology.orchestra.run.smithi173:> import re 2023-11-24T06:29:18.288 DEBUG:teuthology.orchestra.run.smithi173:> import os 2023-11-24T06:29:18.288 DEBUG:teuthology.orchestra.run.smithi173:> import subprocess 2023-11-24T06:29:18.288 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:29:18.289 DEBUG:teuthology.orchestra.run.smithi173:> def _find_admin_socket(client_name): 2023-11-24T06:29:18.289 DEBUG:teuthology.orchestra.run.smithi173:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:29:18.289 DEBUG:teuthology.orchestra.run.smithi173:> files = glob.glob(asok_path) 2023-11-24T06:29:18.289 DEBUG:teuthology.orchestra.run.smithi173:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:29:18.289 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:29:18.289 DEBUG:teuthology.orchestra.run.smithi173:> # Given a non-glob path, it better be there 2023-11-24T06:29:18.289 DEBUG:teuthology.orchestra.run.smithi173:> if "*" not in asok_path: 2023-11-24T06:29:18.290 DEBUG:teuthology.orchestra.run.smithi173:> assert(len(files) == 1) 2023-11-24T06:29:18.290 DEBUG:teuthology.orchestra.run.smithi173:> return files[0] 2023-11-24T06:29:18.290 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:29:18.290 DEBUG:teuthology.orchestra.run.smithi173:> for f in files: 2023-11-24T06:29:18.290 DEBUG:teuthology.orchestra.run.smithi173:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:29:18.290 DEBUG:teuthology.orchestra.run.smithi173:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:29:18.290 DEBUG:teuthology.orchestra.run.smithi173:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:29:18.290 DEBUG:teuthology.orchestra.run.smithi173:> contents = proc_f.read() 2023-11-24T06:29:18.291 DEBUG:teuthology.orchestra.run.smithi173:> if mountpoint in contents: 2023-11-24T06:29:18.291 DEBUG:teuthology.orchestra.run.smithi173:> return f 2023-11-24T06:29:18.291 DEBUG:teuthology.orchestra.run.smithi173:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:29:18.291 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:29:18.291 DEBUG:teuthology.orchestra.run.smithi173:> print(_find_admin_socket("client.1")) 2023-11-24T06:29:18.292 DEBUG:teuthology.orchestra.run.smithi173:> ' 2023-11-24T06:29:18.483 INFO:teuthology.orchestra.run.smithi173.stdout:/var/run/ceph/ceph-client.1.96197.asok 2023-11-24T06:29:18.488 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:29:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:29:18.500 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.96197.asok 2023-11-24T06:29:18.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.501 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.96197.asok status 2023-11-24T06:29:18.739 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-11-24T06:29:18.739 INFO:teuthology.orchestra.run.smithi173.stdout: "metadata": { 2023-11-24T06:29:18.739 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:29:18.739 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:29:18.739 INFO:teuthology.orchestra.run.smithi173.stdout: "entity_id": "1", 2023-11-24T06:29:18.739 INFO:teuthology.orchestra.run.smithi173.stdout: "hostname": "smithi173", 2023-11-24T06:29:18.739 INFO:teuthology.orchestra.run.smithi173.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: "pid": "96197", 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: "root": "/" 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_count": 0, 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_pinned_count": 0, 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: "id": 5200, 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: "inst": { 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: "name": { 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "client", 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: "num": 5200 2023-11-24T06:29:18.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 2935043395 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 2935043395 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:29:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: "inst_str": "client.5200 192.168.0.1:0/2935043395", 2023-11-24T06:29:18.742 INFO:teuthology.orchestra.run.smithi173.stdout: "addr_str": "192.168.0.1:0/2935043395", 2023-11-24T06:29:18.742 INFO:teuthology.orchestra.run.smithi173.stdout: "inode_count": 1, 2023-11-24T06:29:18.742 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_epoch": 25, 2023-11-24T06:29:18.742 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch": 46, 2023-11-24T06:29:18.742 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:29:18.742 INFO:teuthology.orchestra.run.smithi173.stdout: "blocklisted": false, 2023-11-24T06:29:18.742 INFO:teuthology.orchestra.run.smithi173.stdout: "fs_name": "cephfs" 2023-11-24T06:29:18.742 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-11-24T06:29:18.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.755 DEBUG:teuthology.orchestra.run.smithi173:> stat --file-system '--printf=%T 2023-11-24T06:29:18.755 DEBUG:teuthology.orchestra.run.smithi173:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:18.813 INFO:teuthology.orchestra.run.smithi173.stdout:fuseblk 2023-11-24T06:29:18.814 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:18.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:18.814 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:18.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:29:19.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.138+0000 7fd96b3ac640 1 Processor -- start 2023-11-24T06:29:19.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.138+0000 7fd96b3ac640 1 -- start start 2023-11-24T06:29:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.140+0000 7fd96b3ac640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd964143c40 0x7fd964071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:19.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.140+0000 7fd96b3ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9640ccc10 0x7fd96406d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.140+0000 7fd96b3ac640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd964070e50 con 0x7fd9640717d0 2023-11-24T06:29:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.140+0000 7fd96b3ac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9640723c0 con 0x7fd9640ccc10 2023-11-24T06:29:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.140+0000 7fd96b3ac640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd964144020 con 0x7fd964143c40 2023-11-24T06:29:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.140+0000 7fd96abab640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9640ccc10 0x7fd96406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.140+0000 7fd969ba9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9640717d0 0x7fd9640cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44276/0 (socket says 172.21.15.16:44276) 2023-11-24T06:29:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.140+0000 7fd969ba9640 1 -- 172.21.15.16:0/3937816631 learned_addr learned my addr 172.21.15.16:0/3937816631 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:19.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.141+0000 7fd96a3aa640 1 --2- 172.21.15.16:0/3937816631 >> v2:172.21.15.173:3300/0 conn(0x7fd964143c40 0x7fd964071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.141+0000 7fd9693a8640 1 -- 172.21.15.16:0/3937816631 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950880329 0 0) 0x7fd964070e50 con 0x7fd9640717d0 2023-11-24T06:29:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.141+0000 7fd96abab640 1 -- 172.21.15.16:0/3937816631 >> v2:172.21.15.173:3300/0 conn(0x7fd964143c40 msgr2=0x7fd964071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.141+0000 7fd96abab640 1 --2- 172.21.15.16:0/3937816631 >> v2:172.21.15.173:3300/0 conn(0x7fd964143c40 0x7fd964071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.141+0000 7fd96abab640 1 -- 172.21.15.16:0/3937816631 >> v1:172.21.15.16:6789/0 conn(0x7fd9640717d0 legacy=0x7fd9640cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.141+0000 7fd96abab640 1 -- 172.21.15.16:0/3937816631 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd964071e00 con 0x7fd9640ccc10 2023-11-24T06:29:19.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.141+0000 7fd96a3aa640 1 --2- 172.21.15.16:0/3937816631 >> v2:172.21.15.173:3300/0 conn(0x7fd964143c40 0x7fd964071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.142+0000 7fd96abab640 1 --2- 172.21.15.16:0/3937816631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9640ccc10 0x7fd96406d160 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd96406e410 tx=0x7fd95c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c878e0d04dfeca3f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.142+0000 7fd9693a8640 1 -- 172.21.15.16:0/3937816631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c02cbe0 con 0x7fd9640ccc10 2023-11-24T06:29:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.142+0000 7fd9693a8640 1 -- 172.21.15.16:0/3937816631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c02cd40 con 0x7fd9640ccc10 2023-11-24T06:29:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.142+0000 7fd9693a8640 1 -- 172.21.15.16:0/3937816631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c034660 con 0x7fd9640ccc10 2023-11-24T06:29:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.143+0000 7fd96b3ac640 1 -- 172.21.15.16:0/3937816631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9640ccc10 msgr2=0x7fd96406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.143+0000 7fd96b3ac640 1 --2- 172.21.15.16:0/3937816631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9640ccc10 0x7fd96406d160 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd96406e410 tx=0x7fd95c02be30 comp rx=0 tx=0).stop 2023-11-24T06:29:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.143+0000 7fd96b3ac640 1 -- 172.21.15.16:0/3937816631 shutdown_connections 2023-11-24T06:29:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.143+0000 7fd96b3ac640 1 --2- 172.21.15.16:0/3937816631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9640ccc10 0x7fd96406d160 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.143+0000 7fd96b3ac640 1 --2- 172.21.15.16:0/3937816631 >> v2:172.21.15.173:3300/0 conn(0x7fd964143c40 0x7fd964071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.143+0000 7fd96b3ac640 1 -- 172.21.15.16:0/3937816631 >> 172.21.15.16:0/3937816631 conn(0x7fd96406a800 msgr2=0x7fd96406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.143+0000 7fd96b3ac640 1 -- 172.21.15.16:0/3937816631 shutdown_connections 2023-11-24T06:29:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.143+0000 7fd96b3ac640 1 -- 172.21.15.16:0/3937816631 wait complete. 2023-11-24T06:29:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.145+0000 7fd96b3ac640 1 Processor -- start 2023-11-24T06:29:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.145+0000 7fd96b3ac640 1 -- start start 2023-11-24T06:29:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.146+0000 7fd96b3ac640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9640717d0 0x7fd9641424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.146+0000 7fd96b3ac640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9640ccc10 0x7fd9640c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.146+0000 7fd96b3ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd964143c40 0x7fd9640ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.146+0000 7fd96b3ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640c3d70 con 0x7fd964143c40 2023-11-24T06:29:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.146+0000 7fd96b3ac640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640ad5b0 con 0x7fd9640717d0 2023-11-24T06:29:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.146+0000 7fd96b3ac640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640ca950 con 0x7fd9640ccc10 2023-11-24T06:29:19.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.146+0000 7fd96abab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd964143c40 0x7fd9640ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.146+0000 7fd96abab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd964143c40 0x7fd9640ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49654/0 (socket says 172.21.15.16:49654) 2023-11-24T06:29:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.146+0000 7fd96abab640 1 -- 172.21.15.16:0/1359874184 learned_addr learned my addr 172.21.15.16:0/1359874184 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.147+0000 7fd96a3aa640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9640717d0 0x7fd9641424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.147+0000 7fd969ba9640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9640ccc10 0x7fd9640c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.147+0000 7fd96abab640 1 -- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9640ccc10 msgr2=0x7fd9640c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:19.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.147+0000 7fd96abab640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9640ccc10 0x7fd9640c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.147+0000 7fd96abab640 1 -- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9640717d0 msgr2=0x7fd9641424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.147+0000 7fd96abab640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9640717d0 0x7fd9641424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.147+0000 7fd96abab640 1 -- 172.21.15.16:0/1359874184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c009a70 con 0x7fd964143c40 2023-11-24T06:29:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.147+0000 7fd96abab640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd964143c40 0x7fd9640ca2b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd95c02c440 tx=0x7fd95c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.148+0000 7fd95affd640 1 -- 172.21.15.16:0/1359874184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c040650 con 0x7fd964143c40 2023-11-24T06:29:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.148+0000 7fd95affd640 1 -- 172.21.15.16:0/1359874184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c0407d0 con 0x7fd964143c40 2023-11-24T06:29:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.148+0000 7fd95affd640 1 -- 172.21.15.16:0/1359874184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c040b80 con 0x7fd964143c40 2023-11-24T06:29:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.148+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9640cac30 con 0x7fd964143c40 2023-11-24T06:29:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.148+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9640c42a0 con 0x7fd964143c40 2023-11-24T06:29:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.149+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9640ae7c0 con 0x7fd964143c40 2023-11-24T06:29:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.150+0000 7fd95affd640 1 -- 172.21.15.16:0/1359874184 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fd95c040ce0 con 0x7fd964143c40 2023-11-24T06:29:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.151+0000 7fd95affd640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd94806aed0 0x7fd94806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.151+0000 7fd95affd640 1 -- 172.21.15.16:0/1359874184 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd95c0ac550 con 0x7fd964143c40 2023-11-24T06:29:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.152+0000 7fd96a3aa640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd94806aed0 0x7fd94806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.153+0000 7fd96a3aa640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd94806aed0 0x7fd94806d390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd960006fd0 tx=0x7fd960008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.156+0000 7fd95affd640 1 -- 172.21.15.16:0/1359874184 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd95c077510 con 0x7fd964143c40 2023-11-24T06:29:19.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:19.387+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd964071290 con 0x7fd964143c40 2023-11-24T06:29:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.014+0000 7fd95affd640 1 -- 172.21.15.16:0/1359874184 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd95c07ab30 con 0x7fd964143c40 2023-11-24T06:29:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd94806aed0 msgr2=0x7fd94806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd94806aed0 0x7fd94806d390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd960006fd0 tx=0x7fd960008040 comp rx=0 tx=0).stop 2023-11-24T06:29:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd964143c40 msgr2=0x7fd9640ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd964143c40 0x7fd9640ca2b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd95c02c440 tx=0x7fd95c004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 shutdown_connections 2023-11-24T06:29:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd94806aed0 0x7fd94806d390 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd964143c40 0x7fd9640ca2b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9640ccc10 0x7fd9640c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 --2- 172.21.15.16:0/1359874184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9640717d0 0x7fd9641424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 >> 172.21.15.16:0/1359874184 conn(0x7fd96406a800 msgr2=0x7fd9640cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 shutdown_connections 2023-11-24T06:29:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.018+0000 7fd96b3ac640 1 -- 172.21.15.16:0/1359874184 wait complete. 2023-11-24T06:29:20.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:29:20.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.277+0000 7faa5e94e640 1 Processor -- start 2023-11-24T06:29:20.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.277+0000 7faa5e94e640 1 -- start start 2023-11-24T06:29:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.279+0000 7faa5e94e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 0x7faa580b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.279+0000 7faa5e94e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa58162810 0x7faa58071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.279+0000 7faa5e94e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa580702e0 con 0x7faa580b3650 2023-11-24T06:29:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.279+0000 7faa5e94e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa580705c0 con 0x7faa58162810 2023-11-24T06:29:20.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.279+0000 7faa5e94e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa580708a0 con 0x7faa58162430 2023-11-24T06:29:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.279+0000 7faa5d14b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa58162810 0x7faa58071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.279+0000 7faa5d14b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa58162810 0x7faa58071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49662/0 (socket says 172.21.15.16:49662) 2023-11-24T06:29:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.279+0000 7faa5d14b640 1 -- 172.21.15.16:0/3916583350 learned_addr learned my addr 172.21.15.16:0/3916583350 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.279+0000 7faa5d94c640 1 --2- 172.21.15.16:0/3916583350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 0x7faa580b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.280+0000 7faa5d14b640 1 -- 172.21.15.16:0/3916583350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 msgr2=0x7faa580b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:20.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.280+0000 7faa5d14b640 1 --2- 172.21.15.16:0/3916583350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 0x7faa580b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.280+0000 7faa5d14b640 1 -- 172.21.15.16:0/3916583350 >> v1:172.21.15.173:6789/0 conn(0x7faa580b3650 legacy=0x7faa581650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.280+0000 7faa5d14b640 1 -- 172.21.15.16:0/3916583350 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa58070b80 con 0x7faa58162810 2023-11-24T06:29:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.281+0000 7faa5d14b640 1 --2- 172.21.15.16:0/3916583350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa58162810 0x7faa58071240 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faa48009870 tx=0x7faa4802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f1aafd6e5c827b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.281+0000 7faa5c94a640 1 -- 172.21.15.16:0/3916583350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48031640 con 0x7faa58162810 2023-11-24T06:29:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.281+0000 7faa5c94a640 1 -- 172.21.15.16:0/3916583350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa480317a0 con 0x7faa58162810 2023-11-24T06:29:20.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.281+0000 7faa5c94a640 1 -- 172.21.15.16:0/3916583350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48031b30 con 0x7faa58162810 2023-11-24T06:29:20.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.282+0000 7faa5e94e640 1 -- 172.21.15.16:0/3916583350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa58162810 msgr2=0x7faa58071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.282+0000 7faa5e94e640 1 --2- 172.21.15.16:0/3916583350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa58162810 0x7faa58071240 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faa48009870 tx=0x7faa4802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.282+0000 7faa5e94e640 1 -- 172.21.15.16:0/3916583350 shutdown_connections 2023-11-24T06:29:20.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.282+0000 7faa5e94e640 1 --2- 172.21.15.16:0/3916583350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa58162810 0x7faa58071240 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.282+0000 7faa5e94e640 1 --2- 172.21.15.16:0/3916583350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 0x7faa580b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.282+0000 7faa5e94e640 1 -- 172.21.15.16:0/3916583350 >> 172.21.15.16:0/3916583350 conn(0x7faa5806a800 msgr2=0x7faa5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.282+0000 7faa5e94e640 1 -- 172.21.15.16:0/3916583350 shutdown_connections 2023-11-24T06:29:20.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.282+0000 7faa5e94e640 1 -- 172.21.15.16:0/3916583350 wait complete. 2023-11-24T06:29:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.284+0000 7faa5e94e640 1 Processor -- start 2023-11-24T06:29:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.284+0000 7faa5e94e640 1 -- start start 2023-11-24T06:29:20.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.285+0000 7faa5e94e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa580b3650 0x7faa58145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.285+0000 7faa5e94e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 0x7faa581ec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.285+0000 7faa5e94e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa581ec710 0x7faa58159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.285+0000 7faa5e94e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa581eaee0 con 0x7faa580b3650 2023-11-24T06:29:20.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.285+0000 7faa5e94e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa581ebd40 con 0x7faa581ec710 2023-11-24T06:29:20.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.285+0000 7faa5e94e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa581eb130 con 0x7faa58162430 2023-11-24T06:29:20.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.285+0000 7faa5d94c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa580b3650 0x7faa58145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.285+0000 7faa5d94c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa580b3650 0x7faa58145f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49674/0 (socket says 172.21.15.16:49674) 2023-11-24T06:29:20.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.285+0000 7faa5d94c640 1 -- 172.21.15.16:0/2175806219 learned_addr learned my addr 172.21.15.16:0/2175806219 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:20.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5d94c640 1 -- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 msgr2=0x7faa581ec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5d14b640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 0x7faa581ec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5e14d640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa581ec710 0x7faa58159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5d94c640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 0x7faa581ec1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5d94c640 1 -- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa581ec710 msgr2=0x7faa58159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5d94c640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa581ec710 0x7faa58159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5d94c640 1 -- 172.21.15.16:0/2175806219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480094d0 con 0x7faa580b3650 2023-11-24T06:29:20.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5d14b640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 0x7faa581ec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:20.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5d94c640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa580b3650 0x7faa58145f30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7faa5400c7f0 tx=0x7faa5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:20.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa4e7fc640 1 -- 172.21.15.16:0/2175806219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54004220 con 0x7faa580b3650 2023-11-24T06:29:20.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa4e7fc640 1 -- 172.21.15.16:0/2175806219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa540043a0 con 0x7faa580b3650 2023-11-24T06:29:20.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa581eb410 con 0x7faa580b3650 2023-11-24T06:29:20.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa4e7fc640 1 -- 172.21.15.16:0/2175806219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54007860 con 0x7faa580b3650 2023-11-24T06:29:20.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.286+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa581eb9d0 con 0x7faa580b3650 2023-11-24T06:29:20.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.288+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa580ce490 con 0x7faa580b3650 2023-11-24T06:29:20.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.288+0000 7faa4e7fc640 1 -- 172.21.15.16:0/2175806219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7faa54007bd0 con 0x7faa580b3650 2023-11-24T06:29:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.289+0000 7faa4e7fc640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa3c06aed0 0x7faa3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.290+0000 7faa4e7fc640 1 -- 172.21.15.16:0/2175806219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa54089880 con 0x7faa580b3650 2023-11-24T06:29:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.290+0000 7faa5d14b640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa3c06aed0 0x7faa3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.291+0000 7faa5d14b640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa3c06aed0 0x7faa3c06d390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faa48036000 tx=0x7faa4803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.294+0000 7faa4e7fc640 1 -- 172.21.15.16:0/2175806219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa54054730 con 0x7faa580b3650 2023-11-24T06:29:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.518+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faa580b2440 con 0x7faa3c06aed0 2023-11-24T06:29:20.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.521+0000 7faa4e7fc640 1 -- 172.21.15.16:0/2175806219 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7faa580b2440 con 0x7faa3c06aed0 2023-11-24T06:29:20.525 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:29:20.525 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:20.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephfs" 2023-11-24T06:29:20.525 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:20.525 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:29:20.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa3c06aed0 msgr2=0x7faa3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa3c06aed0 0x7faa3c06d390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faa48036000 tx=0x7faa4803c040 comp rx=0 tx=0).stop 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa580b3650 msgr2=0x7faa58145f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa580b3650 0x7faa58145f30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7faa5400c7f0 tx=0x7faa5400ccb0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 shutdown_connections 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa3c06aed0 0x7faa3c06d390 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa581ec710 0x7faa58159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa58162430 0x7faa581ec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 --2- 172.21.15.16:0/2175806219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa580b3650 0x7faa58145f30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 >> 172.21.15.16:0/2175806219 conn(0x7faa5806a800 msgr2=0x7faa580af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 shutdown_connections 2023-11-24T06:29:20.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.524+0000 7faa5e94e640 1 -- 172.21.15.16:0/2175806219 wait complete. 2023-11-24T06:29:20.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:29:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.775+0000 7f3288a14640 1 Processor -- start 2023-11-24T06:29:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.775+0000 7f3288a14640 1 -- start start 2023-11-24T06:29:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.777+0000 7f3288a14640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840cf100 0x7f32840d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.777+0000 7f3288a14640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32840b23c0 con 0x7f32840d0220 2023-11-24T06:29:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.777+0000 7f3288a14640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32840d0800 con 0x7f328415d3c0 2023-11-24T06:29:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.777+0000 7f3288a14640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32840ce5d0 con 0x7f32840cf100 2023-11-24T06:29:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.777+0000 7f327a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32840d0220 0x7f32840ce9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44282/0 (socket says 172.21.15.16:44282) 2023-11-24T06:29:20.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.777+0000 7f327a59c640 1 -- 172.21.15.16:0/1352206070 learned_addr learned my addr 172.21.15.16:0/1352206070 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.777+0000 7f328359e640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840cf100 0x7f32840d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678559749 0 0) 0x7f32840b23c0 con 0x7f32840d0220 2023-11-24T06:29:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3270003520 con 0x7f32840d0220 2023-11-24T06:29:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1874208297 0 0) 0x7f3270003520 con 0x7f32840d0220 2023-11-24T06:29:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840cf100 msgr2=0x7f32840d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840cf100 0x7f32840d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 >> v1:172.21.15.173:6789/0 conn(0x7f328415d3c0 legacy=0x7f32840afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840cdf10 con 0x7f32840d0220 2023-11-24T06:29:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f326c001670 con 0x7f32840d0220 2023-11-24T06:29:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 >> v1:172.21.15.16:6789/0 conn(0x7f32840d0220 legacy=0x7f32840ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840cf100 0x7f32700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3270003f20 0x7f32700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3270004d00 0x7f3270005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840ce5d0 con 0x7f3270003f20 2023-11-24T06:29:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327000bb10 con 0x7f3270004d00 2023-11-24T06:29:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.778+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327000bcd0 con 0x7f32840cf100 2023-11-24T06:29:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.779+0000 7f327a59c640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3270003f20 0x7f32700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.779+0000 7f3282d9d640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840cf100 0x7f32700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.779+0000 7f328359e640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3270004d00 0x7f3270005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.779+0000 7f327a59c640 1 -- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840cf100 msgr2=0x7f32700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.779+0000 7f327a59c640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840cf100 0x7f32700039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.779+0000 7f327a59c640 1 -- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3270004d00 msgr2=0x7f3270005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.779+0000 7f327a59c640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3270004d00 0x7f3270005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.779+0000 7f327a59c640 1 -- 172.21.15.16:0/1352206070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270003520 con 0x7f3270003f20 2023-11-24T06:29:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.779+0000 7f328359e640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3270004d00 0x7f3270005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.781+0000 7f327a59c640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3270003f20 0x7f32700047c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c240 tx=0x7f32700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8897a55715a8bd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.781+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c005ad0 con 0x7f3270003f20 2023-11-24T06:29:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.781+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c005c30 con 0x7f3270003f20 2023-11-24T06:29:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.781+0000 7f328259c640 1 -- 172.21.15.16:0/1352206070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c02c8b0 con 0x7f3270003f20 2023-11-24T06:29:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.782+0000 7f3288a14640 1 -- 172.21.15.16:0/1352206070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3270003f20 msgr2=0x7f32700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.782+0000 7f3288a14640 1 --2- 172.21.15.16:0/1352206070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3270003f20 0x7f32700047c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c240 tx=0x7f32700057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.782+0000 7f328359e640 1 -- 172.21.15.16:0/1352206070 reap_dead start 2023-11-24T06:29:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.782+0000 7f3288a14640 1 -- 172.21.15.16:0/1352206070 shutdown_connections 2023-11-24T06:29:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.782+0000 7f3288a14640 1 -- 172.21.15.16:0/1352206070 >> 172.21.15.16:0/1352206070 conn(0x7f32840d30b0 msgr2=0x7f32840d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.782+0000 7f3288a14640 1 -- 172.21.15.16:0/1352206070 shutdown_connections 2023-11-24T06:29:20.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.782+0000 7f3288a14640 1 -- 172.21.15.16:0/1352206070 wait complete. 2023-11-24T06:29:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.784+0000 7f3288a14640 1 Processor -- start 2023-11-24T06:29:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.784+0000 7f3288a14640 1 -- start start 2023-11-24T06:29:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.785+0000 7f3288a14640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f328414f580 0x7f32840b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.785+0000 7f3288a14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32840c9570 0x7f32840b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.785+0000 7f3288a14640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840b7c70 0x7f32840b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.785+0000 7f3288a14640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840b8630 con 0x7f32840c9570 2023-11-24T06:29:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.785+0000 7f3288a14640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840b61f0 con 0x7f328414f580 2023-11-24T06:29:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.785+0000 7f3288a14640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32840b7030 con 0x7f32840b7c70 2023-11-24T06:29:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f327a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32840c9570 0x7f32840b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f327a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32840c9570 0x7f32840b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49690/0 (socket says 172.21.15.16:49690) 2023-11-24T06:29:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f327a59c640 1 -- 172.21.15.16:0/1327048577 learned_addr learned my addr 172.21.15.16:0/1327048577 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f3282d9d640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f328414f580 0x7f32840b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f328359e640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840b7c70 0x7f32840b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f327a59c640 1 -- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840b7c70 msgr2=0x7f32840b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f327a59c640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840b7c70 0x7f32840b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f327a59c640 1 -- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f328414f580 msgr2=0x7f32840b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f327a59c640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f328414f580 0x7f32840b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.786+0000 7f327a59c640 1 -- 172.21.15.16:0/1327048577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840c99f0 con 0x7f32840c9570 2023-11-24T06:29:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.787+0000 7f327a59c640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32840c9570 0x7f32840b5cb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f326c02c520 tx=0x7f326c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.787+0000 7f327bfff640 1 -- 172.21.15.16:0/1327048577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c03c040 con 0x7f32840c9570 2023-11-24T06:29:20.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.787+0000 7f327bfff640 1 -- 172.21.15.16:0/1327048577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c001630 con 0x7f32840c9570 2023-11-24T06:29:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.787+0000 7f327bfff640 1 -- 172.21.15.16:0/1327048577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c007270 con 0x7f32840c9570 2023-11-24T06:29:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.787+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32840b6420 con 0x7f32840c9570 2023-11-24T06:29:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.787+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32840b69e0 con 0x7f32840c9570 2023-11-24T06:29:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.788+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328415e120 con 0x7f32840c9570 2023-11-24T06:29:20.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.789+0000 7f327bfff640 1 -- 172.21.15.16:0/1327048577 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f326c032070 con 0x7f32840c9570 2023-11-24T06:29:20.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.791+0000 7f327bfff640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f326406af00 0x7f326406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.791+0000 7f327bfff640 1 -- 172.21.15.16:0/1327048577 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f326c0aa900 con 0x7f32840c9570 2023-11-24T06:29:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.792+0000 7f3282d9d640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f326406af00 0x7f326406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.793+0000 7f3282d9d640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f326406af00 0x7f326406d3c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3274001770 tx=0x7f3274009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:20.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.795+0000 7f327bfff640 1 -- 172.21.15.16:0/1327048577 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f326c075a80 con 0x7f32840c9570 2023-11-24T06:29:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.988+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3284067b60 con 0x7f32840c9570 2023-11-24T06:29:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.989+0000 7f327bfff640 1 -- 172.21.15.16:0/1327048577 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f326c0790a0 con 0x7f32840c9570 2023-11-24T06:29:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f326406af00 msgr2=0x7f326406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f326406af00 0x7f326406d3c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3274001770 tx=0x7f3274009040 comp rx=0 tx=0).stop 2023-11-24T06:29:20.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32840c9570 msgr2=0x7f32840b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:20.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32840c9570 0x7f32840b5cb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f326c02c520 tx=0x7f326c02e070 comp rx=0 tx=0).stop 2023-11-24T06:29:20.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 shutdown_connections 2023-11-24T06:29:20.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f326406af00 0x7f326406d3c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32840b7c70 0x7f32840b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32840c9570 0x7f32840b5cb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 --2- 172.21.15.16:0/1327048577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f328414f580 0x7f32840b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:20.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 >> 172.21.15.16:0/1327048577 conn(0x7f32840d30b0 msgr2=0x7f32840cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:20.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.991+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 shutdown_connections 2023-11-24T06:29:20.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:20.992+0000 7f3288a14640 1 -- 172.21.15.16:0/1327048577 wait complete. 2023-11-24T06:29:21.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp5190 2023-11-24T06:29:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.254+0000 7fe6ba947640 1 Processor -- start 2023-11-24T06:29:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.255+0000 7fe6ba947640 1 -- start start 2023-11-24T06:29:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.256+0000 7fe6ba947640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6b415d4b0 0x7fe6b40d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.257+0000 7fe6ba947640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b40b2730 con 0x7fe6b40b41e0 2023-11-24T06:29:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.257+0000 7fe6ba947640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b40d1ec0 con 0x7fe6b40cf9a0 2023-11-24T06:29:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.257+0000 7fe6ba947640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b40b29d0 con 0x7fe6b415d4b0 2023-11-24T06:29:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.257+0000 7fe6b9945640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6b415d4b0 0x7fe6b40d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.257+0000 7fe6ba146640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe6b40b41e0 0x7fe6b40ce500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44296/0 (socket says 172.21.15.16:44296) 2023-11-24T06:29:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.257+0000 7fe6ba146640 1 -- 172.21.15.16:0/3046190238 learned_addr learned my addr 172.21.15.16:0/3046190238 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.258+0000 7fe6b9945640 1 -- 172.21.15.16:0/3046190238 >> v1:172.21.15.173:6789/0 conn(0x7fe6b40cf9a0 legacy=0x7fe6b40cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.258+0000 7fe6b9945640 1 -- 172.21.15.16:0/3046190238 >> v1:172.21.15.16:6789/0 conn(0x7fe6b40b41e0 legacy=0x7fe6b40ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.258+0000 7fe6b9945640 1 -- 172.21.15.16:0/3046190238 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40b2cb0 con 0x7fe6b415d4b0 2023-11-24T06:29:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.259+0000 7fe6b9945640 1 --2- 172.21.15.16:0/3046190238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 0x7fe6b40d1550 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00097a0 tx=0x7fe6b002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6bc548311df0f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.259+0000 7fe6b8943640 1 -- 172.21.15.16:0/3046190238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0030610 con 0x7fe6b415d4b0 2023-11-24T06:29:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.259+0000 7fe6b8943640 1 -- 172.21.15.16:0/3046190238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0030770 con 0x7fe6b415d4b0 2023-11-24T06:29:21.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.259+0000 7fe6b8943640 1 -- 172.21.15.16:0/3046190238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0030b00 con 0x7fe6b415d4b0 2023-11-24T06:29:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.260+0000 7fe6ba947640 1 -- 172.21.15.16:0/3046190238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 msgr2=0x7fe6b40d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.260+0000 7fe6ba947640 1 --2- 172.21.15.16:0/3046190238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 0x7fe6b40d1550 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00097a0 tx=0x7fe6b002b960 comp rx=0 tx=0).stop 2023-11-24T06:29:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.260+0000 7fe6ba947640 1 -- 172.21.15.16:0/3046190238 shutdown_connections 2023-11-24T06:29:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.260+0000 7fe6ba947640 1 --2- 172.21.15.16:0/3046190238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 0x7fe6b40d1550 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.260+0000 7fe6ba947640 1 -- 172.21.15.16:0/3046190238 >> 172.21.15.16:0/3046190238 conn(0x7fe6b40d3000 msgr2=0x7fe6b40d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.260+0000 7fe6ba947640 1 -- 172.21.15.16:0/3046190238 shutdown_connections 2023-11-24T06:29:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.260+0000 7fe6ba947640 1 -- 172.21.15.16:0/3046190238 wait complete. 2023-11-24T06:29:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.263+0000 7fe6ba947640 1 Processor -- start 2023-11-24T06:29:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.263+0000 7fe6ba947640 1 -- start start 2023-11-24T06:29:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.264+0000 7fe6ba947640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6b40b41e0 0x7fe6b4154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.264+0000 7fe6ba947640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6b40cf9a0 0x7fe6b41588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.264+0000 7fe6ba947640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 0x7fe6b40cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.264+0000 7fe6ba947640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40cd3f0 con 0x7fe6b415d4b0 2023-11-24T06:29:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.264+0000 7fe6ba947640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b409cce0 con 0x7fe6b40b41e0 2023-11-24T06:29:21.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.264+0000 7fe6ba947640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4155110 con 0x7fe6b40cf9a0 2023-11-24T06:29:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6ba146640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 0x7fe6b40cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6ba146640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 0x7fe6b40cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49698/0 (socket says 172.21.15.16:49698) 2023-11-24T06:29:21.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6ba146640 1 -- 172.21.15.16:0/2639114954 learned_addr learned my addr 172.21.15.16:0/2639114954 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6ba146640 1 -- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6b40cf9a0 msgr2=0x7fe6b41588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6b9144640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6b40cf9a0 0x7fe6b41588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6b9945640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6b40b41e0 0x7fe6b4154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6ba146640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6b40cf9a0 0x7fe6b41588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6ba146640 1 -- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6b40b41e0 msgr2=0x7fe6b4154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6ba146640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6b40b41e0 0x7fe6b4154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.265+0000 7fe6ba146640 1 -- 172.21.15.16:0/2639114954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b0009400 con 0x7fe6b415d4b0 2023-11-24T06:29:21.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.266+0000 7fe6ba146640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 0x7fe6b40cbde0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00caf0 tx=0x7fe6ac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.266+0000 7fe6aa7fc640 1 -- 172.21.15.16:0/2639114954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac00aca0 con 0x7fe6b415d4b0 2023-11-24T06:29:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.266+0000 7fe6aa7fc640 1 -- 172.21.15.16:0/2639114954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac00ae00 con 0x7fe6b415d4b0 2023-11-24T06:29:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.266+0000 7fe6aa7fc640 1 -- 172.21.15.16:0/2639114954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac012600 con 0x7fe6b415d4b0 2023-11-24T06:29:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.266+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b415da10 con 0x7fe6b415d4b0 2023-11-24T06:29:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.266+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b409d210 con 0x7fe6b415d4b0 2023-11-24T06:29:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.267+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b4067450 con 0x7fe6b415d4b0 2023-11-24T06:29:21.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.268+0000 7fe6aa7fc640 1 -- 172.21.15.16:0/2639114954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe6ac007070 con 0x7fe6b415d4b0 2023-11-24T06:29:21.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.269+0000 7fe6aa7fc640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe69806aed0 0x7fe69806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.270+0000 7fe6aa7fc640 1 -- 172.21.15.16:0/2639114954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6ac0893c0 con 0x7fe6b415d4b0 2023-11-24T06:29:21.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.270+0000 7fe6b9945640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe69806aed0 0x7fe69806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.271+0000 7fe6b9945640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe69806aed0 0x7fe69806d390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0036000 tx=0x7fe6b003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:21.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.274+0000 7fe6aa7fc640 1 -- 172.21.15.16:0/2639114954 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6ac0545c0 con 0x7fe6b415d4b0 2023-11-24T06:29:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.499+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp5190", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6b40d2780 con 0x7fe69806aed0 2023-11-24T06:29:21.518 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:21.516+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:21.519 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:21.516+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:21.519 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:21.516+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:21.519 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:21.516+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:21.519 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:21.516+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:21.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.555+0000 7fe6aa7fc640 1 -- 172.21.15.16:0/2639114954 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe6b40d2780 con 0x7fe69806aed0 2023-11-24T06:29:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.558+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe69806aed0 msgr2=0x7fe69806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.558+0000 7fe6ba947640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe69806aed0 0x7fe69806d390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0036000 tx=0x7fe6b003b040 comp rx=0 tx=0).stop 2023-11-24T06:29:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 msgr2=0x7fe6b40cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 0x7fe6b40cbde0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00caf0 tx=0x7fe6ac00a060 comp rx=0 tx=0).stop 2023-11-24T06:29:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 shutdown_connections 2023-11-24T06:29:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe69806aed0 0x7fe69806d390 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b415d4b0 0x7fe6b40cbde0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6b40cf9a0 0x7fe6b41588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 --2- 172.21.15.16:0/2639114954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6b40b41e0 0x7fe6b4154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 >> 172.21.15.16:0/2639114954 conn(0x7fe6b40d3000 msgr2=0x7fe6b40cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 shutdown_connections 2023-11-24T06:29:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.559+0000 7fe6ba947640 1 -- 172.21.15.16:0/2639114954 wait complete. 2023-11-24T06:29:21.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol6306 subvol_grp5190 2023-11-24T06:29:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.809+0000 7fefe7239640 1 Processor -- start 2023-11-24T06:29:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.809+0000 7fefe7239640 1 -- start start 2023-11-24T06:29:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.811+0000 7fefe7239640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe0143c40 0x7fefe0071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.811+0000 7fefe7239640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fefe00ccc10 0x7fefe006d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.811+0000 7fefe7239640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe0070e50 con 0x7fefe00717d0 2023-11-24T06:29:21.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.811+0000 7fefe7239640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe00723c0 con 0x7fefe00ccc10 2023-11-24T06:29:21.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.811+0000 7fefe7239640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0144020 con 0x7fefe0143c40 2023-11-24T06:29:21.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.812+0000 7fefe5a36640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fefe00717d0 0x7fefe00cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:36196/0 (socket says 172.21.15.16:36196) 2023-11-24T06:29:21.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.812+0000 7fefe5a36640 1 -- 172.21.15.16:0/3100443164 learned_addr learned my addr 172.21.15.16:0/3100443164 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:21.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.812+0000 7fefe6a38640 1 --2- 172.21.15.16:0/3100443164 >> v2:172.21.15.173:3300/0 conn(0x7fefe00ccc10 0x7fefe006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.812+0000 7fefe6237640 1 --2- 172.21.15.16:0/3100443164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe0143c40 0x7fefe0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.813+0000 7fefe6a38640 1 -- 172.21.15.16:0/3100443164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe0143c40 msgr2=0x7fefe0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.813+0000 7fefe5235640 1 -- 172.21.15.16:0/3100443164 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507274364 0 0) 0x7fefe0070e50 con 0x7fefe00717d0 2023-11-24T06:29:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.813+0000 7fefe6a38640 1 --2- 172.21.15.16:0/3100443164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe0143c40 0x7fefe0071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.813+0000 7fefe6a38640 1 -- 172.21.15.16:0/3100443164 >> v1:172.21.15.173:6789/0 conn(0x7fefe00717d0 legacy=0x7fefe00cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.813+0000 7fefe6a38640 1 -- 172.21.15.16:0/3100443164 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0071e00 con 0x7fefe00ccc10 2023-11-24T06:29:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.814+0000 7fefe6a38640 1 --2- 172.21.15.16:0/3100443164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe00ccc10 0x7fefe006d160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefd80097a0 tx=0x7fefd802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ee4c0ba7ce32bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:21.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.814+0000 7fefe5235640 1 -- 172.21.15.16:0/3100443164 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd802e070 con 0x7fefe00ccc10 2023-11-24T06:29:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.815+0000 7fefe5235640 1 -- 172.21.15.16:0/3100443164 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd80315a0 con 0x7fefe00ccc10 2023-11-24T06:29:21.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.815+0000 7fefe5235640 1 -- 172.21.15.16:0/3100443164 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8031930 con 0x7fefe00ccc10 2023-11-24T06:29:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.815+0000 7fefe7239640 1 -- 172.21.15.16:0/3100443164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe00ccc10 msgr2=0x7fefe006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.815+0000 7fefe7239640 1 --2- 172.21.15.16:0/3100443164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe00ccc10 0x7fefe006d160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefd80097a0 tx=0x7fefd802b840 comp rx=0 tx=0).stop 2023-11-24T06:29:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.815+0000 7fefe7239640 1 -- 172.21.15.16:0/3100443164 shutdown_connections 2023-11-24T06:29:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.815+0000 7fefe7239640 1 --2- 172.21.15.16:0/3100443164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe00ccc10 0x7fefe006d160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.815+0000 7fefe7239640 1 --2- 172.21.15.16:0/3100443164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe0143c40 0x7fefe0071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.815+0000 7fefe7239640 1 -- 172.21.15.16:0/3100443164 >> 172.21.15.16:0/3100443164 conn(0x7fefe006a800 msgr2=0x7fefe006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.816+0000 7fefe7239640 1 -- 172.21.15.16:0/3100443164 shutdown_connections 2023-11-24T06:29:21.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.816+0000 7fefe7239640 1 -- 172.21.15.16:0/3100443164 wait complete. 2023-11-24T06:29:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.817+0000 7fefe7239640 1 Processor -- start 2023-11-24T06:29:21.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.818+0000 7fefe7239640 1 -- start start 2023-11-24T06:29:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe7239640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe00717d0 0x7fefe0159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe7239640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe00ccc10 0x7fefe01f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe7239640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0143c40 0x7fefe00a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe7239640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01f2730 con 0x7fefe0143c40 2023-11-24T06:29:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe7239640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01f29d0 con 0x7fefe00ccc10 2023-11-24T06:29:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe7239640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01f2cb0 con 0x7fefe00717d0 2023-11-24T06:29:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe6a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0143c40 0x7fefe00a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe6a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0143c40 0x7fefe00a8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49712/0 (socket says 172.21.15.16:49712) 2023-11-24T06:29:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe6a38640 1 -- 172.21.15.16:0/1373151292 learned_addr learned my addr 172.21.15.16:0/1373151292 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe5a36640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe00ccc10 0x7fefe01f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.819+0000 7fefe6237640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe00717d0 0x7fefe0159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefe6a38640 1 -- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe00717d0 msgr2=0x7fefe0159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefe6a38640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe00717d0 0x7fefe0159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefe6a38640 1 -- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe00ccc10 msgr2=0x7fefe01f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:21.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefe6a38640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe00ccc10 0x7fefe01f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefe6a38640 1 -- 172.21.15.16:0/1373151292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8009400 con 0x7fefe0143c40 2023-11-24T06:29:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefe6a38640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0143c40 0x7fefe00a8bb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fefd80097a0 tx=0x7fefd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefd6ffd640 1 -- 172.21.15.16:0/1373151292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8004230 con 0x7fefe0143c40 2023-11-24T06:29:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefd6ffd640 1 -- 172.21.15.16:0/1373151292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd8004390 con 0x7fefe0143c40 2023-11-24T06:29:21.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe01f2f90 con 0x7fefe0143c40 2023-11-24T06:29:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefd6ffd640 1 -- 172.21.15.16:0/1373151292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8003850 con 0x7fefe0143c40 2023-11-24T06:29:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.820+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe015a020 con 0x7fefe0143c40 2023-11-24T06:29:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.822+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe01f34b0 con 0x7fefe0143c40 2023-11-24T06:29:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.823+0000 7fefd6ffd640 1 -- 172.21.15.16:0/1373151292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fefd80039b0 con 0x7fefe0143c40 2023-11-24T06:29:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.824+0000 7fefd6ffd640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefc406aed0 0x7fefc406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.824+0000 7fefd6ffd640 1 -- 172.21.15.16:0/1373151292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefd80ab6e0 con 0x7fefe0143c40 2023-11-24T06:29:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.824+0000 7fefe6237640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefc406aed0 0x7fefc406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.825+0000 7fefe6237640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefc406aed0 0x7fefc406d390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0073f0 tx=0x7fefdc00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:21.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:21.828+0000 7fefd6ffd640 1 -- 172.21.15.16:0/1373151292 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fefe01f34b0 con 0x7fefe0143c40 2023-11-24T06:29:22.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.052+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol6306", "group_name": "subvol_grp5190", "target": ["mon-mgr", ""]}) v1 -- 0x7fefe006cc40 con 0x7fefc406aed0 2023-11-24T06:29:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.112+0000 7fefd6ffd640 1 -- 172.21.15.16:0/1373151292 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fefdc00f050 con 0x7fefc406aed0 2023-11-24T06:29:22.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefc406aed0 msgr2=0x7fefc406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefc406aed0 0x7fefc406d390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0073f0 tx=0x7fefdc00b040 comp rx=0 tx=0).stop 2023-11-24T06:29:22.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0143c40 msgr2=0x7fefe00a8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0143c40 0x7fefe00a8bb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fefd80097a0 tx=0x7fefd80046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 shutdown_connections 2023-11-24T06:29:22.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefc406aed0 0x7fefc406d390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe0143c40 0x7fefe00a8bb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe00ccc10 0x7fefe01f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 --2- 172.21.15.16:0/1373151292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe00717d0 0x7fefe0159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 >> 172.21.15.16:0/1373151292 conn(0x7fefe006a800 msgr2=0x7fefe00afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:22.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 shutdown_connections 2023-11-24T06:29:22.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.115+0000 7fefe7239640 1 -- 172.21.15.16:0/1373151292 wait complete. 2023-11-24T06:29:22.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol6306 subvol_snap5952 subvol_grp5190 2023-11-24T06:29:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.368+0000 7ff194def640 1 Processor -- start 2023-11-24T06:29:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.368+0000 7ff194def640 1 -- start start 2023-11-24T06:29:22.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.370+0000 7ff194def640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff190144410 0x7ff19006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.370+0000 7ff194def640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff190144870 0x7ff19006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.370+0000 7ff194def640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1900acda0 con 0x7ff1900b2600 2023-11-24T06:29:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.370+0000 7ff194def640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1900af740 con 0x7ff190144410 2023-11-24T06:29:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.370+0000 7ff194def640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1900ad5b0 con 0x7ff190144870 2023-11-24T06:29:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.370+0000 7ff18f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff190144410 0x7ff19006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.371+0000 7ff18f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff190144410 0x7ff19006a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49728/0 (socket says 172.21.15.16:49728) 2023-11-24T06:29:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.371+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1973284774 learned_addr learned my addr 172.21.15.16:0/1973284774 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.371+0000 7ff18ffff640 1 --2- 172.21.15.16:0/1973284774 >> v2:172.21.15.173:3300/0 conn(0x7ff190144870 0x7ff19006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.371+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1973284774 >> v2:172.21.15.173:3300/0 conn(0x7ff190144870 msgr2=0x7ff19006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.371+0000 7ff18f7fe640 1 --2- 172.21.15.16:0/1973284774 >> v2:172.21.15.173:3300/0 conn(0x7ff190144870 0x7ff19006a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.371+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1973284774 >> v1:172.21.15.173:6789/0 conn(0x7ff1900b2600 legacy=0x7ff190071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.371+0000 7ff18f7fe640 1 -- 172.21.15.16:0/1973284774 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900b3240 con 0x7ff190144410 2023-11-24T06:29:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.372+0000 7ff18f7fe640 1 --2- 172.21.15.16:0/1973284774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff190144410 0x7ff19006a3e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff19006cb90 tx=0x7ff18002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d391355a77082b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff18effd640 1 -- 172.21.15.16:0/1973284774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18002ca80 con 0x7ff190144410 2023-11-24T06:29:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff18effd640 1 -- 172.21.15.16:0/1973284774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18002cbe0 con 0x7ff190144410 2023-11-24T06:29:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff18effd640 1 -- 172.21.15.16:0/1973284774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1800343f0 con 0x7ff190144410 2023-11-24T06:29:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff194def640 1 -- 172.21.15.16:0/1973284774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff190144410 msgr2=0x7ff19006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff194def640 1 --2- 172.21.15.16:0/1973284774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff190144410 0x7ff19006a3e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7ff19006cb90 tx=0x7ff18002bee0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff194def640 1 -- 172.21.15.16:0/1973284774 shutdown_connections 2023-11-24T06:29:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff194def640 1 --2- 172.21.15.16:0/1973284774 >> v2:172.21.15.173:3300/0 conn(0x7ff190144870 0x7ff19006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff194def640 1 --2- 172.21.15.16:0/1973284774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff190144410 0x7ff19006a3e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff194def640 1 -- 172.21.15.16:0/1973284774 >> 172.21.15.16:0/1973284774 conn(0x7ff1900b19c0 msgr2=0x7ff1900b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:22.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff194def640 1 -- 172.21.15.16:0/1973284774 shutdown_connections 2023-11-24T06:29:22.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.373+0000 7ff194def640 1 -- 172.21.15.16:0/1973284774 wait complete. 2023-11-24T06:29:22.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.375+0000 7ff194def640 1 Processor -- start 2023-11-24T06:29:22.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.375+0000 7ff194def640 1 -- start start 2023-11-24T06:29:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.376+0000 7ff194def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1900b2600 0x7ff19006d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.376+0000 7ff194def640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff190144410 0x7ff190157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.376+0000 7ff194def640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190144870 0x7ff1901582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.376+0000 7ff194def640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19006e2d0 con 0x7ff1900b2600 2023-11-24T06:29:22.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.376+0000 7ff194def640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1901f2e50 con 0x7ff190144870 2023-11-24T06:29:22.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.376+0000 7ff194def640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19006d650 con 0x7ff190144410 2023-11-24T06:29:22.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1900b2600 0x7ff19006d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1900b2600 0x7ff19006d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49744/0 (socket says 172.21.15.16:49744) 2023-11-24T06:29:22.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18f7fe640 1 -- 172.21.15.16:0/4143811689 learned_addr learned my addr 172.21.15.16:0/4143811689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:22.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18ffff640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190144870 0x7ff1901582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18f7fe640 1 -- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff190144410 msgr2=0x7ff190157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff187fff640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff190144410 0x7ff190157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18f7fe640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff190144410 0x7ff190157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18f7fe640 1 -- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190144870 msgr2=0x7ff1901582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18f7fe640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190144870 0x7ff1901582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18f7fe640 1 -- 172.21.15.16:0/4143811689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180009b30 con 0x7ff1900b2600 2023-11-24T06:29:22.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff18ffff640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190144870 0x7ff1901582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:22.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.377+0000 7ff187fff640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff190144410 0x7ff190157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.378+0000 7ff18f7fe640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1900b2600 0x7ff19006d000 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff180000f80 tx=0x7ff180004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.378+0000 7ff18cff9640 1 -- 172.21.15.16:0/4143811689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180009510 con 0x7ff1900b2600 2023-11-24T06:29:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.378+0000 7ff18cff9640 1 -- 172.21.15.16:0/4143811689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18003e450 con 0x7ff1900b2600 2023-11-24T06:29:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.378+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19006d930 con 0x7ff1900b2600 2023-11-24T06:29:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.378+0000 7ff18cff9640 1 -- 172.21.15.16:0/4143811689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18003e800 con 0x7ff1900b2600 2023-11-24T06:29:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.378+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19006def0 con 0x7ff1900b2600 2023-11-24T06:29:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.379+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff190067b60 con 0x7ff1900b2600 2023-11-24T06:29:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.381+0000 7ff18cff9640 1 -- 172.21.15.16:0/4143811689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff18002cd50 con 0x7ff1900b2600 2023-11-24T06:29:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.381+0000 7ff18cff9640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff17006af00 0x7ff17006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.382+0000 7ff18cff9640 1 -- 172.21.15.16:0/4143811689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1800ac250 con 0x7ff1900b2600 2023-11-24T06:29:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.382+0000 7ff187fff640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff17006af00 0x7ff17006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.383+0000 7ff187fff640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff17006af00 0x7ff17006d3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff1901f6490 tx=0x7ff178009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:22.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.387+0000 7ff18cff9640 1 -- 172.21.15.16:0/4143811689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff180077210 con 0x7ff1900b2600 2023-11-24T06:29:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.621+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol6306", "snap_name": "subvol_snap5952", "group_name": "subvol_grp5190", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1900cf7a0 con 0x7ff17006af00 2023-11-24T06:29:22.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.657+0000 7ff18cff9640 1 -- 172.21.15.16:0/4143811689 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff1900cf7a0 con 0x7ff17006af00 2023-11-24T06:29:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff17006af00 msgr2=0x7ff17006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff17006af00 0x7ff17006d3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff1901f6490 tx=0x7ff178009040 comp rx=0 tx=0).stop 2023-11-24T06:29:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1900b2600 msgr2=0x7ff19006d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1900b2600 0x7ff19006d000 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff180000f80 tx=0x7ff180004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 shutdown_connections 2023-11-24T06:29:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff17006af00 0x7ff17006d3c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190144870 0x7ff1901582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff190144410 0x7ff190157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 --2- 172.21.15.16:0/4143811689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1900b2600 0x7ff19006d000 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 >> 172.21.15.16:0/4143811689 conn(0x7ff1900b19c0 msgr2=0x7ff19006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 shutdown_connections 2023-11-24T06:29:22.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.660+0000 7ff194def640 1 -- 172.21.15.16:0/4143811689 wait complete. 2023-11-24T06:29:22.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot metadata set cephfs subvol6306 subvol_snap5952 key value subvol_grp5190 2023-11-24T06:29:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.918+0000 7f34e5933640 1 Processor -- start 2023-11-24T06:29:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.918+0000 7f34e5933640 1 -- start start 2023-11-24T06:29:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.920+0000 7f34e5933640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e0155760 0x7f34e00b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.920+0000 7f34e5933640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e015ed80 con 0x7f34e00b1190 2023-11-24T06:29:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.920+0000 7f34e5933640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e015f020 con 0x7f34e0072200 2023-11-24T06:29:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.920+0000 7f34e5933640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00b1610 con 0x7f34e0155760 2023-11-24T06:29:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.920+0000 7f34d7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f34e00b1190 0x7f34e0071af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42676/0 (socket says 172.21.15.16:42676) 2023-11-24T06:29:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.920+0000 7f34d7fff640 1 -- 172.21.15.16:0/2646436703 learned_addr learned my addr 172.21.15.16:0/2646436703 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34deffd640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e0155760 0x7f34e00b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039363447 0 0) 0x7f34e015ed80 con 0x7f34e00b1190 2023-11-24T06:29:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34cc003520 con 0x7f34e00b1190 2023-11-24T06:29:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2060373161 0 0) 0x7f34cc003520 con 0x7f34e00b1190 2023-11-24T06:29:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e0155760 msgr2=0x7f34e00b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34de7fc640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e0155760 0x7f34e00b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 >> v1:172.21.15.173:6789/0 conn(0x7f34e0072200 legacy=0x7f34e00702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e015f300 con 0x7f34e00b1190 2023-11-24T06:29:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f34c8001670 con 0x7f34e00b1190 2023-11-24T06:29:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.921+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 >> v1:172.21.15.16:6789/0 conn(0x7f34e00b1190 legacy=0x7f34e0071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34de7fc640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34de7fc640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34cc0047c0 0x7f34cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34de7fc640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e0155760 0x7f34cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00b1610 con 0x7f34cc0039a0 2023-11-24T06:29:22.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc00bb10 con 0x7f34cc0047c0 2023-11-24T06:29:22.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc00bcd0 con 0x7f34e0155760 2023-11-24T06:29:22.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34deffd640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34d7fff640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34cc0047c0 0x7f34cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34deffd640 1 -- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e0155760 msgr2=0x7f34cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:22.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34deffd640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e0155760 0x7f34cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34deffd640 1 -- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34cc0047c0 msgr2=0x7f34cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:22.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34deffd640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34cc0047c0 0x7f34cc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.922+0000 7f34deffd640 1 -- 172.21.15.16:0/2646436703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc003520 con 0x7f34cc0039a0 2023-11-24T06:29:22.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.923+0000 7f34deffd640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34cc003dc0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f34d000ba30 tx=0x7f34cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a2806a4ab3694e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:22.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.924+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0032680 con 0x7f34cc0039a0 2023-11-24T06:29:22.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.924+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d00327e0 con 0x7f34cc0039a0 2023-11-24T06:29:22.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.924+0000 7f34de7fc640 1 -- 172.21.15.16:0/2646436703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0032b70 con 0x7f34cc0039a0 2023-11-24T06:29:22.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.924+0000 7f34e5933640 1 -- 172.21.15.16:0/2646436703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 msgr2=0x7f34cc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.924+0000 7f34e5933640 1 --2- 172.21.15.16:0/2646436703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34cc003dc0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f34d000ba30 tx=0x7f34cc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.924+0000 7f34df7fe640 1 -- 172.21.15.16:0/2646436703 reap_dead start 2023-11-24T06:29:22.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.924+0000 7f34e5933640 1 -- 172.21.15.16:0/2646436703 shutdown_connections 2023-11-24T06:29:22.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.924+0000 7f34e5933640 1 -- 172.21.15.16:0/2646436703 >> 172.21.15.16:0/2646436703 conn(0x7f34e006db10 msgr2=0x7f34e006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.925+0000 7f34e5933640 1 -- 172.21.15.16:0/2646436703 shutdown_connections 2023-11-24T06:29:22.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.925+0000 7f34e5933640 1 -- 172.21.15.16:0/2646436703 wait complete. 2023-11-24T06:29:22.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.926+0000 7f34e5933640 1 Processor -- start 2023-11-24T06:29:22.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.927+0000 7f34e5933640 1 -- start start 2023-11-24T06:29:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.927+0000 7f34e5933640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34e0159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.927+0000 7f34e5933640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e01583b0 0x7f34e013f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34e5933640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34e01f2750 0x7f34e01f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34e5933640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e00b3290 con 0x7f34cc0039a0 2023-11-24T06:29:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34e5933640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e01f3110 con 0x7f34e01f2750 2023-11-24T06:29:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34e5933640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0158be0 con 0x7f34e01583b0 2023-11-24T06:29:22.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34e0159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34e0159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59910/0 (socket says 172.21.15.16:59910) 2023-11-24T06:29:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34deffd640 1 -- 172.21.15.16:0/751273977 learned_addr learned my addr 172.21.15.16:0/751273977 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34df7fe640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34e01f2750 0x7f34e01f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34deffd640 1 -- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e01583b0 msgr2=0x7f34e013f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34d7fff640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e01583b0 0x7f34e013f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34deffd640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e01583b0 0x7f34e013f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34deffd640 1 -- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34e01f2750 msgr2=0x7f34e01f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34deffd640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34e01f2750 0x7f34e01f2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.928+0000 7f34deffd640 1 -- 172.21.15.16:0/751273977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00b36b0 con 0x7f34cc0039a0 2023-11-24T06:29:22.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.929+0000 7f34deffd640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34e0159960 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f34d0035040 tx=0x7f34d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.929+0000 7f34dcff9640 1 -- 172.21.15.16:0/751273977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0030aa0 con 0x7f34cc0039a0 2023-11-24T06:29:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.929+0000 7f34dcff9640 1 -- 172.21.15.16:0/751273977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d0030c20 con 0x7f34cc0039a0 2023-11-24T06:29:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.929+0000 7f34dcff9640 1 -- 172.21.15.16:0/751273977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d003b420 con 0x7f34cc0039a0 2023-11-24T06:29:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.929+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e00b28f0 con 0x7f34cc0039a0 2023-11-24T06:29:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.929+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e00b2eb0 con 0x7f34cc0039a0 2023-11-24T06:29:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.930+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e00676e0 con 0x7f34cc0039a0 2023-11-24T06:29:22.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.931+0000 7f34dcff9640 1 -- 172.21.15.16:0/751273977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f34d003b580 con 0x7f34cc0039a0 2023-11-24T06:29:22.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.933+0000 7f34dcff9640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c006aed0 0x7f34c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:22.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.933+0000 7f34dcff9640 1 -- 172.21.15.16:0/751273977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34d00ac0c0 con 0x7f34cc0039a0 2023-11-24T06:29:22.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.933+0000 7f34d7fff640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c006aed0 0x7f34c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:22.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.934+0000 7f34d7fff640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c006aed0 0x7f34c006d390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f34c8001eb0 tx=0x7f34c8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:22.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:22.937+0000 7f34dcff9640 1 -- 172.21.15.16:0/751273977 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34d0077120 con 0x7f34cc0039a0 2023-11-24T06:29:23.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.052+0000 7f34dcff9640 1 -- 172.21.15.16:0/751273977 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f34d00774f0 con 0x7f34cc0039a0 2023-11-24T06:29:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.172+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot metadata set", "vol_name": "cephfs", "sub_name": "subvol6306", "snap_name": "subvol_snap5952", "key_name": "key", "value": "value", "group_name": "subvol_grp5190", "target": ["mon-mgr", ""]}) v1 -- 0x7f34e01482b0 con 0x7f34c006aed0 2023-11-24T06:29:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.208+0000 7f34dcff9640 1 -- 172.21.15.16:0/751273977 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f34e01482b0 con 0x7f34c006aed0 2023-11-24T06:29:23.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c006aed0 msgr2=0x7f34c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:23.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c006aed0 0x7f34c006d390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f34c8001eb0 tx=0x7f34c8009040 comp rx=0 tx=0).stop 2023-11-24T06:29:23.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 msgr2=0x7f34e0159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34e0159960 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f34d0035040 tx=0x7f34d0004730 comp rx=0 tx=0).stop 2023-11-24T06:29:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 shutdown_connections 2023-11-24T06:29:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c006aed0 0x7f34c006d390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34e01f2750 0x7f34e01f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34e01583b0 0x7f34e013f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 --2- 172.21.15.16:0/751273977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34cc0039a0 0x7f34e0159960 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 >> 172.21.15.16:0/751273977 conn(0x7f34e006db10 msgr2=0x7f34e015d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 shutdown_connections 2023-11-24T06:29:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:23.211+0000 7f34e5933640 1 -- 172.21.15.16:0/751273977 wait complete. 2023-11-24T06:29:23.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:23.233 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp5190/subvol6306/.meta ./volumes/subvol_grp5190/subvol6306/.meta.stale_snap_section) 2023-11-24T06:29:23.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:29:23.355 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:29:25.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol6306 subvol_snap5952 subvol_grp5190 2023-11-24T06:29:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.278+0000 7fc07487b640 1 Processor -- start 2023-11-24T06:29:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.278+0000 7fc07487b640 1 -- start start 2023-11-24T06:29:25.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.279+0000 7fc07487b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc07013cfa0 0x7fc07015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.279+0000 7fc07487b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc07013fcb0 0x7fc0700b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.279+0000 7fc07487b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0700cdec0 con 0x7fc0700cd530 2023-11-24T06:29:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.279+0000 7fc07487b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0700ce1a0 con 0x7fc07013fcb0 2023-11-24T06:29:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.279+0000 7fc07487b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0700ce480 con 0x7fc07013cfa0 2023-11-24T06:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc06f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc07013fcb0 0x7fc0700b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc067fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc0700cd530 0x7fc07013f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42678/0 (socket says 172.21.15.16:42678) 2023-11-24T06:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc06f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc07013fcb0 0x7fc0700b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59914/0 (socket says 172.21.15.16:59914) 2023-11-24T06:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc067fff640 1 -- 172.21.15.16:0/2236093989 learned_addr learned my addr 172.21.15.16:0/2236093989 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc06ed9d640 1 --2- 172.21.15.16:0/2236093989 >> v2:172.21.15.173:3300/0 conn(0x7fc07013cfa0 0x7fc07015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc06f59e640 1 -- 172.21.15.16:0/2236093989 >> v2:172.21.15.173:3300/0 conn(0x7fc07013cfa0 msgr2=0x7fc07015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc06f59e640 1 --2- 172.21.15.16:0/2236093989 >> v2:172.21.15.173:3300/0 conn(0x7fc07013cfa0 0x7fc07015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc06f59e640 1 -- 172.21.15.16:0/2236093989 >> v1:172.21.15.16:6789/0 conn(0x7fc0700cd530 legacy=0x7fc07013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc06f59e640 1 -- 172.21.15.16:0/2236093989 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0700ce760 con 0x7fc07013fcb0 2023-11-24T06:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.280+0000 7fc06ed9d640 1 --2- 172.21.15.16:0/2236093989 >> v2:172.21.15.173:3300/0 conn(0x7fc07013cfa0 0x7fc07015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.282+0000 7fc06f59e640 1 --2- 172.21.15.16:0/2236093989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 0x7fc0700b2040 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc07006cf00 tx=0x7fc06802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64beede852f02ca5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.282+0000 7fc06e59c640 1 -- 172.21.15.16:0/2236093989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06802cbe0 con 0x7fc07013fcb0 2023-11-24T06:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.282+0000 7fc06e59c640 1 -- 172.21.15.16:0/2236093989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06802cd40 con 0x7fc07013fcb0 2023-11-24T06:29:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.282+0000 7fc06e59c640 1 -- 172.21.15.16:0/2236093989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068034660 con 0x7fc07013fcb0 2023-11-24T06:29:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.283+0000 7fc07487b640 1 -- 172.21.15.16:0/2236093989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 msgr2=0x7fc0700b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.283+0000 7fc07487b640 1 --2- 172.21.15.16:0/2236093989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 0x7fc0700b2040 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc07006cf00 tx=0x7fc06802be30 comp rx=0 tx=0).stop 2023-11-24T06:29:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.283+0000 7fc07487b640 1 -- 172.21.15.16:0/2236093989 shutdown_connections 2023-11-24T06:29:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.283+0000 7fc07487b640 1 --2- 172.21.15.16:0/2236093989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 0x7fc0700b2040 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.283+0000 7fc07487b640 1 --2- 172.21.15.16:0/2236093989 >> v2:172.21.15.173:3300/0 conn(0x7fc07013cfa0 0x7fc07015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.283+0000 7fc07487b640 1 -- 172.21.15.16:0/2236093989 >> 172.21.15.16:0/2236093989 conn(0x7fc07006a800 msgr2=0x7fc07006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.283+0000 7fc07487b640 1 -- 172.21.15.16:0/2236093989 shutdown_connections 2023-11-24T06:29:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.283+0000 7fc07487b640 1 -- 172.21.15.16:0/2236093989 wait complete. 2023-11-24T06:29:25.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.285+0000 7fc07487b640 1 Processor -- start 2023-11-24T06:29:25.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.285+0000 7fc07487b640 1 -- start start 2023-11-24T06:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.286+0000 7fc07487b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0700cd530 0x7fc070157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.286+0000 7fc07487b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc07013cfa0 0x7fc0701f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.286+0000 7fc07487b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 0x7fc07013bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.286+0000 7fc07487b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc070137ef0 con 0x7fc07013fcb0 2023-11-24T06:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.286+0000 7fc07487b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc070157750 con 0x7fc0700cd530 2023-11-24T06:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.286+0000 7fc07487b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc070156df0 con 0x7fc07013cfa0 2023-11-24T06:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.286+0000 7fc06f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 0x7fc07013bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.286+0000 7fc06f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 0x7fc07013bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59924/0 (socket says 172.21.15.16:59924) 2023-11-24T06:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.286+0000 7fc06f59e640 1 -- 172.21.15.16:0/3661596073 learned_addr learned my addr 172.21.15.16:0/3661596073 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.287+0000 7fc06ed9d640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0700cd530 0x7fc070157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.287+0000 7fc067fff640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc07013cfa0 0x7fc0701f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.287+0000 7fc06f59e640 1 -- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc07013cfa0 msgr2=0x7fc0701f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.287+0000 7fc06f59e640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc07013cfa0 0x7fc0701f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.287+0000 7fc06f59e640 1 -- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0700cd530 msgr2=0x7fc070157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.287+0000 7fc06f59e640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0700cd530 0x7fc070157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.287+0000 7fc06f59e640 1 -- 172.21.15.16:0/3661596073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068009a70 con 0x7fc07013fcb0 2023-11-24T06:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.287+0000 7fc06f59e640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 0x7fc07013bf20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc06802c440 tx=0x7fc068004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.288+0000 7fc0677fe640 1 -- 172.21.15.16:0/3661596073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068040650 con 0x7fc07013fcb0 2023-11-24T06:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.288+0000 7fc0677fe640 1 -- 172.21.15.16:0/3661596073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0680407d0 con 0x7fc07013fcb0 2023-11-24T06:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.288+0000 7fc0677fe640 1 -- 172.21.15.16:0/3661596073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068040b80 con 0x7fc07013fcb0 2023-11-24T06:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.288+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0701f2830 con 0x7fc07013fcb0 2023-11-24T06:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.288+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0701f2df0 con 0x7fc07013fcb0 2023-11-24T06:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.289+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0701380a0 con 0x7fc07013fcb0 2023-11-24T06:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.290+0000 7fc0677fe640 1 -- 172.21.15.16:0/3661596073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fc068040ce0 con 0x7fc07013fcb0 2023-11-24T06:29:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.291+0000 7fc0677fe640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc05006af60 0x7fc05006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.291+0000 7fc0677fe640 1 -- 172.21.15.16:0/3661596073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0680aca50 con 0x7fc07013fcb0 2023-11-24T06:29:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.291+0000 7fc06ed9d640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc05006af60 0x7fc05006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:25.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.293+0000 7fc06ed9d640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc05006af60 0x7fc05006d420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc06000a990 tx=0x7fc060008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:25.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.295+0000 7fc0677fe640 1 -- 172.21.15.16:0/3661596073 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc068077b00 con 0x7fc07013fcb0 2023-11-24T06:29:25.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.534+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol6306", "snap_name": "subvol_snap5952", "group_name": "subvol_grp5190", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0700b2580 con 0x7fc05006af60 2023-11-24T06:29:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.608+0000 7fc0677fe640 1 -- 172.21.15.16:0/3661596073 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc06000a660 con 0x7fc05006af60 2023-11-24T06:29:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc05006af60 msgr2=0x7fc05006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc05006af60 0x7fc05006d420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc06000a990 tx=0x7fc060008040 comp rx=0 tx=0).stop 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 msgr2=0x7fc07013bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 0x7fc07013bf20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc06802c440 tx=0x7fc068004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 shutdown_connections 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc05006af60 0x7fc05006d420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc07013fcb0 0x7fc07013bf20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc07013cfa0 0x7fc0701f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 --2- 172.21.15.16:0/3661596073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0700cd530 0x7fc070157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 >> 172.21.15.16:0/3661596073 conn(0x7fc07006a800 msgr2=0x7fc0700aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 shutdown_connections 2023-11-24T06:29:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:25.612+0000 7fc07487b640 1 -- 172.21.15.16:0/3661596073 wait complete. 2023-11-24T06:29:25.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:25.638 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp5190/subvol6306/.meta.stale_snap_section ./volumes/subvol_grp5190/subvol6306/.meta) 2023-11-24T06:29:25.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:25.761 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap5952 ./volumes/subvol_grp5190/subvol6306/.meta) 2023-11-24T06:29:25.861 INFO:teuthology.orchestra.run.smithi016.stdout:[SNAP_METADATA_subvol_snap5952] 2023-11-24T06:29:25.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol6306 subvol_grp5190 2023-11-24T06:29:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.084+0000 7f14a89b3640 1 Processor -- start 2023-11-24T06:29:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.084+0000 7f14a89b3640 1 -- start start 2023-11-24T06:29:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.086+0000 7f14a89b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14a4143c40 0x7f14a4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.086+0000 7f14a89b3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40ccc10 0x7f14a406d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.086+0000 7f14a89b3640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a4070e50 con 0x7f14a40717d0 2023-11-24T06:29:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.086+0000 7f14a89b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a40723c0 con 0x7f14a4143c40 2023-11-24T06:29:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.086+0000 7f14a89b3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a4144020 con 0x7f14a40ccc10 2023-11-24T06:29:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.086+0000 7f14a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14a4143c40 0x7f14a4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.086+0000 7f14a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14a4143c40 0x7f14a4071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59930/0 (socket says 172.21.15.16:59930) 2023-11-24T06:29:26.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.086+0000 7f14a37fe640 1 -- 172.21.15.16:0/222836137 learned_addr learned my addr 172.21.15.16:0/222836137 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.087+0000 7f14a3fff640 1 --2- 172.21.15.16:0/222836137 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40ccc10 0x7f14a406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.087+0000 7f14a37fe640 1 -- 172.21.15.16:0/222836137 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40ccc10 msgr2=0x7f14a406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.087+0000 7f14a37fe640 1 --2- 172.21.15.16:0/222836137 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40ccc10 0x7f14a406d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.087+0000 7f14a37fe640 1 -- 172.21.15.16:0/222836137 >> v1:172.21.15.173:6789/0 conn(0x7f14a40717d0 legacy=0x7f14a40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.087+0000 7f14a37fe640 1 -- 172.21.15.16:0/222836137 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a4071e00 con 0x7f14a4143c40 2023-11-24T06:29:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.087+0000 7f14a3fff640 1 --2- 172.21.15.16:0/222836137 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40ccc10 0x7f14a406d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:26.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.088+0000 7f14a37fe640 1 --2- 172.21.15.16:0/222836137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 0x7f14a4071290 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1494009870 tx=0x7f149402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c47460ec5745c68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.088+0000 7f14a2ffd640 1 -- 172.21.15.16:0/222836137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14940317c0 con 0x7f14a4143c40 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.088+0000 7f14a2ffd640 1 -- 172.21.15.16:0/222836137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1494031920 con 0x7f14a4143c40 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.088+0000 7f14a2ffd640 1 -- 172.21.15.16:0/222836137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494031cb0 con 0x7f14a4143c40 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.089+0000 7f14a89b3640 1 -- 172.21.15.16:0/222836137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 msgr2=0x7f14a4071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.089+0000 7f14a89b3640 1 --2- 172.21.15.16:0/222836137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 0x7f14a4071290 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1494009870 tx=0x7f149402b860 comp rx=0 tx=0).stop 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.089+0000 7f14a89b3640 1 -- 172.21.15.16:0/222836137 shutdown_connections 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.089+0000 7f14a89b3640 1 --2- 172.21.15.16:0/222836137 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40ccc10 0x7f14a406d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.089+0000 7f14a89b3640 1 --2- 172.21.15.16:0/222836137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 0x7f14a4071290 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.089+0000 7f14a89b3640 1 -- 172.21.15.16:0/222836137 >> 172.21.15.16:0/222836137 conn(0x7f14a406a800 msgr2=0x7f14a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.089+0000 7f14a89b3640 1 -- 172.21.15.16:0/222836137 shutdown_connections 2023-11-24T06:29:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.089+0000 7f14a89b3640 1 -- 172.21.15.16:0/222836137 wait complete. 2023-11-24T06:29:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.091+0000 7f14a89b3640 1 Processor -- start 2023-11-24T06:29:26.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.091+0000 7f14a89b3640 1 -- start start 2023-11-24T06:29:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.092+0000 7f14a89b3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40717d0 0x7f14a41f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.092+0000 7f14a89b3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14a40ccc10 0x7f14a415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.092+0000 7f14a89b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 0x7f14a40af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.092+0000 7f14a89b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a4131dd0 con 0x7f14a4143c40 2023-11-24T06:29:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.092+0000 7f14a89b3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a415af60 con 0x7f14a40ccc10 2023-11-24T06:29:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.092+0000 7f14a89b3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a415ac80 con 0x7f14a40717d0 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.092+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 0x7f14a40af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.092+0000 7f14a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 0x7f14a40af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59938/0 (socket says 172.21.15.16:59938) 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.092+0000 7f14a3fff640 1 -- 172.21.15.16:0/3303964472 learned_addr learned my addr 172.21.15.16:0/3303964472 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.093+0000 7f149bfff640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14a40ccc10 0x7f14a415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.093+0000 7f14a37fe640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40717d0 0x7f14a41f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.093+0000 7f14a3fff640 1 -- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40717d0 msgr2=0x7f14a41f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.093+0000 7f14a3fff640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40717d0 0x7f14a41f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.093+0000 7f14a3fff640 1 -- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14a40ccc10 msgr2=0x7f14a415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.093+0000 7f14a3fff640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14a40ccc10 0x7f14a415a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.093+0000 7f14a3fff640 1 -- 172.21.15.16:0/3303964472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940094d0 con 0x7f14a4143c40 2023-11-24T06:29:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.093+0000 7f14a3fff640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 0x7f14a40af5b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f149c00cef0 tx=0x7f149c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.094+0000 7f14a0ff9640 1 -- 172.21.15.16:0/3303964472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00d840 con 0x7f14a4143c40 2023-11-24T06:29:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.094+0000 7f14a0ff9640 1 -- 172.21.15.16:0/3303964472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c00d9c0 con 0x7f14a4143c40 2023-11-24T06:29:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.094+0000 7f14a0ff9640 1 -- 172.21.15.16:0/3303964472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00dd70 con 0x7f14a4143c40 2023-11-24T06:29:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.094+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a4132000 con 0x7f14a4143c40 2023-11-24T06:29:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.094+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a41325c0 con 0x7f14a4143c40 2023-11-24T06:29:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.095+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a4067450 con 0x7f14a4143c40 2023-11-24T06:29:26.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.096+0000 7f14a0ff9640 1 -- 172.21.15.16:0/3303964472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f149c01e3b0 con 0x7f14a4143c40 2023-11-24T06:29:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.097+0000 7f14a0ff9640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f148406afb0 0x7f148406d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.097+0000 7f14a0ff9640 1 -- 172.21.15.16:0/3303964472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f149c08a2d0 con 0x7f14a4143c40 2023-11-24T06:29:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.098+0000 7f14a37fe640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f148406afb0 0x7f148406d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.099+0000 7f14a37fe640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f148406afb0 0x7f148406d470 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1494036000 tx=0x7f149403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.101+0000 7f14a0ff9640 1 -- 172.21.15.16:0/3303964472 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f149c055310 con 0x7f14a4143c40 2023-11-24T06:29:26.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.327+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol6306", "group_name": "subvol_grp5190", "target": ["mon-mgr", ""]}) v1 -- 0x7f14a406cc40 con 0x7f148406afb0 2023-11-24T06:29:26.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.395+0000 7f14a0ff9640 1 -- 172.21.15.16:0/3303964472 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+704 (secure 0 0 0) 0x7f1494031c10 con 0x7f148406afb0 2023-11-24T06:29:26.399 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:29:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "atime": "2023-11-24 06:29:22", 2023-11-24T06:29:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "bytes_pcent": "undefined", 2023-11-24T06:29:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "bytes_quota": "infinite", 2023-11-24T06:29:26.399 INFO:teuthology.orchestra.run.smithi016.stdout: "bytes_used": 0, 2023-11-24T06:29:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "created_at": "2023-11-24 06:29:22", 2023-11-24T06:29:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "ctime": "2023-11-24 06:29:22", 2023-11-24T06:29:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "data_pool": "cephfs_data", 2023-11-24T06:29:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "features": [ 2023-11-24T06:29:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot-clone", 2023-11-24T06:29:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot-autoprotect", 2023-11-24T06:29:26.400 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot-retention" 2023-11-24T06:29:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-11-24T06:29:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 0, 2023-11-24T06:29:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": 16877, 2023-11-24T06:29:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mon_addrs": [ 2023-11-24T06:29:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "172.21.15.16:6789", 2023-11-24T06:29:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "172.21.15.173:6789", 2023-11-24T06:29:26.401 INFO:teuthology.orchestra.run.smithi016.stdout: "172.21.15.173:6790" 2023-11-24T06:29:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-11-24T06:29:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "mtime": "2023-11-24 06:29:22", 2023-11-24T06:29:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/volumes/subvol_grp5190/subvol6306/703d4fd0-80ee-43bf-bef0-de83042490c7", 2023-11-24T06:29:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_namespace": "", 2023-11-24T06:29:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "complete", 2023-11-24T06:29:26.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "subvolume", 2023-11-24T06:29:26.403 INFO:teuthology.orchestra.run.smithi016.stdout: "uid": 0 2023-11-24T06:29:26.403 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:29:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f148406afb0 msgr2=0x7f148406d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f148406afb0 0x7f148406d470 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1494036000 tx=0x7f149403d040 comp rx=0 tx=0).stop 2023-11-24T06:29:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 msgr2=0x7f14a40af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 0x7f14a40af5b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f149c00cef0 tx=0x7f149c00a710 comp rx=0 tx=0).stop 2023-11-24T06:29:26.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 shutdown_connections 2023-11-24T06:29:26.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f148406afb0 0x7f148406d470 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14a4143c40 0x7f14a40af5b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14a40ccc10 0x7f14a415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 --2- 172.21.15.16:0/3303964472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14a40717d0 0x7f14a41f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 >> 172.21.15.16:0/3303964472 conn(0x7f14a406a800 msgr2=0x7f14a40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:26.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.399+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 shutdown_connections 2023-11-24T06:29:26.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.400+0000 7f14a89b3640 1 -- 172.21.15.16:0/3303964472 wait complete. 2023-11-24T06:29:26.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:26.423 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap5952 ./volumes/subvol_grp5190/subvol6306/.meta) 2023-11-24T06:29:26.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:29:26.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol6306 subvol_grp5190 2023-11-24T06:29:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.765+0000 7f0ff1be4640 1 Processor -- start 2023-11-24T06:29:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.765+0000 7f0ff1be4640 1 -- start start 2023-11-24T06:29:26.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.767+0000 7f0ff1be4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec143c40 0x7f0fec071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.767+0000 7f0ff1be4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fec070e50 con 0x7f0fec0717d0 2023-11-24T06:29:26.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.767+0000 7f0ff1be4640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fec144020 con 0x7f0fec0ccc10 2023-11-24T06:29:26.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.767+0000 7f0ff1be4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0723c0 con 0x7f0fec143c40 2023-11-24T06:29:26.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.767+0000 7f0febfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0fec0717d0 0x7f0fec0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42692/0 (socket says 172.21.15.16:42692) 2023-11-24T06:29:26.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.767+0000 7f0febfff640 1 -- 172.21.15.16:0/2871002077 learned_addr learned my addr 172.21.15.16:0/2871002077 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:26.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.767+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583472777 0 0) 0x7f0fec070e50 con 0x7f0fec0717d0 2023-11-24T06:29:26.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fdc003520 con 0x7f0fec0717d0 2023-11-24T06:29:26.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0ff0be2640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec143c40 0x7f0fec071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 389049371 0 0) 0x7f0fdc003520 con 0x7f0fec0717d0 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec143c40 msgr2=0x7f0fec071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec143c40 0x7f0fec071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 >> v1:172.21.15.173:6789/0 conn(0x7f0fec0ccc10 legacy=0x7f0fec06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec071e00 con 0x7f0fec0717d0 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f0fd8001670 con 0x7f0fec0717d0 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 >> v1:172.21.15.16:6789/0 conn(0x7f0fec0717d0 legacy=0x7f0fec0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fdc0039a0 0x7f0fdc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fdc0047c0 0x7f0fdc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec143c40 0x7f0fdc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0723c0 con 0x7f0fdc0039a0 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc0054e0 con 0x7f0fdc0047c0 2023-11-24T06:29:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc00bec0 con 0x7f0fec143c40 2023-11-24T06:29:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.768+0000 7f0ff0be2640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fdc0039a0 0x7f0fdc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.769+0000 7f0febfff640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fdc0047c0 0x7f0fdc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.769+0000 7f0ff0be2640 1 -- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec143c40 msgr2=0x7f0fdc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.769+0000 7f0ff0be2640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec143c40 0x7f0fdc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.769+0000 7f0ff0be2640 1 -- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fdc0047c0 msgr2=0x7f0fdc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.769+0000 7f0ff0be2640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fdc0047c0 0x7f0fdc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.769+0000 7f0ff0be2640 1 -- 172.21.15.16:0/2871002077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc003520 con 0x7f0fdc0039a0 2023-11-24T06:29:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.770+0000 7f0ff0be2640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fdc0039a0 0x7f0fdc003dc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000ba30 tx=0x7f0fdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad18c05ab398a867 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.770+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0032680 con 0x7f0fdc0039a0 2023-11-24T06:29:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.770+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe00327e0 con 0x7f0fdc0039a0 2023-11-24T06:29:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.770+0000 7f0feb7fe640 1 -- 172.21.15.16:0/2871002077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0032b70 con 0x7f0fdc0039a0 2023-11-24T06:29:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.771+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2871002077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fdc0039a0 msgr2=0x7f0fdc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.771+0000 7f0ff1be4640 1 --2- 172.21.15.16:0/2871002077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fdc0039a0 0x7f0fdc003dc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000ba30 tx=0x7f0fdc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.771+0000 7f0ff13e3640 1 -- 172.21.15.16:0/2871002077 reap_dead start 2023-11-24T06:29:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.771+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2871002077 shutdown_connections 2023-11-24T06:29:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.771+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2871002077 >> 172.21.15.16:0/2871002077 conn(0x7f0fec06a800 msgr2=0x7f0fec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.771+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2871002077 shutdown_connections 2023-11-24T06:29:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.771+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2871002077 wait complete. 2023-11-24T06:29:26.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.773+0000 7f0ff1be4640 1 Processor -- start 2023-11-24T06:29:26.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.773+0000 7f0ff1be4640 1 -- start start 2023-11-24T06:29:26.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.774+0000 7f0ff1be4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fdc0039a0 0x7f0fec12b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.774+0000 7f0ff1be4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec0c7250 0x7f0fec0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.774+0000 7f0ff1be4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec142120 0x7f0fec1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.774+0000 7f0ff1be4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec12b880 con 0x7f0fec142120 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.774+0000 7f0ff1be4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec141380 con 0x7f0fdc0039a0 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.774+0000 7f0ff1be4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0c7bd0 con 0x7f0fec0c7250 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.774+0000 7f0ff13e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec142120 0x7f0fec1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.774+0000 7f0ff13e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec142120 0x7f0fec1425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59960/0 (socket says 172.21.15.16:59960) 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.774+0000 7f0ff13e3640 1 -- 172.21.15.16:0/2367678144 learned_addr learned my addr 172.21.15.16:0/2367678144 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0ff0be2640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fdc0039a0 0x7f0fec12b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0febfff640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec0c7250 0x7f0fec0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0ff13e3640 1 -- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec0c7250 msgr2=0x7f0fec0c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0ff13e3640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec0c7250 0x7f0fec0c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0ff13e3640 1 -- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fdc0039a0 msgr2=0x7f0fec12b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0ff13e3640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fdc0039a0 0x7f0fec12b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0ff13e3640 1 -- 172.21.15.16:0/2367678144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec0caae0 con 0x7f0fec142120 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0ff13e3640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec142120 0x7f0fec1425a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0fec06e410 tx=0x7f0fe400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0fe97fa640 1 -- 172.21.15.16:0/2367678144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4005d40 con 0x7f0fec142120 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0fe97fa640 1 -- 172.21.15.16:0/2367678144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe400ced0 con 0x7f0fec142120 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.775+0000 7f0fe97fa640 1 -- 172.21.15.16:0/2367678144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400a850 con 0x7f0fec142120 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.776+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec141640 con 0x7f0fec142120 2023-11-24T06:29:26.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.776+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec141c00 con 0x7f0fec142120 2023-11-24T06:29:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.777+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec141e20 con 0x7f0fec142120 2023-11-24T06:29:26.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.778+0000 7f0fe97fa640 1 -- 172.21.15.16:0/2367678144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f0fe400a9b0 con 0x7f0fec142120 2023-11-24T06:29:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.779+0000 7f0fe97fa640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0fd006af90 0x7f0fd006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.779+0000 7f0fe97fa640 1 -- 172.21.15.16:0/2367678144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fe4088a30 con 0x7f0fec142120 2023-11-24T06:29:26.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.779+0000 7f0ff0be2640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0fd006af90 0x7f0fd006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:26.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.781+0000 7f0ff0be2640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0fd006af90 0x7f0fd006d450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000ba00 tx=0x7f0fe003d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:26.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:26.783+0000 7f0fe97fa640 1 -- 172.21.15.16:0/2367678144 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fe40538a0 con 0x7f0fec142120 2023-11-24T06:29:27.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.006+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol6306", "group_name": "subvol_grp5190", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fec06cc40 con 0x7f0fd006af90 2023-11-24T06:29:27.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.041+0000 7f0fe97fa640 1 -- 172.21.15.16:0/2367678144 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0fe0032950 con 0x7f0fd006af90 2023-11-24T06:29:27.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0fd006af90 msgr2=0x7f0fd006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0fd006af90 0x7f0fd006d450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000ba00 tx=0x7f0fe003d040 comp rx=0 tx=0).stop 2023-11-24T06:29:27.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec142120 msgr2=0x7f0fec1425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec142120 0x7f0fec1425a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0fec06e410 tx=0x7f0fe400cb90 comp rx=0 tx=0).stop 2023-11-24T06:29:27.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 shutdown_connections 2023-11-24T06:29:27.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0fd006af90 0x7f0fd006d450 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec142120 0x7f0fec1425a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0fec0c7250 0x7f0fec0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 --2- 172.21.15.16:0/2367678144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0fdc0039a0 0x7f0fec12b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 >> 172.21.15.16:0/2367678144 conn(0x7f0fec06a800 msgr2=0x7f0fec0cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:27.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 shutdown_connections 2023-11-24T06:29:27.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.044+0000 7f0ff1be4640 1 -- 172.21.15.16:0/2367678144 wait complete. 2023-11-24T06:29:27.051 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.049+0000 7fd2b8b04640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.052 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.049+0000 7fd2b8b04640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.052 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.049+0000 7fd2b8b04640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.052 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.049+0000 7fd2b8b04640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.052 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.049+0000 7fd2b8b04640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp5190 2023-11-24T06:29:27.080 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.078+0000 7fd2b7301640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.081 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.078+0000 7fd2b7301640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.081 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.078+0000 7fd2b7301640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.081 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.078+0000 7fd2b7301640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.081 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:29:27.078+0000 7fd2b7301640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:29:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.277+0000 7f2c97bde640 1 Processor -- start 2023-11-24T06:29:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.277+0000 7f2c97bde640 1 -- start start 2023-11-24T06:29:27.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.279+0000 7f2c97bde640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2c90160f80 0x7f2c90161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.279+0000 7f2c97bde640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ced50 0x7f2c900d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.279+0000 7f2c97bde640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c901618a0 con 0x7f2c900ce8f0 2023-11-24T06:29:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.279+0000 7f2c97bde640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c900cf1e0 con 0x7f2c90160f80 2023-11-24T06:29:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.279+0000 7f2c97bde640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900d0de0 con 0x7f2c900ced50 2023-11-24T06:29:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.279+0000 7f2c95179640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2c900ce8f0 0x7f2c900cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42696/0 (socket says 172.21.15.16:42696) 2023-11-24T06:29:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.279+0000 7f2c95179640 1 -- 172.21.15.16:0/1808416968 learned_addr learned my addr 172.21.15.16:0/1808416968 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.280+0000 7f2c9617b640 1 --2- 172.21.15.16:0/1808416968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ced50 0x7f2c900d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.280+0000 7f2c9597a640 1 --2- 172.21.15.16:0/1808416968 >> v2:172.21.15.173:3300/0 conn(0x7f2c90160f80 0x7f2c90161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.280+0000 7f2c94978640 1 -- 172.21.15.16:0/1808416968 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313637739 0 0) 0x7f2c901618a0 con 0x7f2c900ce8f0 2023-11-24T06:29:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.280+0000 7f2c94978640 1 -- 172.21.15.16:0/1808416968 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c7c003520 con 0x7f2c900ce8f0 2023-11-24T06:29:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.280+0000 7f2c9617b640 1 -- 172.21.15.16:0/1808416968 >> v2:172.21.15.173:3300/0 conn(0x7f2c90160f80 msgr2=0x7f2c90161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.280+0000 7f2c9617b640 1 --2- 172.21.15.16:0/1808416968 >> v2:172.21.15.173:3300/0 conn(0x7f2c90160f80 0x7f2c90161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.280+0000 7f2c94978640 1 -- 172.21.15.16:0/1808416968 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 22565835 0 0) 0x7f2c7c003520 con 0x7f2c900ce8f0 2023-11-24T06:29:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.280+0000 7f2c9617b640 1 -- 172.21.15.16:0/1808416968 >> v1:172.21.15.16:6789/0 conn(0x7f2c900ce8f0 legacy=0x7f2c900cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.280+0000 7f2c9617b640 1 -- 172.21.15.16:0/1808416968 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c900d10c0 con 0x7f2c900ced50 2023-11-24T06:29:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.282+0000 7f2c9617b640 1 --2- 172.21.15.16:0/1808416968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ced50 0x7f2c900d02f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009d90 tx=0x7f2c8802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f654e3fea55cd8b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.282+0000 7f2c94978640 1 -- 172.21.15.16:0/1808416968 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8802cc50 con 0x7f2c900ced50 2023-11-24T06:29:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.282+0000 7f2c94978640 1 -- 172.21.15.16:0/1808416968 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8802cdb0 con 0x7f2c900ced50 2023-11-24T06:29:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.282+0000 7f2c94978640 1 -- 172.21.15.16:0/1808416968 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88034640 con 0x7f2c900ced50 2023-11-24T06:29:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.283+0000 7f2c97bde640 1 -- 172.21.15.16:0/1808416968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ced50 msgr2=0x7f2c900d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.283+0000 7f2c97bde640 1 --2- 172.21.15.16:0/1808416968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ced50 0x7f2c900d02f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009d90 tx=0x7f2c8802bea0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.283+0000 7f2c97bde640 1 -- 172.21.15.16:0/1808416968 shutdown_connections 2023-11-24T06:29:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.283+0000 7f2c97bde640 1 --2- 172.21.15.16:0/1808416968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ced50 0x7f2c900d02f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.283+0000 7f2c97bde640 1 --2- 172.21.15.16:0/1808416968 >> v2:172.21.15.173:3300/0 conn(0x7f2c90160f80 0x7f2c90161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.283+0000 7f2c97bde640 1 -- 172.21.15.16:0/1808416968 >> 172.21.15.16:0/1808416968 conn(0x7f2c9009c550 msgr2=0x7f2c900d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.283+0000 7f2c97bde640 1 -- 172.21.15.16:0/1808416968 shutdown_connections 2023-11-24T06:29:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.283+0000 7f2c97bde640 1 -- 172.21.15.16:0/1808416968 wait complete. 2023-11-24T06:29:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.285+0000 7f2c97bde640 1 Processor -- start 2023-11-24T06:29:27.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.285+0000 7f2c97bde640 1 -- start start 2023-11-24T06:29:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.286+0000 7f2c97bde640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ce8f0 0x7f2c90158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.286+0000 7f2c97bde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c900ced50 0x7f2c900ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.286+0000 7f2c97bde640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c90160f80 0x7f2c900b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.286+0000 7f2c97bde640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9015da80 con 0x7f2c900ced50 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.286+0000 7f2c97bde640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9015dd60 con 0x7f2c90160f80 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.286+0000 7f2c97bde640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9015e040 con 0x7f2c900ce8f0 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c95179640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c900ced50 0x7f2c900ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c95179640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c900ced50 0x7f2c900ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59970/0 (socket says 172.21.15.16:59970) 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c95179640 1 -- 172.21.15.16:0/861743457 learned_addr learned my addr 172.21.15.16:0/861743457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c9617b640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c90160f80 0x7f2c900b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c9597a640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ce8f0 0x7f2c90158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c95179640 1 -- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ce8f0 msgr2=0x7f2c90158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c95179640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ce8f0 0x7f2c90158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c95179640 1 -- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c90160f80 msgr2=0x7f2c900b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c95179640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c90160f80 0x7f2c900b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c95179640 1 -- 172.21.15.16:0/861743457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c88009a70 con 0x7f2c900ced50 2023-11-24T06:29:27.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.287+0000 7f2c95179640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c900ced50 0x7f2c900ccca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c3e0 tx=0x7f2c8000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:27.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.288+0000 7f2c867fc640 1 -- 172.21.15.16:0/861743457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80005ce0 con 0x7f2c900ced50 2023-11-24T06:29:27.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.288+0000 7f2c867fc640 1 -- 172.21.15.16:0/861743457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80005e60 con 0x7f2c900ced50 2023-11-24T06:29:27.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.288+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9015e320 con 0x7f2c900ced50 2023-11-24T06:29:27.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.288+0000 7f2c867fc640 1 -- 172.21.15.16:0/861743457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80007310 con 0x7f2c900ced50 2023-11-24T06:29:27.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.288+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c90160290 con 0x7f2c900ced50 2023-11-24T06:29:27.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.289+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9015e840 con 0x7f2c900ced50 2023-11-24T06:29:27.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.290+0000 7f2c867fc640 1 -- 172.21.15.16:0/861743457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f2c80007470 con 0x7f2c900ced50 2023-11-24T06:29:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.292+0000 7f2c867fc640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2c7406af60 0x7f2c7406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.292+0000 7f2c867fc640 1 -- 172.21.15.16:0/861743457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c80088e70 con 0x7f2c900ced50 2023-11-24T06:29:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.293+0000 7f2c9597a640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2c7406af60 0x7f2c7406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.294+0000 7f2c9597a640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2c7406af60 0x7f2c7406d420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009ad0 tx=0x7f2c8c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.297+0000 7f2c867fc640 1 -- 172.21.15.16:0/861743457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c9015e840 con 0x7f2c900ced50 2023-11-24T06:29:27.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.525+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp5190", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c900d1960 con 0x7f2c7406af60 2023-11-24T06:29:27.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.536+0000 7f2c867fc640 1 -- 172.21.15.16:0/861743457 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2c8c009800 con 0x7f2c7406af60 2023-11-24T06:29:27.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.538+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2c7406af60 msgr2=0x7f2c7406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.538+0000 7f2c97bde640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2c7406af60 0x7f2c7406d420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009ad0 tx=0x7f2c8c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:27.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c900ced50 msgr2=0x7f2c900ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:27.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c900ced50 0x7f2c900ccca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c3e0 tx=0x7f2c8000c8a0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 shutdown_connections 2023-11-24T06:29:27.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2c7406af60 0x7f2c7406d420 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c90160f80 0x7f2c900b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c900ced50 0x7f2c900ccca0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 --2- 172.21.15.16:0/861743457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c900ce8f0 0x7f2c90158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:27.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 >> 172.21.15.16:0/861743457 conn(0x7f2c9009c550 msgr2=0x7f2c9006c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:27.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 shutdown_connections 2023-11-24T06:29:27.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:27.539+0000 7f2c97bde640 1 -- 172.21.15.16:0/861743457 wait complete. 2023-11-24T06:29:27.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:27.562 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T06:29:27.633 INFO:teuthology.orchestra.run.smithi016.stdout:2 2023-11-24T06:29:27.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:27.634 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rm -f ./volumes/subvol_grp5190/subvol6306/.meta.stale_snap_section) 2023-11-24T06:29:27.740 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:29:27.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:29:27.863 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:29:28.007 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:29:28.007 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:29:28.007 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:28.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.291+0000 7fd0fa1c5640 1 Processor -- start 2023-11-24T06:29:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.291+0000 7fd0fa1c5640 1 -- start start 2023-11-24T06:29:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.292+0000 7fd0fa1c5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 0x7fd0f415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.292+0000 7fd0fa1c5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0f413e360 0x7fd0f40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.292+0000 7fd0fa1c5640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd0f415f9c0 0x7fd0f40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.292+0000 7fd0fa1c5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f406c470 con 0x7fd0f413e360 2023-11-24T06:29:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.292+0000 7fd0fa1c5640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f406c750 con 0x7fd0f415f9c0 2023-11-24T06:29:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.292+0000 7fd0fa1c5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f406ca30 con 0x7fd0f413df80 2023-11-24T06:29:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.292+0000 7fd0f2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0f413e360 0x7fd0f40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.293+0000 7fd0f2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0f413e360 0x7fd0f40cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59976/0 (socket says 172.21.15.16:59976) 2023-11-24T06:29:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.293+0000 7fd0f2ffd640 1 -- 172.21.15.16:0/3818719013 learned_addr learned my addr 172.21.15.16:0/3818719013 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.293+0000 7fd0f37fe640 1 --2- 172.21.15.16:0/3818719013 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 0x7fd0f415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.293+0000 7fd0f3fff640 1 --2- 172.21.15.16:0/3818719013 >> v2:172.21.15.173:3300/0 conn(0x7fd0f415f9c0 0x7fd0f40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.293+0000 7fd0f2ffd640 1 -- 172.21.15.16:0/3818719013 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 msgr2=0x7fd0f415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.293+0000 7fd0f2ffd640 1 --2- 172.21.15.16:0/3818719013 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 0x7fd0f415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.293+0000 7fd0f2ffd640 1 -- 172.21.15.16:0/3818719013 >> v2:172.21.15.173:3300/0 conn(0x7fd0f415f9c0 msgr2=0x7fd0f40cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.293+0000 7fd0f2ffd640 1 --2- 172.21.15.16:0/3818719013 >> v2:172.21.15.173:3300/0 conn(0x7fd0f415f9c0 0x7fd0f40cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.293+0000 7fd0f2ffd640 1 -- 172.21.15.16:0/3818719013 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f406cd10 con 0x7fd0f413e360 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.294+0000 7fd0f2ffd640 1 --2- 172.21.15.16:0/3818719013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f413e360 0x7fd0f40cf290 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd0f406e6d0 tx=0x7fd0dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122219c7310c9cc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.294+0000 7fd0f27fc640 1 -- 172.21.15.16:0/3818719013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc02ca70 con 0x7fd0f413e360 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0f27fc640 1 -- 172.21.15.16:0/3818719013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc02cbd0 con 0x7fd0f413e360 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0f27fc640 1 -- 172.21.15.16:0/3818719013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0343f0 con 0x7fd0f413e360 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3818719013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f413e360 msgr2=0x7fd0f40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3818719013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f413e360 0x7fd0f40cf290 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd0f406e6d0 tx=0x7fd0dc02bed0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3818719013 shutdown_connections 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3818719013 >> v2:172.21.15.173:3300/0 conn(0x7fd0f415f9c0 0x7fd0f40cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3818719013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f413e360 0x7fd0f40cf290 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3818719013 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 0x7fd0f415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3818719013 >> 172.21.15.16:0/3818719013 conn(0x7fd0f406a800 msgr2=0x7fd0f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3818719013 shutdown_connections 2023-11-24T06:29:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.295+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3818719013 wait complete. 2023-11-24T06:29:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.297+0000 7fd0fa1c5640 1 Processor -- start 2023-11-24T06:29:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.297+0000 7fd0fa1c5640 1 -- start start 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.298+0000 7fd0fa1c5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 0x7fd0f41f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.298+0000 7fd0fa1c5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f413e360 0x7fd0f40b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.298+0000 7fd0fa1c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f415f9c0 0x7fd0f40ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.298+0000 7fd0fa1c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f41f0c20 con 0x7fd0f415f9c0 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.298+0000 7fd0fa1c5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f413d3e0 con 0x7fd0f413e360 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.298+0000 7fd0fa1c5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f415ad80 con 0x7fd0f413df80 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.299+0000 7fd0f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f415f9c0 0x7fd0f40ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.299+0000 7fd0f3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f415f9c0 0x7fd0f40ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59986/0 (socket says 172.21.15.16:59986) 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.299+0000 7fd0f3fff640 1 -- 172.21.15.16:0/3985910999 learned_addr learned my addr 172.21.15.16:0/3985910999 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.299+0000 7fd0f37fe640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 0x7fd0f41f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.299+0000 7fd0f3fff640 1 -- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 msgr2=0x7fd0f41f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.299+0000 7fd0f3fff640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 0x7fd0f41f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.299+0000 7fd0f3fff640 1 -- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f413e360 msgr2=0x7fd0f40b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.299+0000 7fd0f3fff640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f413e360 0x7fd0f40b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.299+0000 7fd0f3fff640 1 -- 172.21.15.16:0/3985910999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc009b30 con 0x7fd0f415f9c0 2023-11-24T06:29:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.300+0000 7fd0f3fff640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f415f9c0 0x7fd0f40ad7d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c7f0 tx=0x7fd0ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.300+0000 7fd0ebfff640 1 -- 172.21.15.16:0/3985910999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec004220 con 0x7fd0f415f9c0 2023-11-24T06:29:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.300+0000 7fd0ebfff640 1 -- 172.21.15.16:0/3985910999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ec004380 con 0x7fd0f415f9c0 2023-11-24T06:29:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.300+0000 7fd0ebfff640 1 -- 172.21.15.16:0/3985910999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec007860 con 0x7fd0f415f9c0 2023-11-24T06:29:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.300+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f41f0e90 con 0x7fd0f415f9c0 2023-11-24T06:29:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.300+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f415b060 con 0x7fd0f415f9c0 2023-11-24T06:29:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.302+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f40675d0 con 0x7fd0f415f9c0 2023-11-24T06:29:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.302+0000 7fd0ebfff640 1 -- 172.21.15.16:0/3985910999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fd0ec007bd0 con 0x7fd0f415f9c0 2023-11-24T06:29:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.303+0000 7fd0ebfff640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd0d406af60 0x7fd0d406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.303+0000 7fd0ebfff640 1 -- 172.21.15.16:0/3985910999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0ec089880 con 0x7fd0f415f9c0 2023-11-24T06:29:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.304+0000 7fd0f37fe640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd0d406af60 0x7fd0d406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.305+0000 7fd0f37fe640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd0d406af60 0x7fd0d406d420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4006fd0 tx=0x7fd0e4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.308+0000 7fd0ebfff640 1 -- 172.21.15.16:0/3985910999 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0ec054670 con 0x7fd0f415f9c0 2023-11-24T06:29:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.502+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0f406df00 con 0x7fd0f415f9c0 2023-11-24T06:29:28.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.505+0000 7fd0ebfff640 1 -- 172.21.15.16:0/3985910999 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16513 (secure 0 0 0) 0x7fd0ec01e090 con 0x7fd0f415f9c0 2023-11-24T06:29:28.509 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:28.510 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:07.314174+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:03.705566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:04.794925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd0d406af60 msgr2=0x7fd0d406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd0d406af60 0x7fd0d406d420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4006fd0 tx=0x7fd0e4008040 comp rx=0 tx=0).stop 2023-11-24T06:29:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f415f9c0 msgr2=0x7fd0f40ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f415f9c0 0x7fd0f40ad7d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c7f0 tx=0x7fd0ec00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 shutdown_connections 2023-11-24T06:29:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd0d406af60 0x7fd0d406d420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0f415f9c0 0x7fd0f40ad7d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0f413e360 0x7fd0f40b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 --2- 172.21.15.16:0/3985910999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0f413df80 0x7fd0f41f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 >> 172.21.15.16:0/3985910999 conn(0x7fd0f406a800 msgr2=0x7fd0f406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:28.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 shutdown_connections 2023-11-24T06:29:28.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.508+0000 7fd0fa1c5640 1 -- 172.21.15.16:0/3985910999 wait complete. 2023-11-24T06:29:28.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:29:03.705566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:29:04.794925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:28.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.765+0000 7fac6b02e640 1 Processor -- start 2023-11-24T06:29:28.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.765+0000 7fac6b02e640 1 -- start start 2023-11-24T06:29:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.767+0000 7fac6b02e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac640adf20 0x7fac64070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.767+0000 7fac6b02e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fac6415bf10 0x7fac640cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.767+0000 7fac6b02e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac640cc980 0x7fac64143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.767+0000 7fac6b02e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6415f970 con 0x7fac640adf20 2023-11-24T06:29:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.767+0000 7fac6b02e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6415f4e0 con 0x7fac6415bf10 2023-11-24T06:29:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.767+0000 7fac6b02e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac640cce10 con 0x7fac640cc980 2023-11-24T06:29:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.767+0000 7fac6a02c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac640adf20 0x7fac64070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.767+0000 7fac6a02c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fac640adf20 0x7fac64070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59990/0 (socket says 172.21.15.16:59990) 2023-11-24T06:29:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.767+0000 7fac6a02c640 1 -- 172.21.15.16:0/778620234 learned_addr learned my addr 172.21.15.16:0/778620234 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.768+0000 7fac6a02c640 1 -- 172.21.15.16:0/778620234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac640cc980 msgr2=0x7fac64143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.768+0000 7fac6a02c640 1 --2- 172.21.15.16:0/778620234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac640cc980 0x7fac64143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.768+0000 7fac6a02c640 1 -- 172.21.15.16:0/778620234 >> v2:172.21.15.173:3300/0 conn(0x7fac6415bf10 msgr2=0x7fac640cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.768+0000 7fac6a02c640 1 --2- 172.21.15.16:0/778620234 >> v2:172.21.15.173:3300/0 conn(0x7fac6415bf10 0x7fac640cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.768+0000 7fac6a02c640 1 -- 172.21.15.16:0/778620234 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac64071130 con 0x7fac640adf20 2023-11-24T06:29:28.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.769+0000 7fac6a02c640 1 --2- 172.21.15.16:0/778620234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 0x7fac64070a30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fac6415e9e0 tx=0x7fac6002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29088ceebc9775d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.769+0000 7fac6902a640 1 -- 172.21.15.16:0/778620234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac600317d0 con 0x7fac640adf20 2023-11-24T06:29:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.769+0000 7fac6902a640 1 -- 172.21.15.16:0/778620234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac60031930 con 0x7fac640adf20 2023-11-24T06:29:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.769+0000 7fac6902a640 1 -- 172.21.15.16:0/778620234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60031cc0 con 0x7fac640adf20 2023-11-24T06:29:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.770+0000 7fac6b02e640 1 -- 172.21.15.16:0/778620234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 msgr2=0x7fac64070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.770+0000 7fac6b02e640 1 --2- 172.21.15.16:0/778620234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 0x7fac64070a30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fac6415e9e0 tx=0x7fac6002b870 comp rx=0 tx=0).stop 2023-11-24T06:29:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.770+0000 7fac6b02e640 1 -- 172.21.15.16:0/778620234 shutdown_connections 2023-11-24T06:29:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.770+0000 7fac6b02e640 1 --2- 172.21.15.16:0/778620234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac640cc980 0x7fac64143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.770+0000 7fac6b02e640 1 --2- 172.21.15.16:0/778620234 >> v2:172.21.15.173:3300/0 conn(0x7fac6415bf10 0x7fac640cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.770+0000 7fac6b02e640 1 --2- 172.21.15.16:0/778620234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 0x7fac64070a30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.770+0000 7fac6b02e640 1 -- 172.21.15.16:0/778620234 >> 172.21.15.16:0/778620234 conn(0x7fac6406a7d0 msgr2=0x7fac6406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.770+0000 7fac6b02e640 1 -- 172.21.15.16:0/778620234 shutdown_connections 2023-11-24T06:29:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.770+0000 7fac6b02e640 1 -- 172.21.15.16:0/778620234 wait complete. 2023-11-24T06:29:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.772+0000 7fac6b02e640 1 Processor -- start 2023-11-24T06:29:28.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.772+0000 7fac6b02e640 1 -- start start 2023-11-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6b02e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 0x7fac641368a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6b02e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac640cc980 0x7fac6415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6b02e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6415bf10 0x7fac640aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6b02e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640b25f0 con 0x7fac640adf20 2023-11-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6b02e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640ad520 con 0x7fac640cc980 2023-11-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6b02e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac640c91e0 con 0x7fac6415bf10 2023-11-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6a02c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 0x7fac641368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6a02c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 0x7fac641368a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59992/0 (socket says 172.21.15.16:59992) 2023-11-24T06:29:28.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6a02c640 1 -- 172.21.15.16:0/527229210 learned_addr learned my addr 172.21.15.16:0/527229210 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6a02c640 1 -- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6415bf10 msgr2=0x7fac640aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6a02c640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6415bf10 0x7fac640aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6a02c640 1 -- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac640cc980 msgr2=0x7fac6415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.774+0000 7fac6982b640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac640cc980 0x7fac6415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.775+0000 7fac6a02c640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac640cc980 0x7fac6415aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.775+0000 7fac6a02c640 1 -- 172.21.15.16:0/527229210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac600094d0 con 0x7fac640adf20 2023-11-24T06:29:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.775+0000 7fac6982b640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac640cc980 0x7fac6415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:28.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.775+0000 7fac6a02c640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 0x7fac641368a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fac60000c00 tx=0x7fac600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.775+0000 7fac5affd640 1 -- 172.21.15.16:0/527229210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac600328f0 con 0x7fac640adf20 2023-11-24T06:29:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.775+0000 7fac5affd640 1 -- 172.21.15.16:0/527229210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac60032a50 con 0x7fac640adf20 2023-11-24T06:29:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.775+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac640acf70 con 0x7fac640adf20 2023-11-24T06:29:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.775+0000 7fac5affd640 1 -- 172.21.15.16:0/527229210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60032dc0 con 0x7fac640adf20 2023-11-24T06:29:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.775+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac640b1fa0 con 0x7fac640adf20 2023-11-24T06:29:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.777+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac64067450 con 0x7fac640adf20 2023-11-24T06:29:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.777+0000 7fac5affd640 1 -- 172.21.15.16:0/527229210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fac60030070 con 0x7fac640adf20 2023-11-24T06:29:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.778+0000 7fac5affd640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fac4806af60 0x7fac4806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.779+0000 7fac5affd640 1 -- 172.21.15.16:0/527229210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac600ab900 con 0x7fac640adf20 2023-11-24T06:29:28.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.779+0000 7fac6982b640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fac4806af60 0x7fac4806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:28.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.780+0000 7fac6982b640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fac4806af60 0x7fac4806d420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fac54006fd0 tx=0x7fac54008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:28.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.783+0000 7fac5affd640 1 -- 172.21.15.16:0/527229210 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac60076790 con 0x7fac640adf20 2023-11-24T06:29:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.974+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac6406b3c0 con 0x7fac640adf20 2023-11-24T06:29:28.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.975+0000 7fac5affd640 1 -- 172.21.15.16:0/527229210 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac60079db0 con 0x7fac640adf20 2023-11-24T06:29:28.978 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:29:28.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fac4806af60 msgr2=0x7fac4806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fac4806af60 0x7fac4806d420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fac54006fd0 tx=0x7fac54008040 comp rx=0 tx=0).stop 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 msgr2=0x7fac641368a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 0x7fac641368a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fac60000c00 tx=0x7fac600046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 shutdown_connections 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fac4806af60 0x7fac4806d420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6415bf10 0x7fac640aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac640cc980 0x7fac6415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 --2- 172.21.15.16:0/527229210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac640adf20 0x7fac641368a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 >> 172.21.15.16:0/527229210 conn(0x7fac6406a7d0 msgr2=0x7fac640afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 shutdown_connections 2023-11-24T06:29:28.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:28.978+0000 7fac6b02e640 1 -- 172.21.15.16:0/527229210 wait complete. 2023-11-24T06:29:29.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:29:29.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.227+0000 7f308e409640 1 Processor -- start 2023-11-24T06:29:29.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.227+0000 7f308e409640 1 -- start start 2023-11-24T06:29:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.229+0000 7f308e409640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f308813e360 0x7f30880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.229+0000 7f308e409640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308815c5f0 con 0x7f308815f9c0 2023-11-24T06:29:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.229+0000 7f308e409640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308806c470 con 0x7f308813df80 2023-11-24T06:29:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.229+0000 7f308e409640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308806c750 con 0x7f308813e360 2023-11-24T06:29:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.229+0000 7f308c9a6640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f308815f9c0 0x7f30880cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42708/0 (socket says 172.21.15.16:42708) 2023-11-24T06:29:29.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.229+0000 7f308c9a6640 1 -- 172.21.15.16:0/424601857 learned_addr learned my addr 172.21.15.16:0/424601857 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.229+0000 7f307ffff640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f308813e360 0x7f30880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384824276 0 0) 0x7f308815c5f0 con 0x7f308815f9c0 2023-11-24T06:29:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3074003520 con 0x7f308815f9c0 2023-11-24T06:29:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 148764002 0 0) 0x7f3074003520 con 0x7f308815f9c0 2023-11-24T06:29:29.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f308813e360 msgr2=0x7f30880cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f308813e360 0x7f30880cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 >> v1:172.21.15.173:6789/0 conn(0x7f308813df80 legacy=0x7f308815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308806ca30 con 0x7f308815f9c0 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f30800019f0 con 0x7f308815f9c0 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 >> v1:172.21.15.16:6789/0 conn(0x7f308815f9c0 legacy=0x7f30880cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30740039a0 0x7f3074003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30740047c0 0x7f3074004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f308813e360 0x7f3074005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308806c750 con 0x7f30740047c0 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307400ba30 con 0x7f30740039a0 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307400bbc0 con 0x7f308813e360 2023-11-24T06:29:29.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.230+0000 7f307ffff640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30740047c0 0x7f3074004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.231+0000 7f3087fff640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30740039a0 0x7f3074003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.231+0000 7f307ffff640 1 -- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f308813e360 msgr2=0x7f3074005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.231+0000 7f307ffff640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f308813e360 0x7f3074005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.231+0000 7f307ffff640 1 -- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30740039a0 msgr2=0x7f3074003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.231+0000 7f307ffff640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30740039a0 0x7f3074003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.231+0000 7f307ffff640 1 -- 172.21.15.16:0/424601857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074003520 con 0x7f30740047c0 2023-11-24T06:29:29.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.232+0000 7f307ffff640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30740047c0 0x7f3074004c40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3070009a90 tx=0x7f30740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbe7df17385593f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307002c6b0 con 0x7f30740047c0 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307002c810 con 0x7f30740047c0 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f30877fe640 1 -- 172.21.15.16:0/424601857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307002cba0 con 0x7f30740047c0 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f308e409640 1 -- 172.21.15.16:0/424601857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30740047c0 msgr2=0x7f3074004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f308e409640 1 --2- 172.21.15.16:0/424601857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30740047c0 0x7f3074004c40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3070009a90 tx=0x7f30740057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f308c9a6640 1 -- 172.21.15.16:0/424601857 reap_dead start 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f308e409640 1 -- 172.21.15.16:0/424601857 shutdown_connections 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f308e409640 1 -- 172.21.15.16:0/424601857 >> 172.21.15.16:0/424601857 conn(0x7f308806a800 msgr2=0x7f308806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f308e409640 1 -- 172.21.15.16:0/424601857 shutdown_connections 2023-11-24T06:29:29.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.233+0000 7f308e409640 1 -- 172.21.15.16:0/424601857 wait complete. 2023-11-24T06:29:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.235+0000 7f308e409640 1 Processor -- start 2023-11-24T06:29:29.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.235+0000 7f308e409640 1 -- start start 2023-11-24T06:29:29.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.236+0000 7f308e409640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30740047c0 0x7f308813d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.236+0000 7f308e409640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308814df10 0x7f30881509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.236+0000 7f308e409640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3088152a80 0x7f3088152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.236+0000 7f308e409640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3088153450 con 0x7f308814df10 2023-11-24T06:29:29.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.236+0000 7f308e409640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308814f510 con 0x7f30740047c0 2023-11-24T06:29:29.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.236+0000 7f308e409640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3088151230 con 0x7f3088152a80 2023-11-24T06:29:29.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f307ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308814df10 0x7f30881509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f307ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308814df10 0x7f30881509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60018/0 (socket says 172.21.15.16:60018) 2023-11-24T06:29:29.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f307ffff640 1 -- 172.21.15.16:0/1495246603 learned_addr learned my addr 172.21.15.16:0/1495246603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f3087fff640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30740047c0 0x7f308813d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f308c9a6640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3088152a80 0x7f3088152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f307ffff640 1 -- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3088152a80 msgr2=0x7f3088152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f307ffff640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3088152a80 0x7f3088152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f307ffff640 1 -- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30740047c0 msgr2=0x7f308813d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f307ffff640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30740047c0 0x7f308813d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.237+0000 7f307ffff640 1 -- 172.21.15.16:0/1495246603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30881f0ca0 con 0x7f308814df10 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.238+0000 7f307ffff640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308814df10 0x7f30881509c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f307002c250 tx=0x7f3070004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.238+0000 7f30857fa640 1 -- 172.21.15.16:0/1495246603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070003810 con 0x7f308814df10 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.238+0000 7f30857fa640 1 -- 172.21.15.16:0/1495246603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070003970 con 0x7f308814df10 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.238+0000 7f30857fa640 1 -- 172.21.15.16:0/1495246603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070003d00 con 0x7f308814df10 2023-11-24T06:29:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.238+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30881f0f80 con 0x7f308814df10 2023-11-24T06:29:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.238+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30881f1540 con 0x7f308814df10 2023-11-24T06:29:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.239+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30881f16d0 con 0x7f308814df10 2023-11-24T06:29:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.240+0000 7f30857fa640 1 -- 172.21.15.16:0/1495246603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3070040450 con 0x7f308814df10 2023-11-24T06:29:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.241+0000 7f30857fa640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f306806b0b0 0x7f306806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.241+0000 7f30857fa640 1 -- 172.21.15.16:0/1495246603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30700ab950 con 0x7f308814df10 2023-11-24T06:29:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.242+0000 7f3087fff640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f306806b0b0 0x7f306806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.243+0000 7f3087fff640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f306806b0b0 0x7f306806d570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f308814d990 tx=0x7f3078007c50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.246+0000 7f30857fa640 1 -- 172.21.15.16:0/1495246603 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3070076690 con 0x7f308814df10 2023-11-24T06:29:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.444+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30880675d0 con 0x7f308814df10 2023-11-24T06:29:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.445+0000 7f30857fa640 1 -- 172.21.15.16:0/1495246603 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3070003e60 con 0x7f308814df10 2023-11-24T06:29:29.448 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f306806b0b0 msgr2=0x7f306806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f306806b0b0 0x7f306806d570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f308814d990 tx=0x7f3078007c50 comp rx=0 tx=0).stop 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308814df10 msgr2=0x7f30881509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308814df10 0x7f30881509c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f307002c250 tx=0x7f3070004730 comp rx=0 tx=0).stop 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 shutdown_connections 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f306806b0b0 0x7f306806d570 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3088152a80 0x7f3088152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308814df10 0x7f30881509c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 --2- 172.21.15.16:0/1495246603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30740047c0 0x7f308813d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.447+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 >> 172.21.15.16:0/1495246603 conn(0x7f308806a800 msgr2=0x7f308815c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:29.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.448+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 shutdown_connections 2023-11-24T06:29:29.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.448+0000 7f308e409640 1 -- 172.21.15.16:0/1495246603 wait complete. 2023-11-24T06:29:29.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:29:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.690+0000 7facc6d16640 1 Processor -- start 2023-11-24T06:29:29.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.691+0000 7facc6d16640 1 -- start start 2023-11-24T06:29:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.692+0000 7facc6d16640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7facc0143810 0x7facc0071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.693+0000 7facc6d16640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc00ae620 con 0x7facc00ae170 2023-11-24T06:29:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.693+0000 7facc6d16640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc006c730 con 0x7facc015d770 2023-11-24T06:29:29.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.693+0000 7facc6d16640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc0143bf0 con 0x7facc0143810 2023-11-24T06:29:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.693+0000 7facc5513640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7facc00ae170 0x7facc015e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42716/0 (socket says 172.21.15.16:42716) 2023-11-24T06:29:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.693+0000 7facc5513640 1 -- 172.21.15.16:0/1398411666 learned_addr learned my addr 172.21.15.16:0/1398411666 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.693+0000 7facc5d14640 1 --2- 172.21.15.16:0/1398411666 >> v2:172.21.15.173:3300/0 conn(0x7facc0143810 0x7facc0071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.693+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190578027 0 0) 0x7facc00ae620 con 0x7facc00ae170 2023-11-24T06:29:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facac003520 con 0x7facc00ae170 2023-11-24T06:29:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 301729884 0 0) 0x7facac003520 con 0x7facc00ae170 2023-11-24T06:29:29.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 >> v2:172.21.15.173:3300/0 conn(0x7facc0143810 msgr2=0x7facc0071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 --2- 172.21.15.16:0/1398411666 >> v2:172.21.15.173:3300/0 conn(0x7facc0143810 0x7facc0071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 >> v1:172.21.15.173:6789/0 conn(0x7facc015d770 legacy=0x7facc015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00724d0 con 0x7facc00ae170 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7facb0001670 con 0x7facc00ae170 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 >> v1:172.21.15.16:6789/0 conn(0x7facc00ae170 legacy=0x7facc015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 --2- 172.21.15.16:0/1398411666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facac003970 0x7facac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 --2- 172.21.15.16:0/1398411666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 --2- 172.21.15.16:0/1398411666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7facac0052a0 0x7facac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc0143bf0 con 0x7facac0047c0 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac00c170 con 0x7facac003970 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac00c330 con 0x7facac0052a0 2023-11-24T06:29:29.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.694+0000 7facc5513640 1 --2- 172.21.15.16:0/1398411666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.695+0000 7facc5513640 1 -- 172.21.15.16:0/1398411666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7facac0052a0 msgr2=0x7facac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.695+0000 7facc5513640 1 --2- 172.21.15.16:0/1398411666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7facac0052a0 0x7facac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.695+0000 7facc5513640 1 -- 172.21.15.16:0/1398411666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facac003970 msgr2=0x7facac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:29.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.695+0000 7facc5513640 1 --2- 172.21.15.16:0/1398411666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facac003970 0x7facac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.695+0000 7facc5513640 1 -- 172.21.15.16:0/1398411666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac003520 con 0x7facac0047c0 2023-11-24T06:29:29.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.695+0000 7facc6515640 1 -- 172.21.15.16:0/1398411666 reap_dead start 2023-11-24T06:29:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.696+0000 7facc5513640 1 --2- 172.21.15.16:0/1398411666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facac004c40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7facb0007fd0 tx=0x7facac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2c7b917417db1e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.696+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0005ad0 con 0x7facac0047c0 2023-11-24T06:29:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.696+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb0005c30 con 0x7facac0047c0 2023-11-24T06:29:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.696+0000 7facc4d12640 1 -- 172.21.15.16:0/1398411666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb002ca90 con 0x7facac0047c0 2023-11-24T06:29:29.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.697+0000 7facc6d16640 1 -- 172.21.15.16:0/1398411666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 msgr2=0x7facac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.697+0000 7facc6d16640 1 --2- 172.21.15.16:0/1398411666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facac004c40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7facb0007fd0 tx=0x7facac0057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.697+0000 7facc6d16640 1 -- 172.21.15.16:0/1398411666 shutdown_connections 2023-11-24T06:29:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.697+0000 7facc6d16640 1 --2- 172.21.15.16:0/1398411666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facac004c40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.697+0000 7facc6d16640 1 -- 172.21.15.16:0/1398411666 >> 172.21.15.16:0/1398411666 conn(0x7facc006a7d0 msgr2=0x7facc006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.697+0000 7facc6d16640 1 -- 172.21.15.16:0/1398411666 shutdown_connections 2023-11-24T06:29:29.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.697+0000 7facc6d16640 1 -- 172.21.15.16:0/1398411666 wait complete. 2023-11-24T06:29:29.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.699+0000 7facc6d16640 1 Processor -- start 2023-11-24T06:29:29.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.699+0000 7facc6d16640 1 -- start start 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.700+0000 7facc6d16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facc00af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.700+0000 7facc6d16640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7facc00b1390 0x7facc015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.700+0000 7facc6d16640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facc0145f00 0x7facc01463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.700+0000 7facc6d16640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01468f0 con 0x7facac0047c0 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.700+0000 7facc6d16640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00b0300 con 0x7facc0145f00 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.700+0000 7facc6d16640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc00afa60 con 0x7facc00b1390 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.700+0000 7facc5d14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facc00af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc5d14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facc00af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60036/0 (socket says 172.21.15.16:60036) 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc5d14640 1 -- 172.21.15.16:0/1431048733 learned_addr learned my addr 172.21.15.16:0/1431048733 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:29.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc6515640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facc0145f00 0x7facc01463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc5513640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7facc00b1390 0x7facc015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc5d14640 1 -- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7facc00b1390 msgr2=0x7facc015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc5d14640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7facc00b1390 0x7facc015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc5d14640 1 -- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facc0145f00 msgr2=0x7facc01463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc5d14640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facc0145f00 0x7facc01463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc5d14640 1 -- 172.21.15.16:0/1431048733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00afd40 con 0x7facac0047c0 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc6515640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facc0145f00 0x7facc01463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.701+0000 7facc5d14640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facc00af3f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7facbc00e770 tx=0x7facbc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.702+0000 7facb67fc640 1 -- 172.21.15.16:0/1431048733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00c730 con 0x7facac0047c0 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.702+0000 7facb67fc640 1 -- 172.21.15.16:0/1431048733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc00c890 con 0x7facac0047c0 2023-11-24T06:29:29.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.702+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc00b0020 con 0x7facac0047c0 2023-11-24T06:29:29.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.702+0000 7facb67fc640 1 -- 172.21.15.16:0/1431048733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00cc40 con 0x7facac0047c0 2023-11-24T06:29:29.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.702+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc015cfd0 con 0x7facac0047c0 2023-11-24T06:29:29.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.703+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc0067450 con 0x7facac0047c0 2023-11-24T06:29:29.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.704+0000 7facb67fc640 1 -- 172.21.15.16:0/1431048733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7facbc013470 con 0x7facac0047c0 2023-11-24T06:29:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.705+0000 7facb67fc640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faca406b0b0 0x7faca406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:29.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.705+0000 7facb67fc640 1 -- 172.21.15.16:0/1431048733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facbc089a40 con 0x7facac0047c0 2023-11-24T06:29:29.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.706+0000 7facc5513640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faca406b0b0 0x7faca406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:29.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.707+0000 7facc5513640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faca406b0b0 0x7faca406d570 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7facb0034000 tx=0x7facb0009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:29.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.710+0000 7facb67fc640 1 -- 172.21.15.16:0/1431048733 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7facbc08f020 con 0x7facac0047c0 2023-11-24T06:29:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.906+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facc0067b60 con 0x7facac0047c0 2023-11-24T06:29:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.907+0000 7facb67fc640 1 -- 172.21.15.16:0/1431048733 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7facbc0170a0 con 0x7facac0047c0 2023-11-24T06:29:29.910 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:29:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faca406b0b0 msgr2=0x7faca406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faca406b0b0 0x7faca406d570 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7facb0034000 tx=0x7facb0009030 comp rx=0 tx=0).stop 2023-11-24T06:29:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 msgr2=0x7facc00af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facc00af3f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7facbc00e770 tx=0x7facbc00ec30 comp rx=0 tx=0).stop 2023-11-24T06:29:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 shutdown_connections 2023-11-24T06:29:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faca406b0b0 0x7faca406d570 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facc0145f00 0x7facc01463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7facc00b1390 0x7facc015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 --2- 172.21.15.16:0/1431048733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facac0047c0 0x7facc00af3f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 >> 172.21.15.16:0/1431048733 conn(0x7facc006a7d0 msgr2=0x7facc00b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 shutdown_connections 2023-11-24T06:29:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:29.910+0000 7facc6d16640 1 -- 172.21.15.16:0/1431048733 wait complete. 2023-11-24T06:29:29.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:29:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.159+0000 7fe6cb59e640 1 Processor -- start 2023-11-24T06:29:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.159+0000 7fe6cb59e640 1 -- start start 2023-11-24T06:29:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.161+0000 7fe6cb59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6cc143c40 0x7fe6cc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.161+0000 7fe6cb59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe6cc0717d0 0x7fe6cc0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.161+0000 7fe6cb59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6cc070e50 con 0x7fe6cc0cca40 2023-11-24T06:29:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.161+0000 7fe6cb59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6cc0723c0 con 0x7fe6cc143c40 2023-11-24T06:29:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.161+0000 7fe6cb59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6cc0cce40 con 0x7fe6cc0717d0 2023-11-24T06:29:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.161+0000 7fe6ca59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6cc143c40 0x7fe6cc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.162+0000 7fe6ca59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6cc143c40 0x7fe6cc071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60038/0 (socket says 172.21.15.16:60038) 2023-11-24T06:29:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.162+0000 7fe6ca59c640 1 -- 172.21.15.16:0/1167577658 learned_addr learned my addr 172.21.15.16:0/1167577658 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.162+0000 7fe6c9d9b640 1 --2- 172.21.15.16:0/1167577658 >> v2:172.21.15.173:3300/0 conn(0x7fe6cc0717d0 0x7fe6cc0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.162+0000 7fe6ca59c640 1 -- 172.21.15.16:0/1167577658 >> v2:172.21.15.173:3300/0 conn(0x7fe6cc0717d0 msgr2=0x7fe6cc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.162+0000 7fe6ca59c640 1 --2- 172.21.15.16:0/1167577658 >> v2:172.21.15.173:3300/0 conn(0x7fe6cc0717d0 0x7fe6cc0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.162+0000 7fe6ca59c640 1 -- 172.21.15.16:0/1167577658 >> v1:172.21.15.173:6789/0 conn(0x7fe6cc0cca40 legacy=0x7fe6cc15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.162+0000 7fe6ca59c640 1 -- 172.21.15.16:0/1167577658 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc06e170 con 0x7fe6cc143c40 2023-11-24T06:29:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.162+0000 7fe6c9d9b640 1 --2- 172.21.15.16:0/1167577658 >> v2:172.21.15.173:3300/0 conn(0x7fe6cc0717d0 0x7fe6cc0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.163+0000 7fe6ca59c640 1 --2- 172.21.15.16:0/1167577658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc143c40 0x7fe6cc071290 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0b44a0 tx=0x7fe6bc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb97a898f520688d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.163+0000 7fe6c959a640 1 -- 172.21.15.16:0/1167577658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0317e0 con 0x7fe6cc143c40 2023-11-24T06:29:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.163+0000 7fe6c959a640 1 -- 172.21.15.16:0/1167577658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc031940 con 0x7fe6cc143c40 2023-11-24T06:29:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.163+0000 7fe6c959a640 1 -- 172.21.15.16:0/1167577658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc031cd0 con 0x7fe6cc143c40 2023-11-24T06:29:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.164+0000 7fe6cb59e640 1 -- 172.21.15.16:0/1167577658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc143c40 msgr2=0x7fe6cc071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.164+0000 7fe6cb59e640 1 --2- 172.21.15.16:0/1167577658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc143c40 0x7fe6cc071290 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0b44a0 tx=0x7fe6bc02b880 comp rx=0 tx=0).stop 2023-11-24T06:29:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.164+0000 7fe6cb59e640 1 -- 172.21.15.16:0/1167577658 shutdown_connections 2023-11-24T06:29:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.164+0000 7fe6cb59e640 1 --2- 172.21.15.16:0/1167577658 >> v2:172.21.15.173:3300/0 conn(0x7fe6cc0717d0 0x7fe6cc0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.164+0000 7fe6cb59e640 1 --2- 172.21.15.16:0/1167577658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc143c40 0x7fe6cc071290 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.164+0000 7fe6cb59e640 1 -- 172.21.15.16:0/1167577658 >> 172.21.15.16:0/1167577658 conn(0x7fe6cc06a800 msgr2=0x7fe6cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.164+0000 7fe6cb59e640 1 -- 172.21.15.16:0/1167577658 shutdown_connections 2023-11-24T06:29:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.164+0000 7fe6cb59e640 1 -- 172.21.15.16:0/1167577658 wait complete. 2023-11-24T06:29:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.166+0000 7fe6cb59e640 1 Processor -- start 2023-11-24T06:29:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.166+0000 7fe6cb59e640 1 -- start start 2023-11-24T06:29:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.167+0000 7fe6cb59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6cc0717d0 0x7fe6cc15a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.167+0000 7fe6cb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc0cca40 0x7fe6cc0c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.167+0000 7fe6cb59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6cc143c40 0x7fe6cc0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.167+0000 7fe6cb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0cabe0 con 0x7fe6cc0cca40 2023-11-24T06:29:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.167+0000 7fe6cb59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc15a920 con 0x7fe6cc143c40 2023-11-24T06:29:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.167+0000 7fe6cb59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0c9fd0 con 0x7fe6cc0717d0 2023-11-24T06:29:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc0cca40 0x7fe6cc0c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc0cca40 0x7fe6cc0c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60048/0 (socket says 172.21.15.16:60048) 2023-11-24T06:29:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6c9d9b640 1 -- 172.21.15.16:0/3613754790 learned_addr learned my addr 172.21.15.16:0/3613754790 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6ca59c640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6cc0717d0 0x7fe6cc15a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6c9d9b640 1 -- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6cc0717d0 msgr2=0x7fe6cc15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6c9d9b640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6cc0717d0 0x7fe6cc15a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6c9d9b640 1 -- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6cc143c40 msgr2=0x7fe6cc0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6cad9d640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6cc143c40 0x7fe6cc0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6c9d9b640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6cc143c40 0x7fe6cc0b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6c9d9b640 1 -- 172.21.15.16:0/3613754790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0094d0 con 0x7fe6cc0cca40 2023-11-24T06:29:30.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.168+0000 7fe6cad9d640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6cc143c40 0x7fe6cc0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.169+0000 7fe6c9d9b640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc0cca40 0x7fe6cc0c96a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000c7f0 tx=0x7fe6c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.169+0000 7fe6baffd640 1 -- 172.21.15.16:0/3613754790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0007a30 con 0x7fe6cc0cca40 2023-11-24T06:29:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.169+0000 7fe6baffd640 1 -- 172.21.15.16:0/3613754790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c0007b90 con 0x7fe6cc0cca40 2023-11-24T06:29:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.169+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc0ca2b0 con 0x7fe6cc0cca40 2023-11-24T06:29:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.169+0000 7fe6baffd640 1 -- 172.21.15.16:0/3613754790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000ce40 con 0x7fe6cc0cca40 2023-11-24T06:29:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.169+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc0ca870 con 0x7fe6cc0cca40 2023-11-24T06:29:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.170+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6cc0cad90 con 0x7fe6cc0cca40 2023-11-24T06:29:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.171+0000 7fe6baffd640 1 -- 172.21.15.16:0/3613754790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fe6c00113f0 con 0x7fe6cc0cca40 2023-11-24T06:29:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.172+0000 7fe6baffd640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6ac06b100 0x7fe6ac06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.172+0000 7fe6baffd640 1 -- 172.21.15.16:0/3613754790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe6c0089d40 con 0x7fe6cc0cca40 2023-11-24T06:29:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.173+0000 7fe6ca59c640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6ac06b100 0x7fe6ac06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.174+0000 7fe6ca59c640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6ac06b100 0x7fe6ac06d5c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc036000 tx=0x7fe6bc03d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.177+0000 7fe6baffd640 1 -- 172.21.15.16:0/3613754790 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6cc0cad90 con 0x7fe6cc0cca40 2023-11-24T06:29:30.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.393+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe6cc067450 con 0x7fe6cc0cca40 2023-11-24T06:29:30.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.394+0000 7fe6baffd640 1 -- 172.21.15.16:0/3613754790 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe6c001f080 con 0x7fe6cc0cca40 2023-11-24T06:29:30.398 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:29:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6ac06b100 msgr2=0x7fe6ac06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6ac06b100 0x7fe6ac06d5c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc036000 tx=0x7fe6bc03d040 comp rx=0 tx=0).stop 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc0cca40 msgr2=0x7fe6cc0c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc0cca40 0x7fe6cc0c96a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000c7f0 tx=0x7fe6c000cbf0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 shutdown_connections 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6ac06b100 0x7fe6ac06d5c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe6cc143c40 0x7fe6cc0b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6cc0cca40 0x7fe6cc0c96a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 --2- 172.21.15.16:0/3613754790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6cc0717d0 0x7fe6cc15a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 >> 172.21.15.16:0/3613754790 conn(0x7fe6cc06a800 msgr2=0x7fe6cc06d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 shutdown_connections 2023-11-24T06:29:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.397+0000 7fe6cb59e640 1 -- 172.21.15.16:0/3613754790 wait complete. 2023-11-24T06:29:30.420 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-24T06:29:30.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:30.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:29:30.514 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[104605]: fuse finished with error 0 and tester_r 0 2023-11-24T06:29:30.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:29:30.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:30.521 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:29:30.574 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:29:30.574 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:30.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:30.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.861+0000 7ff1d085c640 1 Processor -- start 2023-11-24T06:29:30.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.861+0000 7ff1d085c640 1 -- start start 2023-11-24T06:29:30.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.863+0000 7ff1d085c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff1cc143ae0 0x7ff1cc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.863+0000 7ff1d085c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc15e3b0 0x7ff1cc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.863+0000 7ff1d085c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc0cfc30 con 0x7ff1cc0cf7d0 2023-11-24T06:29:30.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.863+0000 7ff1d085c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b1ee0 con 0x7ff1cc143ae0 2023-11-24T06:29:30.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.863+0000 7ff1d085c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b2780 con 0x7ff1cc15e3b0 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1c1d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff1cc0cf7d0 0x7ff1cc15dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42718/0 (socket says 172.21.15.16:42718) 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1c1d9b640 1 -- 172.21.15.16:0/3696741391 learned_addr learned my addr 172.21.15.16:0/3696741391 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1ca59c640 1 --2- 172.21.15.16:0/3696741391 >> v2:172.21.15.173:3300/0 conn(0x7ff1cc143ae0 0x7ff1cc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1cad9d640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc15e3b0 0x7ff1cc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172043657 0 0) 0x7ff1cc0cfc30 con 0x7ff1cc0cf7d0 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b8003520 con 0x7ff1cc0cf7d0 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1758311556 0 0) 0x7ff1b8003520 con 0x7ff1cc0cf7d0 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc15e3b0 msgr2=0x7ff1cc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1c9d9b640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc15e3b0 0x7ff1cc0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 >> v2:172.21.15.173:3300/0 conn(0x7ff1cc143ae0 msgr2=0x7ff1cc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.864+0000 7ff1c9d9b640 1 --2- 172.21.15.16:0/3696741391 >> v2:172.21.15.173:3300/0 conn(0x7ff1cc143ae0 0x7ff1cc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc15e990 con 0x7ff1cc0cf7d0 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7ff1b4001670 con 0x7ff1cc0cf7d0 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1cad9d640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc15e3b0 0x7ff1cc0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 >> v1:172.21.15.16:6789/0 conn(0x7ff1cc0cf7d0 legacy=0x7ff1cc15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c9d9b640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c9d9b640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1b80047c0 0x7ff1b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c9d9b640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc15e3b0 0x7ff1b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b1ee0 con 0x7ff1b8003970 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b2780 con 0x7ff1b80047c0 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b800db90 con 0x7ff1cc15e3b0 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1ca59c640 1 --2- 172.21.15.16:0/3696741391 >> v2:172.21.15.173:3300/0 conn(0x7ff1cc143ae0 0x7ff1cc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1ca59c640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1c1d9b640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1b80047c0 0x7ff1b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.865+0000 7ff1cad9d640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc15e3b0 0x7ff1b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.866+0000 7ff1ca59c640 1 -- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc15e3b0 msgr2=0x7ff1b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.866+0000 7ff1ca59c640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc15e3b0 0x7ff1b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.866+0000 7ff1ca59c640 1 -- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1b80047c0 msgr2=0x7ff1b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.866+0000 7ff1ca59c640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1b80047c0 0x7ff1b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.866+0000 7ff1ca59c640 1 -- 172.21.15.16:0/3696741391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c4009400 con 0x7ff1b8003970 2023-11-24T06:29:30.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.867+0000 7ff1ca59c640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1b8003d90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00e350 tx=0x7ff1b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67d0ed53ec05298 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.867+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc02cde0 con 0x7ff1b8003970 2023-11-24T06:29:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.867+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc00f040 con 0x7ff1b8003970 2023-11-24T06:29:30.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.867+0000 7ff1c9d9b640 1 -- 172.21.15.16:0/3696741391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc003b90 con 0x7ff1b8003970 2023-11-24T06:29:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.868+0000 7ff1d085c640 1 -- 172.21.15.16:0/3696741391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 msgr2=0x7ff1b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.868+0000 7ff1d085c640 1 --2- 172.21.15.16:0/3696741391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1b8003d90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00e350 tx=0x7ff1b80057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.868+0000 7ff1cad9d640 1 -- 172.21.15.16:0/3696741391 reap_dead start 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.868+0000 7ff1d085c640 1 -- 172.21.15.16:0/3696741391 shutdown_connections 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.868+0000 7ff1d085c640 1 -- 172.21.15.16:0/3696741391 >> 172.21.15.16:0/3696741391 conn(0x7ff1cc06a800 msgr2=0x7ff1cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.868+0000 7ff1d085c640 1 -- 172.21.15.16:0/3696741391 shutdown_connections 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.868+0000 7ff1d085c640 1 -- 172.21.15.16:0/3696741391 wait complete. 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.870+0000 7ff1d085c640 1 Processor -- start 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.870+0000 7ff1d085c640 1 -- start start 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.871+0000 7ff1d085c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1cc1382f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.871+0000 7ff1d085c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc138830 0x7ff1cc0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.871+0000 7ff1d085c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1cc0cde40 0x7ff1cc1efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.871+0000 7ff1d085c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc070cb0 con 0x7ff1b8003970 2023-11-24T06:29:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.871+0000 7ff1d085c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0cc610 con 0x7ff1cc0cde40 2023-11-24T06:29:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.871+0000 7ff1d085c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0cd470 con 0x7ff1cc138830 2023-11-24T06:29:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.871+0000 7ff1ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1cc1382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.871+0000 7ff1ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1cc1382f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60068/0 (socket says 172.21.15.16:60068) 2023-11-24T06:29:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.871+0000 7ff1ca59c640 1 -- 172.21.15.16:0/1809613074 learned_addr learned my addr 172.21.15.16:0/1809613074 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.872+0000 7ff1c1d9b640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc138830 0x7ff1cc0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.872+0000 7ff1cad9d640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1cc0cde40 0x7ff1cc1efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.872+0000 7ff1ca59c640 1 -- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc138830 msgr2=0x7ff1cc0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.872+0000 7ff1ca59c640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc138830 0x7ff1cc0cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.872+0000 7ff1ca59c640 1 -- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1cc0cde40 msgr2=0x7ff1cc1efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.872+0000 7ff1ca59c640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1cc0cde40 0x7ff1cc1efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.872+0000 7ff1ca59c640 1 -- 172.21.15.16:0/1809613074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc15a1b0 con 0x7ff1b8003970 2023-11-24T06:29:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.872+0000 7ff1ca59c640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1cc1382f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc02c850 tx=0x7ff1bc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.872+0000 7ff1c37fe640 1 -- 172.21.15.16:0/1809613074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc034dc0 con 0x7ff1b8003970 2023-11-24T06:29:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.873+0000 7ff1c37fe640 1 -- 172.21.15.16:0/1809613074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc02cbe0 con 0x7ff1b8003970 2023-11-24T06:29:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.873+0000 7ff1c37fe640 1 -- 172.21.15.16:0/1809613074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc040470 con 0x7ff1b8003970 2023-11-24T06:29:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.873+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc0cc860 con 0x7ff1b8003970 2023-11-24T06:29:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.873+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc0cce20 con 0x7ff1b8003970 2023-11-24T06:29:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.874+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc067000 con 0x7ff1b8003970 2023-11-24T06:29:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.875+0000 7ff1c37fe640 1 -- 172.21.15.16:0/1809613074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff1bc0405d0 con 0x7ff1b8003970 2023-11-24T06:29:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.876+0000 7ff1c37fe640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1ac06b0b0 0x7ff1ac06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.876+0000 7ff1c37fe640 1 -- 172.21.15.16:0/1809613074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff1bc0acbe0 con 0x7ff1b8003970 2023-11-24T06:29:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.876+0000 7ff1c1d9b640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1ac06b0b0 0x7ff1ac06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.878+0000 7ff1c1d9b640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1ac06b0b0 0x7ff1ac06d570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc1f1ff0 tx=0x7ff1b4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:30.881+0000 7ff1c37fe640 1 -- 172.21.15.16:0/1809613074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1bc077970 con 0x7ff1b8003970 2023-11-24T06:29:31.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.077+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1cc0675d0 con 0x7ff1b8003970 2023-11-24T06:29:31.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.080+0000 7ff1c37fe640 1 -- 172.21.15.16:0/1809613074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16621 (secure 0 0 0) 0x7ff1bc009dc0 con 0x7ff1b8003970 2023-11-24T06:29:31.083 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:31.084 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:30.006201+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:03.705566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:04.794925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1ac06b0b0 msgr2=0x7ff1ac06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1ac06b0b0 0x7ff1ac06d570 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc1f1ff0 tx=0x7ff1b4009040 comp rx=0 tx=0).stop 2023-11-24T06:29:31.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 msgr2=0x7ff1cc1382f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1cc1382f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc02c850 tx=0x7ff1bc009750 comp rx=0 tx=0).stop 2023-11-24T06:29:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 shutdown_connections 2023-11-24T06:29:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff1ac06b0b0 0x7ff1ac06d570 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1cc0cde40 0x7ff1cc1efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1cc138830 0x7ff1cc0cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 --2- 172.21.15.16:0/1809613074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b8003970 0x7ff1cc1382f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 >> 172.21.15.16:0/1809613074 conn(0x7ff1cc06a800 msgr2=0x7ff1cc0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 shutdown_connections 2023-11-24T06:29:31.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.083+0000 7ff1d085c640 1 -- 172.21.15.16:0/1809613074 wait complete. 2023-11-24T06:29:31.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:29:03.705566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:29:04.794925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 47, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:31.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.327+0000 7f61d3fff640 1 Processor -- start 2023-11-24T06:29:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.327+0000 7f61d3fff640 1 -- start start 2023-11-24T06:29:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.328+0000 7f61d3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61d415d4b0 0x7f61d40d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.329+0000 7f61d3fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d40b2730 con 0x7f61d40b41e0 2023-11-24T06:29:31.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.329+0000 7f61d3fff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d40d1ec0 con 0x7f61d40cf9a0 2023-11-24T06:29:31.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.329+0000 7f61d3fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d40b29d0 con 0x7f61d415d4b0 2023-11-24T06:29:31.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.329+0000 7f61d2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61d415d4b0 0x7f61d40d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.329+0000 7f61d2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61d415d4b0 0x7f61d40d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60084/0 (socket says 172.21.15.16:60084) 2023-11-24T06:29:31.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.329+0000 7f61d2ffd640 1 -- 172.21.15.16:0/4138533082 learned_addr learned my addr 172.21.15.16:0/4138533082 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:31.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.329+0000 7f61d2ffd640 1 -- 172.21.15.16:0/4138533082 >> v1:172.21.15.173:6789/0 conn(0x7f61d40cf9a0 legacy=0x7f61d40cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:31.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.329+0000 7f61d2ffd640 1 -- 172.21.15.16:0/4138533082 >> v1:172.21.15.16:6789/0 conn(0x7f61d40b41e0 legacy=0x7f61d40ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:31.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.329+0000 7f61d2ffd640 1 -- 172.21.15.16:0/4138533082 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40b2cb0 con 0x7f61d415d4b0 2023-11-24T06:29:31.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.330+0000 7f61d2ffd640 1 --2- 172.21.15.16:0/4138533082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d415d4b0 0x7f61d40d1550 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009e50 tx=0x7f61c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23490dc315296adc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.331+0000 7f61d1ffb640 1 -- 172.21.15.16:0/4138533082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c402ca80 con 0x7f61d415d4b0 2023-11-24T06:29:31.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.331+0000 7f61d1ffb640 1 -- 172.21.15.16:0/4138533082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c402cbe0 con 0x7f61d415d4b0 2023-11-24T06:29:31.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.331+0000 7f61d1ffb640 1 -- 172.21.15.16:0/4138533082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c40343f0 con 0x7f61d415d4b0 2023-11-24T06:29:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.332+0000 7f61d3fff640 1 -- 172.21.15.16:0/4138533082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d415d4b0 msgr2=0x7f61d40d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.332+0000 7f61d3fff640 1 --2- 172.21.15.16:0/4138533082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d415d4b0 0x7f61d40d1550 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009e50 tx=0x7f61c402bee0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.332+0000 7f61d3fff640 1 -- 172.21.15.16:0/4138533082 shutdown_connections 2023-11-24T06:29:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.332+0000 7f61d3fff640 1 --2- 172.21.15.16:0/4138533082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d415d4b0 0x7f61d40d1550 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.332+0000 7f61d3fff640 1 -- 172.21.15.16:0/4138533082 >> 172.21.15.16:0/4138533082 conn(0x7f61d40d3000 msgr2=0x7f61d40d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:31.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.332+0000 7f61d3fff640 1 -- 172.21.15.16:0/4138533082 shutdown_connections 2023-11-24T06:29:31.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.332+0000 7f61d3fff640 1 -- 172.21.15.16:0/4138533082 wait complete. 2023-11-24T06:29:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.334+0000 7f61d3fff640 1 Processor -- start 2023-11-24T06:29:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.334+0000 7f61d3fff640 1 -- start start 2023-11-24T06:29:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.336+0000 7f61d3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d40b41e0 0x7f61d409fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.336+0000 7f61d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40cf9a0 0x7f61d40a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.336+0000 7f61d3fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61d415d4b0 0x7f61d409dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.336+0000 7f61d3fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d409d8d0 con 0x7f61d40cf9a0 2023-11-24T06:29:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.336+0000 7f61d3fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d409cce0 con 0x7f61d40b41e0 2023-11-24T06:29:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.336+0000 7f61d3fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40a0950 con 0x7f61d415d4b0 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.336+0000 7f61d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40cf9a0 0x7f61d40a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.336+0000 7f61d27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40cf9a0 0x7f61d40a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60096/0 (socket says 172.21.15.16:60096) 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.336+0000 7f61d27fc640 1 -- 172.21.15.16:0/3901961851 learned_addr learned my addr 172.21.15.16:0/3901961851 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d2ffd640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d40b41e0 0x7f61d409fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d37fe640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61d415d4b0 0x7f61d409dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d27fc640 1 -- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61d415d4b0 msgr2=0x7f61d409dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d27fc640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61d415d4b0 0x7f61d409dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d27fc640 1 -- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d40b41e0 msgr2=0x7f61d409fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d27fc640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d40b41e0 0x7f61d409fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d27fc640 1 -- 172.21.15.16:0/3901961851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4009b30 con 0x7f61d40cf9a0 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d37fe640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61d415d4b0 0x7f61d409dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d27fc640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40cf9a0 0x7f61d40a0410 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c440 tx=0x7f61bc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61cb7fe640 1 -- 172.21.15.16:0/3901961851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc005480 con 0x7f61d40cf9a0 2023-11-24T06:29:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61cb7fe640 1 -- 172.21.15.16:0/3901961851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc0035f0 con 0x7f61d40cf9a0 2023-11-24T06:29:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d415da10 con 0x7f61d40cf9a0 2023-11-24T06:29:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.337+0000 7f61cb7fe640 1 -- 172.21.15.16:0/3901961851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0039c0 con 0x7f61d40cf9a0 2023-11-24T06:29:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.338+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d409cf30 con 0x7f61d40cf9a0 2023-11-24T06:29:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.339+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d409da10 con 0x7f61d40cf9a0 2023-11-24T06:29:31.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.340+0000 7f61cb7fe640 1 -- 172.21.15.16:0/3901961851 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f61bc003b20 con 0x7f61d40cf9a0 2023-11-24T06:29:31.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.341+0000 7f61cb7fe640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61b406b080 0x7f61b406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.342+0000 7f61cb7fe640 1 -- 172.21.15.16:0/3901961851 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f61bc088fe0 con 0x7f61d40cf9a0 2023-11-24T06:29:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.342+0000 7f61d2ffd640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61b406b080 0x7f61b406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.343+0000 7f61d2ffd640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61b406b080 0x7f61b406d540 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f61c4035000 tx=0x7f61c4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.347+0000 7f61cb7fe640 1 -- 172.21.15.16:0/3901961851 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61d409da10 con 0x7f61d40cf9a0 2023-11-24T06:29:31.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.549+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61d409da10 con 0x7f61d40cf9a0 2023-11-24T06:29:31.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.550+0000 7f61cb7fe640 1 -- 172.21.15.16:0/3901961851 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61d409da10 con 0x7f61d40cf9a0 2023-11-24T06:29:31.553 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:29:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.552+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61b406b080 msgr2=0x7f61b406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.552+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61b406b080 0x7f61b406d540 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f61c4035000 tx=0x7f61c4030000 comp rx=0 tx=0).stop 2023-11-24T06:29:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.552+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40cf9a0 msgr2=0x7f61d40a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.552+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40cf9a0 0x7f61d40a0410 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c440 tx=0x7f61bc00c900 comp rx=0 tx=0).stop 2023-11-24T06:29:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.553+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 shutdown_connections 2023-11-24T06:29:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.553+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f61b406b080 0x7f61b406d540 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.553+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61d415d4b0 0x7f61d409dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.553+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d40cf9a0 0x7f61d40a0410 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.553+0000 7f61d3fff640 1 --2- 172.21.15.16:0/3901961851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61d40b41e0 0x7f61d409fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.553+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 >> 172.21.15.16:0/3901961851 conn(0x7f61d40d3000 msgr2=0x7f61d406c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.553+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 shutdown_connections 2023-11-24T06:29:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.553+0000 7f61d3fff640 1 -- 172.21.15.16:0/3901961851 wait complete. 2023-11-24T06:29:31.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:29:31.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.804+0000 7f715b59e640 1 Processor -- start 2023-11-24T06:29:31.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.805+0000 7f715b59e640 1 -- start start 2023-11-24T06:29:31.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.806+0000 7f715b59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f715c071e40 0x7f715c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.806+0000 7f715b59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715c06c730 con 0x7f715c143810 2023-11-24T06:29:31.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.806+0000 7f715b59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715c0724d0 con 0x7f715c15d770 2023-11-24T06:29:31.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.806+0000 7f715b59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c143bf0 con 0x7f715c071e40 2023-11-24T06:29:31.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.806+0000 7f715a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f715c143810 0x7f715c15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42738/0 (socket says 172.21.15.16:42738) 2023-11-24T06:29:31.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.807+0000 7f715a59c640 1 -- 172.21.15.16:0/907576349 learned_addr learned my addr 172.21.15.16:0/907576349 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:31.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.807+0000 7f7159d9b640 1 --2- 172.21.15.16:0/907576349 >> v2:172.21.15.16:3300/0 conn(0x7f715c071e40 0x7f715c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.807+0000 7f715959a640 1 -- 172.21.15.16:0/907576349 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455115344 0 0) 0x7f715c06c730 con 0x7f715c143810 2023-11-24T06:29:31.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.807+0000 7f7159d9b640 1 -- 172.21.15.16:0/907576349 >> v1:172.21.15.173:6789/0 conn(0x7f715c15d770 legacy=0x7f715c15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:31.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.807+0000 7f7159d9b640 1 -- 172.21.15.16:0/907576349 >> v1:172.21.15.16:6789/0 conn(0x7f715c143810 legacy=0x7f715c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.807+0000 7f7159d9b640 1 -- 172.21.15.16:0/907576349 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c06c9f0 con 0x7f715c071e40 2023-11-24T06:29:31.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.808+0000 7f7159d9b640 1 --2- 172.21.15.16:0/907576349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 0x7f715c0ae170 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f71500097a0 tx=0x7f715002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ae07d030505bed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.808+0000 7f715959a640 1 -- 172.21.15.16:0/907576349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715002f050 con 0x7f715c071e40 2023-11-24T06:29:31.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.809+0000 7f715959a640 1 -- 172.21.15.16:0/907576349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7150031700 con 0x7f715c071e40 2023-11-24T06:29:31.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.809+0000 7f715959a640 1 -- 172.21.15.16:0/907576349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150031a90 con 0x7f715c071e40 2023-11-24T06:29:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.809+0000 7f715b59e640 1 -- 172.21.15.16:0/907576349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 msgr2=0x7f715c0ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.809+0000 7f715b59e640 1 --2- 172.21.15.16:0/907576349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 0x7f715c0ae170 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f71500097a0 tx=0x7f715002b7a0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.809+0000 7f715b59e640 1 -- 172.21.15.16:0/907576349 shutdown_connections 2023-11-24T06:29:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.809+0000 7f715b59e640 1 --2- 172.21.15.16:0/907576349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 0x7f715c0ae170 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.809+0000 7f715b59e640 1 -- 172.21.15.16:0/907576349 >> 172.21.15.16:0/907576349 conn(0x7f715c06a7d0 msgr2=0x7f715c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.809+0000 7f715b59e640 1 -- 172.21.15.16:0/907576349 shutdown_connections 2023-11-24T06:29:31.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.809+0000 7f715b59e640 1 -- 172.21.15.16:0/907576349 wait complete. 2023-11-24T06:29:31.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.811+0000 7f715b59e640 1 Processor -- start 2023-11-24T06:29:31.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.811+0000 7f715b59e640 1 -- start start 2023-11-24T06:29:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.812+0000 7f715b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 0x7f715c1f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.812+0000 7f715b59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f715c143810 0x7f715c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.812+0000 7f715b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f715c15d770 0x7f715c141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.812+0000 7f715b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0722f0 con 0x7f715c071e40 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.812+0000 7f715b59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0ce9d0 con 0x7f715c143810 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.812+0000 7f715b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0ce010 con 0x7f715c15d770 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.812+0000 7f715a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 0x7f715c1f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f715a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 0x7f715c1f24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60124/0 (socket says 172.21.15.16:60124) 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f715a59c640 1 -- 172.21.15.16:0/1771051918 learned_addr learned my addr 172.21.15.16:0/1771051918 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f7159d9b640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f715c143810 0x7f715c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f715ad9d640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f715c15d770 0x7f715c141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f715a59c640 1 -- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f715c15d770 msgr2=0x7f715c141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f715a59c640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f715c15d770 0x7f715c141e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f715a59c640 1 -- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f715c143810 msgr2=0x7f715c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f715a59c640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f715c143810 0x7f715c0af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f715a59c640 1 -- 172.21.15.16:0/1771051918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7150009400 con 0x7f715c071e40 2023-11-24T06:29:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.813+0000 7f715a59c640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 0x7f715c1f24d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c530 tx=0x7f714c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.814+0000 7f714affd640 1 -- 172.21.15.16:0/1771051918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c0074e0 con 0x7f715c071e40 2023-11-24T06:29:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.814+0000 7f714affd640 1 -- 172.21.15.16:0/1771051918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c007660 con 0x7f715c071e40 2023-11-24T06:29:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.814+0000 7f714affd640 1 -- 172.21.15.16:0/1771051918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c004a20 con 0x7f715c071e40 2023-11-24T06:29:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.814+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c0af950 con 0x7f715c071e40 2023-11-24T06:29:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.814+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c0b0410 con 0x7f715c071e40 2023-11-24T06:29:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.815+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c0b0270 con 0x7f715c071e40 2023-11-24T06:29:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.816+0000 7f714affd640 1 -- 172.21.15.16:0/1771051918 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f714c004b80 con 0x7f715c071e40 2023-11-24T06:29:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.817+0000 7f714affd640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f713c06b0b0 0x7f713c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.817+0000 7f714affd640 1 -- 172.21.15.16:0/1771051918 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f714c088bc0 con 0x7f715c071e40 2023-11-24T06:29:31.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.818+0000 7f7159d9b640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f713c06b0b0 0x7f713c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:31.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.819+0000 7f7159d9b640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f713c06b0b0 0x7f713c06d570 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7150035000 tx=0x7f7150008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:31.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:31.822+0000 7f714affd640 1 -- 172.21.15.16:0/1771051918 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f715c0b0270 con 0x7f715c071e40 2023-11-24T06:29:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.022+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f715c143d30 con 0x7f715c071e40 2023-11-24T06:29:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.023+0000 7f714affd640 1 -- 172.21.15.16:0/1771051918 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f715c143d30 con 0x7f715c071e40 2023-11-24T06:29:32.026 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:29:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.025+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f713c06b0b0 msgr2=0x7f713c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.025+0000 7f715b59e640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f713c06b0b0 0x7f713c06d570 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7150035000 tx=0x7f7150008030 comp rx=0 tx=0).stop 2023-11-24T06:29:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.025+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 msgr2=0x7f715c1f24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.025+0000 7f715b59e640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 0x7f715c1f24d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c530 tx=0x7f714c00c9f0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.026+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 shutdown_connections 2023-11-24T06:29:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.026+0000 7f715b59e640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f713c06b0b0 0x7f713c06d570 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.026+0000 7f715b59e640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f715c15d770 0x7f715c141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.026+0000 7f715b59e640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f715c143810 0x7f715c0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.026+0000 7f715b59e640 1 --2- 172.21.15.16:0/1771051918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c071e40 0x7f715c1f24d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.026+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 >> 172.21.15.16:0/1771051918 conn(0x7f715c06a7d0 msgr2=0x7f715c06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.026+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 shutdown_connections 2023-11-24T06:29:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.026+0000 7f715b59e640 1 -- 172.21.15.16:0/1771051918 wait complete. 2023-11-24T06:29:32.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:29:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.271+0000 7f14ca2f1640 1 Processor -- start 2023-11-24T06:29:32.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.272+0000 7f14ca2f1640 1 -- start start 2023-11-24T06:29:32.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.273+0000 7f14ca2f1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14c40cf9a0 0x7f14c40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.273+0000 7f14ca2f1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f14c40b31d0 0x7f14c40b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.273+0000 7f14ca2f1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c40d0320 con 0x7f14c415d4b0 2023-11-24T06:29:32.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.273+0000 7f14ca2f1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c40d1c60 con 0x7f14c40cf9a0 2023-11-24T06:29:32.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.273+0000 7f14ca2f1640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c40b2730 con 0x7f14c40b31d0 2023-11-24T06:29:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.274+0000 7f14c8aee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14c40cf9a0 0x7f14c40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.274+0000 7f14c92ef640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f14c415d4b0 0x7f14c40d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42754/0 (socket says 172.21.15.16:42754) 2023-11-24T06:29:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.274+0000 7f14c92ef640 1 -- 172.21.15.16:0/1782615597 learned_addr learned my addr 172.21.15.16:0/1782615597 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.274+0000 7f14c9af0640 1 --2- 172.21.15.16:0/1782615597 >> v2:172.21.15.173:3300/0 conn(0x7f14c40b31d0 0x7f14c40b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.274+0000 7f14c8aee640 1 -- 172.21.15.16:0/1782615597 >> v2:172.21.15.173:3300/0 conn(0x7f14c40b31d0 msgr2=0x7f14c40b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.274+0000 7f14c8aee640 1 --2- 172.21.15.16:0/1782615597 >> v2:172.21.15.173:3300/0 conn(0x7f14c40b31d0 0x7f14c40b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.274+0000 7f14c8aee640 1 -- 172.21.15.16:0/1782615597 >> v1:172.21.15.16:6789/0 conn(0x7f14c415d4b0 legacy=0x7f14c40d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.274+0000 7f14c8aee640 1 -- 172.21.15.16:0/1782615597 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c40d1ec0 con 0x7f14c40cf9a0 2023-11-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.274+0000 7f14bbfff640 1 -- 172.21.15.16:0/1782615597 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021718380 0 0) 0x7f14c40d0320 con 0x7f14c415d4b0 2023-11-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.276+0000 7f14c8aee640 1 --2- 172.21.15.16:0/1782615597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40cf9a0 0x7f14c40cfde0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f14c4072a30 tx=0x7f14b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5207342165f71c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.276+0000 7f14bbfff640 1 -- 172.21.15.16:0/1782615597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4031830 con 0x7f14c40cf9a0 2023-11-24T06:29:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.276+0000 7f14bbfff640 1 -- 172.21.15.16:0/1782615597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b4031990 con 0x7f14c40cf9a0 2023-11-24T06:29:32.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.276+0000 7f14bbfff640 1 -- 172.21.15.16:0/1782615597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4031d20 con 0x7f14c40cf9a0 2023-11-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.277+0000 7f14ca2f1640 1 -- 172.21.15.16:0/1782615597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40cf9a0 msgr2=0x7f14c40cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.277+0000 7f14ca2f1640 1 --2- 172.21.15.16:0/1782615597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40cf9a0 0x7f14c40cfde0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f14c4072a30 tx=0x7f14b402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.277+0000 7f14ca2f1640 1 -- 172.21.15.16:0/1782615597 shutdown_connections 2023-11-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.277+0000 7f14ca2f1640 1 --2- 172.21.15.16:0/1782615597 >> v2:172.21.15.173:3300/0 conn(0x7f14c40b31d0 0x7f14c40b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.277+0000 7f14ca2f1640 1 --2- 172.21.15.16:0/1782615597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c40cf9a0 0x7f14c40cfde0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.277+0000 7f14ca2f1640 1 -- 172.21.15.16:0/1782615597 >> 172.21.15.16:0/1782615597 conn(0x7f14c40d3000 msgr2=0x7f14c40d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.277+0000 7f14ca2f1640 1 -- 172.21.15.16:0/1782615597 shutdown_connections 2023-11-24T06:29:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.277+0000 7f14ca2f1640 1 -- 172.21.15.16:0/1782615597 wait complete. 2023-11-24T06:29:32.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.279+0000 7f14ca2f1640 1 Processor -- start 2023-11-24T06:29:32.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.279+0000 7f14ca2f1640 1 -- start start 2023-11-24T06:29:32.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.280+0000 7f14ca2f1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14c40b31d0 0x7f14c415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.280+0000 7f14ca2f1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14c40cf9a0 0x7f14c40ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.280+0000 7f14ca2f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c415d4b0 0x7f14c409ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.280+0000 7f14ca2f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c415ca00 con 0x7f14c415d4b0 2023-11-24T06:29:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.280+0000 7f14ca2f1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40caae0 con 0x7f14c40b31d0 2023-11-24T06:29:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.280+0000 7f14ca2f1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40cadc0 con 0x7f14c40cf9a0 2023-11-24T06:29:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.280+0000 7f14c9af0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c415d4b0 0x7f14c409ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c9af0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c415d4b0 0x7f14c409ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60144/0 (socket says 172.21.15.16:60144) 2023-11-24T06:29:32.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c9af0640 1 -- 172.21.15.16:0/4268004724 learned_addr learned my addr 172.21.15.16:0/4268004724 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c8aee640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14c40cf9a0 0x7f14c40ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c92ef640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14c40b31d0 0x7f14c415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c9af0640 1 -- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14c40cf9a0 msgr2=0x7f14c40ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c9af0640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14c40cf9a0 0x7f14c40ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c9af0640 1 -- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14c40b31d0 msgr2=0x7f14c415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c9af0640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14c40b31d0 0x7f14c415c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c9af0640 1 -- 172.21.15.16:0/4268004724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b4009400 con 0x7f14c415d4b0 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.281+0000 7f14c9af0640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c415d4b0 0x7f14c409ea40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f14bc007e60 tx=0x7f14bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.282+0000 7f14b9ffb640 1 -- 172.21.15.16:0/4268004724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc014420 con 0x7f14c415d4b0 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.282+0000 7f14b9ffb640 1 -- 172.21.15.16:0/4268004724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14bc0145a0 con 0x7f14c415d4b0 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.282+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c40cb0a0 con 0x7f14c415d4b0 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.282+0000 7f14b9ffb640 1 -- 172.21.15.16:0/4268004724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc014950 con 0x7f14c415d4b0 2023-11-24T06:29:32.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.282+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c40cb660 con 0x7f14c415d4b0 2023-11-24T06:29:32.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.283+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c4067450 con 0x7f14c415d4b0 2023-11-24T06:29:32.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.284+0000 7f14b9ffb640 1 -- 172.21.15.16:0/4268004724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f14bc014ab0 con 0x7f14c415d4b0 2023-11-24T06:29:32.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.285+0000 7f14b9ffb640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14a806b080 0x7f14a806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.285+0000 7f14b9ffb640 1 -- 172.21.15.16:0/4268004724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f14bc08a310 con 0x7f14c415d4b0 2023-11-24T06:29:32.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.286+0000 7f14c92ef640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14a806b080 0x7f14a806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.287+0000 7f14c92ef640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14a806b080 0x7f14a806d540 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f14c0000f60 tx=0x7f14c0007c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.290+0000 7f14b9ffb640 1 -- 172.21.15.16:0/4268004724 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14bc0550e0 con 0x7f14c415d4b0 2023-11-24T06:29:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.489+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14c40cb7f0 con 0x7f14c415d4b0 2023-11-24T06:29:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.490+0000 7f14b9ffb640 1 -- 172.21.15.16:0/4268004724 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14bc058700 con 0x7f14c415d4b0 2023-11-24T06:29:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:29:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14a806b080 msgr2=0x7f14a806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14a806b080 0x7f14a806d540 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f14c0000f60 tx=0x7f14c0007c00 comp rx=0 tx=0).stop 2023-11-24T06:29:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c415d4b0 msgr2=0x7f14c409ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c415d4b0 0x7f14c409ea40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f14bc007e60 tx=0x7f14bc00e4b0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 shutdown_connections 2023-11-24T06:29:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f14a806b080 0x7f14a806d540 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14c415d4b0 0x7f14c409ea40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14c40cf9a0 0x7f14c40ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 --2- 172.21.15.16:0/4268004724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14c40b31d0 0x7f14c415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.492+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 >> 172.21.15.16:0/4268004724 conn(0x7f14c40d3000 msgr2=0x7f14c406b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.493+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 shutdown_connections 2023-11-24T06:29:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.493+0000 7f14ca2f1640 1 -- 172.21.15.16:0/4268004724 wait complete. 2023-11-24T06:29:32.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:29:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.736+0000 7fef933e6640 1 Processor -- start 2023-11-24T06:29:32.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.736+0000 7fef933e6640 1 -- start start 2023-11-24T06:29:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.738+0000 7fef933e6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c15d440 0x7fef8c0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.738+0000 7fef933e6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c0cfd00 con 0x7fef8c06c1f0 2023-11-24T06:29:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.738+0000 7fef933e6640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8c0cffe0 con 0x7fef8c06b700 2023-11-24T06:29:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.738+0000 7fef933e6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0cfaf0 con 0x7fef8c15d440 2023-11-24T06:29:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.738+0000 7fef92be5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fef8c06c1f0 0x7fef8c0cb950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53214/0 (socket says 172.21.15.16:53214) 2023-11-24T06:29:32.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.738+0000 7fef92be5640 1 -- 172.21.15.16:0/2100093790 learned_addr learned my addr 172.21.15.16:0/2100093790 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:32.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef923e4640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c15d440 0x7fef8c0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564925913 0 0) 0x7fef8c0cfd00 con 0x7fef8c06c1f0 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef78003520 con 0x7fef8c06c1f0 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 643596982 0 0) 0x7fef78003520 con 0x7fef8c06c1f0 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c15d440 msgr2=0x7fef8c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c15d440 0x7fef8c0d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 >> v1:172.21.15.173:6789/0 conn(0x7fef8c06b700 legacy=0x7fef8c06bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0d02c0 con 0x7fef8c06c1f0 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fef840019f0 con 0x7fef8c06c1f0 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 >> v1:172.21.15.16:6789/0 conn(0x7fef8c06c1f0 legacy=0x7fef8c0cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef780039a0 0x7fef78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef780047c0 0x7fef78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c15d440 0x7fef78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c0cfaf0 con 0x7fef780047c0 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7800ba30 con 0x7fef780039a0 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.739+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7800bbf0 con 0x7fef8c15d440 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.740+0000 7fef91be3640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef780047c0 0x7fef78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.740+0000 7fef91be3640 1 -- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c15d440 msgr2=0x7fef78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.740+0000 7fef91be3640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c15d440 0x7fef78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.740+0000 7fef91be3640 1 -- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef780039a0 msgr2=0x7fef78003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:32.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.740+0000 7fef923e4640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef780039a0 0x7fef78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.740+0000 7fef91be3640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef780039a0 0x7fef78003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.740+0000 7fef91be3640 1 -- 172.21.15.16:0/2100093790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78003520 con 0x7fef780047c0 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.741+0000 7fef91be3640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef780047c0 0x7fef78004c40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fef7c001720 tx=0x7fef780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17568e3e09cf3183 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.741+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0052d0 con 0x7fef780047c0 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.741+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c02e050 con 0x7fef780047c0 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.741+0000 7fef913e2640 1 -- 172.21.15.16:0/2100093790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0035f0 con 0x7fef780047c0 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.742+0000 7fef933e6640 1 -- 172.21.15.16:0/2100093790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef780047c0 msgr2=0x7fef78004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.742+0000 7fef933e6640 1 --2- 172.21.15.16:0/2100093790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef780047c0 0x7fef78004c40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fef7c001720 tx=0x7fef780057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.742+0000 7fef92be5640 1 -- 172.21.15.16:0/2100093790 reap_dead start 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.742+0000 7fef933e6640 1 -- 172.21.15.16:0/2100093790 shutdown_connections 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.742+0000 7fef933e6640 1 -- 172.21.15.16:0/2100093790 >> 172.21.15.16:0/2100093790 conn(0x7fef8c0d0de0 msgr2=0x7fef8c0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.742+0000 7fef933e6640 1 -- 172.21.15.16:0/2100093790 shutdown_connections 2023-11-24T06:29:32.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.742+0000 7fef933e6640 1 -- 172.21.15.16:0/2100093790 wait complete. 2023-11-24T06:29:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.744+0000 7fef933e6640 1 Processor -- start 2023-11-24T06:29:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.744+0000 7fef933e6640 1 -- start start 2023-11-24T06:29:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.745+0000 7fef933e6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef780047c0 0x7fef8c0b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.745+0000 7fef933e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8c0adb50 0x7fef8c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.745+0000 7fef933e6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c0c5090 0x7fef8c0c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.745+0000 7fef933e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c14cc10 con 0x7fef8c0adb50 2023-11-24T06:29:32.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.745+0000 7fef933e6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c14d1a0 con 0x7fef780047c0 2023-11-24T06:29:32.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.745+0000 7fef933e6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c14ce50 con 0x7fef8c0c5090 2023-11-24T06:29:32.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.745+0000 7fef91be3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8c0adb50 0x7fef8c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.745+0000 7fef91be3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8c0adb50 0x7fef8c0b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50020/0 (socket says 172.21.15.16:50020) 2023-11-24T06:29:32.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.745+0000 7fef91be3640 1 -- 172.21.15.16:0/1070861238 learned_addr learned my addr 172.21.15.16:0/1070861238 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:32.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.746+0000 7fef923e4640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef780047c0 0x7fef8c0b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.746+0000 7fef91be3640 1 -- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c0c5090 msgr2=0x7fef8c0c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.746+0000 7fef92be5640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c0c5090 0x7fef8c0c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.746+0000 7fef91be3640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c0c5090 0x7fef8c0c54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.746+0000 7fef91be3640 1 -- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef780047c0 msgr2=0x7fef8c0b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.746+0000 7fef91be3640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef780047c0 0x7fef8c0b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.746+0000 7fef91be3640 1 -- 172.21.15.16:0/1070861238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0c6080 con 0x7fef8c0adb50 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.746+0000 7fef91be3640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8c0adb50 0x7fef8c0b0700 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005040 tx=0x7fef7c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.747+0000 7fef82ffd640 1 -- 172.21.15.16:0/1070861238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0076c0 con 0x7fef8c0adb50 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.747+0000 7fef82ffd640 1 -- 172.21.15.16:0/1070861238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c007a30 con 0x7fef8c0adb50 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.747+0000 7fef82ffd640 1 -- 172.21.15.16:0/1070861238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c006be0 con 0x7fef8c0adb50 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.747+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c0c6360 con 0x7fef8c0adb50 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.747+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c0c5a30 con 0x7fef8c0adb50 2023-11-24T06:29:32.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.748+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c06c010 con 0x7fef8c0adb50 2023-11-24T06:29:32.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.749+0000 7fef82ffd640 1 -- 172.21.15.16:0/1070861238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fef7c006d40 con 0x7fef8c0adb50 2023-11-24T06:29:32.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.750+0000 7fef82ffd640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef7006b080 0x7fef7006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:32.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.750+0000 7fef82ffd640 1 -- 172.21.15.16:0/1070861238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fef7c0ab870 con 0x7fef8c0adb50 2023-11-24T06:29:32.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.751+0000 7fef923e4640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef7006b080 0x7fef7006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:32.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.752+0000 7fef923e4640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef7006b080 0x7fef7006d540 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fef880099f0 tx=0x7fef88008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:32.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.754+0000 7fef82ffd640 1 -- 172.21.15.16:0/1070861238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef7c076650 con 0x7fef8c0adb50 2023-11-24T06:29:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.962+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef8c0c6880 con 0x7fef8c0adb50 2023-11-24T06:29:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.963+0000 7fef82ffd640 1 -- 172.21.15.16:0/1070861238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7fef7c079c70 con 0x7fef8c0adb50 2023-11-24T06:29:32.967 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:29:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.966+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef7006b080 msgr2=0x7fef7006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.966+0000 7fef933e6640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef7006b080 0x7fef7006d540 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fef880099f0 tx=0x7fef88008040 comp rx=0 tx=0).stop 2023-11-24T06:29:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.966+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8c0adb50 msgr2=0x7fef8c0b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.966+0000 7fef933e6640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8c0adb50 0x7fef8c0b0700 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fef7c005040 tx=0x7fef7c001120 comp rx=0 tx=0).stop 2023-11-24T06:29:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.967+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 shutdown_connections 2023-11-24T06:29:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.967+0000 7fef933e6640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef7006b080 0x7fef7006d540 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.967+0000 7fef933e6640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef8c0c5090 0x7fef8c0c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.967+0000 7fef933e6640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8c0adb50 0x7fef8c0b0700 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.967+0000 7fef933e6640 1 --2- 172.21.15.16:0/1070861238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef780047c0 0x7fef8c0b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.967+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 >> 172.21.15.16:0/1070861238 conn(0x7fef8c0d0de0 msgr2=0x7fef8c15db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.967+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 shutdown_connections 2023-11-24T06:29:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:32.967+0000 7fef933e6640 1 -- 172.21.15.16:0/1070861238 wait complete. 2023-11-24T06:29:32.988 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi173.front.sepia.ceph.com... 2023-11-24T06:29:32.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:32.988 DEBUG:teuthology.orchestra.run.smithi173:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:29:33.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[96197]: fuse finished with error 0 and tester_r 0 2023-11-24T06:29:33.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:29:33.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:33.094 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:29:33.278 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:29:33.279 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:33.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:29:33.344 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:29:33.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:33.345 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:29:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:29:33.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:29:33.407 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:29:33.407 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:33.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:29:33.472 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:29:33.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:29:33.473 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:29:33.527 INFO:teuthology.orchestra.run.smithi173.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:29:33.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:29:33.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.755+0000 7f02b889a640 1 Processor -- start 2023-11-24T06:29:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.755+0000 7f02b889a640 1 -- start start 2023-11-24T06:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.757+0000 7f02b889a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02b413cfa0 0x7f02b415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.757+0000 7f02b889a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b413fcb0 0x7f02b40b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.757+0000 7f02b889a640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b40cdec0 con 0x7f02b40cd530 2023-11-24T06:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.757+0000 7f02b889a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b40ce1a0 con 0x7f02b413cfa0 2023-11-24T06:29:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.757+0000 7f02b889a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40ce480 con 0x7f02b413fcb0 2023-11-24T06:29:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.757+0000 7f02b2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02b413cfa0 0x7f02b415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.757+0000 7f02b2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02b413cfa0 0x7f02b415afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50034/0 (socket says 172.21.15.16:50034) 2023-11-24T06:29:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.757+0000 7f02b2d9d640 1 -- 172.21.15.16:0/3471832260 learned_addr learned my addr 172.21.15.16:0/3471832260 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.758+0000 7f02b359e640 1 --2- 172.21.15.16:0/3471832260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b413fcb0 0x7f02b40b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.758+0000 7f02b2d9d640 1 -- 172.21.15.16:0/3471832260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b413fcb0 msgr2=0x7f02b40b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.758+0000 7f02b2d9d640 1 --2- 172.21.15.16:0/3471832260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b413fcb0 0x7f02b40b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.758+0000 7f02b2d9d640 1 -- 172.21.15.16:0/3471832260 >> v1:172.21.15.173:6789/0 conn(0x7f02b40cd530 legacy=0x7f02b413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.758+0000 7f02b2d9d640 1 -- 172.21.15.16:0/3471832260 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b40ce760 con 0x7f02b413cfa0 2023-11-24T06:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.759+0000 7f02b2d9d640 1 --2- 172.21.15.16:0/3471832260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 0x7f02b415afb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009870 tx=0x7f02a402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c097e60766b4e66 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.759+0000 7f02b259c640 1 -- 172.21.15.16:0/3471832260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4031640 con 0x7f02b413cfa0 2023-11-24T06:29:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.759+0000 7f02b259c640 1 -- 172.21.15.16:0/3471832260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a40317a0 con 0x7f02b413cfa0 2023-11-24T06:29:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.759+0000 7f02b259c640 1 -- 172.21.15.16:0/3471832260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4031b30 con 0x7f02b413cfa0 2023-11-24T06:29:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.760+0000 7f02b889a640 1 -- 172.21.15.16:0/3471832260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 msgr2=0x7f02b415afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.760+0000 7f02b889a640 1 --2- 172.21.15.16:0/3471832260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 0x7f02b415afb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009870 tx=0x7f02a402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.760+0000 7f02b889a640 1 -- 172.21.15.16:0/3471832260 shutdown_connections 2023-11-24T06:29:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.760+0000 7f02b889a640 1 --2- 172.21.15.16:0/3471832260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b413fcb0 0x7f02b40b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.760+0000 7f02b889a640 1 --2- 172.21.15.16:0/3471832260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 0x7f02b415afb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.760+0000 7f02b889a640 1 -- 172.21.15.16:0/3471832260 >> 172.21.15.16:0/3471832260 conn(0x7f02b406a800 msgr2=0x7f02b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.760+0000 7f02b889a640 1 -- 172.21.15.16:0/3471832260 shutdown_connections 2023-11-24T06:29:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.760+0000 7f02b889a640 1 -- 172.21.15.16:0/3471832260 wait complete. 2023-11-24T06:29:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.762+0000 7f02b889a640 1 Processor -- start 2023-11-24T06:29:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.762+0000 7f02b889a640 1 -- start start 2023-11-24T06:29:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.763+0000 7f02b889a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b40cd530 0x7f02b4137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.763+0000 7f02b889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 0x7f02b413e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.763+0000 7f02b889a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02b413fcb0 0x7f02b413e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.763+0000 7f02b889a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b4155000 con 0x7f02b413cfa0 2023-11-24T06:29:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.763+0000 7f02b889a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b4137f00 con 0x7f02b413fcb0 2023-11-24T06:29:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.763+0000 7f02b889a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b4137740 con 0x7f02b40cd530 2023-11-24T06:29:33.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 0x7f02b413e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 0x7f02b413e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50038/0 (socket says 172.21.15.16:50038) 2023-11-24T06:29:33.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02abfff640 1 -- 172.21.15.16:0/242132712 learned_addr learned my addr 172.21.15.16:0/242132712 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:33.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02b359e640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02b413fcb0 0x7f02b413e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02b2d9d640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b40cd530 0x7f02b4137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02abfff640 1 -- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b40cd530 msgr2=0x7f02b4137200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02abfff640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b40cd530 0x7f02b4137200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02abfff640 1 -- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02b413fcb0 msgr2=0x7f02b413e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:33.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02abfff640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02b413fcb0 0x7f02b413e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:33.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02abfff640 1 -- 172.21.15.16:0/242132712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a40094d0 con 0x7f02b413cfa0 2023-11-24T06:29:33.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.764+0000 7f02abfff640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 0x7f02b413e460 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f029c00c450 tx=0x7f029c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:33.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.765+0000 7f02ab7fe640 1 -- 172.21.15.16:0/242132712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c0052d0 con 0x7f02b413cfa0 2023-11-24T06:29:33.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.765+0000 7f02ab7fe640 1 -- 172.21.15.16:0/242132712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c005430 con 0x7f02b413cfa0 2023-11-24T06:29:33.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.765+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b4154c60 con 0x7f02b413cfa0 2023-11-24T06:29:33.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.765+0000 7f02ab7fe640 1 -- 172.21.15.16:0/242132712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c003830 con 0x7f02b413cfa0 2023-11-24T06:29:33.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.765+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b4138110 con 0x7f02b413cfa0 2023-11-24T06:29:33.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.766+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b4155140 con 0x7f02b413cfa0 2023-11-24T06:29:33.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.767+0000 7f02ab7fe640 1 -- 172.21.15.16:0/242132712 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f029c003990 con 0x7f02b413cfa0 2023-11-24T06:29:33.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.768+0000 7f02ab7fe640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f029406b080 0x7f029406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:33.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.769+0000 7f02ab7fe640 1 -- 172.21.15.16:0/242132712 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f029c088ff0 con 0x7f02b413cfa0 2023-11-24T06:29:33.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.769+0000 7f02b2d9d640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f029406b080 0x7f029406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:33.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.770+0000 7f02b2d9d640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f029406b080 0x7f029406d540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f02a4036000 tx=0x7f02a403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:33.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:33.773+0000 7f02ab7fe640 1 -- 172.21.15.16:0/242132712 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02b4155140 con 0x7f02b413cfa0 2023-11-24T06:29:34.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.007+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02b4155140 con 0x7f02b413cfa0 2023-11-24T06:29:34.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.007+0000 7f02ab7fe640 1 -- 172.21.15.16:0/242132712 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f02b4155140 con 0x7f02b413cfa0 2023-11-24T06:29:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:34.014 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110540698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110540698},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110540698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1298790428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1298790428},{"type":"v1","addr":"172.21.15.16:6835","nonce":1298790428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2876598187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2876598187},{"type":"v1","addr":"172.21.15.173:6836","nonce":2876598187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:05.754412+0000","modified":"2023-11-24T06:29:08.837901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/2818758442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2818758442},{"type":"v1","addr":"172.21.15.16:6833","nonce":2818758442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:29:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2023-11-24T06:29:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f029406b080 msgr2=0x7f029406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f029406b080 0x7f029406d540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f02a4036000 tx=0x7f02a403c040 comp rx=0 tx=0).stop 2023-11-24T06:29:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 msgr2=0x7f02b413e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 0x7f02b413e460 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f029c00c450 tx=0x7f029c00c910 comp rx=0 tx=0).stop 2023-11-24T06:29:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 shutdown_connections 2023-11-24T06:29:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f029406b080 0x7f029406d540 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02b413fcb0 0x7f02b413e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b413cfa0 0x7f02b413e460 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 --2- 172.21.15.16:0/242132712 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02b40cd530 0x7f02b4137200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.010+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 >> 172.21.15.16:0/242132712 conn(0x7f02b406a800 msgr2=0x7f02b41537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:34.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.011+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 shutdown_connections 2023-11-24T06:29:34.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.011+0000 7f02b889a640 1 -- 172.21.15.16:0/242132712 wait complete. 2023-11-24T06:29:34.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:34.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.250+0000 7f380a1ef640 1 Processor -- start 2023-11-24T06:29:34.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.250+0000 7f380a1ef640 1 -- start start 2023-11-24T06:29:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.252+0000 7f380a1ef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38040cc500 0x7f38040cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.252+0000 7f380a1ef640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f380406d160 0x7f380406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.252+0000 7f380a1ef640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3804070e50 con 0x7f3804143c40 2023-11-24T06:29:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.252+0000 7f380a1ef640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38040723c0 con 0x7f380406d160 2023-11-24T06:29:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.252+0000 7f380a1ef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3804144020 con 0x7f38040cc500 2023-11-24T06:29:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.253+0000 7f38091ed640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f3804143c40 0x7f3804071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:34484/0 (socket says 172.21.15.16:34484) 2023-11-24T06:29:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.253+0000 7f38091ed640 1 -- 172.21.15.16:0/1697370783 learned_addr learned my addr 172.21.15.16:0/1697370783 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.253+0000 7f38089ec640 1 --2- 172.21.15.16:0/1697370783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38040cc500 0x7f38040cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.253+0000 7f38099ee640 1 --2- 172.21.15.16:0/1697370783 >> v2:172.21.15.173:3300/0 conn(0x7f380406d160 0x7f380406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.254+0000 7f37fbfff640 1 -- 172.21.15.16:0/1697370783 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936204866 0 0) 0x7f3804070e50 con 0x7f3804143c40 2023-11-24T06:29:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.254+0000 7f37fbfff640 1 -- 172.21.15.16:0/1697370783 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f0003520 con 0x7f3804143c40 2023-11-24T06:29:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.254+0000 7f38089ec640 1 -- 172.21.15.16:0/1697370783 >> v2:172.21.15.173:3300/0 conn(0x7f380406d160 msgr2=0x7f380406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.254+0000 7f38089ec640 1 --2- 172.21.15.16:0/1697370783 >> v2:172.21.15.173:3300/0 conn(0x7f380406d160 0x7f380406d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.254+0000 7f38089ec640 1 -- 172.21.15.16:0/1697370783 >> v1:172.21.15.173:6789/0 conn(0x7f3804143c40 legacy=0x7f3804071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.254+0000 7f38089ec640 1 -- 172.21.15.16:0/1697370783 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38040ccea0 con 0x7f38040cc500 2023-11-24T06:29:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.254+0000 7f38099ee640 1 --2- 172.21.15.16:0/1697370783 >> v2:172.21.15.173:3300/0 conn(0x7f380406d160 0x7f380406d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.255+0000 7f38089ec640 1 --2- 172.21.15.16:0/1697370783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38040cc500 0x7f38040cc960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37f40097a0 tx=0x7f37f402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4105e5aa503baa6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.255+0000 7f37fbfff640 1 -- 172.21.15.16:0/1697370783 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4031780 con 0x7f38040cc500 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.255+0000 7f37fbfff640 1 -- 172.21.15.16:0/1697370783 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f40318e0 con 0x7f38040cc500 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.255+0000 7f37fbfff640 1 -- 172.21.15.16:0/1697370783 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4031c70 con 0x7f38040cc500 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.256+0000 7f380a1ef640 1 -- 172.21.15.16:0/1697370783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38040cc500 msgr2=0x7f38040cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.256+0000 7f380a1ef640 1 --2- 172.21.15.16:0/1697370783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38040cc500 0x7f38040cc960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37f40097a0 tx=0x7f37f402b820 comp rx=0 tx=0).stop 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.256+0000 7f380a1ef640 1 -- 172.21.15.16:0/1697370783 shutdown_connections 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.256+0000 7f380a1ef640 1 --2- 172.21.15.16:0/1697370783 >> v2:172.21.15.173:3300/0 conn(0x7f380406d160 0x7f380406d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.256+0000 7f380a1ef640 1 --2- 172.21.15.16:0/1697370783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38040cc500 0x7f38040cc960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.256+0000 7f380a1ef640 1 -- 172.21.15.16:0/1697370783 >> 172.21.15.16:0/1697370783 conn(0x7f380406a800 msgr2=0x7f380406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.256+0000 7f380a1ef640 1 -- 172.21.15.16:0/1697370783 shutdown_connections 2023-11-24T06:29:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.256+0000 7f380a1ef640 1 -- 172.21.15.16:0/1697370783 wait complete. 2023-11-24T06:29:34.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.258+0000 7f380a1ef640 1 Processor -- start 2023-11-24T06:29:34.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.258+0000 7f380a1ef640 1 -- start start 2023-11-24T06:29:34.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.259+0000 7f380a1ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f380406d160 0x7f380414d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.259+0000 7f380a1ef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38040cc500 0x7f3804138620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.259+0000 7f380a1ef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3804143c40 0x7f38041eea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.259+0000 7f380a1ef640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3804139050 con 0x7f380406d160 2023-11-24T06:29:34.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.259+0000 7f380a1ef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3804140e80 con 0x7f38040cc500 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.259+0000 7f380a1ef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3804141160 con 0x7f3804143c40 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.259+0000 7f38091ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f380406d160 0x7f380414d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.259+0000 7f38091ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f380406d160 0x7f380414d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50052/0 (socket says 172.21.15.16:50052) 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.259+0000 7f38091ed640 1 -- 172.21.15.16:0/3425503172 learned_addr learned my addr 172.21.15.16:0/3425503172 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.260+0000 7f38089ec640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38040cc500 0x7f3804138620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.260+0000 7f38099ee640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3804143c40 0x7f38041eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.260+0000 7f38091ed640 1 -- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3804143c40 msgr2=0x7f38041eea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.260+0000 7f38091ed640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3804143c40 0x7f38041eea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.260+0000 7f38091ed640 1 -- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38040cc500 msgr2=0x7f3804138620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.260+0000 7f38091ed640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38040cc500 0x7f3804138620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.260+0000 7f38091ed640 1 -- 172.21.15.16:0/3425503172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc009a70 con 0x7f380406d160 2023-11-24T06:29:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.260+0000 7f38099ee640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3804143c40 0x7f38041eea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.260+0000 7f38091ed640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f380406d160 0x7f380414d590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f380000c390 tx=0x7f380000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.261+0000 7f37f9ffb640 1 -- 172.21.15.16:0/3425503172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800005ce0 con 0x7f380406d160 2023-11-24T06:29:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.261+0000 7f37f9ffb640 1 -- 172.21.15.16:0/3425503172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800005e40 con 0x7f380406d160 2023-11-24T06:29:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.261+0000 7f37f9ffb640 1 -- 172.21.15.16:0/3425503172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38000072f0 con 0x7f380406d160 2023-11-24T06:29:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.261+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f4009400 con 0x7f380406d160 2023-11-24T06:29:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.261+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380414dad0 con 0x7f380406d160 2023-11-24T06:29:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.262+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38041391c0 con 0x7f380406d160 2023-11-24T06:29:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.263+0000 7f37f9ffb640 1 -- 172.21.15.16:0/3425503172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3800007450 con 0x7f380406d160 2023-11-24T06:29:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.264+0000 7f37f9ffb640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f37e806b080 0x7f37e806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.264+0000 7f37f9ffb640 1 -- 172.21.15.16:0/3425503172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3800089140 con 0x7f380406d160 2023-11-24T06:29:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.264+0000 7f38089ec640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f37e806b080 0x7f37e806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.266+0000 7f38089ec640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f37e806b080 0x7f37e806d540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f37f4036000 tx=0x7f37f403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.268+0000 7f37f9ffb640 1 -- 172.21.15.16:0/3425503172 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3800053db0 con 0x7f380406d160 2023-11-24T06:29:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.460+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3804138ea0 con 0x7f380406d160 2023-11-24T06:29:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.463+0000 7f37f9ffb640 1 -- 172.21.15.16:0/3425503172 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16621 (secure 0 0 0) 0x7f38000573d0 con 0x7f380406d160 2023-11-24T06:29:34.467 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:34.467 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:30.006201+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:03.705566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:04.794925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f37e806b080 msgr2=0x7f37e806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f37e806b080 0x7f37e806d540 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f37f4036000 tx=0x7f37f403d040 comp rx=0 tx=0).stop 2023-11-24T06:29:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f380406d160 msgr2=0x7f380414d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f380406d160 0x7f380414d590 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f380000c390 tx=0x7f380000c850 comp rx=0 tx=0).stop 2023-11-24T06:29:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 shutdown_connections 2023-11-24T06:29:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f37e806b080 0x7f37e806d540 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3804143c40 0x7f38041eea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38040cc500 0x7f3804138620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 --2- 172.21.15.16:0/3425503172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f380406d160 0x7f380414d590 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.466+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 >> 172.21.15.16:0/3425503172 conn(0x7f380406a800 msgr2=0x7f380406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.467+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 shutdown_connections 2023-11-24T06:29:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.467+0000 7f380a1ef640 1 -- 172.21.15.16:0/3425503172 wait complete. 2023-11-24T06:29:34.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:29:03.705566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:29:04.794925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 47, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:34.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.709+0000 7f9deb59e640 1 Processor -- start 2023-11-24T06:29:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.709+0000 7f9deb59e640 1 -- start start 2023-11-24T06:29:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.711+0000 7f9deb59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9dec13fcb0 0x7f9dec0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.711+0000 7f9deb59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dec0ce440 con 0x7f9dec13cf10 2023-11-24T06:29:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.711+0000 7f9deb59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dec0ce720 con 0x7f9dec0cdc20 2023-11-24T06:29:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.711+0000 7f9deb59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dec0cea00 con 0x7f9dec13fcb0 2023-11-24T06:29:34.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.711+0000 7f9dea59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9dec13cf10 0x7f9dec13f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53224/0 (socket says 172.21.15.16:53224) 2023-11-24T06:29:34.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.711+0000 7f9dea59c640 1 -- 172.21.15.16:0/1805499750 learned_addr learned my addr 172.21.15.16:0/1805499750 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de9d9b640 1 --2- 172.21.15.16:0/1805499750 >> v2:172.21.15.173:3300/0 conn(0x7f9dec13fcb0 0x7f9dec0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180351772 0 0) 0x7f9dec0ce440 con 0x7f9dec13cf10 2023-11-24T06:29:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd4003520 con 0x7f9dec13cf10 2023-11-24T06:29:34.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4127455681 0 0) 0x7f9dd4003520 con 0x7f9dec13cf10 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 >> v2:172.21.15.173:3300/0 conn(0x7f9dec13fcb0 msgr2=0x7f9dec0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 --2- 172.21.15.16:0/1805499750 >> v2:172.21.15.173:3300/0 conn(0x7f9dec13fcb0 0x7f9dec0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 >> v1:172.21.15.173:6789/0 conn(0x7f9dec0cdc20 legacy=0x7f9dec0b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0b1d90 con 0x7f9dec13cf10 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f9ddc001670 con 0x7f9dec13cf10 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 >> v1:172.21.15.16:6789/0 conn(0x7f9dec13cf10 legacy=0x7f9dec13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dd40047c0 0x7f9dd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.712+0000 7f9de959a640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dd4005180 0x7f9dd400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec0cea00 con 0x7f9dd40039c0 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd400e260 con 0x7f9dd40047c0 2023-11-24T06:29:34.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd400e420 con 0x7f9dd4005180 2023-11-24T06:29:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9dea59c640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9de9d9b640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dd40047c0 0x7f9dd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9dea59c640 1 -- 172.21.15.16:0/1805499750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dd4005180 msgr2=0x7f9dd400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9dea59c640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dd4005180 0x7f9dd400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9dea59c640 1 -- 172.21.15.16:0/1805499750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dd40047c0 msgr2=0x7f9dd4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9dea59c640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dd40047c0 0x7f9dd4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9dea59c640 1 -- 172.21.15.16:0/1805499750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd4003520 con 0x7f9dd40039c0 2023-11-24T06:29:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.713+0000 7f9dead9d640 1 -- 172.21.15.16:0/1805499750 reap_dead start 2023-11-24T06:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.714+0000 7f9dea59c640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003dc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00c190 tx=0x7f9dd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11bf7b201d978e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc005ad0 con 0x7f9dd40039c0 2023-11-24T06:29:34.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc005c30 con 0x7f9dd40039c0 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9de959a640 1 -- 172.21.15.16:0/1805499750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc02c810 con 0x7f9dd40039c0 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9deb59e640 1 -- 172.21.15.16:0/1805499750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 msgr2=0x7f9dd4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9deb59e640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003dc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00c190 tx=0x7f9dd40057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9deb59e640 1 -- 172.21.15.16:0/1805499750 shutdown_connections 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9deb59e640 1 --2- 172.21.15.16:0/1805499750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003dc0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9deb59e640 1 -- 172.21.15.16:0/1805499750 >> 172.21.15.16:0/1805499750 conn(0x7f9dec06a7d0 msgr2=0x7f9dec06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9deb59e640 1 -- 172.21.15.16:0/1805499750 shutdown_connections 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.715+0000 7f9deb59e640 1 -- 172.21.15.16:0/1805499750 wait complete. 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.717+0000 7f9deb59e640 1 Processor -- start 2023-11-24T06:29:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.717+0000 7f9deb59e640 1 -- start start 2023-11-24T06:29:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.718+0000 7f9deb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dec0af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.718+0000 7f9deb59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dec15cc20 0x7f9dec13b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.718+0000 7f9deb59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dec15d0a0 0x7f9dec13b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.718+0000 7f9deb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec156770 con 0x7f9dd40039c0 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.718+0000 7f9deb59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec154a80 con 0x7f9dec15d0a0 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.718+0000 7f9deb59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dec13bda0 con 0x7f9dec15cc20 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9dea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dec0af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9dea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dec0af200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50068/0 (socket says 172.21.15.16:50068) 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9dea59c640 1 -- 172.21.15.16:0/3840065857 learned_addr learned my addr 172.21.15.16:0/3840065857 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9dead9d640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dec15d0a0 0x7f9dec13b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9de9d9b640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dec15cc20 0x7f9dec13b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9dea59c640 1 -- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dec15cc20 msgr2=0x7f9dec13b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9dea59c640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dec15cc20 0x7f9dec13b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9dea59c640 1 -- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dec15d0a0 msgr2=0x7f9dec13b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9dea59c640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dec15d0a0 0x7f9dec13b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9dea59c640 1 -- 172.21.15.16:0/3840065857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec1fb1c0 con 0x7f9dd40039c0 2023-11-24T06:29:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.719+0000 7f9de9d9b640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dec15cc20 0x7f9dec13b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.720+0000 7f9dea59c640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dec0af200 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00c2b0 tx=0x7f9ddc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.720+0000 7f9ddaffd640 1 -- 172.21.15.16:0/3840065857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc03c040 con 0x7f9dd40039c0 2023-11-24T06:29:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.720+0000 7f9ddaffd640 1 -- 172.21.15.16:0/3840065857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc007480 con 0x7f9dd40039c0 2023-11-24T06:29:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.720+0000 7f9ddaffd640 1 -- 172.21.15.16:0/3840065857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc005a40 con 0x7f9dd40039c0 2023-11-24T06:29:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.720+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dec1fa2d0 con 0x7f9dd40039c0 2023-11-24T06:29:34.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.720+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dec1fa890 con 0x7f9dd40039c0 2023-11-24T06:29:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.721+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dec1590b0 con 0x7f9dd40039c0 2023-11-24T06:29:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.722+0000 7f9ddaffd640 1 -- 172.21.15.16:0/3840065857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f9ddc032070 con 0x7f9dd40039c0 2023-11-24T06:29:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.723+0000 7f9ddaffd640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9dcc06b0b0 0x7f9dcc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:34.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.723+0000 7f9ddaffd640 1 -- 172.21.15.16:0/3840065857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9ddc0ab410 con 0x7f9dd40039c0 2023-11-24T06:29:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.724+0000 7f9de9d9b640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9dcc06b0b0 0x7f9dcc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.725+0000 7f9de9d9b640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9dcc06b0b0 0x7f9dcc06d570 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9dec15cbb0 tx=0x7f9de0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:34.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.727+0000 7f9ddaffd640 1 -- 172.21.15.16:0/3840065857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9dec1590b0 con 0x7f9dd40039c0 2023-11-24T06:29:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.912+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9dec1590b0 con 0x7f9dd40039c0 2023-11-24T06:29:34.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.913+0000 7f9ddaffd640 1 -- 172.21.15.16:0/3840065857 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9dec1590b0 con 0x7f9dd40039c0 2023-11-24T06:29:34.916 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:29:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9dcc06b0b0 msgr2=0x7f9dcc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9dcc06b0b0 0x7f9dcc06d570 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9dec15cbb0 tx=0x7f9de0008040 comp rx=0 tx=0).stop 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 msgr2=0x7f9dec0af200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dec0af200 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00c2b0 tx=0x7f9ddc02e070 comp rx=0 tx=0).stop 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 shutdown_connections 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9dcc06b0b0 0x7f9dcc06d570 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dec15d0a0 0x7f9dec13b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dec15cc20 0x7f9dec13b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 --2- 172.21.15.16:0/3840065857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dd40039c0 0x7f9dec0af200 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 >> 172.21.15.16:0/3840065857 conn(0x7f9dec06a7d0 msgr2=0x7f9dec06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 shutdown_connections 2023-11-24T06:29:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:34.916+0000 7f9deb59e640 1 -- 172.21.15.16:0/3840065857 wait complete. 2023-11-24T06:29:34.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:29:35.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.156+0000 7f90c2623640 1 Processor -- start 2023-11-24T06:29:35.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.156+0000 7f90c2623640 1 -- start start 2023-11-24T06:29:35.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.158+0000 7f90c2623640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f90bc13df80 0x7f90bc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.158+0000 7f90c2623640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90bc0cf9a0 0x7f90bc0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.158+0000 7f90c2623640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc06c470 con 0x7f90bc13e360 2023-11-24T06:29:35.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.158+0000 7f90c2623640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc06c750 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.158+0000 7f90c2623640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc06ca30 con 0x7f90bc13df80 2023-11-24T06:29:35.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.158+0000 7f90c0bc0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90bc0cf9a0 0x7f90bc0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.158+0000 7f90c0bc0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90bc0cf9a0 0x7f90bc0cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50082/0 (socket says 172.21.15.16:50082) 2023-11-24T06:29:35.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.158+0000 7f90c0bc0640 1 -- 172.21.15.16:0/1527428187 learned_addr learned my addr 172.21.15.16:0/1527428187 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:35.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.159+0000 7f90bbfff640 1 --2- 172.21.15.16:0/1527428187 >> v2:172.21.15.173:3300/0 conn(0x7f90bc13df80 0x7f90bc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.159+0000 7f90c0bc0640 1 -- 172.21.15.16:0/1527428187 >> v2:172.21.15.173:3300/0 conn(0x7f90bc13df80 msgr2=0x7f90bc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.159+0000 7f90c0bc0640 1 --2- 172.21.15.16:0/1527428187 >> v2:172.21.15.173:3300/0 conn(0x7f90bc13df80 0x7f90bc15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.159+0000 7f90c0bc0640 1 -- 172.21.15.16:0/1527428187 >> v1:172.21.15.16:6789/0 conn(0x7f90bc13e360 legacy=0x7f90bc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.159+0000 7f90c0bc0640 1 -- 172.21.15.16:0/1527428187 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc06cd10 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.160+0000 7f90c0bc0640 1 --2- 172.21.15.16:0/1527428187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 0x7f90bc0cd840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f90bc072770 tx=0x7f90b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d3f3975aebfbd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.160+0000 7f90baffd640 1 -- 172.21.15.16:0/1527428187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4030860 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.160+0000 7f90baffd640 1 -- 172.21.15.16:0/1527428187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b40309c0 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.160+0000 7f90baffd640 1 -- 172.21.15.16:0/1527428187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4030d50 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.161+0000 7f90c2623640 1 -- 172.21.15.16:0/1527428187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 msgr2=0x7f90bc0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.161+0000 7f90c2623640 1 --2- 172.21.15.16:0/1527428187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 0x7f90bc0cd840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f90bc072770 tx=0x7f90b402b9a0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.161+0000 7f90c2623640 1 -- 172.21.15.16:0/1527428187 shutdown_connections 2023-11-24T06:29:35.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.161+0000 7f90c2623640 1 --2- 172.21.15.16:0/1527428187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 0x7f90bc0cd840 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.161+0000 7f90c2623640 1 --2- 172.21.15.16:0/1527428187 >> v2:172.21.15.173:3300/0 conn(0x7f90bc13df80 0x7f90bc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.161+0000 7f90c2623640 1 -- 172.21.15.16:0/1527428187 >> 172.21.15.16:0/1527428187 conn(0x7f90bc06a800 msgr2=0x7f90bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:35.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.161+0000 7f90c2623640 1 -- 172.21.15.16:0/1527428187 shutdown_connections 2023-11-24T06:29:35.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.161+0000 7f90c2623640 1 -- 172.21.15.16:0/1527428187 wait complete. 2023-11-24T06:29:35.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.163+0000 7f90c2623640 1 Processor -- start 2023-11-24T06:29:35.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.163+0000 7f90c2623640 1 -- start start 2023-11-24T06:29:35.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.164+0000 7f90c2623640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 0x7f90bc13c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.164+0000 7f90c2623640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90bc13df80 0x7f90bc1f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.164+0000 7f90c2623640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90bc13e360 0x7f90bc152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.164+0000 7f90c2623640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc13c9d0 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.164+0000 7f90c2623640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc13d820 con 0x7f90bc13df80 2023-11-24T06:29:35.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.164+0000 7f90c2623640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc1f3cc0 con 0x7f90bc13e360 2023-11-24T06:29:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.164+0000 7f90bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 0x7f90bc13c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.164+0000 7f90bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 0x7f90bc13c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50084/0 (socket says 172.21.15.16:50084) 2023-11-24T06:29:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.164+0000 7f90bbfff640 1 -- 172.21.15.16:0/1004880037 learned_addr learned my addr 172.21.15.16:0/1004880037 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90bb7fe640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90bc13df80 0x7f90bc1f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90bbfff640 1 -- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90bc13e360 msgr2=0x7f90bc152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90bbfff640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90bc13e360 0x7f90bc152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90bbfff640 1 -- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90bc13df80 msgr2=0x7f90bc1f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90bbfff640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90bc13df80 0x7f90bc1f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90bbfff640 1 -- 172.21.15.16:0/1004880037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b40094d0 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90bb7fe640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90bc13df80 0x7f90bc1f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90bbfff640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 0x7f90bc13c490 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f90ac007ef0 tx=0x7f90ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90b8ff9640 1 -- 172.21.15.16:0/1004880037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac014620 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90b8ff9640 1 -- 172.21.15.16:0/1004880037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ac00ee90 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90b8ff9640 1 -- 172.21.15.16:0/1004880037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0149c0 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc13cc10 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.165+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc13d1d0 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.167+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc0675d0 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.167+0000 7f90b8ff9640 1 -- 172.21.15.16:0/1004880037 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f90ac014b20 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.169+0000 7f90b8ff9640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f909c06b080 0x7f909c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.169+0000 7f90b8ff9640 1 -- 172.21.15.16:0/1004880037 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f90ac08a310 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.170+0000 7f90bb7fe640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f909c06b080 0x7f909c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.171+0000 7f90bb7fe640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f909c06b080 0x7f909c06d540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f90bc06b0e0 tx=0x7f90a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.174+0000 7f90b8ff9640 1 -- 172.21.15.16:0/1004880037 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90ac0550f0 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.360+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90bc06df00 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.361+0000 7f90b8ff9640 1 -- 172.21.15.16:0/1004880037 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f90ac058710 con 0x7f90bc0cf9a0 2023-11-24T06:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:29:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f909c06b080 msgr2=0x7f909c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f909c06b080 0x7f909c06d540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f90bc06b0e0 tx=0x7f90a4009040 comp rx=0 tx=0).stop 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 msgr2=0x7f90bc13c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 0x7f90bc13c490 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f90ac007ef0 tx=0x7f90ac00e4b0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 shutdown_connections 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f909c06b080 0x7f909c06d540 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90bc13e360 0x7f90bc152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90bc13df80 0x7f90bc1f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 --2- 172.21.15.16:0/1004880037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90bc0cf9a0 0x7f90bc13c490 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 >> 172.21.15.16:0/1004880037 conn(0x7f90bc06a800 msgr2=0x7f90bc154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.363+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 shutdown_connections 2023-11-24T06:29:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.364+0000 7f90c2623640 1 -- 172.21.15.16:0/1004880037 wait complete. 2023-11-24T06:29:35.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:29:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.605+0000 7f71a31e6640 1 Processor -- start 2023-11-24T06:29:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.606+0000 7f71a31e6640 1 -- start start 2023-11-24T06:29:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.607+0000 7f71a31e6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0adf20 0x7f719c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.607+0000 7f71a31e6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c15f4e0 con 0x7f719c0ccb50 2023-11-24T06:29:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.607+0000 7f71a31e6640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c071130 con 0x7f719c15bf10 2023-11-24T06:29:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.607+0000 7f71a31e6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c15f970 con 0x7f719c0adf20 2023-11-24T06:29:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.608+0000 7f71a29e5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f719c0ccb50 0x7f719c143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53228/0 (socket says 172.21.15.16:53228) 2023-11-24T06:29:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.608+0000 7f71a29e5640 1 -- 172.21.15.16:0/3282382440 learned_addr learned my addr 172.21.15.16:0/3282382440 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.608+0000 7f71a21e4640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0adf20 0x7f719c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.608+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958942304 0 0) 0x7f719c15f4e0 con 0x7f719c0ccb50 2023-11-24T06:29:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.608+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7188003520 con 0x7f719c0ccb50 2023-11-24T06:29:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2658094323 0 0) 0x7f7188003520 con 0x7f719c0ccb50 2023-11-24T06:29:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0adf20 msgr2=0x7f719c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0adf20 0x7f719c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 >> v1:172.21.15.173:6789/0 conn(0x7f719c15bf10 legacy=0x7f719c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c15c390 con 0x7f719c0ccb50 2023-11-24T06:29:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f71940019f0 con 0x7f719c0ccb50 2023-11-24T06:29:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 >> v1:172.21.15.16:6789/0 conn(0x7f719c0ccb50 legacy=0x7f719c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0adf20 0x7f71880039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7188003f70 0x7f71880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7188004d00 0x7f7188005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c15f970 con 0x7f7188004d00 2023-11-24T06:29:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718800bb70 con 0x7f7188003f70 2023-11-24T06:29:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718800bd30 con 0x7f719c0adf20 2023-11-24T06:29:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.609+0000 7f71a29e5640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7188004d00 0x7f7188005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.610+0000 7f71a21e4640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0adf20 0x7f71880039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.610+0000 7f71a19e3640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7188003f70 0x7f71880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.610+0000 7f71a29e5640 1 -- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0adf20 msgr2=0x7f71880039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:35.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.610+0000 7f71a29e5640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0adf20 0x7f71880039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.610+0000 7f71a29e5640 1 -- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7188003f70 msgr2=0x7f71880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:35.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.610+0000 7f71a29e5640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7188003f70 0x7f71880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.610+0000 7f71a29e5640 1 -- 172.21.15.16:0/3282382440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188003520 con 0x7f7188004d00 2023-11-24T06:29:35.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.610+0000 7f71a19e3640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7188003f70 0x7f71880047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:35.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.611+0000 7f71a29e5640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7188004d00 0x7f7188005180 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f719400a750 tx=0x7f71880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efad59dd7d1c1a83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.611+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194034630 con 0x7f7188004d00 2023-11-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.611+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719402f070 con 0x7f7188004d00 2023-11-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.611+0000 7f71a11e2640 1 -- 172.21.15.16:0/3282382440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940349d0 con 0x7f7188004d00 2023-11-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.612+0000 7f71a31e6640 1 -- 172.21.15.16:0/3282382440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7188004d00 msgr2=0x7f7188005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.612+0000 7f71a31e6640 1 --2- 172.21.15.16:0/3282382440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7188004d00 0x7f7188005180 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f719400a750 tx=0x7f71880057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.612+0000 7f71a29e5640 1 -- 172.21.15.16:0/3282382440 reap_dead start 2023-11-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.612+0000 7f71a31e6640 1 -- 172.21.15.16:0/3282382440 shutdown_connections 2023-11-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.612+0000 7f71a31e6640 1 -- 172.21.15.16:0/3282382440 >> 172.21.15.16:0/3282382440 conn(0x7f719c06a7d0 msgr2=0x7f719c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.612+0000 7f71a31e6640 1 -- 172.21.15.16:0/3282382440 shutdown_connections 2023-11-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.612+0000 7f71a31e6640 1 -- 172.21.15.16:0/3282382440 wait complete. 2023-11-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.614+0000 7f71a31e6640 1 Processor -- start 2023-11-24T06:29:35.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.614+0000 7f71a31e6640 1 -- start start 2023-11-24T06:29:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a31e6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7188004d00 0x7f719c155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a31e6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0b29c0 0x7f719c1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a31e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0b1240 0x7f719c154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a31e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c149a10 con 0x7f719c0b1240 2023-11-24T06:29:35.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a31e6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c1578f0 con 0x7f7188004d00 2023-11-24T06:29:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a31e6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c157bd0 con 0x7f719c0b29c0 2023-11-24T06:29:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a29e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0b1240 0x7f719c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a29e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0b1240 0x7f719c154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50100/0 (socket says 172.21.15.16:50100) 2023-11-24T06:29:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a29e5640 1 -- 172.21.15.16:0/4030189766 learned_addr learned my addr 172.21.15.16:0/4030189766 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a29e5640 1 -- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0b29c0 msgr2=0x7f719c1586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a29e5640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0b29c0 0x7f719c1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a29e5640 1 -- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7188004d00 msgr2=0x7f719c155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a29e5640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7188004d00 0x7f719c155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.615+0000 7f71a29e5640 1 -- 172.21.15.16:0/4030189766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c157eb0 con 0x7f719c0b1240 2023-11-24T06:29:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.616+0000 7f71a29e5640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0b1240 0x7f719c154500 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f719400a870 tx=0x7f719c15b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.616+0000 7f7192ffd640 1 -- 172.21.15.16:0/4030189766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194001250 con 0x7f719c0b1240 2023-11-24T06:29:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.616+0000 7f7192ffd640 1 -- 172.21.15.16:0/4030189766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7194034640 con 0x7f719c0b1240 2023-11-24T06:29:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.616+0000 7f7192ffd640 1 -- 172.21.15.16:0/4030189766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194034bb0 con 0x7f719c0b1240 2023-11-24T06:29:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.616+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c158190 con 0x7f719c0b1240 2023-11-24T06:29:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.616+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c158bf0 con 0x7f719c0b1240 2023-11-24T06:29:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.618+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c15d470 con 0x7f719c0b1240 2023-11-24T06:29:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.618+0000 7f7192ffd640 1 -- 172.21.15.16:0/4030189766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7194031030 con 0x7f719c0b1240 2023-11-24T06:29:35.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.619+0000 7f7192ffd640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f718006b0b0 0x7f718006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:35.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.619+0000 7f7192ffd640 1 -- 172.21.15.16:0/4030189766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f71940ac080 con 0x7f719c0b1240 2023-11-24T06:29:35.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.620+0000 7f71a21e4640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f718006b0b0 0x7f718006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:35.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.621+0000 7f71a21e4640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f718006b0b0 0x7f718006d570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7198009910 tx=0x7f7198008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.624+0000 7f7192ffd640 1 -- 172.21.15.16:0/4030189766 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7194076d40 con 0x7f719c0b1240 2023-11-24T06:29:35.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.808+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f719c15d470 con 0x7f719c0b1240 2023-11-24T06:29:35.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.809+0000 7f7192ffd640 1 -- 172.21.15.16:0/4030189766 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f719407a360 con 0x7f719c0b1240 2023-11-24T06:29:35.812 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.811+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f718006b0b0 msgr2=0x7f718006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.811+0000 7f71a31e6640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f718006b0b0 0x7f718006d570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7198009910 tx=0x7f7198008040 comp rx=0 tx=0).stop 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.811+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0b1240 msgr2=0x7f719c154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.811+0000 7f71a31e6640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0b1240 0x7f719c154500 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f719400a870 tx=0x7f719c15b490 comp rx=0 tx=0).stop 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.811+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 shutdown_connections 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.811+0000 7f71a31e6640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f718006b0b0 0x7f718006d570 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.811+0000 7f71a31e6640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f719c0b1240 0x7f719c154500 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.811+0000 7f71a31e6640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f719c0b29c0 0x7f719c1586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.812+0000 7f71a31e6640 1 --2- 172.21.15.16:0/4030189766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7188004d00 0x7f719c155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.812+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 >> 172.21.15.16:0/4030189766 conn(0x7f719c06a7d0 msgr2=0x7f719c15cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.812+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 shutdown_connections 2023-11-24T06:29:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:35.812+0000 7f71a31e6640 1 -- 172.21.15.16:0/4030189766 wait complete. 2023-11-24T06:29:35.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.052+0000 7f7c30b2e640 1 Processor -- start 2023-11-24T06:29:36.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.052+0000 7f7c30b2e640 1 -- start start 2023-11-24T06:29:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.054+0000 7f7c30b2e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c2c0cc500 0x7f7c2c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.054+0000 7f7c30b2e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c06d160 0x7f7c2c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.054+0000 7f7c30b2e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c070e50 con 0x7f7c2c143c40 2023-11-24T06:29:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.054+0000 7f7c30b2e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2c0723c0 con 0x7f7c2c0cc500 2023-11-24T06:29:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.054+0000 7f7c30b2e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0cced0 con 0x7f7c2c06d160 2023-11-24T06:29:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.054+0000 7f7c2b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7c2c143c40 0x7f7c2c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53230/0 (socket says 172.21.15.16:53230) 2023-11-24T06:29:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.054+0000 7f7c2b7fe640 1 -- 172.21.15.16:0/2125255661 learned_addr learned my addr 172.21.15.16:0/2125255661 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.054+0000 7f7c2affd640 1 --2- 172.21.15.16:0/2125255661 >> v2:172.21.15.16:3300/0 conn(0x7f7c2c0cc500 0x7f7c2c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.055+0000 7f7c2bfff640 1 --2- 172.21.15.16:0/2125255661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c06d160 0x7f7c2c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.055+0000 7f7c2a7fc640 1 -- 172.21.15.16:0/2125255661 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392733068 0 0) 0x7f7c2c070e50 con 0x7f7c2c143c40 2023-11-24T06:29:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.055+0000 7f7c2a7fc640 1 -- 172.21.15.16:0/2125255661 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c14003520 con 0x7f7c2c143c40 2023-11-24T06:29:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.055+0000 7f7c2affd640 1 -- 172.21.15.16:0/2125255661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c06d160 msgr2=0x7f7c2c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.055+0000 7f7c2affd640 1 --2- 172.21.15.16:0/2125255661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c06d160 0x7f7c2c06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.055+0000 7f7c2affd640 1 -- 172.21.15.16:0/2125255661 >> v1:172.21.15.16:6789/0 conn(0x7f7c2c143c40 legacy=0x7f7c2c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.055+0000 7f7c2affd640 1 -- 172.21.15.16:0/2125255661 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c144020 con 0x7f7c2c0cc500 2023-11-24T06:29:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.056+0000 7f7c2affd640 1 --2- 172.21.15.16:0/2125255661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c0cc500 0x7f7c2c0cc990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7c200097a0 tx=0x7f7c2002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2356fd893ae30c26 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.056+0000 7f7c2a7fc640 1 -- 172.21.15.16:0/2125255661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20031780 con 0x7f7c2c0cc500 2023-11-24T06:29:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c2a7fc640 1 -- 172.21.15.16:0/2125255661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c200318e0 con 0x7f7c2c0cc500 2023-11-24T06:29:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c2a7fc640 1 -- 172.21.15.16:0/2125255661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20031c70 con 0x7f7c2c0cc500 2023-11-24T06:29:36.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c30b2e640 1 -- 172.21.15.16:0/2125255661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c0cc500 msgr2=0x7f7c2c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c30b2e640 1 --2- 172.21.15.16:0/2125255661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c0cc500 0x7f7c2c0cc990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7c200097a0 tx=0x7f7c2002b820 comp rx=0 tx=0).stop 2023-11-24T06:29:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c30b2e640 1 -- 172.21.15.16:0/2125255661 shutdown_connections 2023-11-24T06:29:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c30b2e640 1 --2- 172.21.15.16:0/2125255661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c06d160 0x7f7c2c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c30b2e640 1 --2- 172.21.15.16:0/2125255661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c0cc500 0x7f7c2c0cc990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c30b2e640 1 -- 172.21.15.16:0/2125255661 >> 172.21.15.16:0/2125255661 conn(0x7f7c2c06a800 msgr2=0x7f7c2c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:36.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c30b2e640 1 -- 172.21.15.16:0/2125255661 shutdown_connections 2023-11-24T06:29:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.057+0000 7f7c30b2e640 1 -- 172.21.15.16:0/2125255661 wait complete. 2023-11-24T06:29:36.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.059+0000 7f7c30b2e640 1 Processor -- start 2023-11-24T06:29:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.059+0000 7f7c30b2e640 1 -- start start 2023-11-24T06:29:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.060+0000 7f7c30b2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c06d160 0x7f7c2c1e54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.060+0000 7f7c30b2e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c0cc500 0x7f7c2c14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.060+0000 7f7c30b2e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c2c143c40 0x7f7c2c0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.060+0000 7f7c30b2e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c150040 con 0x7f7c2c06d160 2023-11-24T06:29:36.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.060+0000 7f7c30b2e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0b26f0 con 0x7f7c2c143c40 2023-11-24T06:29:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.060+0000 7f7c30b2e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0b29d0 con 0x7f7c2c0cc500 2023-11-24T06:29:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c06d160 0x7f7c2c1e54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c06d160 0x7f7c2c1e54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50114/0 (socket says 172.21.15.16:50114) 2023-11-24T06:29:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2b7fe640 1 -- 172.21.15.16:0/1818494111 learned_addr learned my addr 172.21.15.16:0/1818494111 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2affd640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c0cc500 0x7f7c2c14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2b7fe640 1 -- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c0cc500 msgr2=0x7f7c2c14f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2b7fe640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c0cc500 0x7f7c2c14f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2b7fe640 1 -- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c2c143c40 msgr2=0x7f7c2c0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2b7fe640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c2c143c40 0x7f7c2c0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2b7fe640 1 -- 172.21.15.16:0/1818494111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c20009400 con 0x7f7c2c06d160 2023-11-24T06:29:36.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.061+0000 7f7c2b7fe640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c06d160 0x7f7c2c1e54e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00cbb0 tx=0x7f7c1c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.062+0000 7f7c13fff640 1 -- 172.21.15.16:0/1818494111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00ad40 con 0x7f7c2c06d160 2023-11-24T06:29:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.062+0000 7f7c13fff640 1 -- 172.21.15.16:0/1818494111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c012400 con 0x7f7c2c06d160 2023-11-24T06:29:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.062+0000 7f7c13fff640 1 -- 172.21.15.16:0/1818494111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c00aea0 con 0x7f7c2c06d160 2023-11-24T06:29:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.062+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c0b2cb0 con 0x7f7c2c06d160 2023-11-24T06:29:36.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.062+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c0b3270 con 0x7f7c2c06d160 2023-11-24T06:29:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.063+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c0cd130 con 0x7f7c2c06d160 2023-11-24T06:29:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.064+0000 7f7c13fff640 1 -- 172.21.15.16:0/1818494111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7c1c0127b0 con 0x7f7c2c06d160 2023-11-24T06:29:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.065+0000 7f7c13fff640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c1806b080 0x7f7c1806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.065+0000 7f7c13fff640 1 -- 172.21.15.16:0/1818494111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7c1c0899c0 con 0x7f7c2c06d160 2023-11-24T06:29:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.066+0000 7f7c2affd640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c1806b080 0x7f7c1806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.067+0000 7f7c2affd640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c1806b080 0x7f7c1806d540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7c20036000 tx=0x7f7c2003d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.070+0000 7f7c13fff640 1 -- 172.21.15.16:0/1818494111 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c1c054770 con 0x7f7c2c06d160 2023-11-24T06:29:36.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.294+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c2c067450 con 0x7f7c2c06d160 2023-11-24T06:29:36.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.296+0000 7f7c13fff640 1 -- 172.21.15.16:0/1818494111 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f7c1c017090 con 0x7f7c2c06d160 2023-11-24T06:29:36.299 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:36.299 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110540698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110540698},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110540698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1298790428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1298790428},{"type":"v1","addr":"172.21.15.16:6835","nonce":1298790428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2876598187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2876598187},{"type":"v1","addr":"172.21.15.173:6836","nonce":2876598187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:05.754412+0000","modified":"2023-11-24T06:29:08.837901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/2818758442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2818758442},{"type":"v1","addr":"172.21.15.16:6833","nonce":2818758442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:29:36.300 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2023-11-24T06:29:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c1806b080 msgr2=0x7f7c1806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c1806b080 0x7f7c1806d540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7c20036000 tx=0x7f7c2003d040 comp rx=0 tx=0).stop 2023-11-24T06:29:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c06d160 msgr2=0x7f7c2c1e54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c06d160 0x7f7c2c1e54e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00cbb0 tx=0x7f7c1c00a430 comp rx=0 tx=0).stop 2023-11-24T06:29:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 shutdown_connections 2023-11-24T06:29:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7c1806b080 0x7f7c1806d540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c2c143c40 0x7f7c2c0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c2c0cc500 0x7f7c2c14f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 --2- 172.21.15.16:0/1818494111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c06d160 0x7f7c2c1e54e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.298+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 >> 172.21.15.16:0/1818494111 conn(0x7f7c2c06a800 msgr2=0x7f7c2c142560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.299+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 shutdown_connections 2023-11-24T06:29:36.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.299+0000 7f7c30b2e640 1 -- 172.21.15.16:0/1818494111 wait complete. 2023-11-24T06:29:36.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:36.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.536+0000 7f23a759e640 1 Processor -- start 2023-11-24T06:29:36.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.537+0000 7f23a759e640 1 -- start start 2023-11-24T06:29:36.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.538+0000 7f23a759e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f23a80d0150 0x7f23a80d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.538+0000 7f23a759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a80ceea0 0x7f23a80b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.538+0000 7f23a759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a80b2c70 con 0x7f23a815d3c0 2023-11-24T06:29:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.538+0000 7f23a759e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a80cf6d0 con 0x7f23a80d0150 2023-11-24T06:29:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.538+0000 7f23a759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80cf3f0 con 0x7f23a80ceea0 2023-11-24T06:29:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.538+0000 7f23a659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f23a815d3c0 0x7f23a80cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53242/0 (socket says 172.21.15.16:53242) 2023-11-24T06:29:36.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.538+0000 7f23a659c640 1 -- 172.21.15.16:0/3731976222 learned_addr learned my addr 172.21.15.16:0/3731976222 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f239ffff640 1 --2- 172.21.15.16:0/3731976222 >> v2:172.21.15.173:3300/0 conn(0x7f23a80d0150 0x7f23a80d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f23a6d9d640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a80ceea0 0x7f23a80b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181355597 0 0) 0x7f23a80b2c70 con 0x7f23a815d3c0 2023-11-24T06:29:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2394003520 con 0x7f23a815d3c0 2023-11-24T06:29:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3367937375 0 0) 0x7f2394003520 con 0x7f23a815d3c0 2023-11-24T06:29:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a80ceea0 msgr2=0x7f23a80b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f23a5d9b640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a80ceea0 0x7f23a80b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 >> v2:172.21.15.173:3300/0 conn(0x7f23a80d0150 msgr2=0x7f23a80d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f23a5d9b640 1 --2- 172.21.15.16:0/3731976222 >> v2:172.21.15.173:3300/0 conn(0x7f23a80d0150 0x7f23a80d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.539+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a80b2620 con 0x7f23a815d3c0 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f2398001670 con 0x7f23a815d3c0 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 >> v1:172.21.15.16:6789/0 conn(0x7f23a815d3c0 legacy=0x7f23a80cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a5d9b640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a80ceea0 0x7f2394003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a5d9b640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394003f40 0x7f23940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a5d9b640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f2394005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80cf6d0 con 0x7f2394004d00 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80cf3f0 con 0x7f2394003f40 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239400ba30 con 0x7f23a80ceea0 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a6d9d640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f2394005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a659c640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a80ceea0 0x7f2394003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f239ffff640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394003f40 0x7f23940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a6d9d640 1 -- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a80ceea0 msgr2=0x7f2394003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a6d9d640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a80ceea0 0x7f2394003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a6d9d640 1 -- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394003f40 msgr2=0x7f23940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a6d9d640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394003f40 0x7f23940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f23a6d9d640 1 -- 172.21.15.16:0/3731976222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2394003520 con 0x7f2394004d00 2023-11-24T06:29:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.540+0000 7f239ffff640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394003f40 0x7f23940047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.541+0000 7f23a659c640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a80ceea0 0x7f2394003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:36.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.542+0000 7f23a6d9d640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f2394005180 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f23a0007360 tx=0x7f23940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7952918fd25c8851 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.542+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a002ccb0 con 0x7f2394004d00 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.542+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a002ce10 con 0x7f2394004d00 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.542+0000 7f23a5d9b640 1 -- 172.21.15.16:0/3731976222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a0034660 con 0x7f2394004d00 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.542+0000 7f23a759e640 1 -- 172.21.15.16:0/3731976222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 msgr2=0x7f2394005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.542+0000 7f23a759e640 1 --2- 172.21.15.16:0/3731976222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f2394005180 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f23a0007360 tx=0x7f23940057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.542+0000 7f23a6d9d640 1 -- 172.21.15.16:0/3731976222 reap_dead start 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.543+0000 7f23a759e640 1 -- 172.21.15.16:0/3731976222 shutdown_connections 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.543+0000 7f23a759e640 1 -- 172.21.15.16:0/3731976222 >> 172.21.15.16:0/3731976222 conn(0x7f23a80d3050 msgr2=0x7f23a80d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.543+0000 7f23a759e640 1 -- 172.21.15.16:0/3731976222 shutdown_connections 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.543+0000 7f23a759e640 1 -- 172.21.15.16:0/3731976222 wait complete. 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.544+0000 7f23a759e640 1 Processor -- start 2023-11-24T06:29:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.545+0000 7f23a759e640 1 -- start start 2023-11-24T06:29:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f23a80c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a80c70c0 0x7f23a806b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a806bea0 0x7f23a806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a80c7540 con 0x7f2394004d00 2023-11-24T06:29:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a759e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a806c870 con 0x7f23a80c70c0 2023-11-24T06:29:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a815f060 con 0x7f23a806bea0 2023-11-24T06:29:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f23a80c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f23a80c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50132/0 (socket says 172.21.15.16:50132) 2023-11-24T06:29:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a659c640 1 -- 172.21.15.16:0/3492005040 learned_addr learned my addr 172.21.15.16:0/3492005040 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a659c640 1 -- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a806bea0 msgr2=0x7f23a806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a659c640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a806bea0 0x7f23a806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a659c640 1 -- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a80c70c0 msgr2=0x7f23a806b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a659c640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a80c70c0 0x7f23a806b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.546+0000 7f23a659c640 1 -- 172.21.15.16:0/3492005040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a815e450 con 0x7f2394004d00 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.547+0000 7f23a659c640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f23a80c6b80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f239800d8b0 tx=0x7f239800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.547+0000 7f239effd640 1 -- 172.21.15.16:0/3492005040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980074e0 con 0x7f2394004d00 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.547+0000 7f239effd640 1 -- 172.21.15.16:0/3492005040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398007640 con 0x7f2394004d00 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.547+0000 7f239effd640 1 -- 172.21.15.16:0/3492005040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398004a00 con 0x7f2394004d00 2023-11-24T06:29:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.547+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a815e730 con 0x7f2394004d00 2023-11-24T06:29:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.547+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a815ecf0 con 0x7f2394004d00 2023-11-24T06:29:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.548+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a808ba80 con 0x7f2394004d00 2023-11-24T06:29:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.549+0000 7f239effd640 1 -- 172.21.15.16:0/3492005040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2398004b60 con 0x7f2394004d00 2023-11-24T06:29:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.550+0000 7f239effd640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238806b0b0 0x7f238806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.550+0000 7f239effd640 1 -- 172.21.15.16:0/3492005040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f23980894e0 con 0x7f2394004d00 2023-11-24T06:29:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.551+0000 7f239ffff640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238806b0b0 0x7f238806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.552+0000 7f239ffff640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238806b0b0 0x7f238806d570 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2390009a60 tx=0x7f2390008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.554+0000 7f239effd640 1 -- 172.21.15.16:0/3492005040 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23980540e0 con 0x7f2394004d00 2023-11-24T06:29:36.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.744+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23a8067450 con 0x7f2394004d00 2023-11-24T06:29:36.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.747+0000 7f239effd640 1 -- 172.21.15.16:0/3492005040 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16621 (secure 0 0 0) 0x7f2398057700 con 0x7f2394004d00 2023-11-24T06:29:36.751 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:36.751 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:30.006201+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:03.705566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:04.794925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238806b0b0 msgr2=0x7f238806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238806b0b0 0x7f238806d570 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2390009a60 tx=0x7f2390008040 comp rx=0 tx=0).stop 2023-11-24T06:29:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 msgr2=0x7f23a80c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f23a80c6b80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f239800d8b0 tx=0x7f239800dd70 comp rx=0 tx=0).stop 2023-11-24T06:29:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 shutdown_connections 2023-11-24T06:29:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f238806b0b0 0x7f238806d570 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23a806bea0 0x7f23a806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23a80c70c0 0x7f23a806b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 --2- 172.21.15.16:0/3492005040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394004d00 0x7f23a80c6b80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.749+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 >> 172.21.15.16:0/3492005040 conn(0x7f23a80d3050 msgr2=0x7f23a80b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:36.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.750+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 shutdown_connections 2023-11-24T06:29:36.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.750+0000 7f23a759e640 1 -- 172.21.15.16:0/3492005040 wait complete. 2023-11-24T06:29:36.772 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:29:36.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.980+0000 7fd67759e640 1 Processor -- start 2023-11-24T06:29:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.980+0000 7fd67759e640 1 -- start start 2023-11-24T06:29:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.981+0000 7fd67759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6780cc500 0x7fd6780cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd67759e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd67806d160 0x7fd67806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd67759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd678070e50 con 0x7fd678143c40 2023-11-24T06:29:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd67759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6780723c0 con 0x7fd6780cc500 2023-11-24T06:29:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd67759e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6780cced0 con 0x7fd67806d160 2023-11-24T06:29:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd675d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6780cc500 0x7fd6780cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd67659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd678143c40 0x7fd678071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53254/0 (socket says 172.21.15.16:53254) 2023-11-24T06:29:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd675d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6780cc500 0x7fd6780cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50146/0 (socket says 172.21.15.16:50146) 2023-11-24T06:29:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd67659c640 1 -- 172.21.15.16:0/2794072445 learned_addr learned my addr 172.21.15.16:0/2794072445 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd675d9b640 1 -- 172.21.15.16:0/2794072445 >> v2:172.21.15.173:3300/0 conn(0x7fd67806d160 msgr2=0x7fd67806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd676d9d640 1 --2- 172.21.15.16:0/2794072445 >> v2:172.21.15.173:3300/0 conn(0x7fd67806d160 0x7fd67806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd675d9b640 1 --2- 172.21.15.16:0/2794072445 >> v2:172.21.15.173:3300/0 conn(0x7fd67806d160 0x7fd67806d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd675d9b640 1 -- 172.21.15.16:0/2794072445 >> v1:172.21.15.16:6789/0 conn(0x7fd678143c40 legacy=0x7fd678071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.982+0000 7fd675d9b640 1 -- 172.21.15.16:0/2794072445 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678144020 con 0x7fd6780cc500 2023-11-24T06:29:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.984+0000 7fd675d9b640 1 --2- 172.21.15.16:0/2794072445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6780cc500 0x7fd6780cc990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd678070690 tx=0x7fd66002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9598a3e6da00fac4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.984+0000 7fd67559a640 1 -- 172.21.15.16:0/2794072445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66002cbe0 con 0x7fd6780cc500 2023-11-24T06:29:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.984+0000 7fd67559a640 1 -- 172.21.15.16:0/2794072445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66002cd40 con 0x7fd6780cc500 2023-11-24T06:29:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.984+0000 7fd67559a640 1 -- 172.21.15.16:0/2794072445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660034660 con 0x7fd6780cc500 2023-11-24T06:29:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.985+0000 7fd67759e640 1 -- 172.21.15.16:0/2794072445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6780cc500 msgr2=0x7fd6780cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.985+0000 7fd67759e640 1 --2- 172.21.15.16:0/2794072445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6780cc500 0x7fd6780cc990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd678070690 tx=0x7fd66002be30 comp rx=0 tx=0).stop 2023-11-24T06:29:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.985+0000 7fd67759e640 1 -- 172.21.15.16:0/2794072445 shutdown_connections 2023-11-24T06:29:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.985+0000 7fd67759e640 1 --2- 172.21.15.16:0/2794072445 >> v2:172.21.15.173:3300/0 conn(0x7fd67806d160 0x7fd67806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.985+0000 7fd67759e640 1 --2- 172.21.15.16:0/2794072445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6780cc500 0x7fd6780cc990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.985+0000 7fd67759e640 1 -- 172.21.15.16:0/2794072445 >> 172.21.15.16:0/2794072445 conn(0x7fd67806a800 msgr2=0x7fd67806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.985+0000 7fd67759e640 1 -- 172.21.15.16:0/2794072445 shutdown_connections 2023-11-24T06:29:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.985+0000 7fd67759e640 1 -- 172.21.15.16:0/2794072445 wait complete. 2023-11-24T06:29:36.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.987+0000 7fd67759e640 1 Processor -- start 2023-11-24T06:29:36.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.987+0000 7fd67759e640 1 -- start start 2023-11-24T06:29:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd67759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67806d160 0x7fd678148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd67759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6780cc500 0x7fd678144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd67759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd678143c40 0x7fd6781452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd67759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6781482f0 con 0x7fd678143c40 2023-11-24T06:29:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd67759e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67815aa50 con 0x7fd67806d160 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd67759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd678147670 con 0x7fd6780cc500 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd676d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd678143c40 0x7fd6781452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd676d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd678143c40 0x7fd6781452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50154/0 (socket says 172.21.15.16:50154) 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd676d9d640 1 -- 172.21.15.16:0/1088467822 learned_addr learned my addr 172.21.15.16:0/1088467822 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd676d9d640 1 -- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6780cc500 msgr2=0x7fd678144db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd675d9b640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6780cc500 0x7fd678144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd676d9d640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6780cc500 0x7fd678144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd676d9d640 1 -- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67806d160 msgr2=0x7fd678148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd676d9d640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67806d160 0x7fd678148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.988+0000 7fd676d9d640 1 -- 172.21.15.16:0/1088467822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660009a70 con 0x7fd678143c40 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.989+0000 7fd676d9d640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd678143c40 0x7fd6781452f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd67000c7f0 tx=0x7fd67000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.989+0000 7fd66effd640 1 -- 172.21.15.16:0/1088467822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670004220 con 0x7fd678143c40 2023-11-24T06:29:36.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.989+0000 7fd66effd640 1 -- 172.21.15.16:0/1088467822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd670004380 con 0x7fd678143c40 2023-11-24T06:29:36.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.989+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd678147950 con 0x7fd678143c40 2023-11-24T06:29:36.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.989+0000 7fd66effd640 1 -- 172.21.15.16:0/1088467822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670007860 con 0x7fd678143c40 2023-11-24T06:29:36.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.989+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd678147f10 con 0x7fd678143c40 2023-11-24T06:29:36.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.991+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd678067450 con 0x7fd678143c40 2023-11-24T06:29:36.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.992+0000 7fd66effd640 1 -- 172.21.15.16:0/1088467822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd670007bd0 con 0x7fd678143c40 2023-11-24T06:29:36.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.993+0000 7fd66effd640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd65806b080 0x7fd65806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:36.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.993+0000 7fd66effd640 1 -- 172.21.15.16:0/1088467822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd670089b00 con 0x7fd678143c40 2023-11-24T06:29:36.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.994+0000 7fd67659c640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd65806b080 0x7fd65806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.995+0000 7fd67659c640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd65806b080 0x7fd65806d540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd67813fca0 tx=0x7fd668007a40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:36.998+0000 7fd66effd640 1 -- 172.21.15.16:0/1088467822 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd67008be60 con 0x7fd678143c40 2023-11-24T06:29:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.233+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6780ceb30 con 0x7fd678143c40 2023-11-24T06:29:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.233+0000 7fd66effd640 1 -- 172.21.15.16:0/1088467822 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fd67001e090 con 0x7fd678143c40 2023-11-24T06:29:37.237 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:37.237 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110540698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110540698},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110540698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1298790428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1298790428},{"type":"v1","addr":"172.21.15.16:6835","nonce":1298790428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2876598187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2876598187},{"type":"v1","addr":"172.21.15.173:6836","nonce":2876598187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:05.754412+0000","modified":"2023-11-24T06:29:08.837901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/2818758442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2818758442},{"type":"v1","addr":"172.21.15.16:6833","nonce":2818758442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:29:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2023-11-24T06:29:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd65806b080 msgr2=0x7fd65806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd65806b080 0x7fd65806d540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd67813fca0 tx=0x7fd668007a40 comp rx=0 tx=0).stop 2023-11-24T06:29:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd678143c40 msgr2=0x7fd6781452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd678143c40 0x7fd6781452f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd67000c7f0 tx=0x7fd67000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 shutdown_connections 2023-11-24T06:29:37.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd65806b080 0x7fd65806d540 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd678143c40 0x7fd6781452f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6780cc500 0x7fd678144db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 --2- 172.21.15.16:0/1088467822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd67806d160 0x7fd678148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.236+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 >> 172.21.15.16:0/1088467822 conn(0x7fd67806a800 msgr2=0x7fd67813fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:37.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.237+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 shutdown_connections 2023-11-24T06:29:37.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.237+0000 7fd67759e640 1 -- 172.21.15.16:0/1088467822 wait complete. 2023-11-24T06:29:37.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.484+0000 7f8efe2e7640 1 Processor -- start 2023-11-24T06:29:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.484+0000 7f8efe2e7640 1 -- start start 2023-11-24T06:29:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.485+0000 7f8efe2e7640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8ef815b2a0 0x7f8ef80cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.485+0000 7f8efe2e7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef80cfaf0 0x7f8ef80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.485+0000 7f8efe2e7640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef80d00a0 con 0x7f8ef80cdea0 2023-11-24T06:29:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.485+0000 7f8efe2e7640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef80ce330 con 0x7f8ef815b2a0 2023-11-24T06:29:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.485+0000 7f8efe2e7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80b4010 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.486+0000 7f8ef7fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8ef815b2a0 0x7f8ef80cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.486+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef80cfaf0 0x7f8ef80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.486+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef80cfaf0 0x7f8ef80cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:54858/0 (socket says 172.21.15.16:54858) 2023-11-24T06:29:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.486+0000 7f8ef7fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8ef815b2a0 0x7f8ef80cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:35840/0 (socket says 172.21.15.16:35840) 2023-11-24T06:29:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.486+0000 7f8efc884640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f8ef80cdea0 0x7f8ef80cbd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:34504/0 (socket says 172.21.15.16:34504) 2023-11-24T06:29:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.486+0000 7f8ef7fff640 1 -- 172.21.15.16:0/2663838179 learned_addr learned my addr 172.21.15.16:0/2663838179 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.487+0000 7f8ef7fff640 1 -- 172.21.15.16:0/2663838179 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef80cfaf0 msgr2=0x7f8ef80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.487+0000 7f8ef7fff640 1 --2- 172.21.15.16:0/2663838179 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef80cfaf0 0x7f8ef80cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.487+0000 7f8ef7fff640 1 -- 172.21.15.16:0/2663838179 >> v1:172.21.15.173:6789/0 conn(0x7f8ef80cdea0 legacy=0x7f8ef80cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.487+0000 7f8ef7fff640 1 -- 172.21.15.16:0/2663838179 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80b42f0 con 0x7f8ef815b2a0 2023-11-24T06:29:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.488+0000 7f8ef7fff640 1 --2- 172.21.15.16:0/2663838179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef815b2a0 0x7f8ef80cf5b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009e50 tx=0x7f8ee802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b758a1adda5d8f8e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.489+0000 7f8ef6ffd640 1 -- 172.21.15.16:0/2663838179 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee802cb00 con 0x7f8ef815b2a0 2023-11-24T06:29:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.489+0000 7f8ef6ffd640 1 -- 172.21.15.16:0/2663838179 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee802cc60 con 0x7f8ef815b2a0 2023-11-24T06:29:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.489+0000 7f8ef6ffd640 1 -- 172.21.15.16:0/2663838179 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80343f0 con 0x7f8ef815b2a0 2023-11-24T06:29:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.489+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2663838179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef815b2a0 msgr2=0x7f8ef80cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.489+0000 7f8efe2e7640 1 --2- 172.21.15.16:0/2663838179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef815b2a0 0x7f8ef80cf5b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009e50 tx=0x7f8ee802bf60 comp rx=0 tx=0).stop 2023-11-24T06:29:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.490+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2663838179 shutdown_connections 2023-11-24T06:29:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.490+0000 7f8efe2e7640 1 --2- 172.21.15.16:0/2663838179 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef80cfaf0 0x7f8ef80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.490+0000 7f8efe2e7640 1 --2- 172.21.15.16:0/2663838179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef815b2a0 0x7f8ef80cf5b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.490+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2663838179 >> 172.21.15.16:0/2663838179 conn(0x7f8ef80d1020 msgr2=0x7f8ef80d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.490+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2663838179 shutdown_connections 2023-11-24T06:29:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.490+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2663838179 wait complete. 2023-11-24T06:29:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.491+0000 7f8efe2e7640 1 Processor -- start 2023-11-24T06:29:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.492+0000 7f8efe2e7640 1 -- start start 2023-11-24T06:29:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.492+0000 7f8efe2e7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef80cdea0 0x7f8ef81f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.492+0000 7f8efe2e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef80cfaf0 0x7f8ef806bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.493+0000 7f8efe2e7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef815b2a0 0x7f8ef809ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.493+0000 7f8efe2e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef81f2860 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.493+0000 7f8efe2e7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef806c9e0 con 0x7f8ef80cdea0 2023-11-24T06:29:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.493+0000 7f8efe2e7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef806ccc0 con 0x7f8ef815b2a0 2023-11-24T06:29:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.493+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef80cfaf0 0x7f8ef806bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.493+0000 7f8ef77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef80cfaf0 0x7f8ef806bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50164/0 (socket says 172.21.15.16:50164) 2023-11-24T06:29:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.493+0000 7f8ef77fe640 1 -- 172.21.15.16:0/2688455460 learned_addr learned my addr 172.21.15.16:0/2688455460 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.493+0000 7f8ef7fff640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef80cdea0 0x7f8ef81f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.493+0000 7f8efc884640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef815b2a0 0x7f8ef809ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8ef77fe640 1 -- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef815b2a0 msgr2=0x7f8ef809ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8ef77fe640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef815b2a0 0x7f8ef809ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8ef77fe640 1 -- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef80cdea0 msgr2=0x7f8ef81f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8ef77fe640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef80cdea0 0x7f8ef81f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8ef77fe640 1 -- 172.21.15.16:0/2688455460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee8009b30 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8ef77fe640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef80cfaf0 0x7f8ef806bf00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000e9b0 tx=0x7f8ee000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8ef4ff9640 1 -- 172.21.15.16:0/2688455460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0009670 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8ef4ff9640 1 -- 172.21.15.16:0/2688455460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee000c9a0 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8ef4ff9640 1 -- 172.21.15.16:0/2688455460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000cd50 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef809d550 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.494+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef809d200 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.496+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef80675d0 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.496+0000 7f8ef4ff9640 1 -- 172.21.15.16:0/2688455460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8ee0013470 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.497+0000 7f8ef4ff9640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ed806b080 0x7f8ed806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.497+0000 7f8ef4ff9640 1 -- 172.21.15.16:0/2688455460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8ee0089c60 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.498+0000 7f8ef7fff640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ed806b080 0x7f8ed806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.499+0000 7f8ef7fff640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ed806b080 0x7f8ed806d540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8035000 tx=0x7f8ee8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.502+0000 7f8ef4ff9640 1 -- 172.21.15.16:0/2688455460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ee00549e0 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.738+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ef815e370 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.739+0000 7f8ef4ff9640 1 -- 172.21.15.16:0/2688455460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f8ee0017090 con 0x7f8ef80cfaf0 2023-11-24T06:29:37.742 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:37.742 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110540698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110540698},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110540698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1298790428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1298790428},{"type":"v1","addr":"172.21.15.16:6835","nonce":1298790428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2876598187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2876598187},{"type":"v1","addr":"172.21.15.173:6836","nonce":2876598187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:05.754412+0000","modified":"2023-11-24T06:29:08.837901+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5727":{"gid":5727,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/2818758442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2818758442},{"type":"v1","addr":"172.21.15.16:6833","nonce":2818758442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:29:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2023-11-24T06:29:37.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ed806b080 msgr2=0x7f8ed806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ed806b080 0x7f8ed806d540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8035000 tx=0x7f8ee8030000 comp rx=0 tx=0).stop 2023-11-24T06:29:37.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef80cfaf0 msgr2=0x7f8ef806bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef80cfaf0 0x7f8ef806bf00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000e9b0 tx=0x7f8ee000ee70 comp rx=0 tx=0).stop 2023-11-24T06:29:37.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 shutdown_connections 2023-11-24T06:29:37.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8ed806b080 0x7f8ed806d540 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ef815b2a0 0x7f8ef809ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef80cfaf0 0x7f8ef806bf00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 --2- 172.21.15.16:0/2688455460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ef80cdea0 0x7f8ef81f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 >> 172.21.15.16:0/2688455460 conn(0x7f8ef80d1020 msgr2=0x7f8ef815d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:37.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 shutdown_connections 2023-11-24T06:29:37.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.742+0000 7f8efe2e7640 1 -- 172.21.15.16:0/2688455460 wait complete. 2023-11-24T06:29:37.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.971+0000 7fd780fff640 1 Processor -- start 2023-11-24T06:29:37.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.971+0000 7fd780fff640 1 -- start start 2023-11-24T06:29:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.973+0000 7fd780fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd77c160920 0x7fd77c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.973+0000 7fd780fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77c0d0a10 con 0x7fd77c0cf9f0 2023-11-24T06:29:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.973+0000 7fd780fff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd77c161e30 con 0x7fd77c0cfea0 2023-11-24T06:29:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.973+0000 7fd780fff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c0d1f50 con 0x7fd77c160920 2023-11-24T06:29:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.973+0000 7fd773fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd77c0cf9f0 0x7fd77c161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53264/0 (socket says 172.21.15.16:53264) 2023-11-24T06:29:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.973+0000 7fd773fff640 1 -- 172.21.15.16:0/1946338349 learned_addr learned my addr 172.21.15.16:0/1946338349 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:37.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77b7fe640 1 --2- 172.21.15.16:0/1946338349 >> v2:172.21.15.173:3300/0 conn(0x7fd77c160920 0x7fd77c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069607032 0 0) 0x7fd77c0d0a10 con 0x7fd77c0cf9f0 2023-11-24T06:29:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd768003520 con 0x7fd77c0cf9f0 2023-11-24T06:29:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3015393235 0 0) 0x7fd768003520 con 0x7fd77c0cf9f0 2023-11-24T06:29:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 >> v2:172.21.15.173:3300/0 conn(0x7fd77c160920 msgr2=0x7fd77c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 --2- 172.21.15.16:0/1946338349 >> v2:172.21.15.173:3300/0 conn(0x7fd77c160920 0x7fd77c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 >> v1:172.21.15.173:6789/0 conn(0x7fd77c0cfea0 legacy=0x7fd77c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c0cf080 con 0x7fd77c0cf9f0 2023-11-24T06:29:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fd764001670 con 0x7fd77c0cf9f0 2023-11-24T06:29:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 >> v1:172.21.15.16:6789/0 conn(0x7fd77c0cf9f0 legacy=0x7fd77c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.974+0000 7fd77affd640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7680039c0 0x7fd768003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd77affd640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7680047c0 0x7fd768004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd77affd640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd768005180 0x7fd76800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0d1f50 con 0x7fd7680047c0 2023-11-24T06:29:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76800c080 con 0x7fd768005180 2023-11-24T06:29:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76800c240 con 0x7fd7680039c0 2023-11-24T06:29:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd773fff640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7680047c0 0x7fd768004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd77b7fe640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7680039c0 0x7fd768003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd77bfff640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd768005180 0x7fd76800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd773fff640 1 -- 172.21.15.16:0/1946338349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7680039c0 msgr2=0x7fd768003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd773fff640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7680039c0 0x7fd768003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd773fff640 1 -- 172.21.15.16:0/1946338349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd768005180 msgr2=0x7fd76800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd773fff640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd768005180 0x7fd76800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd773fff640 1 -- 172.21.15.16:0/1946338349 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768003520 con 0x7fd7680047c0 2023-11-24T06:29:37.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.975+0000 7fd77bfff640 1 -- 172.21.15.16:0/1946338349 reap_dead start 2023-11-24T06:29:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.976+0000 7fd773fff640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7680047c0 0x7fd768004c40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c0e0 tx=0x7fd7680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb0c313feec6155 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.977+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764005ad0 con 0x7fd7680047c0 2023-11-24T06:29:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.977+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd764005c30 con 0x7fd7680047c0 2023-11-24T06:29:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.977+0000 7fd77affd640 1 -- 172.21.15.16:0/1946338349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76402ca50 con 0x7fd7680047c0 2023-11-24T06:29:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.977+0000 7fd780fff640 1 -- 172.21.15.16:0/1946338349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7680047c0 msgr2=0x7fd768004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.977+0000 7fd780fff640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7680047c0 0x7fd768004c40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c0e0 tx=0x7fd7680057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.977+0000 7fd780fff640 1 -- 172.21.15.16:0/1946338349 shutdown_connections 2023-11-24T06:29:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.977+0000 7fd780fff640 1 --2- 172.21.15.16:0/1946338349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7680047c0 0x7fd768004c40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.977+0000 7fd780fff640 1 -- 172.21.15.16:0/1946338349 >> 172.21.15.16:0/1946338349 conn(0x7fd77c0d3090 msgr2=0x7fd77c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.978+0000 7fd780fff640 1 -- 172.21.15.16:0/1946338349 shutdown_connections 2023-11-24T06:29:37.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.978+0000 7fd780fff640 1 -- 172.21.15.16:0/1946338349 wait complete. 2023-11-24T06:29:37.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.980+0000 7fd780fff640 1 Processor -- start 2023-11-24T06:29:37.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.980+0000 7fd780fff640 1 -- start start 2023-11-24T06:29:37.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd780fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7680047c0 0x7fd77c1f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd780fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd77c1f6410 0x7fd77c1f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd780fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd77c0b6640 0x7fd77c0b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd780fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c1f6dd0 con 0x7fd77c1f6410 2023-11-24T06:29:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd780fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0b7950 con 0x7fd77c0b6640 2023-11-24T06:29:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd780fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0b7390 con 0x7fd7680047c0 2023-11-24T06:29:37.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd773fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd77c1f6410 0x7fd77c1f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd773fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd77c1f6410 0x7fd77c1f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50186/0 (socket says 172.21.15.16:50186) 2023-11-24T06:29:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd773fff640 1 -- 172.21.15.16:0/64162648 learned_addr learned my addr 172.21.15.16:0/64162648 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.981+0000 7fd77bfff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd77c0b6640 0x7fd77c0b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd773fff640 1 -- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7680047c0 msgr2=0x7fd77c1f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd773fff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7680047c0 0x7fd77c1f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd773fff640 1 -- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd77c0b6640 msgr2=0x7fd77c0b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd773fff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd77c0b6640 0x7fd77c0b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd773fff640 1 -- 172.21.15.16:0/64162648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c0b7670 con 0x7fd77c1f6410 2023-11-24T06:29:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd773fff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd77c1f6410 0x7fd77c1f6890 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c0e0 tx=0x7fd76402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd778ff9640 1 -- 172.21.15.16:0/64162648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76403c040 con 0x7fd77c1f6410 2023-11-24T06:29:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd778ff9640 1 -- 172.21.15.16:0/64162648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76402cd00 con 0x7fd77c1f6410 2023-11-24T06:29:37.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd778ff9640 1 -- 172.21.15.16:0/64162648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640072b0 con 0x7fd77c1f6410 2023-11-24T06:29:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c1f4080 con 0x7fd77c1f6410 2023-11-24T06:29:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.982+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c0b7030 con 0x7fd77c1f6410 2023-11-24T06:29:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.984+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c067450 con 0x7fd77c1f6410 2023-11-24T06:29:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.984+0000 7fd778ff9640 1 -- 172.21.15.16:0/64162648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd764032070 con 0x7fd77c1f6410 2023-11-24T06:29:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.985+0000 7fd778ff9640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd75c06b0b0 0x7fd75c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.985+0000 7fd778ff9640 1 -- 172.21.15.16:0/64162648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd7640aae70 con 0x7fd77c1f6410 2023-11-24T06:29:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.986+0000 7fd77b7fe640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd75c06b0b0 0x7fd75c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.987+0000 7fd77b7fe640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd75c06b0b0 0x7fd75c06d570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd77c067b00 tx=0x7fd76c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:37.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:37.990+0000 7fd778ff9640 1 -- 172.21.15.16:0/64162648 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd764075b60 con 0x7fd77c1f6410 2023-11-24T06:29:38.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.186+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd77c08c7e0 con 0x7fd77c1f6410 2023-11-24T06:29:38.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.188+0000 7fd778ff9640 1 -- 172.21.15.16:0/64162648 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16621 (secure 0 0 0) 0x7fd764079180 con 0x7fd77c1f6410 2023-11-24T06:29:38.193 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:38.193 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:30.006201+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:03.705566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:04.794925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:38.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd75c06b0b0 msgr2=0x7fd75c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd75c06b0b0 0x7fd75c06d570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd77c067b00 tx=0x7fd76c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:38.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd77c1f6410 msgr2=0x7fd77c1f6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd77c1f6410 0x7fd77c1f6890 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c0e0 tx=0x7fd76402e070 comp rx=0 tx=0).stop 2023-11-24T06:29:38.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 shutdown_connections 2023-11-24T06:29:38.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd75c06b0b0 0x7fd75c06d570 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd77c0b6640 0x7fd77c0b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd77c1f6410 0x7fd77c1f6890 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 --2- 172.21.15.16:0/64162648 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7680047c0 0x7fd77c1f00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 >> 172.21.15.16:0/64162648 conn(0x7fd77c0d3090 msgr2=0x7fd77c15b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 shutdown_connections 2023-11-24T06:29:38.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.191+0000 7fd780fff640 1 -- 172.21.15.16:0/64162648 wait complete. 2023-11-24T06:29:38.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:29:38.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.449+0000 7fd83e478640 1 Processor -- start 2023-11-24T06:29:38.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.449+0000 7fd83e478640 1 -- start start 2023-11-24T06:29:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.450+0000 7fd83e478640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd838160920 0x7fd8380b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd83e478640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd8380cfe70 0x7fd838161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd83e478640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8380d0a10 con 0x7fd8380cf9f0 2023-11-24T06:29:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd83e478640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380d1f50 con 0x7fd8380cfe70 2023-11-24T06:29:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd83e478640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380cf080 con 0x7fd838160920 2023-11-24T06:29:38.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd83cc75640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8380cf9f0 0x7fd838161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53278/0 (socket says 172.21.15.16:53278) 2023-11-24T06:29:38.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd83cc75640 1 -- 172.21.15.16:0/43408883 learned_addr learned my addr 172.21.15.16:0/43408883 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:38.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd83dc77640 1 --2- 172.21.15.16:0/43408883 >> v2:172.21.15.173:3300/0 conn(0x7fd8380cfe70 0x7fd838161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd83d476640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd838160920 0x7fd8380b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51427679 0 0) 0x7fd8380d0a10 con 0x7fd8380cf9f0 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.451+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd824003520 con 0x7fd8380cf9f0 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1638014184 0 0) 0x7fd824003520 con 0x7fd8380cf9f0 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd838160920 msgr2=0x7fd8380b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd838160920 0x7fd8380b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 >> v2:172.21.15.173:3300/0 conn(0x7fd8380cfe70 msgr2=0x7fd838161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 --2- 172.21.15.16:0/43408883 >> v2:172.21.15.173:3300/0 conn(0x7fd8380cfe70 0x7fd838161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380c6be0 con 0x7fd8380cf9f0 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fd828001670 con 0x7fd8380cf9f0 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 >> v1:172.21.15.16:6789/0 conn(0x7fd8380cf9f0 legacy=0x7fd838161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd838160920 0x7fd824003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824004d00 0x7fd824005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380d1f50 con 0x7fd824003f40 2023-11-24T06:29:38.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380cf080 con 0x7fd824004d00 2023-11-24T06:29:38.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82400b980 con 0x7fd838160920 2023-11-24T06:29:38.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.452+0000 7fd83cc75640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.453+0000 7fd83dc77640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824004d00 0x7fd824005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.453+0000 7fd83d476640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd838160920 0x7fd824003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.453+0000 7fd83cc75640 1 -- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd838160920 msgr2=0x7fd824003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.453+0000 7fd83cc75640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd838160920 0x7fd824003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.453+0000 7fd83cc75640 1 -- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824004d00 msgr2=0x7fd824005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.453+0000 7fd83cc75640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824004d00 0x7fd824005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.453+0000 7fd83cc75640 1 -- 172.21.15.16:0/43408883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd824003520 con 0x7fd824003f40 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.453+0000 7fd83d476640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd838160920 0x7fd824003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.454+0000 7fd83cc75640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8240047c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c190 tx=0x7fd8240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdb47f26b354c85 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.454+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828005ad0 con 0x7fd824003f40 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.454+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828005c30 con 0x7fd824003f40 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.454+0000 7fd82ffff640 1 -- 172.21.15.16:0/43408883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82802c810 con 0x7fd824003f40 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.455+0000 7fd83e478640 1 -- 172.21.15.16:0/43408883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 msgr2=0x7fd8240047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.455+0000 7fd83e478640 1 --2- 172.21.15.16:0/43408883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8240047c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c190 tx=0x7fd8240057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.455+0000 7fd83dc77640 1 -- 172.21.15.16:0/43408883 reap_dead start 2023-11-24T06:29:38.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.455+0000 7fd83e478640 1 -- 172.21.15.16:0/43408883 shutdown_connections 2023-11-24T06:29:38.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.455+0000 7fd83e478640 1 -- 172.21.15.16:0/43408883 >> 172.21.15.16:0/43408883 conn(0x7fd8380d3090 msgr2=0x7fd8380d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:38.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.455+0000 7fd83e478640 1 -- 172.21.15.16:0/43408883 shutdown_connections 2023-11-24T06:29:38.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.455+0000 7fd83e478640 1 -- 172.21.15.16:0/43408883 wait complete. 2023-11-24T06:29:38.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.457+0000 7fd83e478640 1 Processor -- start 2023-11-24T06:29:38.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.457+0000 7fd83e478640 1 -- start start 2023-11-24T06:29:38.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.458+0000 7fd83e478640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8381f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.458+0000 7fd83e478640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8380cc850 0x7fd8380b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.458+0000 7fd83e478640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8380b75f0 0x7fd8380c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.458+0000 7fd83e478640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380cbc80 con 0x7fd824003f40 2023-11-24T06:29:38.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.458+0000 7fd83e478640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380b6010 con 0x7fd8380cc850 2023-11-24T06:29:38.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.458+0000 7fd83e478640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380b62f0 con 0x7fd8380b75f0 2023-11-24T06:29:38.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.458+0000 7fd83d476640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8381f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83d476640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8381f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50196/0 (socket says 172.21.15.16:50196) 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83d476640 1 -- 172.21.15.16:0/959639566 learned_addr learned my addr 172.21.15.16:0/959639566 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83d476640 1 -- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8380b75f0 msgr2=0x7fd8380c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83d476640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8380b75f0 0x7fd8380c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83cc75640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8380cc850 0x7fd8380b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83d476640 1 -- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8380cc850 msgr2=0x7fd8380b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83d476640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8380cc850 0x7fd8380b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83d476640 1 -- 172.21.15.16:0/959639566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380b65d0 con 0x7fd824003f40 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83cc75640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8380cc850 0x7fd8380b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.459+0000 7fd83d476640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8381f2740 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd834010830 tx=0x7fd834010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.460+0000 7fd82dffb640 1 -- 172.21.15.16:0/959639566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83400eb60 con 0x7fd824003f40 2023-11-24T06:29:38.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.460+0000 7fd82dffb640 1 -- 172.21.15.16:0/959639566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83400ecc0 con 0x7fd824003f40 2023-11-24T06:29:38.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.460+0000 7fd82dffb640 1 -- 172.21.15.16:0/959639566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83401c400 con 0x7fd824003f40 2023-11-24T06:29:38.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.460+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8380b68b0 con 0x7fd824003f40 2023-11-24T06:29:38.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.460+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8381f2c80 con 0x7fd824003f40 2023-11-24T06:29:38.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.461+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd838067450 con 0x7fd824003f40 2023-11-24T06:29:38.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.462+0000 7fd82dffb640 1 -- 172.21.15.16:0/959639566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd83401c560 con 0x7fd824003f40 2023-11-24T06:29:38.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.463+0000 7fd82dffb640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd81c06b0b0 0x7fd81c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.463+0000 7fd82dffb640 1 -- 172.21.15.16:0/959639566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd834089ff0 con 0x7fd824003f40 2023-11-24T06:29:38.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.464+0000 7fd83cc75640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd81c06b0b0 0x7fd81c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.465+0000 7fd83cc75640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd81c06b0b0 0x7fd81c06d570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c160 tx=0x7fd828007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:38.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.467+0000 7fd82dffb640 1 -- 172.21.15.16:0/959639566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8340bce90 con 0x7fd824003f40 2023-11-24T06:29:38.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.699+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd83808c7e0 con 0x7fd824003f40 2023-11-24T06:29:38.704 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-24T06:29:38.701+0000 7f082ae01640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:29:38.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.705+0000 7fd82dffb640 1 -- 172.21.15.16:0/959639566 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd834054ca0 con 0x7fd824003f40 2023-11-24T06:29:38.709 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:29:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.709+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd81c06b0b0 msgr2=0x7fd81c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.709+0000 7fd83e478640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd81c06b0b0 0x7fd81c06d570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c160 tx=0x7fd828007000 comp rx=0 tx=0).stop 2023-11-24T06:29:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.709+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 msgr2=0x7fd8381f2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.709+0000 7fd83e478640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8381f2740 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd834010830 tx=0x7fd834010cf0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.710+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 shutdown_connections 2023-11-24T06:29:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.710+0000 7fd83e478640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd81c06b0b0 0x7fd81c06d570 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.710+0000 7fd83e478640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8380b75f0 0x7fd8380c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.710+0000 7fd83e478640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8380cc850 0x7fd8380b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.710+0000 7fd83e478640 1 --2- 172.21.15.16:0/959639566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd824003f40 0x7fd8381f2740 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.710+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 >> 172.21.15.16:0/959639566 conn(0x7fd8380d3090 msgr2=0x7fd83806aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.710+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 shutdown_connections 2023-11-24T06:29:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.710+0000 7fd83e478640 1 -- 172.21.15.16:0/959639566 wait complete. 2023-11-24T06:29:38.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:29:38.794 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:29:38.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.937+0000 7f24b2800640 1 Processor -- start 2023-11-24T06:29:38.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.937+0000 7f24b2800640 1 -- start start 2023-11-24T06:29:38.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.939+0000 7f24b2800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24ac0b3c10 0x7f24ac0713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.939+0000 7f24b2800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac0cdf90 con 0x7f24ac0b4280 2023-11-24T06:29:38.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.939+0000 7f24b2800640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac072000 con 0x7f24ac0ce420 2023-11-24T06:29:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.939+0000 7f24b2800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ac071d20 con 0x7f24ac0b3c10 2023-11-24T06:29:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.939+0000 7f24abfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24ac0b4280 0x7f24ac0b4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53298/0 (socket says 172.21.15.16:53298) 2023-11-24T06:29:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.939+0000 7f24abfff640 1 -- 172.21.15.16:0/3425514071 learned_addr learned my addr 172.21.15.16:0/3425514071 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.939+0000 7f24a37fe640 1 --2- 172.21.15.16:0/3425514071 >> v2:172.21.15.16:3300/0 conn(0x7f24ac0b3c10 0x7f24ac0713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.940+0000 7f24ab7fe640 1 -- 172.21.15.16:0/3425514071 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407795890 0 0) 0x7f24ac0cdf90 con 0x7f24ac0b4280 2023-11-24T06:29:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.940+0000 7f24a37fe640 1 -- 172.21.15.16:0/3425514071 >> v1:172.21.15.173:6789/0 conn(0x7f24ac0ce420 legacy=0x7f24ac0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.940+0000 7f24a37fe640 1 -- 172.21.15.16:0/3425514071 >> v1:172.21.15.16:6789/0 conn(0x7f24ac0b4280 legacy=0x7f24ac0b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.940+0000 7f24a37fe640 1 -- 172.21.15.16:0/3425514071 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac0cd940 con 0x7f24ac0b3c10 2023-11-24T06:29:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.941+0000 7f24a37fe640 1 --2- 172.21.15.16:0/3425514071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 0x7f24ac0713d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f24940097a0 tx=0x7f249402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226d4649de8e945b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24ab7fe640 1 -- 172.21.15.16:0/3425514071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494031830 con 0x7f24ac0b3c10 2023-11-24T06:29:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24ab7fe640 1 -- 172.21.15.16:0/3425514071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2494031990 con 0x7f24ac0b3c10 2023-11-24T06:29:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24ab7fe640 1 -- 172.21.15.16:0/3425514071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494031d20 con 0x7f24ac0b3c10 2023-11-24T06:29:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24b2800640 1 -- 172.21.15.16:0/3425514071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 msgr2=0x7f24ac0713d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24b2800640 1 --2- 172.21.15.16:0/3425514071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 0x7f24ac0713d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f24940097a0 tx=0x7f249402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24b2800640 1 -- 172.21.15.16:0/3425514071 shutdown_connections 2023-11-24T06:29:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24b2800640 1 --2- 172.21.15.16:0/3425514071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 0x7f24ac0713d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24b2800640 1 -- 172.21.15.16:0/3425514071 >> 172.21.15.16:0/3425514071 conn(0x7f24ac0cfef0 msgr2=0x7f24ac0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24b2800640 1 -- 172.21.15.16:0/3425514071 shutdown_connections 2023-11-24T06:29:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.942+0000 7f24b2800640 1 -- 172.21.15.16:0/3425514071 wait complete. 2023-11-24T06:29:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.944+0000 7f24b2800640 1 Processor -- start 2023-11-24T06:29:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.944+0000 7f24b2800640 1 -- start start 2023-11-24T06:29:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.945+0000 7f24b2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 0x7f24ac0ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.945+0000 7f24b2800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24ac0b4280 0x7f24ac1e3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.945+0000 7f24b2800640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24ac0ce420 0x7f24ac13d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.945+0000 7f24b2800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac15c520 con 0x7f24ac0b3c10 2023-11-24T06:29:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.945+0000 7f24b2800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac15c800 con 0x7f24ac0b4280 2023-11-24T06:29:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.945+0000 7f24b2800640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac15cae0 con 0x7f24ac0ce420 2023-11-24T06:29:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.946+0000 7f24abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 0x7f24ac0ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.946+0000 7f24abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 0x7f24ac0ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50238/0 (socket says 172.21.15.16:50238) 2023-11-24T06:29:38.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.946+0000 7f24abfff640 1 -- 172.21.15.16:0/3844778021 learned_addr learned my addr 172.21.15.16:0/3844778021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:38.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.946+0000 7f24abfff640 1 -- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24ac0ce420 msgr2=0x7f24ac13d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.946+0000 7f24abfff640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24ac0ce420 0x7f24ac13d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.946+0000 7f24abfff640 1 -- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24ac0b4280 msgr2=0x7f24ac1e3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:38.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.946+0000 7f24abfff640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24ac0b4280 0x7f24ac1e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:38.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.946+0000 7f24abfff640 1 -- 172.21.15.16:0/3844778021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2494009400 con 0x7f24ac0b3c10 2023-11-24T06:29:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.946+0000 7f24abfff640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 0x7f24ac0ceb00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f249c00c7c0 tx=0x7f249c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.947+0000 7f24a97fa640 1 -- 172.21.15.16:0/3844778021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c00a640 con 0x7f24ac0b3c10 2023-11-24T06:29:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.947+0000 7f24a97fa640 1 -- 172.21.15.16:0/3844778021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249c00a7c0 con 0x7f24ac0b3c10 2023-11-24T06:29:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.947+0000 7f24a97fa640 1 -- 172.21.15.16:0/3844778021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c00ab70 con 0x7f24ac0b3c10 2023-11-24T06:29:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.947+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ac15cdc0 con 0x7f24ac0b3c10 2023-11-24T06:29:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.947+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ac15d5c0 con 0x7f24ac0b3c10 2023-11-24T06:29:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.948+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ac15d2e0 con 0x7f24ac0b3c10 2023-11-24T06:29:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.949+0000 7f24a97fa640 1 -- 172.21.15.16:0/3844778021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f249c00acd0 con 0x7f24ac0b3c10 2023-11-24T06:29:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.950+0000 7f24a97fa640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f248c06b080 0x7f248c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.950+0000 7f24a97fa640 1 -- 172.21.15.16:0/3844778021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f249c089070 con 0x7f24ac0b3c10 2023-11-24T06:29:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.950+0000 7f24a37fe640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f248c06b080 0x7f248c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.952+0000 7f24a37fe640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f248c06b080 0x7f248c06d540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2494036000 tx=0x7f2494030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:38.954+0000 7f24a97fa640 1 -- 172.21.15.16:0/3844778021 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f249c0bbe90 con 0x7f24ac0b3c10 2023-11-24T06:29:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.190+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f24ac0675d0 con 0x7f24ac0b3c10 2023-11-24T06:29:39.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.709+0000 7f24a97fa640 1 -- 172.21.15.16:0/3844778021 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f249c018020 con 0x7f24ac0b3c10 2023-11-24T06:29:39.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f248c06b080 msgr2=0x7f248c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f248c06b080 0x7f248c06d540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2494036000 tx=0x7f2494030000 comp rx=0 tx=0).stop 2023-11-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 msgr2=0x7f24ac0ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 0x7f24ac0ceb00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f249c00c7c0 tx=0x7f249c00cc80 comp rx=0 tx=0).stop 2023-11-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 shutdown_connections 2023-11-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f248c06b080 0x7f248c06d540 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24ac0ce420 0x7f24ac13d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24ac0b4280 0x7f24ac1e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 --2- 172.21.15.16:0/3844778021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ac0b3c10 0x7f24ac0ceb00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 >> 172.21.15.16:0/3844778021 conn(0x7f24ac0cfef0 msgr2=0x7f24ac0b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 shutdown_connections 2023-11-24T06:29:39.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.713+0000 7f24b2800640 1 -- 172.21.15.16:0/3844778021 wait complete. 2023-11-24T06:29:39.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:29:39.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.967+0000 7f3fe7fff640 1 Processor -- start 2023-11-24T06:29:39.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.967+0000 7f3fe7fff640 1 -- start start 2023-11-24T06:29:39.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.969+0000 7f3fe7fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3fe81447f0 0x7f3fe815c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.969+0000 7f3fe7fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe815c5c0 con 0x7f3fe80cccf0 2023-11-24T06:29:39.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.969+0000 7f3fe7fff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe806ca10 con 0x7f3fe806c470 2023-11-24T06:29:39.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.969+0000 7f3fe7fff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe806d690 con 0x7f3fe81447f0 2023-11-24T06:29:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.969+0000 7f3fe77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3fe80cccf0 0x7f3fe815c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53306/0 (socket says 172.21.15.16:53306) 2023-11-24T06:29:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.969+0000 7f3fe77fe640 1 -- 172.21.15.16:0/1995216217 learned_addr learned my addr 172.21.15.16:0/1995216217 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.970+0000 7f3fe6ffd640 1 --2- 172.21.15.16:0/1995216217 >> v2:172.21.15.173:3300/0 conn(0x7f3fe81447f0 0x7f3fe815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.970+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644032046 0 0) 0x7f3fe815c5c0 con 0x7f3fe80cccf0 2023-11-24T06:29:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.970+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd0003520 con 0x7f3fe80cccf0 2023-11-24T06:29:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.970+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2883914653 0 0) 0x7f3fd0003520 con 0x7f3fe80cccf0 2023-11-24T06:29:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.970+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 >> v2:172.21.15.173:3300/0 conn(0x7f3fe81447f0 msgr2=0x7f3fe815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.970+0000 7f3fe5ffb640 1 --2- 172.21.15.16:0/1995216217 >> v2:172.21.15.173:3300/0 conn(0x7f3fe81447f0 0x7f3fe815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.970+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 >> v1:172.21.15.173:6789/0 conn(0x7f3fe806c470 legacy=0x7f3fe80cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.970+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe806ccf0 con 0x7f3fe80cccf0 2023-11-24T06:29:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f3fe00019f0 con 0x7f3fe80cccf0 2023-11-24T06:29:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 >> v1:172.21.15.16:6789/0 conn(0x7f3fe80cccf0 legacy=0x7f3fe815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe5ffb640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fd00039c0 0x7f3fd0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe5ffb640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fd00047c0 0x7f3fd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe5ffb640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fd00051d0 0x7f3fd000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe806d690 con 0x7f3fd00051d0 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd000c120 con 0x7f3fd00039c0 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd000c2e0 con 0x7f3fd00047c0 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe77fe640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fd00051d0 0x7f3fd000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe6ffd640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fd00039c0 0x7f3fd0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe77fe640 1 -- 172.21.15.16:0/1995216217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fd00047c0 msgr2=0x7f3fd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe77fe640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fd00047c0 0x7f3fd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe77fe640 1 -- 172.21.15.16:0/1995216217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fd00039c0 msgr2=0x7f3fd0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.971+0000 7f3fe77fe640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fd00039c0 0x7f3fd0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.972+0000 7f3fe77fe640 1 -- 172.21.15.16:0/1995216217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd0003520 con 0x7f3fd00051d0 2023-11-24T06:29:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.972+0000 7f3fe77fe640 1 -- 172.21.15.16:0/1995216217 reap_dead start 2023-11-24T06:29:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.973+0000 7f3fe77fe640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fd00051d0 0x7f3fd000b980 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0002230 tx=0x7f3fd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b3df5ce951c42 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.973+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0034630 con 0x7f3fd00051d0 2023-11-24T06:29:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.973+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe002f070 con 0x7f3fd00051d0 2023-11-24T06:29:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.973+0000 7f3fe5ffb640 1 -- 172.21.15.16:0/1995216217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00349d0 con 0x7f3fd00051d0 2023-11-24T06:29:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.974+0000 7f3fe7fff640 1 -- 172.21.15.16:0/1995216217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fd00051d0 msgr2=0x7f3fd000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.974+0000 7f3fe7fff640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fd00051d0 0x7f3fd000b980 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0002230 tx=0x7f3fd00057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.974+0000 7f3fe7fff640 1 -- 172.21.15.16:0/1995216217 shutdown_connections 2023-11-24T06:29:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.974+0000 7f3fe7fff640 1 --2- 172.21.15.16:0/1995216217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fd00051d0 0x7f3fd000b980 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.974+0000 7f3fe7fff640 1 -- 172.21.15.16:0/1995216217 >> 172.21.15.16:0/1995216217 conn(0x7f3fe806a3e0 msgr2=0x7f3fe806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.974+0000 7f3fe7fff640 1 -- 172.21.15.16:0/1995216217 shutdown_connections 2023-11-24T06:29:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.974+0000 7f3fe7fff640 1 -- 172.21.15.16:0/1995216217 wait complete. 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.976+0000 7f3fe7fff640 1 Processor -- start 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.976+0000 7f3fe7fff640 1 -- start start 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.977+0000 7f3fe7fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fd00051d0 0x7f3fe81f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.977+0000 7f3fe7fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fe813d2d0 0x7f3fe813c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.977+0000 7f3fe7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe80a84c0 0x7f3fe80a8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.977+0000 7f3fe7fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe814b170 con 0x7f3fe80a84c0 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.977+0000 7f3fe7fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81f4300 con 0x7f3fe813d2d0 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe7fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81f45e0 con 0x7f3fd00051d0 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe80a84c0 0x7f3fe80a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe80a84c0 0x7f3fe80a8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50254/0 (socket says 172.21.15.16:50254) 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe77fe640 1 -- 172.21.15.16:0/4092061598 learned_addr learned my addr 172.21.15.16:0/4092061598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:39.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe67fc640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fe813d2d0 0x7f3fe813c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe6ffd640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fd00051d0 0x7f3fe81f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe77fe640 1 -- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fd00051d0 msgr2=0x7f3fe81f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe77fe640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fd00051d0 0x7f3fe81f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe77fe640 1 -- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fe813d2d0 msgr2=0x7f3fe813c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:39.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe77fe640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fe813d2d0 0x7f3fe813c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:39.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.978+0000 7f3fe77fe640 1 -- 172.21.15.16:0/4092061598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe81f48c0 con 0x7f3fe80a84c0 2023-11-24T06:29:39.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.979+0000 7f3fe77fe640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe80a84c0 0x7f3fe80a8940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0002230 tx=0x7f3fe0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:39.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.979+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/4092061598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0004e20 con 0x7f3fe80a84c0 2023-11-24T06:29:39.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.979+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/4092061598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe003f470 con 0x7f3fe80a84c0 2023-11-24T06:29:39.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.979+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe814b410 con 0x7f3fe80a84c0 2023-11-24T06:29:39.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.979+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/4092061598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe003f820 con 0x7f3fe80a84c0 2023-11-24T06:29:39.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.979+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe80a8e80 con 0x7f3fe80a84c0 2023-11-24T06:29:39.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.980+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe8067450 con 0x7f3fe80a84c0 2023-11-24T06:29:39.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.981+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/4092061598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3fe0031030 con 0x7f3fe80a84c0 2023-11-24T06:29:39.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.982+0000 7f3fcf7fe640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3fd406b0b0 0x7f3fd406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:39.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.982+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/4092061598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f3fe00aae00 con 0x7f3fe80a84c0 2023-11-24T06:29:39.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.983+0000 7f3fe6ffd640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3fd406b0b0 0x7f3fd406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.984+0000 7f3fe6ffd640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3fd406b0b0 0x7f3fd406d570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009990 tx=0x7f3fd8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:39.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:39.987+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/4092061598 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fe0075a00 con 0x7f3fe80a84c0 2023-11-24T06:29:40.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:40.181+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3fe8067450 con 0x7f3fe80a84c0 2023-11-24T06:29:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.095+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/4092061598 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3fe0079020 con 0x7f3fe80a84c0 2023-11-24T06:29:41.099 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:29:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.099+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3fd406b0b0 msgr2=0x7f3fd406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.099+0000 7f3fe7fff640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3fd406b0b0 0x7f3fd406d570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009990 tx=0x7f3fd8008040 comp rx=0 tx=0).stop 2023-11-24T06:29:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.099+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe80a84c0 msgr2=0x7f3fe80a8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.099+0000 7f3fe7fff640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe80a84c0 0x7f3fe80a8940 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0002230 tx=0x7f3fe0001cb0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.100+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 shutdown_connections 2023-11-24T06:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.100+0000 7f3fe7fff640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3fd406b0b0 0x7f3fd406d570 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.100+0000 7f3fe7fff640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe80a84c0 0x7f3fe80a8940 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.100+0000 7f3fe7fff640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fe813d2d0 0x7f3fe813c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.100+0000 7f3fe7fff640 1 --2- 172.21.15.16:0/4092061598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3fd00051d0 0x7f3fe81f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.100+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 >> 172.21.15.16:0/4092061598 conn(0x7f3fe806a3e0 msgr2=0x7f3fe815de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.103+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 shutdown_connections 2023-11-24T06:29:41.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.103+0000 7f3fe7fff640 1 -- 172.21.15.16:0/4092061598 wait complete. 2023-11-24T06:29:41.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:29:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.333+0000 7f24dc87e640 1 Processor -- start 2023-11-24T06:29:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.333+0000 7f24dc87e640 1 -- start start 2023-11-24T06:29:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.335+0000 7f24dc87e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24d80cc500 0x7f24d80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.335+0000 7f24dc87e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 0x7f24d806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.335+0000 7f24dc87e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d8070e50 con 0x7f24d8143c40 2023-11-24T06:29:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.335+0000 7f24dc87e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d80723c0 con 0x7f24d80cc500 2023-11-24T06:29:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.335+0000 7f24dc87e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d80cced0 con 0x7f24d806d160 2023-11-24T06:29:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.335+0000 7f24ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24d80cc500 0x7f24d80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.336+0000 7f24ceffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24d80cc500 0x7f24d80cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50266/0 (socket says 172.21.15.16:50266) 2023-11-24T06:29:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.336+0000 7f24ceffd640 1 -- 172.21.15.16:0/614311951 learned_addr learned my addr 172.21.15.16:0/614311951 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.336+0000 7f24ceffd640 1 -- 172.21.15.16:0/614311951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 msgr2=0x7f24d806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.336+0000 7f24d7fff640 1 --2- 172.21.15.16:0/614311951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 0x7f24d806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.336+0000 7f24ceffd640 1 --2- 172.21.15.16:0/614311951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 0x7f24d806d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.336+0000 7f24ceffd640 1 -- 172.21.15.16:0/614311951 >> v1:172.21.15.173:6789/0 conn(0x7f24d8143c40 legacy=0x7f24d8071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.336+0000 7f24ceffd640 1 -- 172.21.15.16:0/614311951 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d8144020 con 0x7f24d80cc500 2023-11-24T06:29:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.337+0000 7f24ceffd640 1 --2- 172.21.15.16:0/614311951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d80cc500 0x7f24d80cc990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f24d80b44a0 tx=0x7f24c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9747ac5be7ed4d50 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.337+0000 7f24d6ffd640 1 -- 172.21.15.16:0/614311951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c002e070 con 0x7f24d80cc500 2023-11-24T06:29:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24d6ffd640 1 -- 172.21.15.16:0/614311951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c00315a0 con 0x7f24d80cc500 2023-11-24T06:29:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24d6ffd640 1 -- 172.21.15.16:0/614311951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0031930 con 0x7f24d80cc500 2023-11-24T06:29:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24dc87e640 1 -- 172.21.15.16:0/614311951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d80cc500 msgr2=0x7f24d80cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24dc87e640 1 --2- 172.21.15.16:0/614311951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d80cc500 0x7f24d80cc990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f24d80b44a0 tx=0x7f24c002b840 comp rx=0 tx=0).stop 2023-11-24T06:29:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24dc87e640 1 -- 172.21.15.16:0/614311951 shutdown_connections 2023-11-24T06:29:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24dc87e640 1 --2- 172.21.15.16:0/614311951 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 0x7f24d806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24dc87e640 1 --2- 172.21.15.16:0/614311951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d80cc500 0x7f24d80cc990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24dc87e640 1 -- 172.21.15.16:0/614311951 >> 172.21.15.16:0/614311951 conn(0x7f24d806a800 msgr2=0x7f24d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24dc87e640 1 -- 172.21.15.16:0/614311951 shutdown_connections 2023-11-24T06:29:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.338+0000 7f24dc87e640 1 -- 172.21.15.16:0/614311951 wait complete. 2023-11-24T06:29:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.340+0000 7f24dc87e640 1 Processor -- start 2023-11-24T06:29:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.340+0000 7f24dc87e640 1 -- start start 2023-11-24T06:29:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.341+0000 7f24dc87e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 0x7f24d8139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.341+0000 7f24dc87e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24d80cc500 0x7f24d815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.341+0000 7f24dc87e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d8143c40 0x7f24d81e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.341+0000 7f24dc87e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d8138eb0 con 0x7f24d8143c40 2023-11-24T06:29:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.341+0000 7f24dc87e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d8139880 con 0x7f24d80cc500 2023-11-24T06:29:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.341+0000 7f24dc87e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d81382c0 con 0x7f24d806d160 2023-11-24T06:29:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d8143c40 0x7f24d81e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d8143c40 0x7f24d81e63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50272/0 (socket says 172.21.15.16:50272) 2023-11-24T06:29:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24d7fff640 1 -- 172.21.15.16:0/1805486741 learned_addr learned my addr 172.21.15.16:0/1805486741 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24ceffd640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24d80cc500 0x7f24d815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24d77fe640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 0x7f24d8139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24d7fff640 1 -- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 msgr2=0x7f24d8139340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24d7fff640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 0x7f24d8139340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24d7fff640 1 -- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24d80cc500 msgr2=0x7f24d815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24d7fff640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24d80cc500 0x7f24d815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24d7fff640 1 -- 172.21.15.16:0/1805486741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c00094d0 con 0x7f24d8143c40 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.342+0000 7f24ceffd640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24d80cc500 0x7f24d815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.343+0000 7f24d7fff640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d8143c40 0x7f24d81e63c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f24d000c7f0 tx=0x7f24d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.343+0000 7f24d4ff9640 1 -- 172.21.15.16:0/1805486741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0004220 con 0x7f24d8143c40 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.343+0000 7f24d4ff9640 1 -- 172.21.15.16:0/1805486741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d0004380 con 0x7f24d8143c40 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.343+0000 7f24d4ff9640 1 -- 172.21.15.16:0/1805486741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0007860 con 0x7f24d8143c40 2023-11-24T06:29:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.343+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d815ae30 con 0x7f24d8143c40 2023-11-24T06:29:41.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.343+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d8138860 con 0x7f24d8143c40 2023-11-24T06:29:41.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.344+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d80ccd40 con 0x7f24d8143c40 2023-11-24T06:29:41.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.345+0000 7f24d4ff9640 1 -- 172.21.15.16:0/1805486741 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f24d0007bd0 con 0x7f24d8143c40 2023-11-24T06:29:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.346+0000 7f24d4ff9640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f24b806b0b0 0x7f24b806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:41.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.347+0000 7f24d4ff9640 1 -- 172.21.15.16:0/1805486741 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5385+0+0 (secure 0 0 0) 0x7f24d0089960 con 0x7f24d8143c40 2023-11-24T06:29:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.347+0000 7f24d77fe640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f24b806b0b0 0x7f24b806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.348+0000 7f24d77fe640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f24b806b0b0 0x7f24b806d570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f24c8007bf0 tx=0x7f24c8007240 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.352+0000 7f24d4ff9640 1 -- 172.21.15.16:0/1805486741 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24d00546d0 con 0x7f24d8143c40 2023-11-24T06:29:41.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.545+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24d8067450 con 0x7f24d8143c40 2023-11-24T06:29:41.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.993+0000 7f24d4ff9640 1 -- 172.21.15.16:0/1805486741 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7f24d001e020 con 0x7f24d8143c40 2023-11-24T06:29:41.997 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f24b806b0b0 msgr2=0x7f24b806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f24b806b0b0 0x7f24b806d570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f24c8007bf0 tx=0x7f24c8007240 comp rx=0 tx=0).stop 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d8143c40 msgr2=0x7f24d81e63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d8143c40 0x7f24d81e63c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f24d000c7f0 tx=0x7f24d000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 shutdown_connections 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f24b806b0b0 0x7f24b806d570 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d8143c40 0x7f24d81e63c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24d80cc500 0x7f24d815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 --2- 172.21.15.16:0/1805486741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24d806d160 0x7f24d8139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.997+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 >> 172.21.15.16:0/1805486741 conn(0x7f24d806a800 msgr2=0x7f24d80aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.998+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 shutdown_connections 2023-11-24T06:29:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:41.999+0000 7f24dc87e640 1 -- 172.21.15.16:0/1805486741 wait complete. 2023-11-24T06:29:42.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:29:42.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.225+0000 7fb8df301640 1 Processor -- start 2023-11-24T06:29:42.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.225+0000 7fb8df301640 1 -- start start 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.227+0000 7fb8df301640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8d814b2c0 0x7fb8d80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.227+0000 7fb8df301640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb8d80cf7d0 0x7fb8d8070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.227+0000 7fb8df301640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d8070ea0 0x7fb8d80b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.227+0000 7fb8df301640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d814b6a0 con 0x7fb8d814b2c0 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.227+0000 7fb8df301640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d80cfbf0 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.227+0000 7fb8df301640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d8071690 con 0x7fb8d8070ea0 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.228+0000 7fb8dd09d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8d814b2c0 0x7fb8d80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.228+0000 7fb8dd09d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8d814b2c0 0x7fb8d80cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50282/0 (socket says 172.21.15.16:50282) 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.228+0000 7fb8dd09d640 1 -- 172.21.15.16:0/134334560 learned_addr learned my addr 172.21.15.16:0/134334560 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.228+0000 7fb8dd89e640 1 --2- 172.21.15.16:0/134334560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d8070ea0 0x7fb8d80b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.228+0000 7fb8dc89c640 1 --2- 172.21.15.16:0/134334560 >> v2:172.21.15.173:3300/0 conn(0x7fb8d80cf7d0 0x7fb8d8070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.228+0000 7fb8dd09d640 1 -- 172.21.15.16:0/134334560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d8070ea0 msgr2=0x7fb8d80b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.228+0000 7fb8dd09d640 1 --2- 172.21.15.16:0/134334560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d8070ea0 0x7fb8d80b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.229+0000 7fb8dd09d640 1 -- 172.21.15.16:0/134334560 >> v2:172.21.15.173:3300/0 conn(0x7fb8d80cf7d0 msgr2=0x7fb8d8070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.229+0000 7fb8dd09d640 1 --2- 172.21.15.16:0/134334560 >> v2:172.21.15.173:3300/0 conn(0x7fb8d80cf7d0 0x7fb8d8070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.229+0000 7fb8dd09d640 1 -- 172.21.15.16:0/134334560 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d8071970 con 0x7fb8d814b2c0 2023-11-24T06:29:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.230+0000 7fb8dd09d640 1 --2- 172.21.15.16:0/134334560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d814b2c0 0x7fb8d80cf290 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80cfea0 tx=0x7fb8d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c939a94d2c50b627 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.231+0000 7fb8cffff640 1 -- 172.21.15.16:0/134334560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4031870 con 0x7fb8d814b2c0 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.231+0000 7fb8cffff640 1 -- 172.21.15.16:0/134334560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d40319d0 con 0x7fb8d814b2c0 2023-11-24T06:29:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.231+0000 7fb8cffff640 1 -- 172.21.15.16:0/134334560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4031d60 con 0x7fb8d814b2c0 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.232+0000 7fb8df301640 1 -- 172.21.15.16:0/134334560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d814b2c0 msgr2=0x7fb8d80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.232+0000 7fb8df301640 1 --2- 172.21.15.16:0/134334560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d814b2c0 0x7fb8d80cf290 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80cfea0 tx=0x7fb8d402b910 comp rx=0 tx=0).stop 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.232+0000 7fb8df301640 1 -- 172.21.15.16:0/134334560 shutdown_connections 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.232+0000 7fb8df301640 1 --2- 172.21.15.16:0/134334560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d8070ea0 0x7fb8d80b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.232+0000 7fb8df301640 1 --2- 172.21.15.16:0/134334560 >> v2:172.21.15.173:3300/0 conn(0x7fb8d80cf7d0 0x7fb8d8070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.232+0000 7fb8df301640 1 --2- 172.21.15.16:0/134334560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d814b2c0 0x7fb8d80cf290 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.232+0000 7fb8df301640 1 -- 172.21.15.16:0/134334560 >> 172.21.15.16:0/134334560 conn(0x7fb8d806a800 msgr2=0x7fb8d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.232+0000 7fb8df301640 1 -- 172.21.15.16:0/134334560 shutdown_connections 2023-11-24T06:29:42.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.232+0000 7fb8df301640 1 -- 172.21.15.16:0/134334560 wait complete. 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.234+0000 7fb8df301640 1 Processor -- start 2023-11-24T06:29:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.234+0000 7fb8df301640 1 -- start start 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8df301640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d8070ea0 0x7fb8d81fa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8df301640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d80cf7d0 0x7fb8d8163fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8df301640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d814b2c0 0x7fb8d8164520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8df301640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d81f93b0 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8df301640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d81f9690 con 0x7fb8d814b2c0 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8df301640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d81f9970 con 0x7fb8d8070ea0 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8dc89c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d80cf7d0 0x7fb8d8163fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8dc89c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d80cf7d0 0x7fb8d8163fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50292/0 (socket says 172.21.15.16:50292) 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8dc89c640 1 -- 172.21.15.16:0/1578663981 learned_addr learned my addr 172.21.15.16:0/1578663981 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8dc89c640 1 -- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d8070ea0 msgr2=0x7fb8d81fa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8dc89c640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d8070ea0 0x7fb8d81fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8dc89c640 1 -- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d814b2c0 msgr2=0x7fb8d8164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8dc89c640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d814b2c0 0x7fb8d8164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.235+0000 7fb8dc89c640 1 -- 172.21.15.16:0/1578663981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40094d0 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.236+0000 7fb8dc89c640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d80cf7d0 0x7fb8d8163fe0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800d800 tx=0x7fb8c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.236+0000 7fb8cdffb640 1 -- 172.21.15.16:0/1578663981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c800b640 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.236+0000 7fb8cdffb640 1 -- 172.21.15.16:0/1578663981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c800b7a0 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.236+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d81f9c50 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.236+0000 7fb8cdffb640 1 -- 172.21.15.16:0/1578663981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c800bb30 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.236+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d81fa990 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.237+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d81fab50 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.238+0000 7fb8cdffb640 1 -- 172.21.15.16:0/1578663981 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fb8c800bc90 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.239+0000 7fb8cdffb640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8bc06b080 0x7fb8bc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.239+0000 7fb8cdffb640 1 -- 172.21.15.16:0/1578663981 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fb8c8089cd0 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.240+0000 7fb8dd09d640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8bc06b080 0x7fb8bc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.241+0000 7fb8dd09d640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8bc06b080 0x7fb8bc06d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4036000 tx=0x7fb8d4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:42.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.244+0000 7fb8cdffb640 1 -- 172.21.15.16:0/1578663981 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8c8054c70 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.443+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb8d80675d0 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.449+0000 7fb8cdffb640 1 -- 172.21.15.16:0/1578663981 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb8c8058290 con 0x7fb8d80cf7d0 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8bc06b080 msgr2=0x7fb8bc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8bc06b080 0x7fb8bc06d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4036000 tx=0x7fb8d4030000 comp rx=0 tx=0).stop 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d80cf7d0 msgr2=0x7fb8d8163fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d80cf7d0 0x7fb8d8163fe0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800d800 tx=0x7fb8c800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 shutdown_connections 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb8bc06b080 0x7fb8bc06d540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8d814b2c0 0x7fb8d8164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8d80cf7d0 0x7fb8d8163fe0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 --2- 172.21.15.16:0/1578663981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8d8070ea0 0x7fb8d81fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.452+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 >> 172.21.15.16:0/1578663981 conn(0x7fb8d806a800 msgr2=0x7fb8d80cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:42.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.453+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 shutdown_connections 2023-11-24T06:29:42.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.453+0000 7fb8df301640 1 -- 172.21.15.16:0/1578663981 wait complete. 2023-11-24T06:29:42.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.679+0000 7f278a485640 1 Processor -- start 2023-11-24T06:29:42.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.680+0000 7f278a485640 1 -- start start 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.681+0000 7f278a485640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784160920 0x7f27840b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.681+0000 7f278a485640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27840d0a10 con 0x7f27840cf9f0 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.682+0000 7f278a485640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2784161e30 con 0x7f27840cfe70 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.682+0000 7f278a485640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27840d1f50 con 0x7f2784160920 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.682+0000 7f2788c82640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f27840cf9f0 0x7f2784161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50948/0 (socket says 172.21.15.16:50948) 2023-11-24T06:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.682+0000 7f2788c82640 1 -- 172.21.15.16:0/1607804326 learned_addr learned my addr 172.21.15.16:0/1607804326 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.682+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453820862 0 0) 0x7f27840d0a10 con 0x7f27840cf9f0 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.682+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2770003520 con 0x7f27840cf9f0 2023-11-24T06:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.682+0000 7f2789483640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784160920 0x7f27840b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1984549681 0 0) 0x7f2770003520 con 0x7f27840cf9f0 2023-11-24T06:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784160920 msgr2=0x7f27840b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784160920 0x7f27840b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 >> v1:172.21.15.173:6789/0 conn(0x7f27840cfe70 legacy=0x7f2784161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840cf080 con 0x7f27840cf9f0 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f2778001670 con 0x7f27840cf9f0 2023-11-24T06:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 >> v1:172.21.15.16:6789/0 conn(0x7f27840cf9f0 legacy=0x7f2784161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27700039a0 0x7f2770003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784160920 0x7f27700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770004d00 0x7f2770005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840d1f50 con 0x7f2770004d00 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277000bab0 con 0x7f27700039a0 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277000bc70 con 0x7f2784160920 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.683+0000 7f2789c84640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770004d00 0x7f2770005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.684+0000 7f2789483640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27700039a0 0x7f2770003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.684+0000 7f2788c82640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784160920 0x7f27700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.684+0000 7f2789c84640 1 -- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784160920 msgr2=0x7f27700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.684+0000 7f2789c84640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2784160920 0x7f27700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.684+0000 7f2789c84640 1 -- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27700039a0 msgr2=0x7f2770003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.684+0000 7f2789c84640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27700039a0 0x7f2770003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.684+0000 7f2789c84640 1 -- 172.21.15.16:0/1607804326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770003520 con 0x7f2770004d00 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.685+0000 7f2789c84640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770004d00 0x7f2770005180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f277c0044b0 tx=0x7f27700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55b3ca7fc6e864 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.685+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c030620 con 0x7f2770004d00 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.685+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c030780 con 0x7f2770004d00 2023-11-24T06:29:42.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.685+0000 7f2777fff640 1 -- 172.21.15.16:0/1607804326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c030b30 con 0x7f2770004d00 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.686+0000 7f278a485640 1 -- 172.21.15.16:0/1607804326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770004d00 msgr2=0x7f2770005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.686+0000 7f278a485640 1 --2- 172.21.15.16:0/1607804326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2770004d00 0x7f2770005180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f277c0044b0 tx=0x7f27700057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.686+0000 7f2789c84640 1 -- 172.21.15.16:0/1607804326 reap_dead start 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.686+0000 7f278a485640 1 -- 172.21.15.16:0/1607804326 shutdown_connections 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.686+0000 7f278a485640 1 -- 172.21.15.16:0/1607804326 >> 172.21.15.16:0/1607804326 conn(0x7f27840d3090 msgr2=0x7f27840d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.686+0000 7f278a485640 1 -- 172.21.15.16:0/1607804326 shutdown_connections 2023-11-24T06:29:42.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.686+0000 7f278a485640 1 -- 172.21.15.16:0/1607804326 wait complete. 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.688+0000 7f278a485640 1 Processor -- start 2023-11-24T06:29:42.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.688+0000 7f278a485640 1 -- start start 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f278a485640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27840b8390 0x7f27840b8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f278a485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840b6980 0x7f278415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f278a485640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f278415ba40 0x7f27841f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f278a485640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840b5c50 con 0x7f27840b6980 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f278a485640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278415be80 con 0x7f278415ba40 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f278a485640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27841f3b50 con 0x7f27840b8390 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f2788c82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840b6980 0x7f278415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f2788c82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840b6980 0x7f278415b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49824/0 (socket says 172.21.15.16:49824) 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f2788c82640 1 -- 172.21.15.16:0/2566938169 learned_addr learned my addr 172.21.15.16:0/2566938169 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f2788c82640 1 -- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27840b8390 msgr2=0x7f27840b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.689+0000 7f2788c82640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27840b8390 0x7f27840b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.690+0000 7f2788c82640 1 -- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f278415ba40 msgr2=0x7f27841f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.690+0000 7f2788c82640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f278415ba40 0x7f27841f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.690+0000 7f2788c82640 1 -- 172.21.15.16:0/2566938169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840b5ea0 con 0x7f27840b6980 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.690+0000 7f2788c82640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840b6980 0x7f278415b500 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f277800eb20 tx=0x7f27780016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.690+0000 7f2775ffb640 1 -- 172.21.15.16:0/2566938169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277800c850 con 0x7f27840b6980 2023-11-24T06:29:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.690+0000 7f2775ffb640 1 -- 172.21.15.16:0/2566938169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277800c9b0 con 0x7f27840b6980 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.690+0000 7f2775ffb640 1 -- 172.21.15.16:0/2566938169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277800cd40 con 0x7f27840b6980 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.690+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27840b6180 con 0x7f27840b6980 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.690+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27840b6e00 con 0x7f27840b6980 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.692+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27840b66a0 con 0x7f27840b6980 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.693+0000 7f2775ffb640 1 -- 172.21.15.16:0/2566938169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f277801d460 con 0x7f27840b6980 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.694+0000 7f2775ffb640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806b080 0x7f276806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.694+0000 7f2775ffb640 1 -- 172.21.15.16:0/2566938169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f2778089720 con 0x7f27840b6980 2023-11-24T06:29:42.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.695+0000 7f2789483640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806b080 0x7f276806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.696+0000 7f2789483640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806b080 0x7f276806d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2780006fd0 tx=0x7f278000b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:42.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.698+0000 7f2775ffb640 1 -- 172.21.15.16:0/2566938169 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27840b66a0 con 0x7f27840b6980 2023-11-24T06:29:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:42.961+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f27840b6f90 con 0x7f27840b6980 2023-11-24T06:29:43.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.108+0000 7f2775ffb640 1 -- 172.21.15.16:0/2566938169 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v175) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2778054780 con 0x7f27840b6980 2023-11-24T06:29:43.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806b080 msgr2=0x7f276806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806b080 0x7f276806d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2780006fd0 tx=0x7f278000b2e0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840b6980 msgr2=0x7f278415b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840b6980 0x7f278415b500 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f277800eb20 tx=0x7f27780016c0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 shutdown_connections 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f276806b080 0x7f276806d540 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f278415ba40 0x7f27841f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27840b6980 0x7f278415b500 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 --2- 172.21.15.16:0/2566938169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27840b8390 0x7f27840b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 >> 172.21.15.16:0/2566938169 conn(0x7f27840d3090 msgr2=0x7f278406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:43.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 shutdown_connections 2023-11-24T06:29:43.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.111+0000 7f278a485640 1 -- 172.21.15.16:0/2566938169 wait complete. 2023-11-24T06:29:43.136 INFO:tasks.cephfs_test_runner:Validate cleaning of stale subvolume snapshot metadata. ... ok 2023-11-24T06:29:43.136 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:29:43.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.356+0000 7f6a21747640 1 Processor -- start 2023-11-24T06:29:43.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.356+0000 7f6a21747640 1 -- start start 2023-11-24T06:29:43.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.357+0000 7f6a21747640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6a1c143c40 0x7f6a1c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.357+0000 7f6a21747640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1c070e50 con 0x7f6a1c0ccc10 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.357+0000 7f6a21747640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1c144020 con 0x7f6a1c0717d0 2023-11-24T06:29:43.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.357+0000 7f6a21747640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c0723c0 con 0x7f6a1c143c40 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.357+0000 7f6a20f46640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a1c0ccc10 0x7f6a1c06d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50964/0 (socket says 172.21.15.16:50964) 2023-11-24T06:29:43.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.357+0000 7f6a20f46640 1 -- 172.21.15.16:0/2950535836 learned_addr learned my addr 172.21.15.16:0/2950535836 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.357+0000 7f6a1bfff640 1 --2- 172.21.15.16:0/2950535836 >> v2:172.21.15.173:3300/0 conn(0x7f6a1c143c40 0x7f6a1c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.357+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916298228 0 0) 0x7f6a1c070e50 con 0x7f6a1c0ccc10 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a08003520 con 0x7f6a1c0ccc10 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 559018235 0 0) 0x7f6a08003520 con 0x7f6a1c0ccc10 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 >> v2:172.21.15.173:3300/0 conn(0x7f6a1c143c40 msgr2=0x7f6a1c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 --2- 172.21.15.16:0/2950535836 >> v2:172.21.15.173:3300/0 conn(0x7f6a1c143c40 0x7f6a1c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 >> v1:172.21.15.173:6789/0 conn(0x7f6a1c0717d0 legacy=0x7f6a1c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c071e00 con 0x7f6a1c0ccc10 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f6a140019f0 con 0x7f6a1c0ccc10 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 >> v1:172.21.15.16:6789/0 conn(0x7f6a1c0ccc10 legacy=0x7f6a1c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08003970 0x7f6a08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a080047c0 0x7f6a08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08005180 0x7f6a0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c0723c0 con 0x7f6a08005180 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0800c120 con 0x7f6a08003970 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0800c2e0 con 0x7f6a080047c0 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.358+0000 7f6a20f46640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08005180 0x7f6a0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.359+0000 7f6a1bfff640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08003970 0x7f6a08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.359+0000 7f6a20f46640 1 -- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a080047c0 msgr2=0x7f6a08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.359+0000 7f6a13fff640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a080047c0 0x7f6a08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.359+0000 7f6a20f46640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a080047c0 0x7f6a08004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.359+0000 7f6a20f46640 1 -- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08003970 msgr2=0x7f6a08003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.359+0000 7f6a20f46640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08003970 0x7f6a08003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.359+0000 7f6a20f46640 1 -- 172.21.15.16:0/2950535836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a08003520 con 0x7f6a08005180 2023-11-24T06:29:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.359+0000 7f6a13fff640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a080047c0 0x7f6a08004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.359+0000 7f6a1bfff640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08003970 0x7f6a08003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a20f46640 1 -- 172.21.15.16:0/2950535836 reap_dead start 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a20f46640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08005180 0x7f6a0800b980 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400a830 tx=0x7f6a080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5643e0a1add9d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a140072d0 con 0x7f6a08005180 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1402f070 con 0x7f6a08005180 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a1b7fe640 1 -- 172.21.15.16:0/2950535836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a140076b0 con 0x7f6a08005180 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a21747640 1 -- 172.21.15.16:0/2950535836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08005180 msgr2=0x7f6a0800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a21747640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08005180 0x7f6a0800b980 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400a830 tx=0x7f6a080057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a21747640 1 -- 172.21.15.16:0/2950535836 shutdown_connections 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a21747640 1 --2- 172.21.15.16:0/2950535836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08005180 0x7f6a0800b980 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a21747640 1 -- 172.21.15.16:0/2950535836 >> 172.21.15.16:0/2950535836 conn(0x7f6a1c06a800 msgr2=0x7f6a1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a21747640 1 -- 172.21.15.16:0/2950535836 shutdown_connections 2023-11-24T06:29:43.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.361+0000 7f6a21747640 1 -- 172.21.15.16:0/2950535836 wait complete. 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.363+0000 7f6a21747640 1 Processor -- start 2023-11-24T06:29:43.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.363+0000 7f6a21747640 1 -- start start 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a21747640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08005180 0x7f6a1c1f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a21747640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1c13fb10 0x7f6a1c137e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a21747640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1c1383d0 0x7f6a1c1f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a21747640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c139780 con 0x7f6a1c13fb10 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a21747640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c1f2b20 con 0x7f6a08005180 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a21747640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c13d980 con 0x7f6a1c1383d0 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a13fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1c13fb10 0x7f6a1c137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a13fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1c13fb10 0x7f6a1c137e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49846/0 (socket says 172.21.15.16:49846) 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a13fff640 1 -- 172.21.15.16:0/3215987919 learned_addr learned my addr 172.21.15.16:0/3215987919 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:43.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a1bfff640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08005180 0x7f6a1c1f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a20f46640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1c1383d0 0x7f6a1c1f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a13fff640 1 -- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1c1383d0 msgr2=0x7f6a1c1f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a13fff640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1c1383d0 0x7f6a1c1f3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a13fff640 1 -- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08005180 msgr2=0x7f6a1c1f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a13fff640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08005180 0x7f6a1c1f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a13fff640 1 -- 172.21.15.16:0/3215987919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c13dc60 con 0x7f6a1c13fb10 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.365+0000 7f6a13fff640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1c13fb10 0x7f6a1c137e90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400d770 tx=0x7f6a0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.366+0000 7f6a197fa640 1 -- 172.21.15.16:0/3215987919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04005270 con 0x7f6a1c13fb10 2023-11-24T06:29:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.366+0000 7f6a197fa640 1 -- 172.21.15.16:0/3215987919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a040053d0 con 0x7f6a1c13fb10 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.366+0000 7f6a197fa640 1 -- 172.21.15.16:0/3215987919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400b8f0 con 0x7f6a1c13fb10 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.366+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c13df40 con 0x7f6a1c13fb10 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.366+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c13d5e0 con 0x7f6a1c13fb10 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.367+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1c1f2990 con 0x7f6a1c13fb10 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.369+0000 7f6a197fa640 1 -- 172.21.15.16:0/3215987919 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6a0400ba50 con 0x7f6a1c13fb10 2023-11-24T06:29:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.370+0000 7f6a197fa640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b0b0 0x7f69fc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.370+0000 7f6a197fa640 1 -- 172.21.15.16:0/3215987919 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f6a04089860 con 0x7f6a1c13fb10 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.370+0000 7f6a1bfff640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b0b0 0x7f69fc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.372+0000 7f6a1bfff640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b0b0 0x7f69fc06d570 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009990 tx=0x7f6a0c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.375+0000 7f6a197fa640 1 -- 172.21.15.16:0/3215987919 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a040546e0 con 0x7f6a1c13fb10 2023-11-24T06:29:43.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.574+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a1c1f2c90 con 0x7f6a1c13fb10 2023-11-24T06:29:43.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.576+0000 7f6a197fa640 1 -- 172.21.15.16:0/3215987919 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13029 (secure 0 0 0) 0x7f6a04019310 con 0x7f6a1c13fb10 2023-11-24T06:29:43.580 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:43.580 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":50,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:41.990723+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6833/2818758442":"2023-11-25T06:29:38.701885+0000","172.21.15.16:6832/2818758442":"2023-11-25T06:29:38.701885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b0b0 msgr2=0x7f69fc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b0b0 0x7f69fc06d570 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009990 tx=0x7f6a0c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1c13fb10 msgr2=0x7f6a1c137e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1c13fb10 0x7f6a1c137e90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400d770 tx=0x7f6a0400dc30 comp rx=0 tx=0).stop 2023-11-24T06:29:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 shutdown_connections 2023-11-24T06:29:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b0b0 0x7f69fc06d570 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1c1383d0 0x7f6a1c1f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1c13fb10 0x7f6a1c137e90 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 --2- 172.21.15.16:0/3215987919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a08005180 0x7f6a1c1f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 >> 172.21.15.16:0/3215987919 conn(0x7f6a1c06a800 msgr2=0x7f6a1c0afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 shutdown_connections 2023-11-24T06:29:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.579+0000 7f6a21747640 1 -- 172.21.15.16:0/3215987919 wait complete. 2023-11-24T06:29:43.602 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:43.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.826+0000 7f2695c46640 1 Processor -- start 2023-11-24T06:29:43.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.826+0000 7f2695c46640 1 -- start start 2023-11-24T06:29:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f2695c46640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f269013df80 0x7f269015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f2695c46640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269006c470 con 0x7f269013e360 2023-11-24T06:29:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f2695c46640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269006c750 con 0x7f26900cf9a0 2023-11-24T06:29:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f2695c46640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269006ca30 con 0x7f269013df80 2023-11-24T06:29:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f268effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f269013e360 0x7f26900cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50970/0 (socket says 172.21.15.16:50970) 2023-11-24T06:29:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f268effd640 1 -- 172.21.15.16:0/1294577776 learned_addr learned my addr 172.21.15.16:0/1294577776 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f268f7fe640 1 --2- 172.21.15.16:0/1294577776 >> v2:172.21.15.16:3300/0 conn(0x7f269013df80 0x7f269015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f268e7fc640 1 -- 172.21.15.16:0/1294577776 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520358459 0 0) 0x7f269006c470 con 0x7f269013e360 2023-11-24T06:29:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f268e7fc640 1 -- 172.21.15.16:0/1294577776 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2678003520 con 0x7f269013e360 2023-11-24T06:29:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f268f7fe640 1 -- 172.21.15.16:0/1294577776 >> v1:172.21.15.173:6789/0 conn(0x7f26900cf9a0 legacy=0x7f26900cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f268f7fe640 1 -- 172.21.15.16:0/1294577776 >> v1:172.21.15.16:6789/0 conn(0x7f269013e360 legacy=0x7f26900cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f268e7fc640 1 -- 172.21.15.16:0/1294577776 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 186682641 0 0) 0x7f2678003520 con 0x7f269013e360 2023-11-24T06:29:43.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.829+0000 7f268f7fe640 1 -- 172.21.15.16:0/1294577776 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269006cd10 con 0x7f269013df80 2023-11-24T06:29:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.830+0000 7f268f7fe640 1 --2- 172.21.15.16:0/1294577776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 0x7f269015bee0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2680009d90 tx=0x7f268002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b3e729f74ba4fa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.830+0000 7f268e7fc640 1 -- 172.21.15.16:0/1294577776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684001670 con 0x7f269013df80 2023-11-24T06:29:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.830+0000 7f268e7fc640 1 -- 172.21.15.16:0/1294577776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268002c940 con 0x7f269013df80 2023-11-24T06:29:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.830+0000 7f268e7fc640 1 -- 172.21.15.16:0/1294577776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268002ccd0 con 0x7f269013df80 2023-11-24T06:29:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.831+0000 7f2695c46640 1 -- 172.21.15.16:0/1294577776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 msgr2=0x7f269015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.831+0000 7f2695c46640 1 --2- 172.21.15.16:0/1294577776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 0x7f269015bee0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2680009d90 tx=0x7f268002be10 comp rx=0 tx=0).stop 2023-11-24T06:29:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.831+0000 7f2695c46640 1 -- 172.21.15.16:0/1294577776 shutdown_connections 2023-11-24T06:29:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.831+0000 7f2695c46640 1 --2- 172.21.15.16:0/1294577776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 0x7f269015bee0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.831+0000 7f2695c46640 1 -- 172.21.15.16:0/1294577776 >> 172.21.15.16:0/1294577776 conn(0x7f269006a800 msgr2=0x7f269006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.831+0000 7f2695c46640 1 -- 172.21.15.16:0/1294577776 shutdown_connections 2023-11-24T06:29:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.831+0000 7f2695c46640 1 -- 172.21.15.16:0/1294577776 wait complete. 2023-11-24T06:29:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.833+0000 7f2695c46640 1 Processor -- start 2023-11-24T06:29:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.833+0000 7f2695c46640 1 -- start start 2023-11-24T06:29:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f2695c46640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26900cf9a0 0x7f269013b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f2695c46640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 0x7f2690139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f2695c46640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269013e360 0x7f2690137990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f2695c46640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269013b600 con 0x7f269013df80 2023-11-24T06:29:43.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f2695c46640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2690139bb0 con 0x7f269013e360 2023-11-24T06:29:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f2695c46640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2690153280 con 0x7f26900cf9a0 2023-11-24T06:29:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 0x7f2690139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 0x7f2690139670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49868/0 (socket says 172.21.15.16:49868) 2023-11-24T06:29:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268effd640 1 -- 172.21.15.16:0/1440690599 learned_addr learned my addr 172.21.15.16:0/1440690599 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268f7fe640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26900cf9a0 0x7f269013b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268ffff640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269013e360 0x7f2690137990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268effd640 1 -- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26900cf9a0 msgr2=0x7f269013b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268effd640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26900cf9a0 0x7f269013b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268effd640 1 -- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269013e360 msgr2=0x7f2690137990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268effd640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269013e360 0x7f2690137990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268effd640 1 -- 172.21.15.16:0/1440690599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2680009a70 con 0x7f269013df80 2023-11-24T06:29:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.835+0000 7f268ffff640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269013e360 0x7f2690137990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:43.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.836+0000 7f268effd640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 0x7f2690139670 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f268400c760 tx=0x7f268400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:43.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.836+0000 7f2677fff640 1 -- 172.21.15.16:0/1440690599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684005ad0 con 0x7f269013df80 2023-11-24T06:29:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.836+0000 7f2677fff640 1 -- 172.21.15.16:0/1440690599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684005c30 con 0x7f269013df80 2023-11-24T06:29:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.836+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269013c490 con 0x7f269013df80 2023-11-24T06:29:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.836+0000 7f2677fff640 1 -- 172.21.15.16:0/1440690599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684007070 con 0x7f269013df80 2023-11-24T06:29:43.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.836+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269013ca50 con 0x7f269013df80 2023-11-24T06:29:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.837+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269006d920 con 0x7f269013df80 2023-11-24T06:29:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.838+0000 7f2677fff640 1 -- 172.21.15.16:0/1440690599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f26840071d0 con 0x7f269013df80 2023-11-24T06:29:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.839+0000 7f2677fff640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06b0b0 0x7f267c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.839+0000 7f2677fff640 1 -- 172.21.15.16:0/1440690599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f2684089530 con 0x7f269013df80 2023-11-24T06:29:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.840+0000 7f268f7fe640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06b0b0 0x7f267c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:43.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.841+0000 7f268f7fe640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06b0b0 0x7f267c06d570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2680036000 tx=0x7f2680030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:43.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:43.845+0000 7f2677fff640 1 -- 172.21.15.16:0/1440690599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26840544c0 con 0x7f269013df80 2023-11-24T06:29:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.037+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26900675d0 con 0x7f269013df80 2023-11-24T06:29:44.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.039+0000 7f2677fff640 1 -- 172.21.15.16:0/1440690599 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2684057ae0 con 0x7f269013df80 2023-11-24T06:29:44.042 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:29:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.041+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06b0b0 msgr2=0x7f267c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.041+0000 7f2695c46640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06b0b0 0x7f267c06d570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2680036000 tx=0x7f2680030000 comp rx=0 tx=0).stop 2023-11-24T06:29:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.041+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 msgr2=0x7f2690139670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.041+0000 7f2695c46640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 0x7f2690139670 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f268400c760 tx=0x7f268400cc20 comp rx=0 tx=0).stop 2023-11-24T06:29:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.042+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 shutdown_connections 2023-11-24T06:29:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.042+0000 7f2695c46640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f267c06b0b0 0x7f267c06d570 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.042+0000 7f2695c46640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269013e360 0x7f2690137990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.042+0000 7f2695c46640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f269013df80 0x7f2690139670 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.042+0000 7f2695c46640 1 --2- 172.21.15.16:0/1440690599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26900cf9a0 0x7f269013b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.042+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 >> 172.21.15.16:0/1440690599 conn(0x7f269006a800 msgr2=0x7f26900cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:44.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.042+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 shutdown_connections 2023-11-24T06:29:44.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.042+0000 7f2695c46640 1 -- 172.21.15.16:0/1440690599 wait complete. 2023-11-24T06:29:44.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T06:29:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.287+0000 7f35e98bb640 1 Processor -- start 2023-11-24T06:29:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.288+0000 7f35e98bb640 1 -- start start 2023-11-24T06:29:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.289+0000 7f35e98bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4162430 0x7f35e40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.289+0000 7f35e98bb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f35e4162810 0x7f35e4071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.289+0000 7f35e98bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e40702e0 con 0x7f35e40b3650 2023-11-24T06:29:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.289+0000 7f35e98bb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e40705c0 con 0x7f35e4162810 2023-11-24T06:29:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.289+0000 7f35e98bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40708a0 con 0x7f35e4162430 2023-11-24T06:29:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.290+0000 7f35e90ba640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f35e40b3650 0x7f35e41650c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50984/0 (socket says 172.21.15.16:50984) 2023-11-24T06:29:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.290+0000 7f35e90ba640 1 -- 172.21.15.16:0/49264769 learned_addr learned my addr 172.21.15.16:0/49264769 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.290+0000 7f35e3fff640 1 --2- 172.21.15.16:0/49264769 >> v2:172.21.15.173:3300/0 conn(0x7f35e4162810 0x7f35e4071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.290+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744261051 0 0) 0x7f35e40702e0 con 0x7f35e40b3650 2023-11-24T06:29:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.290+0000 7f35e88b9640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4162430 0x7f35e40b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.290+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d4003520 con 0x7f35e40b3650 2023-11-24T06:29:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3215629252 0 0) 0x7f35d4003520 con 0x7f35e40b3650 2023-11-24T06:29:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4162430 msgr2=0x7f35e40b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4162430 0x7f35e40b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 >> v2:172.21.15.173:3300/0 conn(0x7f35e4162810 msgr2=0x7f35e4071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 --2- 172.21.15.16:0/49264769 >> v2:172.21.15.173:3300/0 conn(0x7f35e4162810 0x7f35e4071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4070b80 con 0x7f35e40b3650 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f35dc0019f0 con 0x7f35e40b3650 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 >> v1:172.21.15.16:6789/0 conn(0x7f35e40b3650 legacy=0x7f35e41650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d4003970 0x7f35d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4162430 0x7f35d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d4004d00 0x7f35d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40705c0 con 0x7f35d4003970 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40708a0 con 0x7f35d4004d00 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d400ba00 con 0x7f35e4162430 2023-11-24T06:29:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e3fff640 1 --2- 172.21.15.16:0/49264769 >> v2:172.21.15.173:3300/0 conn(0x7f35e4162810 0x7f35e4071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.291+0000 7f35e88b9640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d4003970 0x7f35d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.292+0000 7f35e90ba640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d4004d00 0x7f35d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.292+0000 7f35e3fff640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4162430 0x7f35d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.292+0000 7f35e88b9640 1 -- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4162430 msgr2=0x7f35d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.292+0000 7f35e88b9640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4162430 0x7f35d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.292+0000 7f35e88b9640 1 -- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d4004d00 msgr2=0x7f35d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.292+0000 7f35e88b9640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d4004d00 0x7f35d4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.292+0000 7f35e88b9640 1 -- 172.21.15.16:0/49264769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4003520 con 0x7f35d4003970 2023-11-24T06:29:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.292+0000 7f35e3fff640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35e4162430 0x7f35d40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.293+0000 7f35e88b9640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d4003970 0x7f35d4003d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009ac0 tx=0x7f35d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d984f8a9ebb8f9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.294+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d802c6b0 con 0x7f35d4003970 2023-11-24T06:29:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.294+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d802c810 con 0x7f35d4003970 2023-11-24T06:29:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.294+0000 7f35e37fe640 1 -- 172.21.15.16:0/49264769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d802cba0 con 0x7f35d4003970 2023-11-24T06:29:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.294+0000 7f35e98bb640 1 -- 172.21.15.16:0/49264769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d4003970 msgr2=0x7f35d4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.294+0000 7f35e98bb640 1 --2- 172.21.15.16:0/49264769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d4003970 0x7f35d4003d90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009ac0 tx=0x7f35d40057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.294+0000 7f35e90ba640 1 -- 172.21.15.16:0/49264769 reap_dead start 2023-11-24T06:29:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.295+0000 7f35e98bb640 1 -- 172.21.15.16:0/49264769 shutdown_connections 2023-11-24T06:29:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.295+0000 7f35e98bb640 1 -- 172.21.15.16:0/49264769 >> 172.21.15.16:0/49264769 conn(0x7f35e406a800 msgr2=0x7f35e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.295+0000 7f35e98bb640 1 -- 172.21.15.16:0/49264769 shutdown_connections 2023-11-24T06:29:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.295+0000 7f35e98bb640 1 -- 172.21.15.16:0/49264769 wait complete. 2023-11-24T06:29:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.297+0000 7f35e98bb640 1 Processor -- start 2023-11-24T06:29:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.297+0000 7f35e98bb640 1 -- start start 2023-11-24T06:29:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e98bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d4003970 0x7f35e41f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e98bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4164630 0x7f35e415b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e98bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35e415b540 0x7f35e41f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e98bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e415b9c0 con 0x7f35e4164630 2023-11-24T06:29:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e98bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4164b60 con 0x7f35e415b540 2023-11-24T06:29:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e98bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4164e40 con 0x7f35d4003970 2023-11-24T06:29:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4164630 0x7f35e415b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4164630 0x7f35e415b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49886/0 (socket says 172.21.15.16:49886) 2023-11-24T06:29:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e3fff640 1 -- 172.21.15.16:0/2452207804 learned_addr learned my addr 172.21.15.16:0/2452207804 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.298+0000 7f35e88b9640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d4003970 0x7f35e41f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e3fff640 1 -- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d4003970 msgr2=0x7f35e41f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e3fff640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d4003970 0x7f35e41f67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e3fff640 1 -- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35e415b540 msgr2=0x7f35e41f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e90ba640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35e415b540 0x7f35e41f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e3fff640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35e415b540 0x7f35e41f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e3fff640 1 -- 172.21.15.16:0/2452207804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4165120 con 0x7f35e4164630 2023-11-24T06:29:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e88b9640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d4003970 0x7f35e41f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e3fff640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4164630 0x7f35e415b000 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f35d000f600 tx=0x7f35d000fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e17fa640 1 -- 172.21.15.16:0/2452207804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0014890 con 0x7f35e4164630 2023-11-24T06:29:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e17fa640 1 -- 172.21.15.16:0/2452207804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000c030 con 0x7f35e4164630 2023-11-24T06:29:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e4165400 con 0x7f35e4164630 2023-11-24T06:29:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.299+0000 7f35e17fa640 1 -- 172.21.15.16:0/2452207804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0014c30 con 0x7f35e4164630 2023-11-24T06:29:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.300+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e41f7890 con 0x7f35e4164630 2023-11-24T06:29:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.301+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e4067450 con 0x7f35e4164630 2023-11-24T06:29:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.302+0000 7f35e17fa640 1 -- 172.21.15.16:0/2452207804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f35d000d490 con 0x7f35e4164630 2023-11-24T06:29:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.303+0000 7f35e17fa640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35c806b080 0x7f35c806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.303+0000 7f35e17fa640 1 -- 172.21.15.16:0/2452207804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f35d0089c40 con 0x7f35e4164630 2023-11-24T06:29:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.304+0000 7f35e88b9640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35c806b080 0x7f35c806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.305+0000 7f35e88b9640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35c806b080 0x7f35c806d540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f35d8036000 tx=0x7f35d8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.308+0000 7f35e17fa640 1 -- 172.21.15.16:0/2452207804 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35d0054b60 con 0x7f35e4164630 2023-11-24T06:29:44.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:44.570+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7f35e40b2ef0 con 0x7f35e4164630 2023-11-24T06:29:45.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.124+0000 7f35e17fa640 1 -- 172.21.15.16:0/2452207804 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v177) v1 ==== 169+0+0 (secure 0 0 0) 0x7f35d00220f0 con 0x7f35e4164630 2023-11-24T06:29:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35c806b080 msgr2=0x7f35c806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35c806b080 0x7f35c806d540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f35d8036000 tx=0x7f35d8030000 comp rx=0 tx=0).stop 2023-11-24T06:29:45.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4164630 msgr2=0x7f35e415b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4164630 0x7f35e415b000 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f35d000f600 tx=0x7f35d000fac0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 shutdown_connections 2023-11-24T06:29:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f35c806b080 0x7f35c806d540 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35e415b540 0x7f35e41f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4164630 0x7f35e415b000 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 --2- 172.21.15.16:0/2452207804 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d4003970 0x7f35e41f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 >> 172.21.15.16:0/2452207804 conn(0x7f35e406a800 msgr2=0x7f35e40af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.128+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 shutdown_connections 2023-11-24T06:29:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.129+0000 7f35e98bb640 1 -- 172.21.15.16:0/2452207804 wait complete. 2023-11-24T06:29:45.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:29:45.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.374+0000 7f8a017b3640 1 Processor -- start 2023-11-24T06:29:45.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.374+0000 7f8a017b3640 1 -- start start 2023-11-24T06:29:45.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.376+0000 7f8a017b3640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f89fc1447f0 0x7f89fc15c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.376+0000 7f8a017b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89fc0cccf0 0x7f89fc15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.376+0000 7f8a017b3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc15c5c0 con 0x7f89fc06c470 2023-11-24T06:29:45.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.376+0000 7f8a017b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc06d690 con 0x7f89fc0cccf0 2023-11-24T06:29:45.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.376+0000 7f8a017b3640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc0cebb0 con 0x7f89fc1447f0 2023-11-24T06:29:45.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f8a00fb2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89fc0cccf0 0x7f89fc15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f8a00fb2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89fc0cccf0 0x7f89fc15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49898/0 (socket says 172.21.15.16:49898) 2023-11-24T06:29:45.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f8a00fb2640 1 -- 172.21.15.16:0/856103890 learned_addr learned my addr 172.21.15.16:0/856103890 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:45.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f89f3fff640 1 --1- 172.21.15.16:0/856103890 >> v1:172.21.15.16:6789/0 conn(0x7f89fc06c470 0x7f89fc0cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50986/0 (socket says 172.21.15.16:50986) 2023-11-24T06:29:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f89fbfff640 1 --2- 172.21.15.16:0/856103890 >> v2:172.21.15.173:3300/0 conn(0x7f89fc1447f0 0x7f89fc15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f8a00fb2640 1 -- 172.21.15.16:0/856103890 >> v2:172.21.15.173:3300/0 conn(0x7f89fc1447f0 msgr2=0x7f89fc15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f8a00fb2640 1 --2- 172.21.15.16:0/856103890 >> v2:172.21.15.173:3300/0 conn(0x7f89fc1447f0 0x7f89fc15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f8a00fb2640 1 -- 172.21.15.16:0/856103890 >> v1:172.21.15.16:6789/0 conn(0x7f89fc06c470 legacy=0x7f89fc0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f8a00fb2640 1 -- 172.21.15.16:0/856103890 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc06ca10 con 0x7f89fc0cccf0 2023-11-24T06:29:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f89fb7fe640 1 -- 172.21.15.16:0/856103890 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815883684 0 0) 0x7f89fc15c5c0 con 0x7f89fc06c470 2023-11-24T06:29:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.377+0000 7f89fbfff640 1 --2- 172.21.15.16:0/856103890 >> v2:172.21.15.173:3300/0 conn(0x7f89fc1447f0 0x7f89fc15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.379+0000 7f8a00fb2640 1 --2- 172.21.15.16:0/856103890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc0cccf0 0x7f89fc15c7c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f89fc072520 tx=0x7f89f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d0ed5ab7dc9286 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.379+0000 7f89fb7fe640 1 -- 172.21.15.16:0/856103890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f402ca80 con 0x7f89fc0cccf0 2023-11-24T06:29:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.379+0000 7f89fb7fe640 1 -- 172.21.15.16:0/856103890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f402cbe0 con 0x7f89fc0cccf0 2023-11-24T06:29:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.379+0000 7f89fb7fe640 1 -- 172.21.15.16:0/856103890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f40343f0 con 0x7f89fc0cccf0 2023-11-24T06:29:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.379+0000 7f8a017b3640 1 -- 172.21.15.16:0/856103890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc0cccf0 msgr2=0x7f89fc15c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.380+0000 7f8a017b3640 1 --2- 172.21.15.16:0/856103890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc0cccf0 0x7f89fc15c7c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f89fc072520 tx=0x7f89f402bee0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.380+0000 7f8a017b3640 1 -- 172.21.15.16:0/856103890 shutdown_connections 2023-11-24T06:29:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.380+0000 7f8a017b3640 1 --2- 172.21.15.16:0/856103890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc0cccf0 0x7f89fc15c7c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.380+0000 7f8a017b3640 1 --2- 172.21.15.16:0/856103890 >> v2:172.21.15.173:3300/0 conn(0x7f89fc1447f0 0x7f89fc15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.380+0000 7f8a017b3640 1 -- 172.21.15.16:0/856103890 >> 172.21.15.16:0/856103890 conn(0x7f89fc06a3e0 msgr2=0x7f89fc06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.380+0000 7f8a017b3640 1 -- 172.21.15.16:0/856103890 shutdown_connections 2023-11-24T06:29:45.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.380+0000 7f8a017b3640 1 -- 172.21.15.16:0/856103890 wait complete. 2023-11-24T06:29:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.382+0000 7f8a017b3640 1 Processor -- start 2023-11-24T06:29:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.382+0000 7f8a017b3640 1 -- start start 2023-11-24T06:29:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a017b3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89fc06c470 0x7f89fc13eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a017b3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89fc0cccf0 0x7f89fc13a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a017b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc1447f0 0x7f89fc0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a017b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0aefb0 con 0x7f89fc1447f0 2023-11-24T06:29:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a017b3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc13f400 con 0x7f89fc0cccf0 2023-11-24T06:29:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a017b3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0af190 con 0x7f89fc06c470 2023-11-24T06:29:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a00fb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc1447f0 0x7f89fc0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a00fb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc1447f0 0x7f89fc0b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49904/0 (socket says 172.21.15.16:49904) 2023-11-24T06:29:45.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a00fb2640 1 -- 172.21.15.16:0/267706699 learned_addr learned my addr 172.21.15.16:0/267706699 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a00fb2640 1 -- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89fc06c470 msgr2=0x7f89fc13eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a00fb2640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89fc06c470 0x7f89fc13eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a00fb2640 1 -- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89fc0cccf0 msgr2=0x7f89fc13a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a00fb2640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89fc0cccf0 0x7f89fc13a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.383+0000 7f8a00fb2640 1 -- 172.21.15.16:0/267706699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f4009b30 con 0x7f89fc1447f0 2023-11-24T06:29:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.384+0000 7f8a00fb2640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc1447f0 0x7f89fc0b0c30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f89f4036040 tx=0x7f89f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.384+0000 7f89f97fa640 1 -- 172.21.15.16:0/267706699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4009510 con 0x7f89fc1447f0 2023-11-24T06:29:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.384+0000 7f89f97fa640 1 -- 172.21.15.16:0/267706699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f403e450 con 0x7f89fc1447f0 2023-11-24T06:29:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.384+0000 7f89f97fa640 1 -- 172.21.15.16:0/267706699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f403e800 con 0x7f89fc1447f0 2023-11-24T06:29:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.384+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc0af470 con 0x7f89fc1447f0 2023-11-24T06:29:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.384+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc0af750 con 0x7f89fc1447f0 2023-11-24T06:29:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.385+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc067450 con 0x7f89fc1447f0 2023-11-24T06:29:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.386+0000 7f89f97fa640 1 -- 172.21.15.16:0/267706699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f89f402cd50 con 0x7f89fc1447f0 2023-11-24T06:29:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.387+0000 7f89f97fa640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f89dc06b080 0x7f89dc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.387+0000 7f89f97fa640 1 -- 172.21.15.16:0/267706699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f89f40ac2d0 con 0x7f89fc1447f0 2023-11-24T06:29:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.388+0000 7f89fbfff640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f89dc06b080 0x7f89dc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.389+0000 7f89fbfff640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f89dc06b080 0x7f89dc06d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006fd0 tx=0x7f89ec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.392+0000 7f89f97fa640 1 -- 172.21.15.16:0/267706699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89f4077560 con 0x7f89fc1447f0 2023-11-24T06:29:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.587+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f89fc1e3be0 con 0x7f89fc1447f0 2023-11-24T06:29:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.594+0000 7f89f97fa640 1 -- 172.21.15.16:0/267706699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f89f407ab80 con 0x7f89fc1447f0 2023-11-24T06:29:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f89dc06b080 msgr2=0x7f89dc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f89dc06b080 0x7f89dc06d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006fd0 tx=0x7f89ec008040 comp rx=0 tx=0).stop 2023-11-24T06:29:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc1447f0 msgr2=0x7f89fc0b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc1447f0 0x7f89fc0b0c30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f89f4036040 tx=0x7f89f4004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 shutdown_connections 2023-11-24T06:29:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f89dc06b080 0x7f89dc06d540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc1447f0 0x7f89fc0b0c30 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89fc0cccf0 0x7f89fc13a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 --2- 172.21.15.16:0/267706699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89fc06c470 0x7f89fc13eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.596+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 >> 172.21.15.16:0/267706699 conn(0x7f89fc06a3e0 msgr2=0x7f89fc13b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.597+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 shutdown_connections 2023-11-24T06:29:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:45.597+0000 7f8a017b3640 1 -- 172.21.15.16:0/267706699 wait complete. 2023-11-24T06:29:45.618 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:29:45.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:29:45.739 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:29:45.862 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:29:45.862 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:29:45.863 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:45.928 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:29:45.928 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:29:45.928 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:45.989 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:29:45.990 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:29:45.990 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:46.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:29:46.057 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:29:46.058 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:29:46.120 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:29:46.120 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:29:46.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.356+0000 7f3cbffff640 1 Processor -- start 2023-11-24T06:29:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.356+0000 7f3cbffff640 1 -- start start 2023-11-24T06:29:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.357+0000 7f3cbffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3cc0160920 0x7f3cc00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.357+0000 7f3cbffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3cc00cfea0 0x7f3cc0161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.357+0000 7f3cbffff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc00d0a10 con 0x7f3cc00cf9f0 2023-11-24T06:29:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.357+0000 7f3cbffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc00d1f50 con 0x7f3cc0160920 2023-11-24T06:29:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.357+0000 7f3cbffff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc00cf080 con 0x7f3cc00cfea0 2023-11-24T06:29:46.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.357+0000 7f3cbeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3cc0160920 0x7f3cc00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.357+0000 7f3cbeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3cc0160920 0x7f3cc00b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49912/0 (socket says 172.21.15.16:49912) 2023-11-24T06:29:46.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.357+0000 7f3cbeffd640 1 -- 172.21.15.16:0/15309864 learned_addr learned my addr 172.21.15.16:0/15309864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:46.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.358+0000 7f3cbf7fe640 1 --2- 172.21.15.16:0/15309864 >> v2:172.21.15.173:3300/0 conn(0x7f3cc00cfea0 0x7f3cc0161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.358+0000 7f3cbeffd640 1 -- 172.21.15.16:0/15309864 >> v2:172.21.15.173:3300/0 conn(0x7f3cc00cfea0 msgr2=0x7f3cc0161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.358+0000 7f3cbeffd640 1 --2- 172.21.15.16:0/15309864 >> v2:172.21.15.173:3300/0 conn(0x7f3cc00cfea0 0x7f3cc0161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.358+0000 7f3cbeffd640 1 -- 172.21.15.16:0/15309864 >> v1:172.21.15.173:6789/0 conn(0x7f3cc00cf9f0 legacy=0x7f3cc0161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.358+0000 7f3cbeffd640 1 -- 172.21.15.16:0/15309864 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc00d0310 con 0x7f3cc0160920 2023-11-24T06:29:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.358+0000 7f3cbf7fe640 1 --2- 172.21.15.16:0/15309864 >> v2:172.21.15.173:3300/0 conn(0x7f3cc00cfea0 0x7f3cc0161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.359+0000 7f3cbeffd640 1 --2- 172.21.15.16:0/15309864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 0x7f3cc00b3f30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3cc008c2c0 tx=0x7f3cb002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a998a480239d28 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.359+0000 7f3cbdffb640 1 -- 172.21.15.16:0/15309864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb002ca80 con 0x7f3cc0160920 2023-11-24T06:29:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.359+0000 7f3cbdffb640 1 -- 172.21.15.16:0/15309864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb002cbe0 con 0x7f3cc0160920 2023-11-24T06:29:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.359+0000 7f3cbdffb640 1 -- 172.21.15.16:0/15309864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00343f0 con 0x7f3cc0160920 2023-11-24T06:29:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.360+0000 7f3cbffff640 1 -- 172.21.15.16:0/15309864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 msgr2=0x7f3cc00b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.360+0000 7f3cbffff640 1 --2- 172.21.15.16:0/15309864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 0x7f3cc00b3f30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3cc008c2c0 tx=0x7f3cb002bee0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.360+0000 7f3cbffff640 1 -- 172.21.15.16:0/15309864 shutdown_connections 2023-11-24T06:29:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.360+0000 7f3cbffff640 1 --2- 172.21.15.16:0/15309864 >> v2:172.21.15.173:3300/0 conn(0x7f3cc00cfea0 0x7f3cc0161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.360+0000 7f3cbffff640 1 --2- 172.21.15.16:0/15309864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 0x7f3cc00b3f30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.360+0000 7f3cbffff640 1 -- 172.21.15.16:0/15309864 >> 172.21.15.16:0/15309864 conn(0x7f3cc00d3090 msgr2=0x7f3cc00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.360+0000 7f3cbffff640 1 -- 172.21.15.16:0/15309864 shutdown_connections 2023-11-24T06:29:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.360+0000 7f3cbffff640 1 -- 172.21.15.16:0/15309864 wait complete. 2023-11-24T06:29:46.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.362+0000 7f3cbffff640 1 Processor -- start 2023-11-24T06:29:46.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.362+0000 7f3cbffff640 1 -- start start 2023-11-24T06:29:46.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.363+0000 7f3cbffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc00cf9f0 0x7f3cc01f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.363+0000 7f3cbffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cc00cfea0 0x7f3cc01f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.363+0000 7f3cbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 0x7f3cc00c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.363+0000 7f3cbffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc01f7150 con 0x7f3cc0160920 2023-11-24T06:29:46.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.363+0000 7f3cbffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc015d670 con 0x7f3cc00cf9f0 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.363+0000 7f3cbffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc015d950 con 0x7f3cc00cfea0 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 0x7f3cc00c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 0x7f3cc00c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49928/0 (socket says 172.21.15.16:49928) 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbf7fe640 1 -- 172.21.15.16:0/353108422 learned_addr learned my addr 172.21.15.16:0/353108422 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbeffd640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc00cf9f0 0x7f3cc01f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbe7fc640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cc00cfea0 0x7f3cc01f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbf7fe640 1 -- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cc00cfea0 msgr2=0x7f3cc01f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbf7fe640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cc00cfea0 0x7f3cc01f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbf7fe640 1 -- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc00cf9f0 msgr2=0x7f3cc01f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbf7fe640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc00cf9f0 0x7f3cc01f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbf7fe640 1 -- 172.21.15.16:0/353108422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb0009b30 con 0x7f3cc0160920 2023-11-24T06:29:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbe7fc640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cc00cfea0 0x7f3cc01f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.364+0000 7f3cbf7fe640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 0x7f3cc00c86f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c7c0 tx=0x7f3cb800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.365+0000 7f3ca77fe640 1 -- 172.21.15.16:0/353108422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8004230 con 0x7f3cc0160920 2023-11-24T06:29:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.365+0000 7f3ca77fe640 1 -- 172.21.15.16:0/353108422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb8004390 con 0x7f3cc0160920 2023-11-24T06:29:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.365+0000 7f3ca77fe640 1 -- 172.21.15.16:0/353108422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8007860 con 0x7f3cc0160920 2023-11-24T06:29:46.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.365+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc015dc30 con 0x7f3cc0160920 2023-11-24T06:29:46.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.365+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc01f2970 con 0x7f3cc0160920 2023-11-24T06:29:46.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.366+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc0067450 con 0x7f3cc0160920 2023-11-24T06:29:46.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.367+0000 7f3ca77fe640 1 -- 172.21.15.16:0/353108422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3cb80079c0 con 0x7f3cc0160920 2023-11-24T06:29:46.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.368+0000 7f3ca77fe640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3cac06b080 0x7f3cac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.369+0000 7f3ca77fe640 1 -- 172.21.15.16:0/353108422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f3cb8089b60 con 0x7f3cc0160920 2023-11-24T06:29:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.369+0000 7f3cbeffd640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3cac06b080 0x7f3cac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.370+0000 7f3cbeffd640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3cac06b080 0x7f3cac06d540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0035000 tx=0x7f3cb0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.373+0000 7f3ca77fe640 1 -- 172.21.15.16:0/353108422 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3cb8054aa0 con 0x7f3cc0160920 2023-11-24T06:29:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.602+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cc008c7e0 con 0x7f3cc0160920 2023-11-24T06:29:46.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.603+0000 7f3ca77fe640 1 -- 172.21.15.16:0/353108422 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f3cb8015090 con 0x7f3cc0160920 2023-11-24T06:29:46.606 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:46.606 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4853,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/2110540698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":2110540698},{"type":"v1","addr":"172.21.15.173:6837","nonce":2110540698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1298790428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1298790428},{"type":"v1","addr":"172.21.15.16:6835","nonce":1298790428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5107,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/2876598187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2876598187},{"type":"v1","addr":"172.21.15.173:6836","nonce":2876598187}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6120,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3549188407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3549188407},{"type":"v1","addr":"172.21.15.16:6833","nonce":3549188407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2023-11-24T06:29:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2023-11-24T06:29:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.605+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3cac06b080 msgr2=0x7f3cac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.605+0000 7f3cbffff640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3cac06b080 0x7f3cac06d540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0035000 tx=0x7f3cb0030000 comp rx=0 tx=0).stop 2023-11-24T06:29:46.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 msgr2=0x7f3cc00c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 0x7f3cc00c86f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c7c0 tx=0x7f3cb800cc80 comp rx=0 tx=0).stop 2023-11-24T06:29:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 shutdown_connections 2023-11-24T06:29:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3cac06b080 0x7f3cac06d540 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cc0160920 0x7f3cc00c86f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cc00cfea0 0x7f3cc01f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 --2- 172.21.15.16:0/353108422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cc00cf9f0 0x7f3cc01f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 >> 172.21.15.16:0/353108422 conn(0x7f3cc00d3090 msgr2=0x7f3cc006aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 shutdown_connections 2023-11-24T06:29:46.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.606+0000 7f3cbffff640 1 -- 172.21.15.16:0/353108422 wait complete. 2023-11-24T06:29:46.626 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:29:46.626 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:29:46.627 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:29:46.627 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:29:46.627 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:29:46.627 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:29:46.627 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:29:46.627 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:29:46.627 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:29:46.628 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:29:46.628 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:29:46.628 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:29:46.628 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:29:46.628 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:29:46.628 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:29:46.628 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:29:46.646 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:29:46.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:29:46.647 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:29:46.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:29:46.648 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:29:46.648 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:29:46.654 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:29:46.654 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:29:46.693 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:29:46.694 DEBUG:teuthology.parallel:result is None 2023-11-24T06:29:46.698 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:29:46.698 DEBUG:teuthology.parallel:result is None 2023-11-24T06:29:46.701 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:29:46.702 DEBUG:teuthology.parallel:result is None 2023-11-24T06:29:46.709 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:29:46.710 DEBUG:teuthology.parallel:result is None 2023-11-24T06:29:46.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:29:46.853 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:29:46.864 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-24T06:29:46.865 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2023-11-24T06:29:46.873 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2023-11-24T06:29:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.901+0000 7f370810f640 1 Processor -- start 2023-11-24T06:29:46.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.902+0000 7f370810f640 1 -- start start 2023-11-24T06:29:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.903+0000 7f370810f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37000cc940 0x7f37000cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.903+0000 7f370810f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f370006a3e0 0x7f370006a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.903+0000 7f370810f640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37000cfcc0 con 0x7f3700144c60 2023-11-24T06:29:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.903+0000 7f370810f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370006adb0 con 0x7f37000cc940 2023-11-24T06:29:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.903+0000 7f370810f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3700071a20 con 0x7f370006a3e0 2023-11-24T06:29:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.903+0000 7f37056aa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37000cc940 0x7f37000cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.903+0000 7f37066ac640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f370006a3e0 0x7f370006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.903+0000 7f3705eab640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f3700144c60 0x7f370006e7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:42256/0 (socket says 172.21.15.16:42256) 2023-11-24T06:29:46.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.903+0000 7f3705eab640 1 -- 172.21.15.16:0/4208460540 learned_addr learned my addr 172.21.15.16:0/4208460540 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.904+0000 7f3704ea9640 1 -- 172.21.15.16:0/4208460540 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577010617 0 0) 0x7f36fc001110 con 0x7f3700144c60 2023-11-24T06:29:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.904+0000 7f3704ea9640 1 -- 172.21.15.16:0/4208460540 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36ec003520 con 0x7f3700144c60 2023-11-24T06:29:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.904+0000 7f37066ac640 1 -- 172.21.15.16:0/4208460540 >> v2:172.21.15.16:3300/0 conn(0x7f37000cc940 msgr2=0x7f37000cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.904+0000 7f37066ac640 1 --2- 172.21.15.16:0/4208460540 >> v2:172.21.15.16:3300/0 conn(0x7f37000cc940 0x7f37000cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.904+0000 7f37066ac640 1 -- 172.21.15.16:0/4208460540 >> v1:172.21.15.173:6789/0 conn(0x7f3700144c60 legacy=0x7f370006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.904+0000 7f37066ac640 1 -- 172.21.15.16:0/4208460540 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37000ccd80 con 0x7f370006a3e0 2023-11-24T06:29:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.904+0000 7f37056aa640 1 --2- 172.21.15.16:0/4208460540 >> v2:172.21.15.16:3300/0 conn(0x7f37000cc940 0x7f37000cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:29:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.905+0000 7f37066ac640 1 --2- 172.21.15.16:0/4208460540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f370006a3e0 0x7f370006a870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009d90 tx=0x7f36f802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d606a2c2c3384165 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.905+0000 7f3704ea9640 1 -- 172.21.15.16:0/4208460540 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f802cc60 con 0x7f370006a3e0 2023-11-24T06:29:46.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.905+0000 7f3704ea9640 1 -- 172.21.15.16:0/4208460540 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f802cdc0 con 0x7f370006a3e0 2023-11-24T06:29:46.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.905+0000 7f3704ea9640 1 -- 172.21.15.16:0/4208460540 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8034660 con 0x7f370006a3e0 2023-11-24T06:29:46.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.906+0000 7f370810f640 1 -- 172.21.15.16:0/4208460540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f370006a3e0 msgr2=0x7f370006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.906+0000 7f370810f640 1 --2- 172.21.15.16:0/4208460540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f370006a3e0 0x7f370006a870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009d90 tx=0x7f36f802beb0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.906+0000 7f370810f640 1 -- 172.21.15.16:0/4208460540 shutdown_connections 2023-11-24T06:29:46.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.906+0000 7f370810f640 1 --2- 172.21.15.16:0/4208460540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f370006a3e0 0x7f370006a870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.906+0000 7f370810f640 1 --2- 172.21.15.16:0/4208460540 >> v2:172.21.15.16:3300/0 conn(0x7f37000cc940 0x7f37000cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.906+0000 7f370810f640 1 -- 172.21.15.16:0/4208460540 >> 172.21.15.16:0/4208460540 conn(0x7f37000cfef0 msgr2=0x7f37000d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.906+0000 7f370810f640 1 -- 172.21.15.16:0/4208460540 shutdown_connections 2023-11-24T06:29:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.906+0000 7f370810f640 1 -- 172.21.15.16:0/4208460540 wait complete. 2023-11-24T06:29:46.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.908+0000 7f370810f640 1 Processor -- start 2023-11-24T06:29:46.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.908+0000 7f370810f640 1 -- start start 2023-11-24T06:29:46.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f370810f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f370006a3e0 0x7f370013c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f370810f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37000cc940 0x7f37000c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f370810f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3700144c60 0x7f37000c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f370810f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001f6f10 con 0x7f37000cc940 2023-11-24T06:29:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f370810f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001f6300 con 0x7f370006a3e0 2023-11-24T06:29:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f370810f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37001f65e0 con 0x7f3700144c60 2023-11-24T06:29:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f37056aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37000cc940 0x7f37000c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f37056aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37000cc940 0x7f37000c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49994/0 (socket says 172.21.15.16:49994) 2023-11-24T06:29:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f37056aa640 1 -- 172.21.15.16:0/487351970 learned_addr learned my addr 172.21.15.16:0/487351970 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f37066ac640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3700144c60 0x7f37000c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.909+0000 7f3705eab640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f370006a3e0 0x7f370013c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f37056aa640 1 -- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3700144c60 msgr2=0x7f37000c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f37056aa640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3700144c60 0x7f37000c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f37056aa640 1 -- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f370006a3e0 msgr2=0x7f370013c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:46.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f37056aa640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f370006a3e0 0x7f370013c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:46.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f37056aa640 1 -- 172.21.15.16:0/487351970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f0009400 con 0x7f37000cc940 2023-11-24T06:29:46.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f37056aa640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37000cc940 0x7f37000c92e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f36f000ebe0 tx=0x7f36f000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:46.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f36f67fc640 1 -- 172.21.15.16:0/487351970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0009760 con 0x7f37000cc940 2023-11-24T06:29:46.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f36f67fc640 1 -- 172.21.15.16:0/487351970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f000cc30 con 0x7f37000cc940 2023-11-24T06:29:46.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f36f67fc640 1 -- 172.21.15.16:0/487351970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f000eee0 con 0x7f37000cc940 2023-11-24T06:29:46.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f8009a70 con 0x7f37000cc940 2023-11-24T06:29:46.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.910+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37001f6ba0 con 0x7f37000cc940 2023-11-24T06:29:46.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.912+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37000675d0 con 0x7f37000cc940 2023-11-24T06:29:46.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.913+0000 7f36f67fc640 1 -- 172.21.15.16:0/487351970 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f36f0013470 con 0x7f37000cc940 2023-11-24T06:29:46.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.914+0000 7f36f67fc640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36e406b080 0x7f36e406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:46.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.914+0000 7f36f67fc640 1 -- 172.21.15.16:0/487351970 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f36f0089b00 con 0x7f37000cc940 2023-11-24T06:29:46.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.914+0000 7f3705eab640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36e406b080 0x7f36e406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:46.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.915+0000 7f3705eab640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36e406b080 0x7f36e406d540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f36fc001270 tx=0x7f36fc007bb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:46.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:46.918+0000 7f36f67fc640 1 -- 172.21.15.16:0/487351970 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36f008be60 con 0x7f37000cc940 2023-11-24T06:29:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:47.131+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f37000d0a10 con 0x7f37000cc940 2023-11-24T06:29:48.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.133+0000 7f36f67fc640 1 -- 172.21.15.16:0/487351970 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) v1 ==== 97+0+0 (secure 0 0 0) 0x7f36f0054ac0 con 0x7f37000cc940 2023-11-24T06:29:48.137 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-11-24T06:29:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36e406b080 msgr2=0x7f36e406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36e406b080 0x7f36e406d540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f36fc001270 tx=0x7f36fc007bb0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37000cc940 msgr2=0x7f37000c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37000cc940 0x7f37000c92e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f36f000ebe0 tx=0x7f36f000c520 comp rx=0 tx=0).stop 2023-11-24T06:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 shutdown_connections 2023-11-24T06:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36e406b080 0x7f36e406d540 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3700144c60 0x7f37000c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37000cc940 0x7f37000c92e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 --2- 172.21.15.16:0/487351970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f370006a3e0 0x7f370013c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 >> 172.21.15.16:0/487351970 conn(0x7f37000cfef0 msgr2=0x7f37000d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 shutdown_connections 2023-11-24T06:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.138+0000 7f370810f640 1 -- 172.21.15.16:0/487351970 wait complete. 2023-11-24T06:29:48.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:29:48.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.379+0000 7f7ba4a50640 1 Processor -- start 2023-11-24T06:29:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.379+0000 7f7ba4a50640 1 -- start start 2023-11-24T06:29:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.381+0000 7f7ba4a50640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7ba00d0100 0x7f7ba00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.381+0000 7f7ba4a50640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ba00d0f40 0x7f7ba00d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.381+0000 7f7ba4a50640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba00d0a10 con 0x7f7ba0160920 2023-11-24T06:29:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.381+0000 7f7ba4a50640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba00cf080 con 0x7f7ba00d0f40 2023-11-24T06:29:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.381+0000 7f7ba4a50640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba00c6be0 con 0x7f7ba00d0100 2023-11-24T06:29:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.381+0000 7f7b9ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7ba0160920 0x7f7ba00cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51034/0 (socket says 172.21.15.16:51034) 2023-11-24T06:29:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.381+0000 7f7b9ed9d640 1 -- 172.21.15.16:0/1477283358 learned_addr learned my addr 172.21.15.16:0/1477283358 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.381+0000 7f7b9f59e640 1 --2- 172.21.15.16:0/1477283358 >> v2:172.21.15.16:3300/0 conn(0x7f7ba00d0f40 0x7f7ba00d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.382+0000 7f7b97fff640 1 --2- 172.21.15.16:0/1477283358 >> v2:172.21.15.173:3300/0 conn(0x7f7ba00d0100 0x7f7ba00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.382+0000 7f7b9e59c640 1 -- 172.21.15.16:0/1477283358 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773963801 0 0) 0x7f7ba00d0a10 con 0x7f7ba0160920 2023-11-24T06:29:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.382+0000 7f7b9f59e640 1 -- 172.21.15.16:0/1477283358 >> v2:172.21.15.173:3300/0 conn(0x7f7ba00d0100 msgr2=0x7f7ba00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.382+0000 7f7b9f59e640 1 --2- 172.21.15.16:0/1477283358 >> v2:172.21.15.173:3300/0 conn(0x7f7ba00d0100 0x7f7ba00b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.382+0000 7f7b9f59e640 1 -- 172.21.15.16:0/1477283358 >> v1:172.21.15.16:6789/0 conn(0x7f7ba0160920 legacy=0x7f7ba00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.382+0000 7f7b9f59e640 1 -- 172.21.15.16:0/1477283358 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba00ce7e0 con 0x7f7ba00d0f40 2023-11-24T06:29:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.383+0000 7f7b9f59e640 1 --2- 172.21.15.16:0/1477283358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba00d0f40 0x7f7ba00d1390 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7ba008c1f0 tx=0x7f7b9802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0241be4c4cefe7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.383+0000 7f7b9e59c640 1 -- 172.21.15.16:0/1477283358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9802cc70 con 0x7f7ba00d0f40 2023-11-24T06:29:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.383+0000 7f7b9e59c640 1 -- 172.21.15.16:0/1477283358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9802cdd0 con 0x7f7ba00d0f40 2023-11-24T06:29:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.383+0000 7f7b9e59c640 1 -- 172.21.15.16:0/1477283358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98034660 con 0x7f7ba00d0f40 2023-11-24T06:29:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.384+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1477283358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba00d0f40 msgr2=0x7f7ba00d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.384+0000 7f7ba4a50640 1 --2- 172.21.15.16:0/1477283358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba00d0f40 0x7f7ba00d1390 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7ba008c1f0 tx=0x7f7b9802bec0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.384+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1477283358 shutdown_connections 2023-11-24T06:29:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.384+0000 7f7ba4a50640 1 --2- 172.21.15.16:0/1477283358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba00d0f40 0x7f7ba00d1390 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.384+0000 7f7ba4a50640 1 --2- 172.21.15.16:0/1477283358 >> v2:172.21.15.173:3300/0 conn(0x7f7ba00d0100 0x7f7ba00b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.384+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1477283358 >> 172.21.15.16:0/1477283358 conn(0x7f7ba00d3090 msgr2=0x7f7ba00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.384+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1477283358 shutdown_connections 2023-11-24T06:29:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.384+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1477283358 wait complete. 2023-11-24T06:29:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.386+0000 7f7ba4a50640 1 Processor -- start 2023-11-24T06:29:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.386+0000 7f7ba4a50640 1 -- start start 2023-11-24T06:29:48.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.387+0000 7f7ba4a50640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba00d0100 0x7f7ba00b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.387+0000 7f7ba4a50640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ba00d0f40 0x7f7ba00cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.387+0000 7f7ba4a50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0160920 0x7f7ba0150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.387+0000 7f7ba4a50640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00cb340 con 0x7f7ba0160920 2023-11-24T06:29:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.387+0000 7f7ba4a50640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00b7000 con 0x7f7ba00d0100 2023-11-24T06:29:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.387+0000 7f7ba4a50640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00b72e0 con 0x7f7ba00d0f40 2023-11-24T06:29:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.387+0000 7f7b9f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0160920 0x7f7ba0150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.387+0000 7f7b9f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0160920 0x7f7ba0150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50010/0 (socket says 172.21.15.16:50010) 2023-11-24T06:29:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.387+0000 7f7b9f59e640 1 -- 172.21.15.16:0/1409163663 learned_addr learned my addr 172.21.15.16:0/1409163663 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b97fff640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ba00d0f40 0x7f7ba00cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b9ed9d640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba00d0100 0x7f7ba00b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b9f59e640 1 -- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ba00d0f40 msgr2=0x7f7ba00cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b9f59e640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ba00d0f40 0x7f7ba00cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b9f59e640 1 -- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba00d0100 msgr2=0x7f7ba00b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b9f59e640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba00d0100 0x7f7ba00b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b9f59e640 1 -- 172.21.15.16:0/1409163663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b98009a70 con 0x7f7ba0160920 2023-11-24T06:29:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b9ed9d640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba00d0100 0x7f7ba00b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b9f59e640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0160920 0x7f7ba0150490 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7b9802c4d0 tx=0x7f7b98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b977fe640 1 -- 172.21.15.16:0/1409163663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9803e610 con 0x7f7ba0160920 2023-11-24T06:29:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b977fe640 1 -- 172.21.15.16:0/1409163663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9803e770 con 0x7f7ba0160920 2023-11-24T06:29:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba00b75c0 con 0x7f7ba0160920 2023-11-24T06:29:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.388+0000 7f7b977fe640 1 -- 172.21.15.16:0/1409163663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9803eb20 con 0x7f7ba0160920 2023-11-24T06:29:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.389+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba00b7b80 con 0x7f7ba0160920 2023-11-24T06:29:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.390+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba0067450 con 0x7f7ba0160920 2023-11-24T06:29:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.391+0000 7f7b977fe640 1 -- 172.21.15.16:0/1409163663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7b9803b050 con 0x7f7ba0160920 2023-11-24T06:29:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.392+0000 7f7b977fe640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7b8006b080 0x7f7b8006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.392+0000 7f7b977fe640 1 -- 172.21.15.16:0/1409163663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7b980ac0a0 con 0x7f7ba0160920 2023-11-24T06:29:48.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.393+0000 7f7b9ed9d640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7b8006b080 0x7f7b8006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.394+0000 7f7b9ed9d640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7b8006b080 0x7f7b8006d540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7b90000f60 tx=0x7f7b90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.397+0000 7f7b977fe640 1 -- 172.21.15.16:0/1409163663 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b98077010 con 0x7f7ba0160920 2023-11-24T06:29:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.649+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7ba008c7e0 con 0x7f7ba0160920 2023-11-24T06:29:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.651+0000 7f7b977fe640 1 -- 172.21.15.16:0/1409163663 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7b9807a630 con 0x7f7ba0160920 2023-11-24T06:29:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:48.653 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:29:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-11-24T06:29:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.653 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBl2seAMxAAL8jbBlU0yStjuIivg2ehGQ==", 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBlbyW+GxAAy15inYIiNwnGVHj7Ghs5hg==", 2023-11-24T06:29:48.654 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl79LhCRAALZnwoYyKju4aeWNS+jF31A==", 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl/NBDLBAAR4qYgD5THE/yNeZT9XgK5g==", 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:48.656 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD4QWBlarihIxAAczpFVQAODZogIKH0TTdMhw==", 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.657 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-11-24T06:29:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD8QWBlITYFAhAAMlJyCTojY7WZihUegTcBcQ==", 2023-11-24T06:29:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-11-24T06:29:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD/QWBlA/N0DRAAThU0hfAL0+MggufKZC4K8g==", 2023-11-24T06:29:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQACQmBlM7E3JxAAdmfhQHbjK/JGHgeD0urDZw==", 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-11-24T06:29:48.660 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAKQmBlrM97ARAApYaqyASXJqB5ITNLeBec/w==", 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQANQmBl3s5dHBAA3spWOZQiqazGS6isl8ft3Q==", 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:48.661 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAQQmBlKVmeNhAAuV5j9qZ2NNvEePQx77EM6Q==", 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAUQmBlwADrBRAAvS3M6+H3X+OQD1h6xwhnBQ==", 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA==", 2023-11-24T06:29:48.663 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBlKhfoFxAA8A+MObA/cjawPN7aclqS9Q==", 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:29:48.664 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBliwu9CxAAPcBaB9CWK2QqRD1k2x+4ig==", 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:29:48.665 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBl8HWRIhAAqzUXni0Uqgbmi/imvnNIZg==", 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.666 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDzQWBlo6bBIRAADWG8Peciy2ws4wHagrfFZw==", 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-11-24T06:29:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlz+KlJhAAXtiyVVA72YrbDKpNSBUnwA==", 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlN4GDGhAAF9YIRqu011Xj6oWUv9LYlw==", 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:29:48.668 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:29:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:48.669 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-11-24T06:29:48.669 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:29:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7b8006b080 msgr2=0x7f7b8006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7b8006b080 0x7f7b8006d540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7b90000f60 tx=0x7f7b90009040 comp rx=0 tx=0).stop 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0160920 msgr2=0x7f7ba0150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0160920 0x7f7ba0150490 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7b9802c4d0 tx=0x7f7b98004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 shutdown_connections 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7b8006b080 0x7f7b8006d540 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0160920 0x7f7ba0150490 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ba00d0f40 0x7f7ba00cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 --2- 172.21.15.16:0/1409163663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ba00d0100 0x7f7ba00b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.653+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 >> 172.21.15.16:0/1409163663 conn(0x7f7ba00d3090 msgr2=0x7f7ba006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.654+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 shutdown_connections 2023-11-24T06:29:48.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.654+0000 7f7ba4a50640 1 -- 172.21.15.16:0/1409163663 wait complete. 2023-11-24T06:29:48.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:48.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.892+0000 7f084aa09640 1 Processor -- start 2023-11-24T06:29:48.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.892+0000 7f084aa09640 1 -- start start 2023-11-24T06:29:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.893+0000 7f084aa09640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f084415d540 0x7f08440d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.893+0000 7f084aa09640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08440b1ad0 con 0x7f08440b2000 2023-11-24T06:29:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.893+0000 7f084aa09640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08440b1d20 con 0x7f08441617e0 2023-11-24T06:29:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.893+0000 7f084aa09640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084406b2a0 con 0x7f084415d540 2023-11-24T06:29:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.894+0000 7f0849206640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f08440b2000 0x7f08441610d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51044/0 (socket says 172.21.15.16:51044) 2023-11-24T06:29:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.894+0000 7f0849a07640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f084415d540 0x7f08440d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.894+0000 7f0849206640 1 -- 172.21.15.16:0/150357082 learned_addr learned my addr 172.21.15.16:0/150357082 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.894+0000 7f0849a07640 1 -- 172.21.15.16:0/150357082 >> v1:172.21.15.173:6789/0 conn(0x7f08441617e0 legacy=0x7f08440b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.894+0000 7f0849a07640 1 -- 172.21.15.16:0/150357082 >> v1:172.21.15.16:6789/0 conn(0x7f08440b2000 legacy=0x7f08441610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.894+0000 7f0849a07640 1 -- 172.21.15.16:0/150357082 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0844161ee0 con 0x7f084415d540 2023-11-24T06:29:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.895+0000 7f0849a07640 1 --2- 172.21.15.16:0/150357082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 0x7f08440d01c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0834009d90 tx=0x7f083402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51093fcd2853b1fa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f0848a05640 1 -- 172.21.15.16:0/150357082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083402cc60 con 0x7f084415d540 2023-11-24T06:29:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f0848a05640 1 -- 172.21.15.16:0/150357082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083402cdc0 con 0x7f084415d540 2023-11-24T06:29:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f0848a05640 1 -- 172.21.15.16:0/150357082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0834034660 con 0x7f084415d540 2023-11-24T06:29:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f084aa09640 1 -- 172.21.15.16:0/150357082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 msgr2=0x7f08440d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f084aa09640 1 --2- 172.21.15.16:0/150357082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 0x7f08440d01c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0834009d90 tx=0x7f083402beb0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f084aa09640 1 -- 172.21.15.16:0/150357082 shutdown_connections 2023-11-24T06:29:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f084aa09640 1 --2- 172.21.15.16:0/150357082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 0x7f08440d01c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f084aa09640 1 -- 172.21.15.16:0/150357082 >> 172.21.15.16:0/150357082 conn(0x7f08440b2a30 msgr2=0x7f08440b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f084aa09640 1 -- 172.21.15.16:0/150357082 shutdown_connections 2023-11-24T06:29:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.896+0000 7f084aa09640 1 -- 172.21.15.16:0/150357082 wait complete. 2023-11-24T06:29:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.898+0000 7f084aa09640 1 Processor -- start 2023-11-24T06:29:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.898+0000 7f084aa09640 1 -- start start 2023-11-24T06:29:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.899+0000 7f084aa09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 0x7f0844160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.899+0000 7f084aa09640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08441617e0 0x7f08441f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.899+0000 7f084aa09640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f084415fb00 0x7f08440c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.899+0000 7f084aa09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08441f7230 con 0x7f084415d540 2023-11-24T06:29:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.899+0000 7f084aa09640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084414eb70 con 0x7f084415fb00 2023-11-24T06:29:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.899+0000 7f084aa09640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08441f65b0 con 0x7f08441617e0 2023-11-24T06:29:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.899+0000 7f0849a07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 0x7f0844160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.899+0000 7f0849a07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 0x7f0844160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50018/0 (socket says 172.21.15.16:50018) 2023-11-24T06:29:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.899+0000 7f0849a07640 1 -- 172.21.15.16:0/1884855823 learned_addr learned my addr 172.21.15.16:0/1884855823 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.900+0000 7f0849206640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08441617e0 0x7f08441f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.900+0000 7f084a208640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f084415fb00 0x7f08440c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.900+0000 7f0849a07640 1 -- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08441617e0 msgr2=0x7f08441f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.900+0000 7f0849a07640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08441617e0 0x7f08441f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.900+0000 7f0849a07640 1 -- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f084415fb00 msgr2=0x7f08440c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.900+0000 7f0849a07640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f084415fb00 0x7f08440c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.900+0000 7f0849a07640 1 -- 172.21.15.16:0/1884855823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0834009a70 con 0x7f084415d540 2023-11-24T06:29:48.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.900+0000 7f084a208640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f084415fb00 0x7f08440c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.900+0000 7f0849a07640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 0x7f0844160640 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0834037040 tx=0x7f0834004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.901+0000 7f08427fc640 1 -- 172.21.15.16:0/1884855823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0834040650 con 0x7f084415d540 2023-11-24T06:29:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.901+0000 7f08427fc640 1 -- 172.21.15.16:0/1884855823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08340407b0 con 0x7f084415d540 2023-11-24T06:29:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.901+0000 7f08427fc640 1 -- 172.21.15.16:0/1884855823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0834040b60 con 0x7f084415d540 2023-11-24T06:29:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.901+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08441f6890 con 0x7f084415d540 2023-11-24T06:29:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.901+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08441f6e50 con 0x7f084415d540 2023-11-24T06:29:48.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.902+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08441f7370 con 0x7f084415d540 2023-11-24T06:29:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.903+0000 7f08427fc640 1 -- 172.21.15.16:0/1884855823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0834040cc0 con 0x7f084415d540 2023-11-24T06:29:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.904+0000 7f08427fc640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f083006b080 0x7f083006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.904+0000 7f08427fc640 1 -- 172.21.15.16:0/1884855823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f08340ac630 con 0x7f084415d540 2023-11-24T06:29:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.904+0000 7f0849206640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f083006b080 0x7f083006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.905+0000 7f0849206640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f083006b080 0x7f083006d540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0844160b80 tx=0x7f0838009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:48.908+0000 7f08427fc640 1 -- 172.21.15.16:0/1884855823 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08441f7370 con 0x7f084415d540 2023-11-24T06:29:49.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.095+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08440cfc20 con 0x7f084415d540 2023-11-24T06:29:49.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.096+0000 7f08427fc640 1 -- 172.21.15.16:0/1884855823 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+12857 (secure 0 0 0) 0x7f0834077790 con 0x7f084415d540 2023-11-24T06:29:49.099 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:49.099 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":55,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:48.130059+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f083006b080 msgr2=0x7f083006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f083006b080 0x7f083006d540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0844160b80 tx=0x7f0838009040 comp rx=0 tx=0).stop 2023-11-24T06:29:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 msgr2=0x7f0844160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 0x7f0844160640 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0834037040 tx=0x7f0834004a40 comp rx=0 tx=0).stop 2023-11-24T06:29:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 shutdown_connections 2023-11-24T06:29:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f083006b080 0x7f083006d540 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f084415fb00 0x7f08440c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08441617e0 0x7f08441f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 --2- 172.21.15.16:0/1884855823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f084415d540 0x7f0844160640 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.099+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 >> 172.21.15.16:0/1884855823 conn(0x7f08440b2a30 msgr2=0x7f08440d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.100+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 shutdown_connections 2023-11-24T06:29:49.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.100+0000 7f084aa09640 1 -- 172.21.15.16:0/1884855823 wait complete. 2023-11-24T06:29:49.120 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:29:49.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:29:49.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.332+0000 7fa633fff640 1 Processor -- start 2023-11-24T06:29:49.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.332+0000 7fa633fff640 1 -- start start 2023-11-24T06:29:49.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.334+0000 7fa633fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa634143810 0x7fa634071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.335+0000 7fa633fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6340ae620 con 0x7fa63415d770 2023-11-24T06:29:49.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.335+0000 7fa633fff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63406c730 con 0x7fa6340ae170 2023-11-24T06:29:49.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.335+0000 7fa633fff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa634143bf0 con 0x7fa634143810 2023-11-24T06:29:49.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.335+0000 7fa6337fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa63415d770 0x7fa63415dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51046/0 (socket says 172.21.15.16:51046) 2023-11-24T06:29:49.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.335+0000 7fa6337fe640 1 -- 172.21.15.16:0/1548234627 learned_addr learned my addr 172.21.15.16:0/1548234627 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.335+0000 7fa632ffd640 1 --2- 172.21.15.16:0/1548234627 >> v2:172.21.15.173:3300/0 conn(0x7fa634143810 0x7fa634071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.335+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279996503 0 0) 0x7fa6340ae620 con 0x7fa63415d770 2023-11-24T06:29:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.335+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa620003520 con 0x7fa63415d770 2023-11-24T06:29:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3523009204 0 0) 0x7fa620003520 con 0x7fa63415d770 2023-11-24T06:29:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 >> v2:172.21.15.173:3300/0 conn(0x7fa634143810 msgr2=0x7fa634071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 --2- 172.21.15.16:0/1548234627 >> v2:172.21.15.173:3300/0 conn(0x7fa634143810 0x7fa634071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 >> v1:172.21.15.173:6789/0 conn(0x7fa6340ae170 legacy=0x7fa63415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6340724d0 con 0x7fa63415d770 2023-11-24T06:29:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fa62c0019f0 con 0x7fa63415d770 2023-11-24T06:29:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 >> v1:172.21.15.16:6789/0 conn(0x7fa63415d770 legacy=0x7fa63415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6200039c0 0x7fa620003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6200047c0 0x7fa620004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200052a0 0x7fa62000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa634143bf0 con 0x7fa6200039c0 2023-11-24T06:29:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62000c1a0 con 0x7fa6200052a0 2023-11-24T06:29:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62000c360 con 0x7fa6200047c0 2023-11-24T06:29:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.336+0000 7fa632ffd640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6200039c0 0x7fa620003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa62bfff640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6200047c0 0x7fa620004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa6337fe640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200052a0 0x7fa62000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa632ffd640 1 -- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6200047c0 msgr2=0x7fa620004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa632ffd640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6200047c0 0x7fa620004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa632ffd640 1 -- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200052a0 msgr2=0x7fa62000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa632ffd640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200052a0 0x7fa62000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa632ffd640 1 -- 172.21.15.16:0/1548234627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa620003520 con 0x7fa6200039c0 2023-11-24T06:29:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa62bfff640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6200047c0 0x7fa620004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa6337fe640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200052a0 0x7fa62000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.337+0000 7fa6337fe640 1 -- 172.21.15.16:0/1548234627 reap_dead start 2023-11-24T06:29:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.338+0000 7fa632ffd640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6200039c0 0x7fa620003de0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa624009ac0 tx=0x7fa6200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de00e592a1d0734 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.338+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62402c6b0 con 0x7fa6200039c0 2023-11-24T06:29:49.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.338+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62402c810 con 0x7fa6200039c0 2023-11-24T06:29:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.338+0000 7fa6327fc640 1 -- 172.21.15.16:0/1548234627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62402cba0 con 0x7fa6200039c0 2023-11-24T06:29:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.339+0000 7fa633fff640 1 -- 172.21.15.16:0/1548234627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6200039c0 msgr2=0x7fa620003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.339+0000 7fa633fff640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6200039c0 0x7fa620003de0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa624009ac0 tx=0x7fa6200057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.339+0000 7fa633fff640 1 -- 172.21.15.16:0/1548234627 shutdown_connections 2023-11-24T06:29:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.339+0000 7fa633fff640 1 --2- 172.21.15.16:0/1548234627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6200039c0 0x7fa620003de0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.339+0000 7fa633fff640 1 -- 172.21.15.16:0/1548234627 >> 172.21.15.16:0/1548234627 conn(0x7fa63406a7d0 msgr2=0x7fa63406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.339+0000 7fa633fff640 1 -- 172.21.15.16:0/1548234627 shutdown_connections 2023-11-24T06:29:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.339+0000 7fa633fff640 1 -- 172.21.15.16:0/1548234627 wait complete. 2023-11-24T06:29:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.341+0000 7fa633fff640 1 Processor -- start 2023-11-24T06:29:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.341+0000 7fa633fff640 1 -- start start 2023-11-24T06:29:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.342+0000 7fa633fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200039c0 0x7fa6340aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.342+0000 7fa633fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa63414c7f0 0x7fa6340afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.342+0000 7fa633fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63414cc50 0x7fa6340b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.342+0000 7fa633fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340af8c0 con 0x7fa63414cc50 2023-11-24T06:29:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.342+0000 7fa633fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa634146f10 con 0x7fa6200039c0 2023-11-24T06:29:49.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.342+0000 7fa633fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa634144450 con 0x7fa63414c7f0 2023-11-24T06:29:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.342+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63414cc50 0x7fa6340b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.342+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63414cc50 0x7fa6340b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50040/0 (socket says 172.21.15.16:50040) 2023-11-24T06:29:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa6337fe640 1 -- 172.21.15.16:0/4082422523 learned_addr learned my addr 172.21.15.16:0/4082422523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa632ffd640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200039c0 0x7fa6340aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa62bfff640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa63414c7f0 0x7fa6340afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa6337fe640 1 -- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa63414c7f0 msgr2=0x7fa6340afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa6337fe640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa63414c7f0 0x7fa6340afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa6337fe640 1 -- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200039c0 msgr2=0x7fa6340aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa6337fe640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200039c0 0x7fa6340aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa6337fe640 1 -- 172.21.15.16:0/4082422523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa634143ed0 con 0x7fa63414cc50 2023-11-24T06:29:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa62bfff640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa63414c7f0 0x7fa6340afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.343+0000 7fa6337fe640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63414cc50 0x7fa6340b0220 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa6340ad100 tx=0x7fa62c00b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.344+0000 7fa630ff9640 1 -- 172.21.15.16:0/4082422523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00b8d0 con 0x7fa63414cc50 2023-11-24T06:29:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.344+0000 7fa630ff9640 1 -- 172.21.15.16:0/4082422523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c00ba30 con 0x7fa63414cc50 2023-11-24T06:29:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.344+0000 7fa630ff9640 1 -- 172.21.15.16:0/4082422523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00bda0 con 0x7fa63414cc50 2023-11-24T06:29:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.344+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa634144170 con 0x7fa63414cc50 2023-11-24T06:29:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.344+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6340af3e0 con 0x7fa63414cc50 2023-11-24T06:29:49.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.345+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6340afa00 con 0x7fa63414cc50 2023-11-24T06:29:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.346+0000 7fa630ff9640 1 -- 172.21.15.16:0/4082422523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa62c007070 con 0x7fa63414cc50 2023-11-24T06:29:49.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.348+0000 7fa630ff9640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa61406b0b0 0x7fa61406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.348+0000 7fa630ff9640 1 -- 172.21.15.16:0/4082422523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa62c089b40 con 0x7fa63414cc50 2023-11-24T06:29:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.348+0000 7fa632ffd640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa61406b0b0 0x7fa61406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.349+0000 7fa632ffd640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa61406b0b0 0x7fa61406d570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa624036000 tx=0x7fa624030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.352+0000 7fa630ff9640 1 -- 172.21.15.16:0/4082422523 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6340afa00 con 0x7fa63414cc50 2023-11-24T06:29:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.545+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6340afa00 con 0x7fa63414cc50 2023-11-24T06:29:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.546+0000 7fa630ff9640 1 -- 172.21.15.16:0/4082422523 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6340afa00 con 0x7fa63414cc50 2023-11-24T06:29:49.548 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:29:49.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa61406b0b0 msgr2=0x7fa61406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa61406b0b0 0x7fa61406d570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa624036000 tx=0x7fa624030000 comp rx=0 tx=0).stop 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63414cc50 msgr2=0x7fa6340b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63414cc50 0x7fa6340b0220 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa6340ad100 tx=0x7fa62c00b0d0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 shutdown_connections 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa61406b0b0 0x7fa61406d570 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa63414cc50 0x7fa6340b0220 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa63414c7f0 0x7fa6340afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 --2- 172.21.15.16:0/4082422523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6200039c0 0x7fa6340aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 >> 172.21.15.16:0/4082422523 conn(0x7fa63406a7d0 msgr2=0x7fa634071470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 shutdown_connections 2023-11-24T06:29:49.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.549+0000 7fa633fff640 1 -- 172.21.15.16:0/4082422523 wait complete. 2023-11-24T06:29:49.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:29:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.797+0000 7f3e618ad640 1 Processor -- start 2023-11-24T06:29:49.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.797+0000 7f3e618ad640 1 -- start start 2023-11-24T06:29:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.798+0000 7f3e618ad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c160920 0x7f3e5c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.798+0000 7f3e618ad640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3e5c0cf9f0 0x7f3e5c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.798+0000 7f3e618ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e5c161010 0x7f3e5c1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.798+0000 7f3e618ad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5c0d03b0 con 0x7f3e5c161010 2023-11-24T06:29:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.798+0000 7f3e618ad640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5c0d0a10 con 0x7f3e5c0cf9f0 2023-11-24T06:29:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.798+0000 7f3e618ad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0cf080 con 0x7f3e5c160920 2023-11-24T06:29:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.798+0000 7f3e610ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e5c161010 0x7f3e5c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.798+0000 7f3e610ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3e5c161010 0x7f3e5c1614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50054/0 (socket says 172.21.15.16:50054) 2023-11-24T06:29:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.798+0000 7f3e610ac640 1 -- 172.21.15.16:0/1956421543 learned_addr learned my addr 172.21.15.16:0/1956421543 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.799+0000 7f3e608ab640 1 --2- 172.21.15.16:0/1956421543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c160920 0x7f3e5c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.799+0000 7f3e610ac640 1 -- 172.21.15.16:0/1956421543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c160920 msgr2=0x7f3e5c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.799+0000 7f3e610ac640 1 --2- 172.21.15.16:0/1956421543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c160920 0x7f3e5c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.799+0000 7f3e610ac640 1 -- 172.21.15.16:0/1956421543 >> v2:172.21.15.173:3300/0 conn(0x7f3e5c0cf9f0 msgr2=0x7f3e5c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.799+0000 7f3e5bfff640 1 --2- 172.21.15.16:0/1956421543 >> v2:172.21.15.173:3300/0 conn(0x7f3e5c0cf9f0 0x7f3e5c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.799+0000 7f3e610ac640 1 --2- 172.21.15.16:0/1956421543 >> v2:172.21.15.173:3300/0 conn(0x7f3e5c0cf9f0 0x7f3e5c0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.799+0000 7f3e610ac640 1 -- 172.21.15.16:0/1956421543 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0c6be0 con 0x7f3e5c161010 2023-11-24T06:29:49.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.799+0000 7f3e5bfff640 1 --2- 172.21.15.16:0/1956421543 >> v2:172.21.15.173:3300/0 conn(0x7f3e5c0cf9f0 0x7f3e5c0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.800+0000 7f3e610ac640 1 --2- 172.21.15.16:0/1956421543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c161010 0x7f3e5c1614a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c08d300 tx=0x7f3e5402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec4632823863aac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.800+0000 7f3e5b7fe640 1 -- 172.21.15.16:0/1956421543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54031860 con 0x7f3e5c161010 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.800+0000 7f3e5b7fe640 1 -- 172.21.15.16:0/1956421543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540319c0 con 0x7f3e5c161010 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.800+0000 7f3e5b7fe640 1 -- 172.21.15.16:0/1956421543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54031d50 con 0x7f3e5c161010 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.801+0000 7f3e618ad640 1 -- 172.21.15.16:0/1956421543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c161010 msgr2=0x7f3e5c1614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.801+0000 7f3e618ad640 1 --2- 172.21.15.16:0/1956421543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c161010 0x7f3e5c1614a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c08d300 tx=0x7f3e5402b900 comp rx=0 tx=0).stop 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.801+0000 7f3e618ad640 1 -- 172.21.15.16:0/1956421543 shutdown_connections 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.801+0000 7f3e618ad640 1 --2- 172.21.15.16:0/1956421543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c161010 0x7f3e5c1614a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.801+0000 7f3e618ad640 1 --2- 172.21.15.16:0/1956421543 >> v2:172.21.15.173:3300/0 conn(0x7f3e5c0cf9f0 0x7f3e5c0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.801+0000 7f3e618ad640 1 --2- 172.21.15.16:0/1956421543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c160920 0x7f3e5c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.801+0000 7f3e618ad640 1 -- 172.21.15.16:0/1956421543 >> 172.21.15.16:0/1956421543 conn(0x7f3e5c0d3090 msgr2=0x7f3e5c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.801+0000 7f3e618ad640 1 -- 172.21.15.16:0/1956421543 shutdown_connections 2023-11-24T06:29:49.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.801+0000 7f3e618ad640 1 -- 172.21.15.16:0/1956421543 wait complete. 2023-11-24T06:29:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.803+0000 7f3e618ad640 1 Processor -- start 2023-11-24T06:29:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.803+0000 7f3e618ad640 1 -- start start 2023-11-24T06:29:49.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.804+0000 7f3e618ad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c0cf9f0 0x7f3e5c1f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.804+0000 7f3e618ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c160920 0x7f3e5c0b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.804+0000 7f3e618ad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c161010 0x7f3e5c0b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.804+0000 7f3e618ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1f3720 con 0x7f3e5c160920 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.804+0000 7f3e618ad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0cb3c0 con 0x7f3e5c0cf9f0 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.804+0000 7f3e618ad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0cb6a0 con 0x7f3e5c161010 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.804+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c160920 0x7f3e5c0b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c160920 0x7f3e5c0b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50066/0 (socket says 172.21.15.16:50066) 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e5bfff640 1 -- 172.21.15.16:0/2225137756 learned_addr learned my addr 172.21.15.16:0/2225137756 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e610ac640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c161010 0x7f3e5c0b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e608ab640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c0cf9f0 0x7f3e5c1f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e5bfff640 1 -- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c161010 msgr2=0x7f3e5c0b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e5bfff640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c161010 0x7f3e5c0b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e5bfff640 1 -- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c0cf9f0 msgr2=0x7f3e5c1f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e5bfff640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c0cf9f0 0x7f3e5c1f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e5bfff640 1 -- 172.21.15.16:0/2225137756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540094d0 con 0x7f3e5c160920 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e608ab640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c0cf9f0 0x7f3e5c1f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.805+0000 7f3e5bfff640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c160920 0x7f3e5c0b8500 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400c7c0 tx=0x7f3e4400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.806+0000 7f3e597fa640 1 -- 172.21.15.16:0/2225137756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e44007a30 con 0x7f3e5c160920 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.806+0000 7f3e597fa640 1 -- 172.21.15.16:0/2225137756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e44007b90 con 0x7f3e5c160920 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.806+0000 7f3e597fa640 1 -- 172.21.15.16:0/2225137756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400ce70 con 0x7f3e5c160920 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.806+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c0cb980 con 0x7f3e5c160920 2023-11-24T06:29:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.806+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c0b8a40 con 0x7f3e5c160920 2023-11-24T06:29:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.807+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c0b5a60 con 0x7f3e5c160920 2023-11-24T06:29:49.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.808+0000 7f3e597fa640 1 -- 172.21.15.16:0/2225137756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3e440113f0 con 0x7f3e5c160920 2023-11-24T06:29:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.809+0000 7f3e597fa640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e3c06b080 0x7f3e3c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.809+0000 7f3e597fa640 1 -- 172.21.15.16:0/2225137756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3e44089a20 con 0x7f3e5c160920 2023-11-24T06:29:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.810+0000 7f3e608ab640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e3c06b080 0x7f3e3c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:49.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.811+0000 7f3e608ab640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e3c06b080 0x7f3e3c06d540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00e4a0 tx=0x7f3e4c007650 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:49.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:49.814+0000 7f3e597fa640 1 -- 172.21.15.16:0/2225137756 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e44054880 con 0x7f3e5c160920 2023-11-24T06:29:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.010+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3e5c067450 con 0x7f3e5c160920 2023-11-24T06:29:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.011+0000 7f3e597fa640 1 -- 172.21.15.16:0/2225137756 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3e4401f020 con 0x7f3e5c160920 2023-11-24T06:29:50.014 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:50.014 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:29:50.015 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:29:50.015 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-24T06:29:50.015 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-24T06:29:50.015 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:29:50.016 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:29:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.013+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e3c06b080 msgr2=0x7f3e3c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.013+0000 7f3e618ad640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e3c06b080 0x7f3e3c06d540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00e4a0 tx=0x7f3e4c007650 comp rx=0 tx=0).stop 2023-11-24T06:29:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.013+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c160920 msgr2=0x7f3e5c0b8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.013+0000 7f3e618ad640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c160920 0x7f3e5c0b8500 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400c7c0 tx=0x7f3e4400cc80 comp rx=0 tx=0).stop 2023-11-24T06:29:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.014+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 shutdown_connections 2023-11-24T06:29:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.014+0000 7f3e618ad640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3e3c06b080 0x7f3e3c06d540 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.014+0000 7f3e618ad640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e5c161010 0x7f3e5c0b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.014+0000 7f3e618ad640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e5c160920 0x7f3e5c0b8500 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.014+0000 7f3e618ad640 1 --2- 172.21.15.16:0/2225137756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e5c0cf9f0 0x7f3e5c1f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.014+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 >> 172.21.15.16:0/2225137756 conn(0x7f3e5c0d3090 msgr2=0x7f3e5c0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:50.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.014+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 shutdown_connections 2023-11-24T06:29:50.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.014+0000 7f3e618ad640 1 -- 172.21.15.16:0/2225137756 wait complete. 2023-11-24T06:29:50.036 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:29:50.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:29:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.278+0000 7fb20c083640 1 Processor -- start 2023-11-24T06:29:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.278+0000 7fb20c083640 1 -- start start 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.280+0000 7fb20c083640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb204143ae0 0x7fb2040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.280+0000 7fb20c083640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb2040cf7d0 0x7fb20415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.280+0000 7fb20c083640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb20415e1e0 0x7fb20415a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.280+0000 7fb20c083640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2040cfc30 con 0x7fb20415e1e0 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.280+0000 7fb20c083640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2040b1ee0 con 0x7fb2040cf7d0 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.280+0000 7fb20c083640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040b2780 con 0x7fb204143ae0 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.280+0000 7fb20a620640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb20415e1e0 0x7fb20415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.281+0000 7fb20a620640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb20415e1e0 0x7fb20415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50086/0 (socket says 172.21.15.16:50086) 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.281+0000 7fb20a620640 1 -- 172.21.15.16:0/144542043 learned_addr learned my addr 172.21.15.16:0/144542043 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.281+0000 7fb20961e640 1 --2- 172.21.15.16:0/144542043 >> v2:172.21.15.173:3300/0 conn(0x7fb2040cf7d0 0x7fb20415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.281+0000 7fb209e1f640 1 --2- 172.21.15.16:0/144542043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb204143ae0 0x7fb2040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.281+0000 7fb20a620640 1 -- 172.21.15.16:0/144542043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb204143ae0 msgr2=0x7fb2040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.281+0000 7fb20a620640 1 --2- 172.21.15.16:0/144542043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb204143ae0 0x7fb2040cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.281+0000 7fb20a620640 1 -- 172.21.15.16:0/144542043 >> v2:172.21.15.173:3300/0 conn(0x7fb2040cf7d0 msgr2=0x7fb20415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.281+0000 7fb20a620640 1 --2- 172.21.15.16:0/144542043 >> v2:172.21.15.173:3300/0 conn(0x7fb2040cf7d0 0x7fb20415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.281+0000 7fb20a620640 1 -- 172.21.15.16:0/144542043 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20415e990 con 0x7fb20415e1e0 2023-11-24T06:29:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.282+0000 7fb20a620640 1 --2- 172.21.15.16:0/144542043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20415e1e0 0x7fb20415a670 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb20406e350 tx=0x7fb1fc02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea43438ea307351f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb208e1d640 1 -- 172.21.15.16:0/144542043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc02ea80 con 0x7fb20415e1e0 2023-11-24T06:29:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb208e1d640 1 -- 172.21.15.16:0/144542043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc02ebe0 con 0x7fb20415e1e0 2023-11-24T06:29:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb208e1d640 1 -- 172.21.15.16:0/144542043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc009600 con 0x7fb20415e1e0 2023-11-24T06:29:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb20c083640 1 -- 172.21.15.16:0/144542043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20415e1e0 msgr2=0x7fb20415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb20c083640 1 --2- 172.21.15.16:0/144542043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20415e1e0 0x7fb20415a670 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb20406e350 tx=0x7fb1fc02e0e0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb20c083640 1 -- 172.21.15.16:0/144542043 shutdown_connections 2023-11-24T06:29:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb20c083640 1 --2- 172.21.15.16:0/144542043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb20415e1e0 0x7fb20415a670 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb20c083640 1 --2- 172.21.15.16:0/144542043 >> v2:172.21.15.173:3300/0 conn(0x7fb2040cf7d0 0x7fb20415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb20c083640 1 --2- 172.21.15.16:0/144542043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb204143ae0 0x7fb2040cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.283+0000 7fb20c083640 1 -- 172.21.15.16:0/144542043 >> 172.21.15.16:0/144542043 conn(0x7fb20406a800 msgr2=0x7fb20406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.284+0000 7fb20c083640 1 -- 172.21.15.16:0/144542043 shutdown_connections 2023-11-24T06:29:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.284+0000 7fb20c083640 1 -- 172.21.15.16:0/144542043 wait complete. 2023-11-24T06:29:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.285+0000 7fb20c083640 1 Processor -- start 2023-11-24T06:29:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.285+0000 7fb20c083640 1 -- start start 2023-11-24T06:29:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.286+0000 7fb20c083640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2040cf7d0 0x7fb204159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.286+0000 7fb20c083640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb204143ae0 0x7fb204142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.286+0000 7fb20c083640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb20415e1e0 0x7fb204142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.286+0000 7fb20c083640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040c4200 con 0x7fb204143ae0 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.286+0000 7fb20c083640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204159eb0 con 0x7fb2040cf7d0 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.286+0000 7fb20c083640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040ad4e0 con 0x7fb20415e1e0 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20961e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb204143ae0 0x7fb204142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20961e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb204143ae0 0x7fb204142960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50092/0 (socket says 172.21.15.16:50092) 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20961e640 1 -- 172.21.15.16:0/2852315149 learned_addr learned my addr 172.21.15.16:0/2852315149 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb209e1f640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2040cf7d0 0x7fb204159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20a620640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb20415e1e0 0x7fb204142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20961e640 1 -- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb20415e1e0 msgr2=0x7fb204142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20961e640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb20415e1e0 0x7fb204142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20961e640 1 -- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2040cf7d0 msgr2=0x7fb204159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20961e640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2040cf7d0 0x7fb204159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20961e640 1 -- 172.21.15.16:0/2852315149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc00bd40 con 0x7fb204143ae0 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb209e1f640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2040cf7d0 0x7fb204159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.287+0000 7fb20a620640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb20415e1e0 0x7fb204142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.288+0000 7fb20961e640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb204143ae0 0x7fb204142960 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400d800 tx=0x7fb1f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.288+0000 7fb1fa7fc640 1 -- 172.21.15.16:0/2852315149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40096a0 con 0x7fb204143ae0 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.288+0000 7fb1fa7fc640 1 -- 172.21.15.16:0/2852315149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f4009800 con 0x7fb204143ae0 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.288+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2040ad7c0 con 0x7fb204143ae0 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.288+0000 7fb1fa7fc640 1 -- 172.21.15.16:0/2852315149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4009b90 con 0x7fb204143ae0 2023-11-24T06:29:50.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.288+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2040add80 con 0x7fb204143ae0 2023-11-24T06:29:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.289+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2040675d0 con 0x7fb204143ae0 2023-11-24T06:29:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.291+0000 7fb1fa7fc640 1 -- 172.21.15.16:0/2852315149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fb1f4009cf0 con 0x7fb204143ae0 2023-11-24T06:29:50.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.292+0000 7fb1fa7fc640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb1e806b080 0x7fb1e806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.292+0000 7fb1fa7fc640 1 -- 172.21.15.16:0/2852315149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb1f4089c10 con 0x7fb204143ae0 2023-11-24T06:29:50.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.292+0000 7fb209e1f640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb1e806b080 0x7fb1e806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:50.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.294+0000 7fb209e1f640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb1e806b080 0x7fb1e806d540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb200006fd0 tx=0x7fb200008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.296+0000 7fb1fa7fc640 1 -- 172.21.15.16:0/2852315149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1f4054c10 con 0x7fb204143ae0 2023-11-24T06:29:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:50.493+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb20406b3f0 con 0x7fb204143ae0 2023-11-24T06:29:51.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.170+0000 7fb1fa7fc640 1 -- 172.21.15.16:0/2852315149 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb1f400b440 con 0x7fb204143ae0 2023-11-24T06:29:51.173 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-24T06:29:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.174+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb1e806b080 msgr2=0x7fb1e806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.174+0000 7fb20c083640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb1e806b080 0x7fb1e806d540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb200006fd0 tx=0x7fb200008040 comp rx=0 tx=0).stop 2023-11-24T06:29:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.175+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb204143ae0 msgr2=0x7fb204142960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.175+0000 7fb20c083640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb204143ae0 0x7fb204142960 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400d800 tx=0x7fb1f400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.175+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 shutdown_connections 2023-11-24T06:29:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.175+0000 7fb20c083640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb1e806b080 0x7fb1e806d540 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.175+0000 7fb20c083640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb20415e1e0 0x7fb204142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.175+0000 7fb20c083640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb204143ae0 0x7fb204142960 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.175+0000 7fb20c083640 1 --2- 172.21.15.16:0/2852315149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2040cf7d0 0x7fb204159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.175+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 >> 172.21.15.16:0/2852315149 conn(0x7fb20406a800 msgr2=0x7fb20406ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:51.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.176+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 shutdown_connections 2023-11-24T06:29:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.177+0000 7fb20c083640 1 -- 172.21.15.16:0/2852315149 wait complete. 2023-11-24T06:29:51.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:29:51.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.430+0000 7f71edf68640 1 Processor -- start 2023-11-24T06:29:51.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.430+0000 7f71edf68640 1 -- start start 2023-11-24T06:29:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.431+0000 7f71edf68640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e815ed20 0x7f71e8141360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.431+0000 7f71edf68640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71e806c850 0x7f71e806cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.431+0000 7f71edf68640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e813e440 con 0x7f71e815f100 2023-11-24T06:29:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.431+0000 7f71edf68640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e80ce1d0 con 0x7f71e806c850 2023-11-24T06:29:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.431+0000 7f71edf68640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80ce4b0 con 0x7f71e815ed20 2023-11-24T06:29:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.432+0000 7f71e7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71e806c850 0x7f71e806cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.432+0000 7f71e7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71e806c850 0x7f71e806cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50094/0 (socket says 172.21.15.16:50094) 2023-11-24T06:29:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.432+0000 7f71e7fff640 1 -- 172.21.15.16:0/1625513729 learned_addr learned my addr 172.21.15.16:0/1625513729 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.432+0000 7f71e77fe640 1 --2- 172.21.15.16:0/1625513729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e815ed20 0x7f71e8141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.432+0000 7f71e7fff640 1 -- 172.21.15.16:0/1625513729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e815ed20 msgr2=0x7f71e8141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.432+0000 7f71e7fff640 1 --2- 172.21.15.16:0/1625513729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e815ed20 0x7f71e8141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.432+0000 7f71e7fff640 1 -- 172.21.15.16:0/1625513729 >> v1:172.21.15.173:6789/0 conn(0x7f71e815f100 legacy=0x7f71e80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.432+0000 7f71e7fff640 1 -- 172.21.15.16:0/1625513729 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80ce790 con 0x7f71e806c850 2023-11-24T06:29:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.433+0000 7f71e7fff640 1 --2- 172.21.15.16:0/1625513729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e806c850 0x7f71e806cca0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009e50 tx=0x7f71e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedfb200c4c35bfb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.434+0000 7f71e6ffd640 1 -- 172.21.15.16:0/1625513729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e002c9e0 con 0x7f71e806c850 2023-11-24T06:29:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.434+0000 7f71e6ffd640 1 -- 172.21.15.16:0/1625513729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e002cb40 con 0x7f71e806c850 2023-11-24T06:29:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.434+0000 7f71e6ffd640 1 -- 172.21.15.16:0/1625513729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00343f0 con 0x7f71e806c850 2023-11-24T06:29:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.435+0000 7f71edf68640 1 -- 172.21.15.16:0/1625513729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e806c850 msgr2=0x7f71e806cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.435+0000 7f71edf68640 1 --2- 172.21.15.16:0/1625513729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e806c850 0x7f71e806cca0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009e50 tx=0x7f71e002be40 comp rx=0 tx=0).stop 2023-11-24T06:29:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.435+0000 7f71edf68640 1 -- 172.21.15.16:0/1625513729 shutdown_connections 2023-11-24T06:29:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.435+0000 7f71edf68640 1 --2- 172.21.15.16:0/1625513729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e806c850 0x7f71e806cca0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.435+0000 7f71edf68640 1 --2- 172.21.15.16:0/1625513729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e815ed20 0x7f71e8141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.435+0000 7f71edf68640 1 -- 172.21.15.16:0/1625513729 >> 172.21.15.16:0/1625513729 conn(0x7f71e80b0be0 msgr2=0x7f71e80b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:51.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.435+0000 7f71edf68640 1 -- 172.21.15.16:0/1625513729 shutdown_connections 2023-11-24T06:29:51.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.435+0000 7f71edf68640 1 -- 172.21.15.16:0/1625513729 wait complete. 2023-11-24T06:29:51.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.437+0000 7f71edf68640 1 Processor -- start 2023-11-24T06:29:51.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.437+0000 7f71edf68640 1 -- start start 2023-11-24T06:29:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.438+0000 7f71edf68640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e806c850 0x7f71e80b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.438+0000 7f71edf68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e815ed20 0x7f71e80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.438+0000 7f71edf68640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71e815f100 0x7f71e80b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.438+0000 7f71edf68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8155730 con 0x7f71e815ed20 2023-11-24T06:29:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.438+0000 7f71edf68640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8154b20 con 0x7f71e815f100 2023-11-24T06:29:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.438+0000 7f71edf68640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8154e00 con 0x7f71e806c850 2023-11-24T06:29:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.438+0000 7f71dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e815ed20 0x7f71e80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.438+0000 7f71dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e815ed20 0x7f71e80b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50112/0 (socket says 172.21.15.16:50112) 2023-11-24T06:29:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.438+0000 7f71dffff640 1 -- 172.21.15.16:0/3006182890 learned_addr learned my addr 172.21.15.16:0/3006182890 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71e7fff640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71e815f100 0x7f71e80b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71e77fe640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e806c850 0x7f71e80b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71dffff640 1 -- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e806c850 msgr2=0x7f71e80b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71dffff640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e806c850 0x7f71e80b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71dffff640 1 -- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71e815f100 msgr2=0x7f71e80b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71dffff640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71e815f100 0x7f71e80b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71dffff640 1 -- 172.21.15.16:0/3006182890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e0009b30 con 0x7f71e815ed20 2023-11-24T06:29:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71dffff640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e815ed20 0x7f71e80b32f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f71d000b3f0 tx=0x7f71d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71e4ff9640 1 -- 172.21.15.16:0/3006182890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0011830 con 0x7f71e815ed20 2023-11-24T06:29:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71e4ff9640 1 -- 172.21.15.16:0/3006182890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d0011990 con 0x7f71e815ed20 2023-11-24T06:29:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71e4ff9640 1 -- 172.21.15.16:0/3006182890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0011d40 con 0x7f71e815ed20 2023-11-24T06:29:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.439+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e815f6c0 con 0x7f71e815ed20 2023-11-24T06:29:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.440+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e81553c0 con 0x7f71e815ed20 2023-11-24T06:29:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.441+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e813b330 con 0x7f71e815ed20 2023-11-24T06:29:51.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.442+0000 7f71e4ff9640 1 -- 172.21.15.16:0/3006182890 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f71d0004020 con 0x7f71e815ed20 2023-11-24T06:29:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.443+0000 7f71e4ff9640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f71c806b0d0 0x7f71c806d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.443+0000 7f71e4ff9640 1 -- 172.21.15.16:0/3006182890 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f71d0089b10 con 0x7f71e815ed20 2023-11-24T06:29:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.444+0000 7f71e77fe640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f71c806b0d0 0x7f71c806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.445+0000 7f71e77fe640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f71c806b0d0 0x7f71c806d590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f71d8006fd0 tx=0x7f71d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.447+0000 7f71e4ff9640 1 -- 172.21.15.16:0/3006182890 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71d0054ab0 con 0x7f71e815ed20 2023-11-24T06:29:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:51.643+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f71e8067670 con 0x7f71e815ed20 2023-11-24T06:29:52.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.176+0000 7f71e4ff9640 1 -- 172.21.15.16:0/3006182890 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) v1 ==== 170+0+0 (secure 0 0 0) 0x7f71d00150d0 con 0x7f71e815ed20 2023-11-24T06:29:52.179 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-24T06:29:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.181+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f71c806b0d0 msgr2=0x7f71c806d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.181+0000 7f71edf68640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f71c806b0d0 0x7f71c806d590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f71d8006fd0 tx=0x7f71d8008040 comp rx=0 tx=0).stop 2023-11-24T06:29:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.181+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e815ed20 msgr2=0x7f71e80b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.181+0000 7f71edf68640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e815ed20 0x7f71e80b32f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f71d000b3f0 tx=0x7f71d000b8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.182+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 shutdown_connections 2023-11-24T06:29:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.182+0000 7f71edf68640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f71c806b0d0 0x7f71c806d590 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.182+0000 7f71edf68640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71e815f100 0x7f71e80b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.182+0000 7f71edf68640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e815ed20 0x7f71e80b32f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.182+0000 7f71edf68640 1 --2- 172.21.15.16:0/3006182890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71e806c850 0x7f71e80b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.182+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 >> 172.21.15.16:0/3006182890 conn(0x7f71e80b0be0 msgr2=0x7f71e815cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.183+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 shutdown_connections 2023-11-24T06:29:52.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.183+0000 7f71edf68640 1 -- 172.21.15.16:0/3006182890 wait complete. 2023-11-24T06:29:52.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:29:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.413+0000 7f02c3f3c640 1 Processor -- start 2023-11-24T06:29:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.413+0000 7f02c3f3c640 1 -- start start 2023-11-24T06:29:52.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.415+0000 7f02c3f3c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13df80 0x7f02bc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.415+0000 7f02c3f3c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f02bc0cf9a0 0x7f02bc0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.415+0000 7f02c3f3c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02bc06c470 con 0x7f02bc13e360 2023-11-24T06:29:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.415+0000 7f02c3f3c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02bc06c750 con 0x7f02bc0cf9a0 2023-11-24T06:29:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.415+0000 7f02c3f3c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc06ca30 con 0x7f02bc13df80 2023-11-24T06:29:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.416+0000 7f02c1cd8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13df80 0x7f02bc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.416+0000 7f02c24d9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f02bc0cf9a0 0x7f02bc0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.416+0000 7f02c14d7640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f02bc13e360 0x7f02bc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:42302/0 (socket says 172.21.15.16:42302) 2023-11-24T06:29:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.416+0000 7f02c1cd8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13df80 0x7f02bc15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:34738/0 (socket says 172.21.15.16:34738) 2023-11-24T06:29:52.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.416+0000 7f02c14d7640 1 -- 172.21.15.16:0/1458090872 learned_addr learned my addr 172.21.15.16:0/1458090872 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.417+0000 7f02c0cd6640 1 -- 172.21.15.16:0/1458090872 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226599185 0 0) 0x7f02bc06c470 con 0x7f02bc13e360 2023-11-24T06:29:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.417+0000 7f02c0cd6640 1 -- 172.21.15.16:0/1458090872 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a8003520 con 0x7f02bc13e360 2023-11-24T06:29:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.417+0000 7f02c24d9640 1 -- 172.21.15.16:0/1458090872 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13df80 msgr2=0x7f02bc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.417+0000 7f02c24d9640 1 --2- 172.21.15.16:0/1458090872 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13df80 0x7f02bc15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.417+0000 7f02c24d9640 1 -- 172.21.15.16:0/1458090872 >> v1:172.21.15.173:6789/0 conn(0x7f02bc13e360 legacy=0x7f02bc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.417+0000 7f02c24d9640 1 -- 172.21.15.16:0/1458090872 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc06cd10 con 0x7f02bc0cf9a0 2023-11-24T06:29:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.418+0000 7f02c24d9640 1 --2- 172.21.15.16:0/1458090872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02bc0cf9a0 0x7f02bc0cd840 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009d90 tx=0x7f02b402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff704aabccaf71df server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.419+0000 7f02c0cd6640 1 -- 172.21.15.16:0/1458090872 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b402cc60 con 0x7f02bc0cf9a0 2023-11-24T06:29:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.419+0000 7f02c0cd6640 1 -- 172.21.15.16:0/1458090872 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b402cdc0 con 0x7f02bc0cf9a0 2023-11-24T06:29:52.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.419+0000 7f02c0cd6640 1 -- 172.21.15.16:0/1458090872 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4034640 con 0x7f02bc0cf9a0 2023-11-24T06:29:52.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.420+0000 7f02c3f3c640 1 -- 172.21.15.16:0/1458090872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02bc0cf9a0 msgr2=0x7f02bc0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.420+0000 7f02c3f3c640 1 --2- 172.21.15.16:0/1458090872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02bc0cf9a0 0x7f02bc0cd840 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009d90 tx=0x7f02b402beb0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.420+0000 7f02c3f3c640 1 -- 172.21.15.16:0/1458090872 shutdown_connections 2023-11-24T06:29:52.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.420+0000 7f02c3f3c640 1 --2- 172.21.15.16:0/1458090872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02bc0cf9a0 0x7f02bc0cd840 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.420+0000 7f02c3f3c640 1 --2- 172.21.15.16:0/1458090872 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13df80 0x7f02bc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.420+0000 7f02c3f3c640 1 -- 172.21.15.16:0/1458090872 >> 172.21.15.16:0/1458090872 conn(0x7f02bc06a800 msgr2=0x7f02bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:52.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.420+0000 7f02c3f3c640 1 -- 172.21.15.16:0/1458090872 shutdown_connections 2023-11-24T06:29:52.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.420+0000 7f02c3f3c640 1 -- 172.21.15.16:0/1458090872 wait complete. 2023-11-24T06:29:52.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.422+0000 7f02c3f3c640 1 Processor -- start 2023-11-24T06:29:52.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.422+0000 7f02c3f3c640 1 -- start start 2023-11-24T06:29:52.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.423+0000 7f02c3f3c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02bc0cf9a0 0x7f02bc150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.423+0000 7f02c3f3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc13df80 0x7f02bc14e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.423+0000 7f02c3f3c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13e360 0x7f02bc15e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.423+0000 7f02c3f3c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0a84c0 con 0x7f02bc13df80 2023-11-24T06:29:52.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.423+0000 7f02c3f3c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc14f2e0 con 0x7f02bc0cf9a0 2023-11-24T06:29:52.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.423+0000 7f02c3f3c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc14f580 con 0x7f02bc13e360 2023-11-24T06:29:52.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.423+0000 7f02c14d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc13df80 0x7f02bc14e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.423+0000 7f02c14d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc13df80 0x7f02bc14e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50118/0 (socket says 172.21.15.16:50118) 2023-11-24T06:29:52.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.423+0000 7f02c14d7640 1 -- 172.21.15.16:0/3374005475 learned_addr learned my addr 172.21.15.16:0/3374005475 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:52.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.424+0000 7f02c1cd8640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02bc0cf9a0 0x7f02bc150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.424+0000 7f02c24d9640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13e360 0x7f02bc15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.424+0000 7f02c14d7640 1 -- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13e360 msgr2=0x7f02bc15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.424+0000 7f02c14d7640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13e360 0x7f02bc15e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.424+0000 7f02c14d7640 1 -- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02bc0cf9a0 msgr2=0x7f02bc150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.424+0000 7f02c14d7640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02bc0cf9a0 0x7f02bc150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.424+0000 7f02c14d7640 1 -- 172.21.15.16:0/3374005475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b4009a70 con 0x7f02bc13df80 2023-11-24T06:29:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.424+0000 7f02c14d7640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc13df80 0x7f02bc14e020 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00c500 tx=0x7f02ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.425+0000 7f02b27fc640 1 -- 172.21.15.16:0/3374005475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac005ce0 con 0x7f02bc13df80 2023-11-24T06:29:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.425+0000 7f02b27fc640 1 -- 172.21.15.16:0/3374005475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac005e40 con 0x7f02bc13df80 2023-11-24T06:29:52.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.425+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc14f860 con 0x7f02bc13df80 2023-11-24T06:29:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.425+0000 7f02b27fc640 1 -- 172.21.15.16:0/3374005475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac007330 con 0x7f02bc13df80 2023-11-24T06:29:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.425+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc150100 con 0x7f02bc13df80 2023-11-24T06:29:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.426+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02bc150320 con 0x7f02bc13df80 2023-11-24T06:29:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.428+0000 7f02b27fc640 1 -- 172.21.15.16:0/3374005475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f02ac007550 con 0x7f02bc13df80 2023-11-24T06:29:52.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.428+0000 7f02b27fc640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f02a006b080 0x7f02a006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:52.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.429+0000 7f02b27fc640 1 -- 172.21.15.16:0/3374005475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f02ac088ff0 con 0x7f02bc13df80 2023-11-24T06:29:52.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.429+0000 7f02c1cd8640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f02a006b080 0x7f02a006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:52.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.430+0000 7f02c1cd8640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f02a006b080 0x7f02a006d540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009b40 tx=0x7f02b8009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:52.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.434+0000 7f02b27fc640 1 -- 172.21.15.16:0/3374005475 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02ac0bbe90 con 0x7f02bc13df80 2023-11-24T06:29:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:52.665+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f02bc0675d0 con 0x7f02bc13df80 2023-11-24T06:29:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.203+0000 7f02b27fc640 1 -- 172.21.15.16:0/3374005475 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) v1 ==== 172+0+0 (secure 0 0 0) 0x7f02ac017020 con 0x7f02bc13df80 2023-11-24T06:29:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 8 and data pool 9 2023-11-24T06:29:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.207+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f02a006b080 msgr2=0x7f02a006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.207+0000 7f02c3f3c640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f02a006b080 0x7f02a006d540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f02b8009b40 tx=0x7f02b8009190 comp rx=0 tx=0).stop 2023-11-24T06:29:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.207+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc13df80 msgr2=0x7f02bc14e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.207+0000 7f02c3f3c640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc13df80 0x7f02bc14e020 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00c500 tx=0x7f02ac00c9c0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.207+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 shutdown_connections 2023-11-24T06:29:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.207+0000 7f02c3f3c640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f02a006b080 0x7f02a006d540 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.208+0000 7f02c3f3c640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02bc13e360 0x7f02bc15e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.208+0000 7f02c3f3c640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02bc13df80 0x7f02bc14e020 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.208+0000 7f02c3f3c640 1 --2- 172.21.15.16:0/3374005475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02bc0cf9a0 0x7f02bc150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.208+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 >> 172.21.15.16:0/3374005475 conn(0x7f02bc06a800 msgr2=0x7f02bc15c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.208+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 shutdown_connections 2023-11-24T06:29:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.208+0000 7f02c3f3c640 1 -- 172.21.15.16:0/3374005475 wait complete. 2023-11-24T06:29:53.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:53.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.420+0000 7fed8759e640 1 Processor -- start 2023-11-24T06:29:53.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.420+0000 7fed8759e640 1 -- start start 2023-11-24T06:29:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.422+0000 7fed8759e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fed88143c40 0x7fed88071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.422+0000 7fed8759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fed880717d0 0x7fed880cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.422+0000 7fed8759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed88070e50 con 0x7fed880cca40 2023-11-24T06:29:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.422+0000 7fed8759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880723c0 con 0x7fed880717d0 2023-11-24T06:29:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.422+0000 7fed8759e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880cce40 con 0x7fed88143c40 2023-11-24T06:29:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed86d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fed880cca40 0x7fed8815eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43022/0 (socket says 172.21.15.16:43022) 2023-11-24T06:29:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed86d9d640 1 -- 172.21.15.16:0/2382446358 learned_addr learned my addr 172.21.15.16:0/2382446358 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:53.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed7ffff640 1 --2- 172.21.15.16:0/2382446358 >> v2:172.21.15.16:3300/0 conn(0x7fed880717d0 0x7fed880cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed8659c640 1 --2- 172.21.15.16:0/2382446358 >> v2:172.21.15.173:3300/0 conn(0x7fed88143c40 0x7fed88071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed7ffff640 1 -- 172.21.15.16:0/2382446358 >> v2:172.21.15.173:3300/0 conn(0x7fed88143c40 msgr2=0x7fed88071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed7ffff640 1 --2- 172.21.15.16:0/2382446358 >> v2:172.21.15.173:3300/0 conn(0x7fed88143c40 0x7fed88071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed7ffff640 1 -- 172.21.15.16:0/2382446358 >> v1:172.21.15.16:6789/0 conn(0x7fed880cca40 legacy=0x7fed8815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed7ffff640 1 -- 172.21.15.16:0/2382446358 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8806e170 con 0x7fed880717d0 2023-11-24T06:29:53.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed8659c640 1 --2- 172.21.15.16:0/2382446358 >> v2:172.21.15.173:3300/0 conn(0x7fed88143c40 0x7fed88071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:53.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.423+0000 7fed85d9b640 1 -- 172.21.15.16:0/2382446358 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579116986 0 0) 0x7fed88070e50 con 0x7fed880cca40 2023-11-24T06:29:53.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.424+0000 7fed7ffff640 1 --2- 172.21.15.16:0/2382446358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880717d0 0x7fed880cc500 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fed8806e410 tx=0x7fed7002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913d24e64590fea7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.425+0000 7fed85d9b640 1 -- 172.21.15.16:0/2382446358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7002e070 con 0x7fed880717d0 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.425+0000 7fed85d9b640 1 -- 172.21.15.16:0/2382446358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed700315a0 con 0x7fed880717d0 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.425+0000 7fed85d9b640 1 -- 172.21.15.16:0/2382446358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70031930 con 0x7fed880717d0 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.425+0000 7fed8759e640 1 -- 172.21.15.16:0/2382446358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880717d0 msgr2=0x7fed880cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.425+0000 7fed8759e640 1 --2- 172.21.15.16:0/2382446358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880717d0 0x7fed880cc500 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fed8806e410 tx=0x7fed7002b840 comp rx=0 tx=0).stop 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.425+0000 7fed8759e640 1 -- 172.21.15.16:0/2382446358 shutdown_connections 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.425+0000 7fed8759e640 1 --2- 172.21.15.16:0/2382446358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880717d0 0x7fed880cc500 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.425+0000 7fed8759e640 1 --2- 172.21.15.16:0/2382446358 >> v2:172.21.15.173:3300/0 conn(0x7fed88143c40 0x7fed88071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.425+0000 7fed8759e640 1 -- 172.21.15.16:0/2382446358 >> 172.21.15.16:0/2382446358 conn(0x7fed8806a800 msgr2=0x7fed8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.426+0000 7fed8759e640 1 -- 172.21.15.16:0/2382446358 shutdown_connections 2023-11-24T06:29:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.426+0000 7fed8759e640 1 -- 172.21.15.16:0/2382446358 wait complete. 2023-11-24T06:29:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.427+0000 7fed8759e640 1 Processor -- start 2023-11-24T06:29:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.427+0000 7fed8759e640 1 -- start start 2023-11-24T06:29:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.428+0000 7fed8759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed880717d0 0x7fed88140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.428+0000 7fed8759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880cca40 0x7fed880c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.428+0000 7fed8759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed88143c40 0x7fed88159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.428+0000 7fed8759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8813db70 con 0x7fed880cca40 2023-11-24T06:29:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.428+0000 7fed8759e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c5590 con 0x7fed88143c40 2023-11-24T06:29:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.428+0000 7fed8759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880c4980 con 0x7fed880717d0 2023-11-24T06:29:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.429+0000 7fed7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880cca40 0x7fed880c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.429+0000 7fed7ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880cca40 0x7fed880c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43178/0 (socket says 172.21.15.16:43178) 2023-11-24T06:29:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.429+0000 7fed7ffff640 1 -- 172.21.15.16:0/843789835 learned_addr learned my addr 172.21.15.16:0/843789835 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:53.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.429+0000 7fed7ffff640 1 -- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed880717d0 msgr2=0x7fed88140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.429+0000 7fed7ffff640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed880717d0 0x7fed88140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.429+0000 7fed7ffff640 1 -- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed88143c40 msgr2=0x7fed88159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.429+0000 7fed7ffff640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed88143c40 0x7fed88159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.429+0000 7fed7ffff640 1 -- 172.21.15.16:0/843789835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70009400 con 0x7fed880cca40 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.430+0000 7fed7ffff640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880cca40 0x7fed880c5a20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fed70034040 tx=0x7fed700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.430+0000 7fed7effd640 1 -- 172.21.15.16:0/843789835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70004230 con 0x7fed880cca40 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.430+0000 7fed7effd640 1 -- 172.21.15.16:0/843789835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed70004390 con 0x7fed880cca40 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.430+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed880c4c60 con 0x7fed880cca40 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.430+0000 7fed7effd640 1 -- 172.21.15.16:0/843789835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70032880 con 0x7fed880cca40 2023-11-24T06:29:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.430+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed880c5220 con 0x7fed880cca40 2023-11-24T06:29:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.431+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed88067450 con 0x7fed880cca40 2023-11-24T06:29:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.432+0000 7fed7effd640 1 -- 172.21.15.16:0/843789835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fed700329e0 con 0x7fed880cca40 2023-11-24T06:29:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.433+0000 7fed7effd640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed6806b080 0x7fed6806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.433+0000 7fed7effd640 1 -- 172.21.15.16:0/843789835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed700abd70 con 0x7fed880cca40 2023-11-24T06:29:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.433+0000 7fed8659c640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed6806b080 0x7fed6806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.435+0000 7fed8659c640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed6806b080 0x7fed6806d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fed78008e00 tx=0x7fed78008330 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.437+0000 7fed7effd640 1 -- 172.21.15.16:0/843789835 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed70076910 con 0x7fed880cca40 2023-11-24T06:29:53.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:29:53.502 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:29:53.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.633+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed880c56d0 con 0x7fed880cca40 2023-11-24T06:29:53.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.635+0000 7fed7effd640 1 -- 172.21.15.16:0/843789835 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16578 (secure 0 0 0) 0x7fed70044340 con 0x7fed880cca40 2023-11-24T06:29:53.638 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:53.638 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:52.666647+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:50.546182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:51.698824+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed6806b080 msgr2=0x7fed6806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed6806b080 0x7fed6806d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fed78008e00 tx=0x7fed78008330 comp rx=0 tx=0).stop 2023-11-24T06:29:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880cca40 msgr2=0x7fed880c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880cca40 0x7fed880c5a20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fed70034040 tx=0x7fed700046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 shutdown_connections 2023-11-24T06:29:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fed6806b080 0x7fed6806d540 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed88143c40 0x7fed88159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed880cca40 0x7fed880c5a20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 --2- 172.21.15.16:0/843789835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fed880717d0 0x7fed88140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 >> 172.21.15.16:0/843789835 conn(0x7fed8806a800 msgr2=0x7fed880ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 shutdown_connections 2023-11-24T06:29:53.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.638+0000 7fed8759e640 1 -- 172.21.15.16:0/843789835 wait complete. 2023-11-24T06:29:53.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:53.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.910+0000 7f7e36f89640 1 Processor -- start 2023-11-24T06:29:53.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.911+0000 7f7e36f89640 1 -- start start 2023-11-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.912+0000 7f7e36f89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e3015b130 0x7f7e3015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.912+0000 7f7e36f89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e30070c00 0x7f7e300d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.912+0000 7f7e36f89640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e300cf430 con 0x7f7e300c5f50 2023-11-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.912+0000 7f7e36f89640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e300cec60 con 0x7f7e30070c00 2023-11-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.912+0000 7f7e36f89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300d0890 con 0x7f7e3015b130 2023-11-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.913+0000 7f7e27fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e30070c00 0x7f7e300d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.913+0000 7f7e35526640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7e300c5f50 0x7f7e300c63e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43030/0 (socket says 172.21.15.16:43030) 2023-11-24T06:29:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.913+0000 7f7e35526640 1 -- 172.21.15.16:0/3481031356 learned_addr learned my addr 172.21.15.16:0/3481031356 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.913+0000 7f7e34d25640 1 --2- 172.21.15.16:0/3481031356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e3015b130 0x7f7e3015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.913+0000 7f7e277fe640 1 -- 172.21.15.16:0/3481031356 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782815329 0 0) 0x7f7e300cf430 con 0x7f7e300c5f50 2023-11-24T06:29:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.913+0000 7f7e27fff640 1 -- 172.21.15.16:0/3481031356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e3015b130 msgr2=0x7f7e3015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.913+0000 7f7e27fff640 1 --2- 172.21.15.16:0/3481031356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e3015b130 0x7f7e3015f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.913+0000 7f7e27fff640 1 -- 172.21.15.16:0/3481031356 >> v1:172.21.15.16:6789/0 conn(0x7f7e300c5f50 legacy=0x7f7e300c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.913+0000 7f7e27fff640 1 -- 172.21.15.16:0/3481031356 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3015fbe0 con 0x7f7e30070c00 2023-11-24T06:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.915+0000 7f7e27fff640 1 --2- 172.21.15.16:0/3481031356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 0x7f7e300d1330 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7e200097a0 tx=0x7f7e2002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2436009e1a152e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.915+0000 7f7e277fe640 1 -- 172.21.15.16:0/3481031356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20031830 con 0x7f7e30070c00 2023-11-24T06:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.916+0000 7f7e277fe640 1 -- 172.21.15.16:0/3481031356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e20031990 con 0x7f7e30070c00 2023-11-24T06:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.916+0000 7f7e277fe640 1 -- 172.21.15.16:0/3481031356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20031d20 con 0x7f7e30070c00 2023-11-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.916+0000 7f7e36f89640 1 -- 172.21.15.16:0/3481031356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 msgr2=0x7f7e300d1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.916+0000 7f7e36f89640 1 --2- 172.21.15.16:0/3481031356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 0x7f7e300d1330 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7e200097a0 tx=0x7f7e2002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.916+0000 7f7e36f89640 1 -- 172.21.15.16:0/3481031356 shutdown_connections 2023-11-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.916+0000 7f7e36f89640 1 --2- 172.21.15.16:0/3481031356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 0x7f7e300d1330 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.916+0000 7f7e36f89640 1 --2- 172.21.15.16:0/3481031356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e3015b130 0x7f7e3015f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.916+0000 7f7e36f89640 1 -- 172.21.15.16:0/3481031356 >> 172.21.15.16:0/3481031356 conn(0x7f7e300b2ce0 msgr2=0x7f7e300b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.916+0000 7f7e36f89640 1 -- 172.21.15.16:0/3481031356 shutdown_connections 2023-11-24T06:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.917+0000 7f7e36f89640 1 -- 172.21.15.16:0/3481031356 wait complete. 2023-11-24T06:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.918+0000 7f7e36f89640 1 Processor -- start 2023-11-24T06:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.918+0000 7f7e36f89640 1 -- start start 2023-11-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.919+0000 7f7e36f89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 0x7f7e3014c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.919+0000 7f7e36f89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e300c5f50 0x7f7e300b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.919+0000 7f7e36f89640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e3015b130 0x7f7e300b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.919+0000 7f7e36f89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3014bed0 con 0x7f7e30070c00 2023-11-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.919+0000 7f7e36f89640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300c9d90 con 0x7f7e3015b130 2023-11-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.919+0000 7f7e36f89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300cafd0 con 0x7f7e300c5f50 2023-11-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.920+0000 7f7e34d25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 0x7f7e3014c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.920+0000 7f7e34d25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 0x7f7e3014c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43200/0 (socket says 172.21.15.16:43200) 2023-11-24T06:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.920+0000 7f7e34d25640 1 -- 172.21.15.16:0/2793827164 learned_addr learned my addr 172.21.15.16:0/2793827164 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.920+0000 7f7e34d25640 1 -- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e300c5f50 msgr2=0x7f7e300b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.920+0000 7f7e34d25640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e300c5f50 0x7f7e300b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.920+0000 7f7e34d25640 1 -- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e3015b130 msgr2=0x7f7e300b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.920+0000 7f7e34d25640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e3015b130 0x7f7e300b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.920+0000 7f7e34d25640 1 -- 172.21.15.16:0/2793827164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20009400 con 0x7f7e30070c00 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.920+0000 7f7e34d25640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 0x7f7e3014c0d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c007e40 tx=0x7f7e2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.921+0000 7f7e257fa640 1 -- 172.21.15.16:0/2793827164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c014620 con 0x7f7e30070c00 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.921+0000 7f7e257fa640 1 -- 172.21.15.16:0/2793827164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c00ee90 con 0x7f7e30070c00 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.921+0000 7f7e257fa640 1 -- 172.21.15.16:0/2793827164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0149c0 con 0x7f7e30070c00 2023-11-24T06:29:53.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.921+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e300caa30 con 0x7f7e30070c00 2023-11-24T06:29:53.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.921+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e300c9a20 con 0x7f7e30070c00 2023-11-24T06:29:53.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.922+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e300675d0 con 0x7f7e30070c00 2023-11-24T06:29:53.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.923+0000 7f7e257fa640 1 -- 172.21.15.16:0/2793827164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7e2c014b20 con 0x7f7e30070c00 2023-11-24T06:29:53.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.925+0000 7f7e257fa640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e1c06b080 0x7f7e1c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.925+0000 7f7e257fa640 1 -- 172.21.15.16:0/2793827164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e2c08a380 con 0x7f7e30070c00 2023-11-24T06:29:53.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.925+0000 7f7e27fff640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e1c06b080 0x7f7e1c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:53.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.927+0000 7f7e27fff640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e1c06b080 0x7f7e1c06d540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7e20036000 tx=0x7f7e20030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:53.929+0000 7f7e257fa640 1 -- 172.21.15.16:0/2793827164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e2c055100 con 0x7f7e30070c00 2023-11-24T06:29:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.129+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e300705f0 con 0x7f7e30070c00 2023-11-24T06:29:54.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.131+0000 7f7e257fa640 1 -- 172.21.15.16:0/2793827164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f7e2c058720 con 0x7f7e30070c00 2023-11-24T06:29:54.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:54.134 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":59,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:54.005340+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:50.546182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:51.698824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.133+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e1c06b080 msgr2=0x7f7e1c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.133+0000 7f7e36f89640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e1c06b080 0x7f7e1c06d540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7e20036000 tx=0x7f7e20030000 comp rx=0 tx=0).stop 2023-11-24T06:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.133+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 msgr2=0x7f7e3014c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.133+0000 7f7e36f89640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 0x7f7e3014c0d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c007e40 tx=0x7f7e2c00e4b0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.134+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 shutdown_connections 2023-11-24T06:29:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.134+0000 7f7e36f89640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e1c06b080 0x7f7e1c06d540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.134+0000 7f7e36f89640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e3015b130 0x7f7e300b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.134+0000 7f7e36f89640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e300c5f50 0x7f7e300b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.134+0000 7f7e36f89640 1 --2- 172.21.15.16:0/2793827164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30070c00 0x7f7e3014c0d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.134+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 >> 172.21.15.16:0/2793827164 conn(0x7f7e300b2ce0 msgr2=0x7f7e300d1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.134+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 shutdown_connections 2023-11-24T06:29:54.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.134+0000 7f7e36f89640 1 -- 172.21.15.16:0/2793827164 wait complete. 2023-11-24T06:29:54.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:29:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.372+0000 7f3df8e52640 1 Processor -- start 2023-11-24T06:29:54.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.373+0000 7f3df8e52640 1 -- start start 2023-11-24T06:29:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.374+0000 7f3df8e52640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40cf9a0 0x7f3df40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:54.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.374+0000 7f3df8e52640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3df40b31d0 0x7f3df40b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.374+0000 7f3df8e52640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df40d0320 con 0x7f3df415d4b0 2023-11-24T06:29:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.374+0000 7f3df8e52640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df40d1c60 con 0x7f3df40b31d0 2023-11-24T06:29:54.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.374+0000 7f3df8e52640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40b2730 con 0x7f3df40cf9a0 2023-11-24T06:29:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.375+0000 7f3df3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3df40b31d0 0x7f3df40b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.375+0000 7f3df37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3df415d4b0 0x7f3df40d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43040/0 (socket says 172.21.15.16:43040) 2023-11-24T06:29:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.375+0000 7f3df37fe640 1 -- 172.21.15.16:0/1316035816 learned_addr learned my addr 172.21.15.16:0/1316035816 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.375+0000 7f3debfff640 1 --2- 172.21.15.16:0/1316035816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40cf9a0 0x7f3df40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.375+0000 7f3df3fff640 1 -- 172.21.15.16:0/1316035816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40cf9a0 msgr2=0x7f3df40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.375+0000 7f3df3fff640 1 --2- 172.21.15.16:0/1316035816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40cf9a0 0x7f3df40cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.375+0000 7f3df3fff640 1 -- 172.21.15.16:0/1316035816 >> v1:172.21.15.16:6789/0 conn(0x7f3df415d4b0 legacy=0x7f3df40d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.375+0000 7f3df2ffd640 1 -- 172.21.15.16:0/1316035816 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609901454 0 0) 0x7f3df40d0320 con 0x7f3df415d4b0 2023-11-24T06:29:54.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.376+0000 7f3df3fff640 1 -- 172.21.15.16:0/1316035816 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40d1ec0 con 0x7f3df40b31d0 2023-11-24T06:29:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.377+0000 7f3df3fff640 1 --2- 172.21.15.16:0/1316035816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40b31d0 0x7f3df40b3660 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009dc0 tx=0x7f3dec02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50f98df71bfe9bb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.377+0000 7f3df2ffd640 1 -- 172.21.15.16:0/1316035816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec02cc40 con 0x7f3df40b31d0 2023-11-24T06:29:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.377+0000 7f3df2ffd640 1 -- 172.21.15.16:0/1316035816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dec02cda0 con 0x7f3df40b31d0 2023-11-24T06:29:54.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.377+0000 7f3df2ffd640 1 -- 172.21.15.16:0/1316035816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dec034660 con 0x7f3df40b31d0 2023-11-24T06:29:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.378+0000 7f3df8e52640 1 -- 172.21.15.16:0/1316035816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40b31d0 msgr2=0x7f3df40b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.378+0000 7f3df8e52640 1 --2- 172.21.15.16:0/1316035816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40b31d0 0x7f3df40b3660 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009dc0 tx=0x7f3dec02be90 comp rx=0 tx=0).stop 2023-11-24T06:29:54.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.378+0000 7f3df8e52640 1 -- 172.21.15.16:0/1316035816 shutdown_connections 2023-11-24T06:29:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.378+0000 7f3df8e52640 1 --2- 172.21.15.16:0/1316035816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40b31d0 0x7f3df40b3660 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.378+0000 7f3df8e52640 1 --2- 172.21.15.16:0/1316035816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40cf9a0 0x7f3df40cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.378+0000 7f3df8e52640 1 -- 172.21.15.16:0/1316035816 >> 172.21.15.16:0/1316035816 conn(0x7f3df40d3000 msgr2=0x7f3df40d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.378+0000 7f3df8e52640 1 -- 172.21.15.16:0/1316035816 shutdown_connections 2023-11-24T06:29:54.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.378+0000 7f3df8e52640 1 -- 172.21.15.16:0/1316035816 wait complete. 2023-11-24T06:29:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.380+0000 7f3df8e52640 1 Processor -- start 2023-11-24T06:29:54.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.380+0000 7f3df8e52640 1 -- start start 2023-11-24T06:29:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.381+0000 7f3df8e52640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40b31d0 0x7f3df41f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.381+0000 7f3df8e52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40cf9a0 0x7f3df41f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3df8e52640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3df415d4b0 0x7f3df40cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3df8e52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41f7120 con 0x7f3df40cf9a0 2023-11-24T06:29:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3df8e52640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41f6190 con 0x7f3df415d4b0 2023-11-24T06:29:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3df8e52640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df41f6750 con 0x7f3df40b31d0 2023-11-24T06:29:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3debfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40cf9a0 0x7f3df41f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3debfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40cf9a0 0x7f3df41f2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43212/0 (socket says 172.21.15.16:43212) 2023-11-24T06:29:54.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3debfff640 1 -- 172.21.15.16:0/2652682469 learned_addr learned my addr 172.21.15.16:0/2652682469 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3df37fe640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40b31d0 0x7f3df41f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3debfff640 1 -- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40b31d0 msgr2=0x7f3df41f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3debfff640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40b31d0 0x7f3df41f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3debfff640 1 -- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3df415d4b0 msgr2=0x7f3df40cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3df3fff640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3df415d4b0 0x7f3df40cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3debfff640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3df415d4b0 0x7f3df40cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.382+0000 7f3debfff640 1 -- 172.21.15.16:0/2652682469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec009a70 con 0x7f3df40cf9a0 2023-11-24T06:29:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.383+0000 7f3debfff640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40cf9a0 0x7f3df41f2450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00d5f0 tx=0x7f3ddc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:54.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.383+0000 7f3df0ff9640 1 -- 172.21.15.16:0/2652682469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0125f0 con 0x7f3df40cf9a0 2023-11-24T06:29:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.383+0000 7f3df0ff9640 1 -- 172.21.15.16:0/2652682469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc012750 con 0x7f3df40cf9a0 2023-11-24T06:29:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.383+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df40ccbd0 con 0x7f3df40cf9a0 2023-11-24T06:29:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.383+0000 7f3df0ff9640 1 -- 172.21.15.16:0/2652682469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc012b00 con 0x7f3df40cf9a0 2023-11-24T06:29:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.383+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41f63e0 con 0x7f3df40cf9a0 2023-11-24T06:29:54.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.384+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df41f68e0 con 0x7f3df40cf9a0 2023-11-24T06:29:54.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.386+0000 7f3df0ff9640 1 -- 172.21.15.16:0/2652682469 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3ddc012c60 con 0x7f3df40cf9a0 2023-11-24T06:29:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.387+0000 7f3df0ff9640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3dd406b100 0x7f3dd406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:54.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.387+0000 7f3df0ff9640 1 -- 172.21.15.16:0/2652682469 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ddc089da0 con 0x7f3df40cf9a0 2023-11-24T06:29:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.387+0000 7f3df37fe640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3dd406b100 0x7f3dd406d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:54.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.389+0000 7f3df37fe640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3dd406b100 0x7f3dd406d5c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000f60 tx=0x7f3de4007b50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:54.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.391+0000 7f3df0ff9640 1 -- 172.21.15.16:0/2652682469 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ddc054c60 con 0x7f3df40cf9a0 2023-11-24T06:29:54.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:54.632+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3df4067450 con 0x7f3df40cf9a0 2023-11-24T06:29:55.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.225+0000 7f3df0ff9640 1 -- 172.21.15.16:0/2652682469 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3ddc089020 con 0x7f3df40cf9a0 2023-11-24T06:29:55.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3dd406b100 msgr2=0x7f3dd406d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:55.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3dd406b100 0x7f3dd406d5c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000f60 tx=0x7f3de4007b50 comp rx=0 tx=0).stop 2023-11-24T06:29:55.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40cf9a0 msgr2=0x7f3df41f2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:55.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40cf9a0 0x7f3df41f2450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00d5f0 tx=0x7f3ddc00dab0 comp rx=0 tx=0).stop 2023-11-24T06:29:55.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 shutdown_connections 2023-11-24T06:29:55.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3dd406b100 0x7f3dd406d5c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:55.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3df415d4b0 0x7f3df40cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:55.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df40cf9a0 0x7f3df41f2450 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:55.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 --2- 172.21.15.16:0/2652682469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3df40b31d0 0x7f3df41f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:55.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 >> 172.21.15.16:0/2652682469 conn(0x7f3df40d3000 msgr2=0x7f3df40b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:55.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 shutdown_connections 2023-11-24T06:29:55.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.228+0000 7f3df8e52640 1 -- 172.21.15.16:0/2652682469 wait complete. 2023-11-24T06:29:55.251 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:29:55.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:29:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.472+0000 7f330b50c640 1 Processor -- start 2023-11-24T06:29:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.473+0000 7f330b50c640 1 -- start start 2023-11-24T06:29:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.474+0000 7f330b50c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33040cf9a0 0x7f3304166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:55.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.474+0000 7f330b50c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330415c5f0 con 0x7f330413df80 2023-11-24T06:29:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.474+0000 7f330b50c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330406c470 con 0x7f330413e360 2023-11-24T06:29:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.474+0000 7f330b50c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330406c750 con 0x7f33040cf9a0 2023-11-24T06:29:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.475+0000 7f3309aa9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33040cf9a0 0x7f3304166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.475+0000 7f33092a8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f330413df80 0x7f330415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43052/0 (socket says 172.21.15.16:43052) 2023-11-24T06:29:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.475+0000 7f33092a8640 1 -- 172.21.15.16:0/1175439264 learned_addr learned my addr 172.21.15.16:0/1175439264 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.475+0000 7f3309aa9640 1 -- 172.21.15.16:0/1175439264 >> v1:172.21.15.173:6789/0 conn(0x7f330413e360 legacy=0x7f33040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.475+0000 7f3309aa9640 1 -- 172.21.15.16:0/1175439264 >> v1:172.21.15.16:6789/0 conn(0x7f330413df80 legacy=0x7f330415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.475+0000 7f3309aa9640 1 -- 172.21.15.16:0/1175439264 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330406ca30 con 0x7f33040cf9a0 2023-11-24T06:29:55.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.475+0000 7f3303fff640 1 -- 172.21.15.16:0/1175439264 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646606089 0 0) 0x7f330415c5f0 con 0x7f330413df80 2023-11-24T06:29:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.476+0000 7f3309aa9640 1 --2- 172.21.15.16:0/1175439264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33040cf9a0 0x7f3304166030 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0097a0 tx=0x7f32fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc3ac2e5bad49c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.477+0000 7f3303fff640 1 -- 172.21.15.16:0/1175439264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc031830 con 0x7f33040cf9a0 2023-11-24T06:29:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.477+0000 7f3303fff640 1 -- 172.21.15.16:0/1175439264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32fc031990 con 0x7f33040cf9a0 2023-11-24T06:29:55.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.477+0000 7f3303fff640 1 -- 172.21.15.16:0/1175439264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc031d20 con 0x7f33040cf9a0 2023-11-24T06:29:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.477+0000 7f330b50c640 1 -- 172.21.15.16:0/1175439264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33040cf9a0 msgr2=0x7f3304166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.477+0000 7f330b50c640 1 --2- 172.21.15.16:0/1175439264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33040cf9a0 0x7f3304166030 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0097a0 tx=0x7f32fc02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:29:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.477+0000 7f330b50c640 1 -- 172.21.15.16:0/1175439264 shutdown_connections 2023-11-24T06:29:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.477+0000 7f330b50c640 1 --2- 172.21.15.16:0/1175439264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33040cf9a0 0x7f3304166030 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.477+0000 7f330b50c640 1 -- 172.21.15.16:0/1175439264 >> 172.21.15.16:0/1175439264 conn(0x7f330406a800 msgr2=0x7f330406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.478+0000 7f330b50c640 1 -- 172.21.15.16:0/1175439264 shutdown_connections 2023-11-24T06:29:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.478+0000 7f330b50c640 1 -- 172.21.15.16:0/1175439264 wait complete. 2023-11-24T06:29:55.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.479+0000 7f330b50c640 1 Processor -- start 2023-11-24T06:29:55.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.480+0000 7f330b50c640 1 -- start start 2023-11-24T06:29:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.480+0000 7f330b50c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33040cf9a0 0x7f33041f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.480+0000 7f330b50c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330413df80 0x7f330413d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.480+0000 7f330b50c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330413e360 0x7f3304152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.480+0000 7f330b50c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041f48d0 con 0x7f330413df80 2023-11-24T06:29:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.480+0000 7f330b50c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041503b0 con 0x7f330413e360 2023-11-24T06:29:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.480+0000 7f330b50c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3304150690 con 0x7f33040cf9a0 2023-11-24T06:29:55.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.481+0000 7f3308aa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330413df80 0x7f330413d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.481+0000 7f3308aa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330413df80 0x7f330413d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43238/0 (socket says 172.21.15.16:43238) 2023-11-24T06:29:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.481+0000 7f3308aa7640 1 -- 172.21.15.16:0/514020624 learned_addr learned my addr 172.21.15.16:0/514020624 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.481+0000 7f3308aa7640 1 -- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33040cf9a0 msgr2=0x7f33041f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.481+0000 7f3308aa7640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33040cf9a0 0x7f33041f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.481+0000 7f3308aa7640 1 -- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330413e360 msgr2=0x7f3304152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.481+0000 7f3308aa7640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330413e360 0x7f3304152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:55.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.481+0000 7f3308aa7640 1 -- 172.21.15.16:0/514020624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc009400 con 0x7f330413df80 2023-11-24T06:29:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.481+0000 7f3308aa7640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330413df80 0x7f330413d410 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f32f800c400 tx=0x7f32f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.482+0000 7f3301ffb640 1 -- 172.21.15.16:0/514020624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f8011790 con 0x7f330413df80 2023-11-24T06:29:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.482+0000 7f3301ffb640 1 -- 172.21.15.16:0/514020624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f8011930 con 0x7f330413df80 2023-11-24T06:29:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.482+0000 7f3301ffb640 1 -- 172.21.15.16:0/514020624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f8011ce0 con 0x7f330413df80 2023-11-24T06:29:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.482+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3304150970 con 0x7f330413df80 2023-11-24T06:29:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.482+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330413d950 con 0x7f330413df80 2023-11-24T06:29:55.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.483+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330414abb0 con 0x7f330413df80 2023-11-24T06:29:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.484+0000 7f3301ffb640 1 -- 172.21.15.16:0/514020624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f32f801c460 con 0x7f330413df80 2023-11-24T06:29:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.485+0000 7f3301ffb640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32f006b080 0x7f32f006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.485+0000 7f3301ffb640 1 -- 172.21.15.16:0/514020624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32f8089900 con 0x7f330413df80 2023-11-24T06:29:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.485+0000 7f33092a8640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32f006b080 0x7f32f006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.487+0000 7f33092a8640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32f006b080 0x7f32f006d540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f32f4007fd0 tx=0x7f32f4007ce0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.489+0000 7f3301ffb640 1 -- 172.21.15.16:0/514020624 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f330414abb0 con 0x7f330413df80 2023-11-24T06:29:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:55.721+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f330414abb0 con 0x7f330413df80 2023-11-24T06:29:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.235+0000 7f3301ffb640 1 -- 172.21.15.16:0/514020624 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f330414abb0 con 0x7f330413df80 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.238+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32f006b080 msgr2=0x7f32f006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.238+0000 7f330b50c640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32f006b080 0x7f32f006d540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f32f4007fd0 tx=0x7f32f4007ce0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.238+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330413df80 msgr2=0x7f330413d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.238+0000 7f330b50c640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330413df80 0x7f330413d410 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f32f800c400 tx=0x7f32f800c8c0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.238+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 shutdown_connections 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.238+0000 7f330b50c640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f32f006b080 0x7f32f006d540 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.238+0000 7f330b50c640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330413e360 0x7f3304152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.239+0000 7f330b50c640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f330413df80 0x7f330413d410 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.239+0000 7f330b50c640 1 --2- 172.21.15.16:0/514020624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33040cf9a0 0x7f33041f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.239+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 >> 172.21.15.16:0/514020624 conn(0x7f330406a800 msgr2=0x7f330415cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:56.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.239+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 shutdown_connections 2023-11-24T06:29:56.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.239+0000 7f330b50c640 1 -- 172.21.15.16:0/514020624 wait complete. 2023-11-24T06:29:56.262 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:29:56.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.487+0000 7f0aa3553640 1 Processor -- start 2023-11-24T06:29:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.488+0000 7f0aa3553640 1 -- start start 2023-11-24T06:29:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.489+0000 7f0aa3553640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0a9c06c6b0 0x7f0a9c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.489+0000 7f0aa3553640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c06cea0 con 0x7f0a9c0cd240 2023-11-24T06:29:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.489+0000 7f0aa3553640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c06cbc0 con 0x7f0a9c09a170 2023-11-24T06:29:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.489+0000 7f0aa3553640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c06f350 con 0x7f0a9c06c6b0 2023-11-24T06:29:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.489+0000 7f0aa12ef640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0a9c0cd240 0x7f0a9c099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43058/0 (socket says 172.21.15.16:43058) 2023-11-24T06:29:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.489+0000 7f0aa12ef640 1 -- 172.21.15.16:0/2995790709 learned_addr learned my addr 172.21.15.16:0/2995790709 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318838735 0 0) 0x7f0a9c06cea0 con 0x7f0a9c0cd240 2023-11-24T06:29:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0aa1af0640 1 --2- 172.21.15.16:0/2995790709 >> v2:172.21.15.173:3300/0 conn(0x7f0a9c06c6b0 0x7f0a9c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a84003520 con 0x7f0a9c0cd240 2023-11-24T06:29:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1516118939 0 0) 0x7f0a84003520 con 0x7f0a9c0cd240 2023-11-24T06:29:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 >> v2:172.21.15.173:3300/0 conn(0x7f0a9c06c6b0 msgr2=0x7f0a9c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 --2- 172.21.15.16:0/2995790709 >> v2:172.21.15.173:3300/0 conn(0x7f0a9c06c6b0 0x7f0a9c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 >> v1:172.21.15.173:6789/0 conn(0x7f0a9c09a170 legacy=0x7f0a9c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c06f630 con 0x7f0a9c0cd240 2023-11-24T06:29:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f0a8c0019f0 con 0x7f0a9c0cd240 2023-11-24T06:29:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 >> v1:172.21.15.16:6789/0 conn(0x7f0a9c0cd240 legacy=0x7f0a9c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a84003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a840047c0 0x7f0a84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.490+0000 7f0a9bfff640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84005180 0x7f0a8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c06f350 con 0x7f0a84003970 2023-11-24T06:29:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8400c0b0 con 0x7f0a840047c0 2023-11-24T06:29:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8400c270 con 0x7f0a84005180 2023-11-24T06:29:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0aa12ef640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a84003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0aa0aee640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a840047c0 0x7f0a84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0aa12ef640 1 -- 172.21.15.16:0/2995790709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84005180 msgr2=0x7f0a8400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0aa12ef640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a84005180 0x7f0a8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0aa12ef640 1 -- 172.21.15.16:0/2995790709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a840047c0 msgr2=0x7f0a84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0aa12ef640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a840047c0 0x7f0a84004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0aa12ef640 1 -- 172.21.15.16:0/2995790709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84003520 con 0x7f0a84003970 2023-11-24T06:29:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.491+0000 7f0aa1af0640 1 -- 172.21.15.16:0/2995790709 reap_dead start 2023-11-24T06:29:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.492+0000 7f0aa12ef640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a84003d70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c002230 tx=0x7f0a840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9087a54630caf807 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c034630 con 0x7f0a84003970 2023-11-24T06:29:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c02f070 con 0x7f0a84003970 2023-11-24T06:29:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0a9bfff640 1 -- 172.21.15.16:0/2995790709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c034a30 con 0x7f0a84003970 2023-11-24T06:29:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0aa3553640 1 -- 172.21.15.16:0/2995790709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 msgr2=0x7f0a84003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0aa3553640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a84003d70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c002230 tx=0x7f0a840057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0aa3553640 1 -- 172.21.15.16:0/2995790709 shutdown_connections 2023-11-24T06:29:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0aa3553640 1 --2- 172.21.15.16:0/2995790709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a84003d70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0aa3553640 1 -- 172.21.15.16:0/2995790709 >> 172.21.15.16:0/2995790709 conn(0x7f0a9c0b2d00 msgr2=0x7f0a9c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0aa3553640 1 -- 172.21.15.16:0/2995790709 shutdown_connections 2023-11-24T06:29:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.493+0000 7f0aa3553640 1 -- 172.21.15.16:0/2995790709 wait complete. 2023-11-24T06:29:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.495+0000 7f0aa3553640 1 Processor -- start 2023-11-24T06:29:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.495+0000 7f0aa3553640 1 -- start start 2023-11-24T06:29:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.496+0000 7f0aa3553640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a9c0cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.496+0000 7f0aa3553640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a9c0c69f0 0x7f0a9c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.496+0000 7f0aa3553640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1f63f0 0x7f0a9c1f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.496+0000 7f0aa3553640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1f6db0 con 0x7f0a84003970 2023-11-24T06:29:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.496+0000 7f0aa3553640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0c5c50 con 0x7f0a9c1f63f0 2023-11-24T06:29:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.496+0000 7f0aa3553640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0c6720 con 0x7f0a9c0c69f0 2023-11-24T06:29:56.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa12ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a9c0cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa12ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a9c0cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43268/0 (socket says 172.21.15.16:43268) 2023-11-24T06:29:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa12ef640 1 -- 172.21.15.16:0/1269535391 learned_addr learned my addr 172.21.15.16:0/1269535391 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa0aee640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a9c0c69f0 0x7f0a9c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa1af0640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1f63f0 0x7f0a9c1f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa12ef640 1 -- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a9c0c69f0 msgr2=0x7f0a9c0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa12ef640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a9c0c69f0 0x7f0a9c0c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa12ef640 1 -- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1f63f0 msgr2=0x7f0a9c1f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa12ef640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1f63f0 0x7f0a9c1f6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.497+0000 7f0aa12ef640 1 -- 172.21.15.16:0/1269535391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0c5e80 con 0x7f0a84003970 2023-11-24T06:29:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.498+0000 7f0aa12ef640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a9c0cb190 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c002230 tx=0x7f0a8c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.498+0000 7f0a99ffb640 1 -- 172.21.15.16:0/1269535391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c034840 con 0x7f0a84003970 2023-11-24T06:29:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.498+0000 7f0a99ffb640 1 -- 172.21.15.16:0/1269535391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c0349a0 con 0x7f0a84003970 2023-11-24T06:29:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.498+0000 7f0a99ffb640 1 -- 172.21.15.16:0/1269535391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c034d10 con 0x7f0a84003970 2023-11-24T06:29:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.498+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c0c6160 con 0x7f0a84003970 2023-11-24T06:29:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.498+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c0b4190 con 0x7f0a84003970 2023-11-24T06:29:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.499+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c0675d0 con 0x7f0a84003970 2023-11-24T06:29:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.500+0000 7f0a99ffb640 1 -- 172.21.15.16:0/1269535391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0a8c031030 con 0x7f0a84003970 2023-11-24T06:29:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.502+0000 7f0a99ffb640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a8806b080 0x7f0a8806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.502+0000 7f0a99ffb640 1 -- 172.21.15.16:0/1269535391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a8c0ac100 con 0x7f0a84003970 2023-11-24T06:29:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.502+0000 7f0aa0aee640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a8806b080 0x7f0a8806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.503+0000 7f0aa0aee640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a8806b080 0x7f0a8806d540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0a90007fd0 tx=0x7f0a90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.506+0000 7f0a99ffb640 1 -- 172.21.15.16:0/1269535391 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a8c076ea0 con 0x7f0a84003970 2023-11-24T06:29:56.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.736+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a9c0675d0 con 0x7f0a84003970 2023-11-24T06:29:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.737+0000 7f0a99ffb640 1 -- 172.21.15.16:0/1269535391 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f0a8c04f300 con 0x7f0a84003970 2023-11-24T06:29:56.740 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:56.740 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3782726360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3782726360},{"type":"v1","addr":"172.21.15.173:6835","nonce":3782726360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1586323927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1586323927},{"type":"v1","addr":"172.21.15.16:6835","nonce":1586323927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1730788036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1730788036},{"type":"v1","addr":"172.21.15.173:6837","nonce":1730788036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:52.667087+0000","modified":"2023-11-24T06:29:55.722711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/515502120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":515502120},{"type":"v1","addr":"172.21.15.16:6833","nonce":515502120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:29:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 34 2023-11-24T06:29:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a8806b080 msgr2=0x7f0a8806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a8806b080 0x7f0a8806d540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0a90007fd0 tx=0x7f0a90009040 comp rx=0 tx=0).stop 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 msgr2=0x7f0a9c0cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a9c0cb190 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c002230 tx=0x7f0a8c001d80 comp rx=0 tx=0).stop 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 shutdown_connections 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a8806b080 0x7f0a8806d540 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a9c1f63f0 0x7f0a9c1f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a9c0c69f0 0x7f0a9c0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 --2- 172.21.15.16:0/1269535391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a84003970 0x7f0a9c0cb190 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 >> 172.21.15.16:0/1269535391 conn(0x7f0a9c0b2d00 msgr2=0x7f0a9c0b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 shutdown_connections 2023-11-24T06:29:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.740+0000 7f0aa3553640 1 -- 172.21.15.16:0/1269535391 wait complete. 2023-11-24T06:29:56.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:29:56.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.987+0000 7f0bd6f33640 1 Processor -- start 2023-11-24T06:29:56.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.988+0000 7f0bd6f33640 1 -- start start 2023-11-24T06:29:56.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.989+0000 7f0bd6f33640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd014b2c0 0x7f0bd00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.989+0000 7f0bd6f33640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0bd00cf7d0 0x7f0bd0070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:56.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.989+0000 7f0bd6f33640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd00cfbf0 con 0x7f0bd0070ea0 2023-11-24T06:29:56.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.989+0000 7f0bd6f33640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd014b6a0 con 0x7f0bd00cf7d0 2023-11-24T06:29:56.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.989+0000 7f0bd6f33640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd0071690 con 0x7f0bd014b2c0 2023-11-24T06:29:56.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.990+0000 7f0bc7fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0bd00cf7d0 0x7f0bd0070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:56.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.990+0000 7f0bd4ccf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd014b2c0 0x7f0bd00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:56.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.990+0000 7f0bc7fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0bd00cf7d0 0x7f0bd0070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:33812/0 (socket says 172.21.15.16:33812) 2023-11-24T06:29:56.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.990+0000 7f0bc7fff640 1 -- 172.21.15.16:0/3667622050 learned_addr learned my addr 172.21.15.16:0/3667622050 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:56.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.991+0000 7f0bc7fff640 1 -- 172.21.15.16:0/3667622050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd014b2c0 msgr2=0x7f0bd00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:56.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.991+0000 7f0bc7fff640 1 --2- 172.21.15.16:0/3667622050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd014b2c0 0x7f0bd00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.991+0000 7f0bc7fff640 1 -- 172.21.15.16:0/3667622050 >> v1:172.21.15.173:6789/0 conn(0x7f0bd0070ea0 legacy=0x7f0bd00b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.991+0000 7f0bc7fff640 1 -- 172.21.15.16:0/3667622050 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd0071970 con 0x7f0bd00cf7d0 2023-11-24T06:29:56.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.993+0000 7f0bc7fff640 1 --2- 172.21.15.16:0/3667622050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bd00cf7d0 0x7f0bd0070960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009870 tx=0x7f0bc002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5b2c75801b26f01 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:56.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.993+0000 7f0bc77fe640 1 -- 172.21.15.16:0/3667622050 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0030840 con 0x7f0bd00cf7d0 2023-11-24T06:29:56.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.993+0000 7f0bc77fe640 1 -- 172.21.15.16:0/3667622050 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc00309a0 con 0x7f0bd00cf7d0 2023-11-24T06:29:56.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.993+0000 7f0bc77fe640 1 -- 172.21.15.16:0/3667622050 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0030d30 con 0x7f0bd00cf7d0 2023-11-24T06:29:56.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.994+0000 7f0bd6f33640 1 -- 172.21.15.16:0/3667622050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bd00cf7d0 msgr2=0x7f0bd0070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:56.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.994+0000 7f0bd6f33640 1 --2- 172.21.15.16:0/3667622050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bd00cf7d0 0x7f0bd0070960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009870 tx=0x7f0bc002b980 comp rx=0 tx=0).stop 2023-11-24T06:29:56.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.994+0000 7f0bd6f33640 1 -- 172.21.15.16:0/3667622050 shutdown_connections 2023-11-24T06:29:56.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.994+0000 7f0bd6f33640 1 --2- 172.21.15.16:0/3667622050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bd00cf7d0 0x7f0bd0070960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.994+0000 7f0bd6f33640 1 --2- 172.21.15.16:0/3667622050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd014b2c0 0x7f0bd00cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:56.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.994+0000 7f0bd6f33640 1 -- 172.21.15.16:0/3667622050 >> 172.21.15.16:0/3667622050 conn(0x7f0bd006a800 msgr2=0x7f0bd006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:56.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.994+0000 7f0bd6f33640 1 -- 172.21.15.16:0/3667622050 shutdown_connections 2023-11-24T06:29:56.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.994+0000 7f0bd6f33640 1 -- 172.21.15.16:0/3667622050 wait complete. 2023-11-24T06:29:56.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.996+0000 7f0bd6f33640 1 Processor -- start 2023-11-24T06:29:56.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.996+0000 7f0bd6f33640 1 -- start start 2023-11-24T06:29:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bd6f33640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd0070ea0 0x7f0bd015f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bd6f33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd00cf7d0 0x7f0bd0160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bd6f33640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bd014b2c0 0x7f0bd01fa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bd6f33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd01fb430 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bd6f33640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd0142e60 con 0x7f0bd014b2c0 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bd6f33640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd0143140 con 0x7f0bd0070ea0 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd00cf7d0 0x7f0bd0160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd00cf7d0 0x7f0bd0160f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43282/0 (socket says 172.21.15.16:43282) 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bc7fff640 1 -- 172.21.15.16:0/1053600596 learned_addr learned my addr 172.21.15.16:0/1053600596 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.997+0000 7f0bd4ccf640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd0070ea0 0x7f0bd015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.998+0000 7f0bd54d0640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bd014b2c0 0x7f0bd01fa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.998+0000 7f0bc7fff640 1 -- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd0070ea0 msgr2=0x7f0bd015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.998+0000 7f0bc7fff640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd0070ea0 0x7f0bd015f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.998+0000 7f0bc7fff640 1 -- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bd014b2c0 msgr2=0x7f0bd01fa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.998+0000 7f0bc7fff640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bd014b2c0 0x7f0bd01fa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.998+0000 7f0bc7fff640 1 -- 172.21.15.16:0/1053600596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc00094d0 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.998+0000 7f0bc7fff640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd00cf7d0 0x7f0bd0160f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0000c00 tx=0x7f0bc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.999+0000 7f0bc57fa640 1 -- 172.21.15.16:0/1053600596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0039c90 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.999+0000 7f0bc57fa640 1 -- 172.21.15.16:0/1053600596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc0039e10 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.999+0000 7f0bc57fa640 1 -- 172.21.15.16:0/1053600596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0043640 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.999+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd0143420 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:56.999+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd01439e0 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.000+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd00675d0 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.001+0000 7f0bc57fa640 1 -- 172.21.15.16:0/1053600596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0bc00437a0 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.002+0000 7f0bc57fa640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0bbc06b0b0 0x7f0bbc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.002+0000 7f0bc57fa640 1 -- 172.21.15.16:0/1053600596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bc003e070 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.003+0000 7f0bd4ccf640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0bbc06b0b0 0x7f0bbc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.004+0000 7f0bd4ccf640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0bbc06b0b0 0x7f0bbc06d570 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0073f0 tx=0x7f0bcc00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.007+0000 7f0bc57fa640 1 -- 172.21.15.16:0/1053600596 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bc0077750 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.153 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-11-24T06:29:57.151+0000 7f2c33fe4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:29:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.203+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bd0147da0 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.206+0000 7f0bc57fa640 1 -- 172.21.15.16:0/1053600596 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f0bc0034090 con 0x7f0bd00cf7d0 2023-11-24T06:29:57.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:57.210 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":59,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:54.005340+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:50.546182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:51.698824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:29:57.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.209+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0bbc06b0b0 msgr2=0x7f0bbc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.209+0000 7f0bd6f33640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0bbc06b0b0 0x7f0bbc06d570 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0073f0 tx=0x7f0bcc00b040 comp rx=0 tx=0).stop 2023-11-24T06:29:57.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.209+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd00cf7d0 msgr2=0x7f0bd0160f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.209+0000 7f0bd6f33640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd00cf7d0 0x7f0bd0160f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0000c00 tx=0x7f0bc0004730 comp rx=0 tx=0).stop 2023-11-24T06:29:57.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.210+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 shutdown_connections 2023-11-24T06:29:57.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.210+0000 7f0bd6f33640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0bbc06b0b0 0x7f0bbc06d570 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.210+0000 7f0bd6f33640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bd014b2c0 0x7f0bd01fa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.210+0000 7f0bd6f33640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd00cf7d0 0x7f0bd0160f50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.210+0000 7f0bd6f33640 1 --2- 172.21.15.16:0/1053600596 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bd0070ea0 0x7f0bd015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.210+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 >> 172.21.15.16:0/1053600596 conn(0x7f0bd006a800 msgr2=0x7f0bd00ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:57.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.210+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 shutdown_connections 2023-11-24T06:29:57.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.210+0000 7f0bd6f33640 1 -- 172.21.15.16:0/1053600596 wait complete. 2023-11-24T06:29:57.233 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:29:57.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:57.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.454+0000 7f69f5193640 1 Processor -- start 2023-11-24T06:29:57.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.454+0000 7f69f5193640 1 -- start start 2023-11-24T06:29:57.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.455+0000 7f69f5193640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f69f0160920 0x7f69f00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.456+0000 7f69f5193640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f00d0a10 con 0x7f69f00cf9f0 2023-11-24T06:29:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.456+0000 7f69f5193640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f0161e30 con 0x7f69f00cfea0 2023-11-24T06:29:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.456+0000 7f69f5193640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f00d1f50 con 0x7f69f0160920 2023-11-24T06:29:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.456+0000 7f69e77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f69f00cf9f0 0x7f69f0161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43066/0 (socket says 172.21.15.16:43066) 2023-11-24T06:29:57.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.456+0000 7f69e77fe640 1 -- 172.21.15.16:0/1378756757 learned_addr learned my addr 172.21.15.16:0/1378756757 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:57.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.456+0000 7f69effff640 1 --2- 172.21.15.16:0/1378756757 >> v2:172.21.15.173:3300/0 conn(0x7f69f0160920 0x7f69f00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.456+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490546693 0 0) 0x7f69f00d0a10 con 0x7f69f00cf9f0 2023-11-24T06:29:57.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.456+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69dc003520 con 0x7f69f00cf9f0 2023-11-24T06:29:57.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1354752155 0 0) 0x7f69dc003520 con 0x7f69f00cf9f0 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 >> v2:172.21.15.173:3300/0 conn(0x7f69f0160920 msgr2=0x7f69f00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 --2- 172.21.15.16:0/1378756757 >> v2:172.21.15.173:3300/0 conn(0x7f69f0160920 0x7f69f00b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 >> v1:172.21.15.173:6789/0 conn(0x7f69f00cfea0 legacy=0x7f69f0161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f00cf080 con 0x7f69f00cf9f0 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f69d8001670 con 0x7f69f00cf9f0 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 >> v1:172.21.15.16:6789/0 conn(0x7f69f00cf9f0 legacy=0x7f69f0161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69dc003970 0x7f69dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69dc0047c0 0x7f69dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc005180 0x7f69dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00d1f50 con 0x7f69dc005180 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc00c020 con 0x7f69dc0047c0 2023-11-24T06:29:57.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc00c1e0 con 0x7f69dc003970 2023-11-24T06:29:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.457+0000 7f69f4992640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc005180 0x7f69dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.458+0000 7f69e77fe640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69dc0047c0 0x7f69dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.458+0000 7f69effff640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69dc003970 0x7f69dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.458+0000 7f69f4992640 1 -- 172.21.15.16:0/1378756757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69dc003970 msgr2=0x7f69dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.458+0000 7f69f4992640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69dc003970 0x7f69dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.458+0000 7f69f4992640 1 -- 172.21.15.16:0/1378756757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69dc0047c0 msgr2=0x7f69dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.458+0000 7f69f4992640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69dc0047c0 0x7f69dc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.458+0000 7f69f4992640 1 -- 172.21.15.16:0/1378756757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc003520 con 0x7f69dc005180 2023-11-24T06:29:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.459+0000 7f69f4992640 1 -- 172.21.15.16:0/1378756757 reap_dead start 2023-11-24T06:29:57.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.459+0000 7f69f4992640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc005180 0x7f69dc00b980 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f69e8001e50 tx=0x7f69dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65d0cdfb47fd589 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.459+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8003b70 con 0x7f69dc005180 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.459+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8003d10 con 0x7f69dc005180 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.459+0000 7f69ef7fe640 1 -- 172.21.15.16:0/1378756757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8005310 con 0x7f69dc005180 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.460+0000 7f69f5193640 1 -- 172.21.15.16:0/1378756757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc005180 msgr2=0x7f69dc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.460+0000 7f69f5193640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc005180 0x7f69dc00b980 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f69e8001e50 tx=0x7f69dc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.460+0000 7f69f5193640 1 -- 172.21.15.16:0/1378756757 shutdown_connections 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.460+0000 7f69f5193640 1 --2- 172.21.15.16:0/1378756757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69dc005180 0x7f69dc00b980 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.460+0000 7f69f5193640 1 -- 172.21.15.16:0/1378756757 >> 172.21.15.16:0/1378756757 conn(0x7f69f00d3090 msgr2=0x7f69f00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.460+0000 7f69f5193640 1 -- 172.21.15.16:0/1378756757 shutdown_connections 2023-11-24T06:29:57.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.460+0000 7f69f5193640 1 -- 172.21.15.16:0/1378756757 wait complete. 2023-11-24T06:29:57.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.462+0000 7f69f5193640 1 Processor -- start 2023-11-24T06:29:57.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.462+0000 7f69f5193640 1 -- start start 2023-11-24T06:29:57.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.463+0000 7f69f5193640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69dc005180 0x7f69f00ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.463+0000 7f69f5193640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f01f6350 0x7f69f01540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.463+0000 7f69f5193640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f00c84a0 0x7f69f00c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.463+0000 7f69f5193640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00c8e60 con 0x7f69f01f6350 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.463+0000 7f69f5193640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f01f2c10 con 0x7f69f00c84a0 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.463+0000 7f69f5193640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00cb360 con 0x7f69dc005180 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.463+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f01f6350 0x7f69f01540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.463+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f01f6350 0x7f69f01540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43302/0 (socket says 172.21.15.16:43302) 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.463+0000 7f69e77fe640 1 -- 172.21.15.16:0/97733952 learned_addr learned my addr 172.21.15.16:0/97733952 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.464+0000 7f69effff640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69dc005180 0x7f69f00ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.464+0000 7f69f4992640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f00c84a0 0x7f69f00c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.464+0000 7f69e77fe640 1 -- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69dc005180 msgr2=0x7f69f00ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.464+0000 7f69e77fe640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69dc005180 0x7f69f00ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.464+0000 7f69e77fe640 1 -- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f00c84a0 msgr2=0x7f69f00c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.464+0000 7f69e77fe640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f00c84a0 0x7f69f00c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.464+0000 7f69e77fe640 1 -- 172.21.15.16:0/97733952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f00caa60 con 0x7f69f01f6350 2023-11-24T06:29:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.464+0000 7f69e77fe640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f01f6350 0x7f69f01540f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f69d800e7e0 tx=0x7f69d800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.464+0000 7f69ed7fa640 1 -- 172.21.15.16:0/97733952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800c870 con 0x7f69f01f6350 2023-11-24T06:29:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.465+0000 7f69ed7fa640 1 -- 172.21.15.16:0/97733952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d800c9d0 con 0x7f69f01f6350 2023-11-24T06:29:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.465+0000 7f69ed7fa640 1 -- 172.21.15.16:0/97733952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800cd80 con 0x7f69f01f6350 2023-11-24T06:29:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.465+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f00cad40 con 0x7f69f01f6350 2023-11-24T06:29:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.465+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f00cb620 con 0x7f69f01f6350 2023-11-24T06:29:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.466+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f0067450 con 0x7f69f01f6350 2023-11-24T06:29:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.467+0000 7f69ed7fa640 1 -- 172.21.15.16:0/97733952 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f69d801d460 con 0x7f69f01f6350 2023-11-24T06:29:57.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.468+0000 7f69ed7fa640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69d006b080 0x7f69d006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:57.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.468+0000 7f69ed7fa640 1 -- 172.21.15.16:0/97733952 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69d80899c0 con 0x7f69f01f6350 2023-11-24T06:29:57.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.469+0000 7f69effff640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69d006b080 0x7f69d006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.470+0000 7f69effff640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69d006b080 0x7f69d006d540 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009990 tx=0x7f69e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:57.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.474+0000 7f69ed7fa640 1 -- 172.21.15.16:0/97733952 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69d8054740 con 0x7f69f01f6350 2023-11-24T06:29:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.667+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69f00ce7e0 con 0x7f69d006b080 2023-11-24T06:29:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.696+0000 7f69ed7fa640 1 -- 172.21.15.16:0/97733952 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286452 (secure 0 0 0) 0x7f69e0013960 con 0x7f69d006b080 2023-11-24T06:29:57.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:29:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69d006b080 msgr2=0x7f69d006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69d006b080 0x7f69d006d540 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009990 tx=0x7f69e0008040 comp rx=0 tx=0).stop 2023-11-24T06:29:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f01f6350 msgr2=0x7f69f01540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f01f6350 0x7f69f01540f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f69d800e7e0 tx=0x7f69d800eca0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 shutdown_connections 2023-11-24T06:29:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69d006b080 0x7f69d006d540 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69f00c84a0 0x7f69f00c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69f01f6350 0x7f69f01540f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 --2- 172.21.15.16:0/97733952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69dc005180 0x7f69f00ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 >> 172.21.15.16:0/97733952 conn(0x7f69f00d3090 msgr2=0x7f69f006aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 shutdown_connections 2023-11-24T06:29:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:57.702+0000 7f69f5193640 1 -- 172.21.15.16:0/97733952 wait complete. 2023-11-24T06:29:57.725 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-11-24T06:29:56.163984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243676,"kb_used_data":6664,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731044,"statfs":{"total":773094113280,"available":772844589056,"internally_reserved":0,"allocated":6823936,"data_stored":3498748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.990980"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.015038+0000","last_change":"2023-11-24T06:29:53.046793+0000","last_active":"2023-11-24T06:29:54.015038+0000","last_peered":"2023-11-24T06:29:54.015038+0000","last_clean":"2023-11-24T06:29:54.015038+0000","last_became_active":"2023-11-24T06:29:53.042557+0000","last_became_peered":"2023-11-24T06:29:53.042557+0000","last_unstale":"2023-11-24T06:29:54.015038+0000","last_undegraded":"2023-11-24T06:29:54.015038+0000","last_fullsized":"2023-11-24T06:29:54.015038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:02.210505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.018049+0000","last_change":"2023-11-24T06:29:53.040014+0000","last_active":"2023-11-24T06:29:54.018049+0000","last_peered":"2023-11-24T06:29:54.018049+0000","last_clean":"2023-11-24T06:29:54.018049+0000","last_became_active":"2023-11-24T06:29:53.039690+0000","last_became_peered":"2023-11-24T06:29:53.039690+0000","last_unstale":"2023-11-24T06:29:54.018049+0000","last_undegraded":"2023-11-24T06:29:54.018049+0000","last_fullsized":"2023-11-24T06:29:54.018049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:13.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016653+0000","last_change":"2023-11-24T06:29:53.036294+0000","last_active":"2023-11-24T06:29:54.016653+0000","last_peered":"2023-11-24T06:29:54.016653+0000","last_clean":"2023-11-24T06:29:54.016653+0000","last_became_active":"2023-11-24T06:29:53.035467+0000","last_became_peered":"2023-11-24T06:29:53.035467+0000","last_unstale":"2023-11-24T06:29:54.016653+0000","last_undegraded":"2023-11-24T06:29:54.016653+0000","last_fullsized":"2023-11-24T06:29:54.016653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:51.921598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053540+0000","last_change":"2023-11-24T06:29:53.049704+0000","last_active":"2023-11-24T06:29:53.053540+0000","last_peered":"2023-11-24T06:29:53.053540+0000","last_clean":"2023-11-24T06:29:53.053540+0000","last_became_active":"2023-11-24T06:29:53.049405+0000","last_became_peered":"2023-11-24T06:29:53.049405+0000","last_unstale":"2023-11-24T06:29:53.053540+0000","last_undegraded":"2023-11-24T06:29:53.053540+0000","last_fullsized":"2023-11-24T06:29:53.053540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:02.389910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.014079+0000","last_change":"2023-11-24T06:29:53.037591+0000","last_active":"2023-11-24T06:29:54.014079+0000","last_peered":"2023-11-24T06:29:54.014079+0000","last_clean":"2023-11-24T06:29:54.014079+0000","last_became_active":"2023-11-24T06:29:53.037251+0000","last_became_peered":"2023-11-24T06:29:53.037251+0000","last_unstale":"2023-11-24T06:29:54.014079+0000","last_undegraded":"2023-11-24T06:29:54.014079+0000","last_fullsized":"2023-11-24T06:29:54.014079+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:48.817335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.052862+0000","last_change":"2023-11-24T06:29:53.043815+0000","last_active":"2023-11-24T06:29:53.052862+0000","last_peered":"2023-11-24T06:29:53.052862+0000","last_clean":"2023-11-24T06:29:53.052862+0000","last_became_active":"2023-11-24T06:29:53.043488+0000","last_became_peered":"2023-11-24T06:29:53.043488+0000","last_unstale":"2023-11-24T06:29:53.052862+0000","last_undegraded":"2023-11-24T06:29:53.052862+0000","last_fullsized":"2023-11-24T06:29:53.052862+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:04:32.346084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051029+0000","last_change":"2023-11-24T06:29:53.046134+0000","last_active":"2023-11-24T06:29:53.051029+0000","last_peered":"2023-11-24T06:29:53.051029+0000","last_clean":"2023-11-24T06:29:53.051029+0000","last_became_active":"2023-11-24T06:29:53.045723+0000","last_became_peered":"2023-11-24T06:29:53.045723+0000","last_unstale":"2023-11-24T06:29:53.051029+0000","last_undegraded":"2023-11-24T06:29:53.051029+0000","last_fullsized":"2023-11-24T06:29:53.051029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:40.111154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.015963+0000","last_change":"2023-11-24T06:29:53.046696+0000","last_active":"2023-11-24T06:29:54.015963+0000","last_peered":"2023-11-24T06:29:54.015963+0000","last_clean":"2023-11-24T06:29:54.015963+0000","last_became_active":"2023-11-24T06:29:53.045921+0000","last_became_peered":"2023-11-24T06:29:53.045921+0000","last_unstale":"2023-11-24T06:29:54.015963+0000","last_undegraded":"2023-11-24T06:29:54.015963+0000","last_fullsized":"2023-11-24T06:29:54.015963+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:16.766027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.049219+0000","last_change":"2023-11-24T06:29:53.042523+0000","last_active":"2023-11-24T06:29:53.049219+0000","last_peered":"2023-11-24T06:29:53.049219+0000","last_clean":"2023-11-24T06:29:53.049219+0000","last_became_active":"2023-11-24T06:29:53.042246+0000","last_became_peered":"2023-11-24T06:29:53.042246+0000","last_unstale":"2023-11-24T06:29:53.049219+0000","last_undegraded":"2023-11-24T06:29:53.049219+0000","last_fullsized":"2023-11-24T06:29:53.049219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:05.258049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.045787+0000","last_change":"2023-11-24T06:29:53.038488+0000","last_active":"2023-11-24T06:29:53.045787+0000","last_peered":"2023-11-24T06:29:53.045787+0000","last_clean":"2023-11-24T06:29:53.045787+0000","last_became_active":"2023-11-24T06:29:53.038122+0000","last_became_peered":"2023-11-24T06:29:53.038122+0000","last_unstale":"2023-11-24T06:29:53.045787+0000","last_undegraded":"2023-11-24T06:29:53.045787+0000","last_fullsized":"2023-11-24T06:29:53.045787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:53.958643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.050166+0000","last_change":"2023-11-24T06:29:53.044081+0000","last_active":"2023-11-24T06:29:53.050166+0000","last_peered":"2023-11-24T06:29:53.050166+0000","last_clean":"2023-11-24T06:29:53.050166+0000","last_became_active":"2023-11-24T06:29:53.043733+0000","last_became_peered":"2023-11-24T06:29:53.043733+0000","last_unstale":"2023-11-24T06:29:53.050166+0000","last_undegraded":"2023-11-24T06:29:53.050166+0000","last_fullsized":"2023-11-24T06:29:53.050166+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:13:54.758822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051441+0000","last_change":"2023-11-24T06:29:53.047989+0000","last_active":"2023-11-24T06:29:53.051441+0000","last_peered":"2023-11-24T06:29:53.051441+0000","last_clean":"2023-11-24T06:29:53.051441+0000","last_became_active":"2023-11-24T06:29:53.047632+0000","last_became_peered":"2023-11-24T06:29:53.047632+0000","last_unstale":"2023-11-24T06:29:53.051441+0000","last_undegraded":"2023-11-24T06:29:53.051441+0000","last_fullsized":"2023-11-24T06:29:53.051441+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:45.864860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.043661+0000","last_change":"2023-11-24T06:29:53.038620+0000","last_active":"2023-11-24T06:29:53.043661+0000","last_peered":"2023-11-24T06:29:53.043661+0000","last_clean":"2023-11-24T06:29:53.043661+0000","last_became_active":"2023-11-24T06:29:53.038279+0000","last_became_peered":"2023-11-24T06:29:53.038279+0000","last_unstale":"2023-11-24T06:29:53.043661+0000","last_undegraded":"2023-11-24T06:29:53.043661+0000","last_fullsized":"2023-11-24T06:29:53.043661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:17.383196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.049652+0000","last_change":"2023-11-24T06:29:53.043353+0000","last_active":"2023-11-24T06:29:53.049652+0000","last_peered":"2023-11-24T06:29:53.049652+0000","last_clean":"2023-11-24T06:29:53.049652+0000","last_became_active":"2023-11-24T06:29:53.043066+0000","last_became_peered":"2023-11-24T06:29:53.043066+0000","last_unstale":"2023-11-24T06:29:53.049652+0000","last_undegraded":"2023-11-24T06:29:53.049652+0000","last_fullsized":"2023-11-24T06:29:53.049652+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:14.238180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.048149+0000","last_change":"2023-11-24T06:29:53.040251+0000","last_active":"2023-11-24T06:29:53.048149+0000","last_peered":"2023-11-24T06:29:53.048149+0000","last_clean":"2023-11-24T06:29:53.048149+0000","last_became_active":"2023-11-24T06:29:53.039911+0000","last_became_peered":"2023-11-24T06:29:53.039911+0000","last_unstale":"2023-11-24T06:29:53.048149+0000","last_undegraded":"2023-11-24T06:29:53.048149+0000","last_fullsized":"2023-11-24T06:29:53.048149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:08:55.106042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.052458+0000","last_change":"2023-11-24T06:29:53.035847+0000","last_active":"2023-11-24T06:29:53.052458+0000","last_peered":"2023-11-24T06:29:53.052458+0000","last_clean":"2023-11-24T06:29:53.052458+0000","last_became_active":"2023-11-24T06:29:53.035556+0000","last_became_peered":"2023-11-24T06:29:53.035556+0000","last_unstale":"2023-11-24T06:29:53.052458+0000","last_undegraded":"2023-11-24T06:29:53.052458+0000","last_fullsized":"2023-11-24T06:29:53.052458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:46.839257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017942+0000","last_change":"2023-11-24T06:29:53.042369+0000","last_active":"2023-11-24T06:29:54.017942+0000","last_peered":"2023-11-24T06:29:54.017942+0000","last_clean":"2023-11-24T06:29:54.017942+0000","last_became_active":"2023-11-24T06:29:53.042006+0000","last_became_peered":"2023-11-24T06:29:53.042006+0000","last_unstale":"2023-11-24T06:29:54.017942+0000","last_undegraded":"2023-11-24T06:29:54.017942+0000","last_fullsized":"2023-11-24T06:29:54.017942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:06:48.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016934+0000","last_change":"2023-11-24T06:29:53.044752+0000","last_active":"2023-11-24T06:29:54.016934+0000","last_peered":"2023-11-24T06:29:54.016934+0000","last_clean":"2023-11-24T06:29:54.016934+0000","last_became_active":"2023-11-24T06:29:53.042188+0000","last_became_peered":"2023-11-24T06:29:53.042188+0000","last_unstale":"2023-11-24T06:29:54.016934+0000","last_undegraded":"2023-11-24T06:29:54.016934+0000","last_fullsized":"2023-11-24T06:29:54.016934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:50.660874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.013731+0000","last_change":"2023-11-24T06:29:53.044120+0000","last_active":"2023-11-24T06:29:54.013731+0000","last_peered":"2023-11-24T06:29:54.013731+0000","last_clean":"2023-11-24T06:29:54.013731+0000","last_became_active":"2023-11-24T06:29:53.043786+0000","last_became_peered":"2023-11-24T06:29:53.043786+0000","last_unstale":"2023-11-24T06:29:54.013731+0000","last_undegraded":"2023-11-24T06:29:54.013731+0000","last_fullsized":"2023-11-24T06:29:54.013731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:19.348209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017106+0000","last_change":"2023-11-24T06:29:53.050639+0000","last_active":"2023-11-24T06:29:54.017106+0000","last_peered":"2023-11-24T06:29:54.017106+0000","last_clean":"2023-11-24T06:29:54.017106+0000","last_became_active":"2023-11-24T06:29:53.048846+0000","last_became_peered":"2023-11-24T06:29:53.048846+0000","last_unstale":"2023-11-24T06:29:54.017106+0000","last_undegraded":"2023-11-24T06:29:54.017106+0000","last_fullsized":"2023-11-24T06:29:54.017106+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:30.452190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016973+0000","last_change":"2023-11-24T06:29:53.037352+0000","last_active":"2023-11-24T06:29:54.016973+0000","last_peered":"2023-11-24T06:29:54.016973+0000","last_clean":"2023-11-24T06:29:54.016973+0000","last_became_active":"2023-11-24T06:29:53.036524+0000","last_became_peered":"2023-11-24T06:29:53.036524+0000","last_unstale":"2023-11-24T06:29:54.016973+0000","last_undegraded":"2023-11-24T06:29:54.016973+0000","last_fullsized":"2023-11-24T06:29:54.016973+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:06.524237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017050+0000","last_change":"2023-11-24T06:29:53.046624+0000","last_active":"2023-11-24T06:29:54.017050+0000","last_peered":"2023-11-24T06:29:54.017050+0000","last_clean":"2023-11-24T06:29:54.017050+0000","last_became_active":"2023-11-24T06:29:53.046396+0000","last_became_peered":"2023-11-24T06:29:53.046396+0000","last_unstale":"2023-11-24T06:29:54.017050+0000","last_undegraded":"2023-11-24T06:29:54.017050+0000","last_fullsized":"2023-11-24T06:29:54.017050+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:23:25.711085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016205+0000","last_change":"2023-11-24T06:29:53.048944+0000","last_active":"2023-11-24T06:29:54.016205+0000","last_peered":"2023-11-24T06:29:54.016205+0000","last_clean":"2023-11-24T06:29:54.016205+0000","last_became_active":"2023-11-24T06:29:53.048577+0000","last_became_peered":"2023-11-24T06:29:53.048577+0000","last_unstale":"2023-11-24T06:29:54.016205+0000","last_undegraded":"2023-11-24T06:29:54.016205+0000","last_fullsized":"2023-11-24T06:29:54.016205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:24:50.830782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.046010+0000","last_change":"2023-11-24T06:29:53.039295+0000","last_active":"2023-11-24T06:29:53.046010+0000","last_peered":"2023-11-24T06:29:53.046010+0000","last_clean":"2023-11-24T06:29:53.046010+0000","last_became_active":"2023-11-24T06:29:53.037574+0000","last_became_peered":"2023-11-24T06:29:53.037574+0000","last_unstale":"2023-11-24T06:29:53.046010+0000","last_undegraded":"2023-11-24T06:29:53.046010+0000","last_fullsized":"2023-11-24T06:29:53.046010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:22:38.460485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.015417+0000","last_change":"2023-11-24T06:29:53.034475+0000","last_active":"2023-11-24T06:29:54.015417+0000","last_peered":"2023-11-24T06:29:54.015417+0000","last_clean":"2023-11-24T06:29:54.015417+0000","last_became_active":"2023-11-24T06:29:53.034154+0000","last_became_peered":"2023-11-24T06:29:53.034154+0000","last_unstale":"2023-11-24T06:29:54.015417+0000","last_undegraded":"2023-11-24T06:29:54.015417+0000","last_fullsized":"2023-11-24T06:29:54.015417+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:52.569269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053565+0000","last_change":"2023-11-24T06:29:53.049910+0000","last_active":"2023-11-24T06:29:53.053565+0000","last_peered":"2023-11-24T06:29:53.053565+0000","last_clean":"2023-11-24T06:29:53.053565+0000","last_became_active":"2023-11-24T06:29:53.049577+0000","last_became_peered":"2023-11-24T06:29:53.049577+0000","last_unstale":"2023-11-24T06:29:53.053565+0000","last_undegraded":"2023-11-24T06:29:53.053565+0000","last_fullsized":"2023-11-24T06:29:53.053565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:16:49.009307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053724+0000","last_change":"2023-11-24T06:29:53.052592+0000","last_active":"2023-11-24T06:29:53.053724+0000","last_peered":"2023-11-24T06:29:53.053724+0000","last_clean":"2023-11-24T06:29:53.053724+0000","last_became_active":"2023-11-24T06:29:53.052246+0000","last_became_peered":"2023-11-24T06:29:53.052246+0000","last_unstale":"2023-11-24T06:29:53.053724+0000","last_undegraded":"2023-11-24T06:29:53.053724+0000","last_fullsized":"2023-11-24T06:29:53.053724+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:04.370530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.014655+0000","last_change":"2023-11-24T06:29:53.038466+0000","last_active":"2023-11-24T06:29:54.014655+0000","last_peered":"2023-11-24T06:29:54.014655+0000","last_clean":"2023-11-24T06:29:54.014655+0000","last_became_active":"2023-11-24T06:29:53.038188+0000","last_became_peered":"2023-11-24T06:29:53.038188+0000","last_unstale":"2023-11-24T06:29:54.014655+0000","last_undegraded":"2023-11-24T06:29:54.014655+0000","last_fullsized":"2023-11-24T06:29:54.014655+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:55.314608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017399+0000","last_change":"2023-11-24T06:29:53.047642+0000","last_active":"2023-11-24T06:29:54.017399+0000","last_peered":"2023-11-24T06:29:54.017399+0000","last_clean":"2023-11-24T06:29:54.017399+0000","last_became_active":"2023-11-24T06:29:53.046864+0000","last_became_peered":"2023-11-24T06:29:53.046864+0000","last_unstale":"2023-11-24T06:29:54.017399+0000","last_undegraded":"2023-11-24T06:29:54.017399+0000","last_fullsized":"2023-11-24T06:29:54.017399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:25.572027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016734+0000","last_change":"2023-11-24T06:29:53.037294+0000","last_active":"2023-11-24T06:29:54.016734+0000","last_peered":"2023-11-24T06:29:54.016734+0000","last_clean":"2023-11-24T06:29:54.016734+0000","last_became_active":"2023-11-24T06:29:53.035756+0000","last_became_peered":"2023-11-24T06:29:53.035756+0000","last_unstale":"2023-11-24T06:29:54.016734+0000","last_undegraded":"2023-11-24T06:29:54.016734+0000","last_fullsized":"2023-11-24T06:29:54.016734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:41.311201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.013225+0000","last_change":"2023-11-24T06:29:53.034068+0000","last_active":"2023-11-24T06:29:54.013225+0000","last_peered":"2023-11-24T06:29:54.013225+0000","last_clean":"2023-11-24T06:29:54.013225+0000","last_became_active":"2023-11-24T06:29:53.033563+0000","last_became_peered":"2023-11-24T06:29:53.033563+0000","last_unstale":"2023-11-24T06:29:54.013225+0000","last_undegraded":"2023-11-24T06:29:54.013225+0000","last_fullsized":"2023-11-24T06:29:54.013225+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:23.796861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053150+0000","last_change":"2023-11-24T06:29:53.046246+0000","last_active":"2023-11-24T06:29:53.053150+0000","last_peered":"2023-11-24T06:29:53.053150+0000","last_clean":"2023-11-24T06:29:53.053150+0000","last_became_active":"2023-11-24T06:29:53.045979+0000","last_became_peered":"2023-11-24T06:29:53.045979+0000","last_unstale":"2023-11-24T06:29:53.053150+0000","last_undegraded":"2023-11-24T06:29:53.053150+0000","last_fullsized":"2023-11-24T06:29:53.053150+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:37.548739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053431+0000","last_change":"2023-11-24T06:29:53.052059+0000","last_active":"2023-11-24T06:29:53.053431+0000","last_peered":"2023-11-24T06:29:53.053431+0000","last_clean":"2023-11-24T06:29:53.053431+0000","last_became_active":"2023-11-24T06:29:53.051735+0000","last_became_peered":"2023-11-24T06:29:53.051735+0000","last_unstale":"2023-11-24T06:29:53.053431+0000","last_undegraded":"2023-11-24T06:29:53.053431+0000","last_fullsized":"2023-11-24T06:29:53.053431+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:41.311201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.230833+0000","last_change":"2023-11-24T06:29:52.216329+0000","last_active":"2023-11-24T06:29:53.230833+0000","last_peered":"2023-11-24T06:29:53.230833+0000","last_clean":"2023-11-24T06:29:53.230833+0000","last_became_active":"2023-11-24T06:29:52.215999+0000","last_became_peered":"2023-11-24T06:29:52.215999+0000","last_unstale":"2023-11-24T06:29:53.230833+0000","last_undegraded":"2023-11-24T06:29:53.230833+0000","last_fullsized":"2023-11-24T06:29:53.230833+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:11.511440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.045657+0000","last_change":"2023-11-24T06:29:53.039140+0000","last_active":"2023-11-24T06:29:53.045657+0000","last_peered":"2023-11-24T06:29:53.045657+0000","last_clean":"2023-11-24T06:29:53.045657+0000","last_became_active":"2023-11-24T06:29:53.038799+0000","last_became_peered":"2023-11-24T06:29:53.038799+0000","last_unstale":"2023-11-24T06:29:53.045657+0000","last_undegraded":"2023-11-24T06:29:53.045657+0000","last_fullsized":"2023-11-24T06:29:53.045657+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:45.375004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017729+0000","last_change":"2023-11-24T06:29:52.208760+0000","last_active":"2023-11-24T06:29:54.017729+0000","last_peered":"2023-11-24T06:29:54.017729+0000","last_clean":"2023-11-24T06:29:54.017729+0000","last_became_active":"2023-11-24T06:29:52.208452+0000","last_became_peered":"2023-11-24T06:29:52.208452+0000","last_unstale":"2023-11-24T06:29:54.017729+0000","last_undegraded":"2023-11-24T06:29:54.017729+0000","last_fullsized":"2023-11-24T06:29:54.017729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:37.588360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.052881+0000","last_change":"2023-11-24T06:29:53.044787+0000","last_active":"2023-11-24T06:29:53.052881+0000","last_peered":"2023-11-24T06:29:53.052881+0000","last_clean":"2023-11-24T06:29:53.052881+0000","last_became_active":"2023-11-24T06:29:53.039939+0000","last_became_peered":"2023-11-24T06:29:53.039939+0000","last_unstale":"2023-11-24T06:29:53.052881+0000","last_undegraded":"2023-11-24T06:29:53.052881+0000","last_fullsized":"2023-11-24T06:29:53.052881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:24:50.830782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.220944+0000","last_change":"2023-11-24T06:29:52.214623+0000","last_active":"2023-11-24T06:29:53.220944+0000","last_peered":"2023-11-24T06:29:53.220944+0000","last_clean":"2023-11-24T06:29:53.220944+0000","last_became_active":"2023-11-24T06:29:52.214328+0000","last_became_peered":"2023-11-24T06:29:52.214328+0000","last_unstale":"2023-11-24T06:29:53.220944+0000","last_undegraded":"2023-11-24T06:29:53.220944+0000","last_fullsized":"2023-11-24T06:29:53.220944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:07.906349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053082+0000","last_change":"2023-11-24T06:29:53.045869+0000","last_active":"2023-11-24T06:29:53.053082+0000","last_peered":"2023-11-24T06:29:53.053082+0000","last_clean":"2023-11-24T06:29:53.053082+0000","last_became_active":"2023-11-24T06:29:53.045330+0000","last_became_peered":"2023-11-24T06:29:53.045330+0000","last_unstale":"2023-11-24T06:29:53.053082+0000","last_undegraded":"2023-11-24T06:29:53.053082+0000","last_fullsized":"2023-11-24T06:29:53.053082+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:48:40.169825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.227585+0000","last_change":"2023-11-24T06:29:52.215836+0000","last_active":"2023-11-24T06:29:53.227585+0000","last_peered":"2023-11-24T06:29:53.227585+0000","last_clean":"2023-11-24T06:29:53.227585+0000","last_became_active":"2023-11-24T06:29:52.214260+0000","last_became_peered":"2023-11-24T06:29:52.214260+0000","last_unstale":"2023-11-24T06:29:53.227585+0000","last_undegraded":"2023-11-24T06:29:53.227585+0000","last_fullsized":"2023-11-24T06:29:53.227585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:11.511440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017097+0000","last_change":"2023-11-24T06:29:53.042286+0000","last_active":"2023-11-24T06:29:54.017097+0000","last_peered":"2023-11-24T06:29:54.017097+0000","last_clean":"2023-11-24T06:29:54.017097+0000","last_became_active":"2023-11-24T06:29:53.041991+0000","last_became_peered":"2023-11-24T06:29:53.041991+0000","last_unstale":"2023-11-24T06:29:54.017097+0000","last_undegraded":"2023-11-24T06:29:54.017097+0000","last_fullsized":"2023-11-24T06:29:54.017097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:53.629136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.220811+0000","last_change":"2023-11-24T06:29:52.214566+0000","last_active":"2023-11-24T06:29:53.220811+0000","last_peered":"2023-11-24T06:29:53.220811+0000","last_clean":"2023-11-24T06:29:53.220811+0000","last_became_active":"2023-11-24T06:29:52.214269+0000","last_became_peered":"2023-11-24T06:29:52.214269+0000","last_unstale":"2023-11-24T06:29:53.220811+0000","last_undegraded":"2023-11-24T06:29:53.220811+0000","last_fullsized":"2023-11-24T06:29:53.220811+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:43.465119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017248+0000","last_change":"2023-11-24T06:29:53.040441+0000","last_active":"2023-11-24T06:29:54.017248+0000","last_peered":"2023-11-24T06:29:54.017248+0000","last_clean":"2023-11-24T06:29:54.017248+0000","last_became_active":"2023-11-24T06:29:53.040066+0000","last_became_peered":"2023-11-24T06:29:53.040066+0000","last_unstale":"2023-11-24T06:29:54.017248+0000","last_undegraded":"2023-11-24T06:29:54.017248+0000","last_fullsized":"2023-11-24T06:29:54.017248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:05.887029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.223721+0000","last_change":"2023-11-24T06:29:52.217828+0000","last_active":"2023-11-24T06:29:53.223721+0000","last_peered":"2023-11-24T06:29:53.223721+0000","last_clean":"2023-11-24T06:29:53.223721+0000","last_became_active":"2023-11-24T06:29:52.217458+0000","last_became_peered":"2023-11-24T06:29:52.217458+0000","last_unstale":"2023-11-24T06:29:53.223721+0000","last_undegraded":"2023-11-24T06:29:53.223721+0000","last_fullsized":"2023-11-24T06:29:53.223721+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:58:46.276511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.043648+0000","last_change":"2023-11-24T06:29:53.034170+0000","last_active":"2023-11-24T06:29:53.043648+0000","last_peered":"2023-11-24T06:29:53.043648+0000","last_clean":"2023-11-24T06:29:53.043648+0000","last_became_active":"2023-11-24T06:29:53.033402+0000","last_became_peered":"2023-11-24T06:29:53.033402+0000","last_unstale":"2023-11-24T06:29:53.043648+0000","last_undegraded":"2023-11-24T06:29:53.043648+0000","last_fullsized":"2023-11-24T06:29:53.043648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:09.168419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.036802+0000","last_change":"2023-11-24T06:29:52.213281+0000","last_active":"2023-11-24T06:29:53.036802+0000","last_peered":"2023-11-24T06:29:53.036802+0000","last_clean":"2023-11-24T06:29:53.036802+0000","last_became_active":"2023-11-24T06:29:52.210667+0000","last_became_peered":"2023-11-24T06:29:52.210667+0000","last_unstale":"2023-11-24T06:29:53.036802+0000","last_undegraded":"2023-11-24T06:29:53.036802+0000","last_fullsized":"2023-11-24T06:29:53.036802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:34:11.416727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016467+0000","last_change":"2023-11-24T06:29:53.039889+0000","last_active":"2023-11-24T06:29:54.016467+0000","last_peered":"2023-11-24T06:29:54.016467+0000","last_clean":"2023-11-24T06:29:54.016467+0000","last_became_active":"2023-11-24T06:29:53.039385+0000","last_became_peered":"2023-11-24T06:29:53.039385+0000","last_unstale":"2023-11-24T06:29:54.016467+0000","last_undegraded":"2023-11-24T06:29:54.016467+0000","last_fullsized":"2023-11-24T06:29:54.016467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:59:58.960040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.028958+0000","last_change":"2023-11-24T06:29:52.212440+0000","last_active":"2023-11-24T06:29:53.028958+0000","last_peered":"2023-11-24T06:29:53.028958+0000","last_clean":"2023-11-24T06:29:53.028958+0000","last_became_active":"2023-11-24T06:29:52.212149+0000","last_became_peered":"2023-11-24T06:29:52.212149+0000","last_unstale":"2023-11-24T06:29:53.028958+0000","last_undegraded":"2023-11-24T06:29:53.028958+0000","last_fullsized":"2023-11-24T06:29:53.028958+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:57.138021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.048550+0000","last_change":"2023-11-24T06:29:53.040815+0000","last_active":"2023-11-24T06:29:53.048550+0000","last_peered":"2023-11-24T06:29:53.048550+0000","last_clean":"2023-11-24T06:29:53.048550+0000","last_became_active":"2023-11-24T06:29:53.040451+0000","last_became_peered":"2023-11-24T06:29:53.040451+0000","last_unstale":"2023-11-24T06:29:53.048550+0000","last_undegraded":"2023-11-24T06:29:53.048550+0000","last_fullsized":"2023-11-24T06:29:53.048550+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:22:38.460485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016782+0000","last_change":"2023-11-24T06:29:52.211167+0000","last_active":"2023-11-24T06:29:54.016782+0000","last_peered":"2023-11-24T06:29:54.016782+0000","last_clean":"2023-11-24T06:29:54.016782+0000","last_became_active":"2023-11-24T06:29:52.210900+0000","last_became_peered":"2023-11-24T06:29:52.210900+0000","last_unstale":"2023-11-24T06:29:54.016782+0000","last_undegraded":"2023-11-24T06:29:54.016782+0000","last_fullsized":"2023-11-24T06:29:54.016782+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:04:31.332577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016831+0000","last_change":"2023-11-24T06:29:53.041045+0000","last_active":"2023-11-24T06:29:54.016831+0000","last_peered":"2023-11-24T06:29:54.016831+0000","last_clean":"2023-11-24T06:29:54.016831+0000","last_became_active":"2023-11-24T06:29:53.040725+0000","last_became_peered":"2023-11-24T06:29:53.040725+0000","last_unstale":"2023-11-24T06:29:54.016831+0000","last_undegraded":"2023-11-24T06:29:54.016831+0000","last_fullsized":"2023-11-24T06:29:54.016831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:08.919856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017822+0000","last_change":"2023-11-24T06:29:52.206226+0000","last_active":"2023-11-24T06:29:54.017822+0000","last_peered":"2023-11-24T06:29:54.017822+0000","last_clean":"2023-11-24T06:29:54.017822+0000","last_became_active":"2023-11-24T06:29:52.205678+0000","last_became_peered":"2023-11-24T06:29:52.205678+0000","last_unstale":"2023-11-24T06:29:54.017822+0000","last_undegraded":"2023-11-24T06:29:54.017822+0000","last_fullsized":"2023-11-24T06:29:54.017822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:11.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.046412+0000","last_change":"2023-11-24T06:29:53.039401+0000","last_active":"2023-11-24T06:29:53.046412+0000","last_peered":"2023-11-24T06:29:53.046412+0000","last_clean":"2023-11-24T06:29:53.046412+0000","last_became_active":"2023-11-24T06:29:53.039082+0000","last_became_peered":"2023-11-24T06:29:53.039082+0000","last_unstale":"2023-11-24T06:29:53.046412+0000","last_undegraded":"2023-11-24T06:29:53.046412+0000","last_fullsized":"2023-11-24T06:29:53.046412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:19:45.049987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016955+0000","last_change":"2023-11-24T06:29:52.204934+0000","last_active":"2023-11-24T06:29:54.016955+0000","last_peered":"2023-11-24T06:29:54.016955+0000","last_clean":"2023-11-24T06:29:54.016955+0000","last_became_active":"2023-11-24T06:29:52.204625+0000","last_became_peered":"2023-11-24T06:29:52.204625+0000","last_unstale":"2023-11-24T06:29:54.016955+0000","last_undegraded":"2023-11-24T06:29:54.016955+0000","last_fullsized":"2023-11-24T06:29:54.016955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:36.535231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016660+0000","last_change":"2023-11-24T06:29:53.040364+0000","last_active":"2023-11-24T06:29:54.016660+0000","last_peered":"2023-11-24T06:29:54.016660+0000","last_clean":"2023-11-24T06:29:54.016660+0000","last_became_active":"2023-11-24T06:29:53.039736+0000","last_became_peered":"2023-11-24T06:29:53.039736+0000","last_unstale":"2023-11-24T06:29:54.016660+0000","last_undegraded":"2023-11-24T06:29:54.016660+0000","last_fullsized":"2023-11-24T06:29:54.016660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:04.370530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.234436+0000","last_change":"2023-11-24T06:29:52.213776+0000","last_active":"2023-11-24T06:29:53.234436+0000","last_peered":"2023-11-24T06:29:53.234436+0000","last_clean":"2023-11-24T06:29:53.234436+0000","last_became_active":"2023-11-24T06:29:52.213484+0000","last_became_peered":"2023-11-24T06:29:52.213484+0000","last_unstale":"2023-11-24T06:29:53.234436+0000","last_undegraded":"2023-11-24T06:29:53.234436+0000","last_fullsized":"2023-11-24T06:29:53.234436+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:39.365789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051409+0000","last_change":"2023-11-24T06:29:53.047505+0000","last_active":"2023-11-24T06:29:53.051409+0000","last_peered":"2023-11-24T06:29:53.051409+0000","last_clean":"2023-11-24T06:29:53.051409+0000","last_became_active":"2023-11-24T06:29:53.044791+0000","last_became_peered":"2023-11-24T06:29:53.044791+0000","last_unstale":"2023-11-24T06:29:53.051409+0000","last_undegraded":"2023-11-24T06:29:53.051409+0000","last_fullsized":"2023-11-24T06:29:53.051409+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:14.238180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017330+0000","last_change":"2023-11-24T06:29:52.205001+0000","last_active":"2023-11-24T06:29:54.017330+0000","last_peered":"2023-11-24T06:29:54.017330+0000","last_clean":"2023-11-24T06:29:54.017330+0000","last_became_active":"2023-11-24T06:29:52.204804+0000","last_became_peered":"2023-11-24T06:29:52.204804+0000","last_unstale":"2023-11-24T06:29:54.017330+0000","last_undegraded":"2023-11-24T06:29:54.017330+0000","last_fullsized":"2023-11-24T06:29:54.017330+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:18.913373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017306+0000","last_change":"2023-11-24T06:29:53.038626+0000","last_active":"2023-11-24T06:29:54.017306+0000","last_peered":"2023-11-24T06:29:54.017306+0000","last_clean":"2023-11-24T06:29:54.017306+0000","last_became_active":"2023-11-24T06:29:53.038334+0000","last_became_peered":"2023-11-24T06:29:53.038334+0000","last_unstale":"2023-11-24T06:29:54.017306+0000","last_undegraded":"2023-11-24T06:29:54.017306+0000","last_fullsized":"2023-11-24T06:29:54.017306+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:38.601868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.017786+0000","last_change":"2023-11-24T06:29:52.212030+0000","last_active":"2023-11-24T06:29:53.017786+0000","last_peered":"2023-11-24T06:29:53.017786+0000","last_clean":"2023-11-24T06:29:53.017786+0000","last_became_active":"2023-11-24T06:29:52.211740+0000","last_became_peered":"2023-11-24T06:29:52.211740+0000","last_unstale":"2023-11-24T06:29:53.017786+0000","last_undegraded":"2023-11-24T06:29:53.017786+0000","last_fullsized":"2023-11-24T06:29:53.017786+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:08:54.092535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.047896+0000","last_change":"2023-11-24T06:29:53.039526+0000","last_active":"2023-11-24T06:29:53.047896+0000","last_peered":"2023-11-24T06:29:53.047896+0000","last_clean":"2023-11-24T06:29:53.047896+0000","last_became_active":"2023-11-24T06:29:53.039282+0000","last_became_peered":"2023-11-24T06:29:53.039282+0000","last_unstale":"2023-11-24T06:29:53.047896+0000","last_undegraded":"2023-11-24T06:29:53.047896+0000","last_fullsized":"2023-11-24T06:29:53.047896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:17.383196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.036820+0000","last_change":"2023-11-24T06:29:52.208985+0000","last_active":"2023-11-24T06:29:53.036820+0000","last_peered":"2023-11-24T06:29:53.036820+0000","last_clean":"2023-11-24T06:29:53.036820+0000","last_became_active":"2023-11-24T06:29:52.208478+0000","last_became_peered":"2023-11-24T06:29:52.208478+0000","last_unstale":"2023-11-24T06:29:53.036820+0000","last_undegraded":"2023-11-24T06:29:53.036820+0000","last_fullsized":"2023-11-24T06:29:53.036820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:41.427319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016565+0000","last_change":"2023-11-24T06:29:53.045873+0000","last_active":"2023-11-24T06:29:54.016565+0000","last_peered":"2023-11-24T06:29:54.016565+0000","last_clean":"2023-11-24T06:29:54.016565+0000","last_became_active":"2023-11-24T06:29:53.045549+0000","last_became_peered":"2023-11-24T06:29:53.045549+0000","last_unstale":"2023-11-24T06:29:54.016565+0000","last_undegraded":"2023-11-24T06:29:54.016565+0000","last_fullsized":"2023-11-24T06:29:54.016565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:48:40.169825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016095+0000","last_change":"2023-11-24T06:29:52.206665+0000","last_active":"2023-11-24T06:29:54.016095+0000","last_peered":"2023-11-24T06:29:54.016095+0000","last_clean":"2023-11-24T06:29:54.016095+0000","last_became_active":"2023-11-24T06:29:52.206385+0000","last_became_peered":"2023-11-24T06:29:52.206385+0000","last_unstale":"2023-11-24T06:29:54.016095+0000","last_undegraded":"2023-11-24T06:29:54.016095+0000","last_fullsized":"2023-11-24T06:29:54.016095+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:23:34.198375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017208+0000","last_change":"2023-11-24T06:29:53.036064+0000","last_active":"2023-11-24T06:29:54.017208+0000","last_peered":"2023-11-24T06:29:54.017208+0000","last_clean":"2023-11-24T06:29:54.017208+0000","last_became_active":"2023-11-24T06:29:53.035634+0000","last_became_peered":"2023-11-24T06:29:53.035634+0000","last_unstale":"2023-11-24T06:29:54.017208+0000","last_undegraded":"2023-11-24T06:29:54.017208+0000","last_fullsized":"2023-11-24T06:29:54.017208+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:11.855717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.018125+0000","last_change":"2023-11-24T06:29:52.201939+0000","last_active":"2023-11-24T06:29:54.018125+0000","last_peered":"2023-11-24T06:29:54.018125+0000","last_clean":"2023-11-24T06:29:54.018125+0000","last_became_active":"2023-11-24T06:29:52.201661+0000","last_became_peered":"2023-11-24T06:29:52.201661+0000","last_unstale":"2023-11-24T06:29:54.018125+0000","last_undegraded":"2023-11-24T06:29:54.018125+0000","last_fullsized":"2023-11-24T06:29:54.018125+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:07.675887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.048080+0000","last_change":"2023-11-24T06:29:53.043592+0000","last_active":"2023-11-24T06:29:53.048080+0000","last_peered":"2023-11-24T06:29:53.048080+0000","last_clean":"2023-11-24T06:29:53.048080+0000","last_became_active":"2023-11-24T06:29:53.043335+0000","last_became_peered":"2023-11-24T06:29:53.043335+0000","last_unstale":"2023-11-24T06:29:53.048080+0000","last_undegraded":"2023-11-24T06:29:53.048080+0000","last_fullsized":"2023-11-24T06:29:53.048080+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:45.394544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.221249+0000","last_change":"2023-11-24T06:29:52.211084+0000","last_active":"2023-11-24T06:29:53.221249+0000","last_peered":"2023-11-24T06:29:53.221249+0000","last_clean":"2023-11-24T06:29:53.221249+0000","last_became_active":"2023-11-24T06:29:52.207269+0000","last_became_peered":"2023-11-24T06:29:52.207269+0000","last_unstale":"2023-11-24T06:29:53.221249+0000","last_undegraded":"2023-11-24T06:29:53.221249+0000","last_fullsized":"2023-11-24T06:29:53.221249+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:49.647367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017424+0000","last_change":"2023-11-24T06:29:53.040165+0000","last_active":"2023-11-24T06:29:54.017424+0000","last_peered":"2023-11-24T06:29:54.017424+0000","last_clean":"2023-11-24T06:29:54.017424+0000","last_became_active":"2023-11-24T06:29:53.037665+0000","last_became_peered":"2023-11-24T06:29:53.037665+0000","last_unstale":"2023-11-24T06:29:54.017424+0000","last_undegraded":"2023-11-24T06:29:54.017424+0000","last_fullsized":"2023-11-24T06:29:54.017424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:12.433603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.229382+0000","last_change":"2023-11-24T06:29:52.215444+0000","last_active":"2023-11-24T06:29:53.229382+0000","last_peered":"2023-11-24T06:29:53.229382+0000","last_clean":"2023-11-24T06:29:53.229382+0000","last_became_active":"2023-11-24T06:29:52.213594+0000","last_became_peered":"2023-11-24T06:29:52.213594+0000","last_unstale":"2023-11-24T06:29:53.229382+0000","last_undegraded":"2023-11-24T06:29:53.229382+0000","last_fullsized":"2023-11-24T06:29:53.229382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:44.488029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017398+0000","last_change":"2023-11-24T06:29:53.040048+0000","last_active":"2023-11-24T06:29:54.017398+0000","last_peered":"2023-11-24T06:29:54.017398+0000","last_clean":"2023-11-24T06:29:54.017398+0000","last_became_active":"2023-11-24T06:29:53.039299+0000","last_became_peered":"2023-11-24T06:29:53.039299+0000","last_unstale":"2023-11-24T06:29:54.017398+0000","last_undegraded":"2023-11-24T06:29:54.017398+0000","last_fullsized":"2023-11-24T06:29:54.017398+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:12.505585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.031583+0000","last_change":"2023-11-24T06:29:52.215132+0000","last_active":"2023-11-24T06:29:53.031583+0000","last_peered":"2023-11-24T06:29:53.031583+0000","last_clean":"2023-11-24T06:29:53.031583+0000","last_became_active":"2023-11-24T06:29:52.212194+0000","last_became_peered":"2023-11-24T06:29:52.212194+0000","last_unstale":"2023-11-24T06:29:53.031583+0000","last_undegraded":"2023-11-24T06:29:53.031583+0000","last_fullsized":"2023-11-24T06:29:53.031583+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:50.908090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.047753+0000","last_change":"2023-11-24T06:29:53.040431+0000","last_active":"2023-11-24T06:29:53.047753+0000","last_peered":"2023-11-24T06:29:53.047753+0000","last_clean":"2023-11-24T06:29:53.047753+0000","last_became_active":"2023-11-24T06:29:53.040085+0000","last_became_peered":"2023-11-24T06:29:53.040085+0000","last_unstale":"2023-11-24T06:29:53.047753+0000","last_undegraded":"2023-11-24T06:29:53.047753+0000","last_fullsized":"2023-11-24T06:29:53.047753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:05.258049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017576+0000","last_change":"2023-11-24T06:29:52.211045+0000","last_active":"2023-11-24T06:29:54.017576+0000","last_peered":"2023-11-24T06:29:54.017576+0000","last_clean":"2023-11-24T06:29:54.017576+0000","last_became_active":"2023-11-24T06:29:52.210783+0000","last_became_peered":"2023-11-24T06:29:52.210783+0000","last_unstale":"2023-11-24T06:29:54.017576+0000","last_undegraded":"2023-11-24T06:29:54.017576+0000","last_fullsized":"2023-11-24T06:29:54.017576+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:08.839177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053346+0000","last_change":"2023-11-24T06:29:53.051186+0000","last_active":"2023-11-24T06:29:53.053346+0000","last_peered":"2023-11-24T06:29:53.053346+0000","last_clean":"2023-11-24T06:29:53.053346+0000","last_became_active":"2023-11-24T06:29:53.049439+0000","last_became_peered":"2023-11-24T06:29:53.049439+0000","last_unstale":"2023-11-24T06:29:53.053346+0000","last_undegraded":"2023-11-24T06:29:53.053346+0000","last_fullsized":"2023-11-24T06:29:53.053346+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:40:48.242120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.020785+0000","last_change":"2023-11-24T06:29:52.202046+0000","last_active":"2023-11-24T06:29:53.020785+0000","last_peered":"2023-11-24T06:29:53.020785+0000","last_clean":"2023-11-24T06:29:53.020785+0000","last_became_active":"2023-11-24T06:29:52.201750+0000","last_became_peered":"2023-11-24T06:29:52.201750+0000","last_unstale":"2023-11-24T06:29:53.020785+0000","last_undegraded":"2023-11-24T06:29:53.020785+0000","last_fullsized":"2023-11-24T06:29:53.020785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:45.954019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.052660+0000","last_change":"2023-11-24T06:29:53.038612+0000","last_active":"2023-11-24T06:29:53.052660+0000","last_peered":"2023-11-24T06:29:53.052660+0000","last_clean":"2023-11-24T06:29:53.052660+0000","last_became_active":"2023-11-24T06:29:53.038309+0000","last_became_peered":"2023-11-24T06:29:53.038309+0000","last_unstale":"2023-11-24T06:29:53.052660+0000","last_undegraded":"2023-11-24T06:29:53.052660+0000","last_fullsized":"2023-11-24T06:29:53.052660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:45.501536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.037094+0000","last_change":"2023-11-24T06:29:52.212924+0000","last_active":"2023-11-24T06:29:53.037094+0000","last_peered":"2023-11-24T06:29:53.037094+0000","last_clean":"2023-11-24T06:29:53.037094+0000","last_became_active":"2023-11-24T06:29:52.212374+0000","last_became_peered":"2023-11-24T06:29:52.212374+0000","last_unstale":"2023-11-24T06:29:53.037094+0000","last_undegraded":"2023-11-24T06:29:53.037094+0000","last_fullsized":"2023-11-24T06:29:53.037094+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:36.535231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051532+0000","last_change":"2023-11-24T06:29:53.049139+0000","last_active":"2023-11-24T06:29:53.051532+0000","last_peered":"2023-11-24T06:29:53.051532+0000","last_clean":"2023-11-24T06:29:53.051532+0000","last_became_active":"2023-11-24T06:29:53.048815+0000","last_became_peered":"2023-11-24T06:29:53.048815+0000","last_unstale":"2023-11-24T06:29:53.051532+0000","last_undegraded":"2023-11-24T06:29:53.051532+0000","last_fullsized":"2023-11-24T06:29:53.051532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:42.041698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.220194+0000","last_change":"2023-11-24T06:29:52.197996+0000","last_active":"2023-11-24T06:29:53.220194+0000","last_peered":"2023-11-24T06:29:53.220194+0000","last_clean":"2023-11-24T06:29:53.220194+0000","last_became_active":"2023-11-24T06:29:52.197655+0000","last_became_peered":"2023-11-24T06:29:52.197655+0000","last_unstale":"2023-11-24T06:29:53.220194+0000","last_undegraded":"2023-11-24T06:29:53.220194+0000","last_fullsized":"2023-11-24T06:29:53.220194+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:04.696421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051428+0000","last_change":"2023-11-24T06:29:53.047961+0000","last_active":"2023-11-24T06:29:53.051428+0000","last_peered":"2023-11-24T06:29:53.051428+0000","last_clean":"2023-11-24T06:29:53.051428+0000","last_became_active":"2023-11-24T06:29:53.046978+0000","last_became_peered":"2023-11-24T06:29:53.046978+0000","last_unstale":"2023-11-24T06:29:53.051428+0000","last_undegraded":"2023-11-24T06:29:53.051428+0000","last_fullsized":"2023-11-24T06:29:53.051428+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:23:35.211882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.035086+0000","last_change":"2023-11-24T06:29:52.218275+0000","last_active":"2023-11-24T06:29:53.035086+0000","last_peered":"2023-11-24T06:29:53.035086+0000","last_clean":"2023-11-24T06:29:53.035086+0000","last_became_active":"2023-11-24T06:29:52.217960+0000","last_became_peered":"2023-11-24T06:29:52.217960+0000","last_unstale":"2023-11-24T06:29:53.035086+0000","last_undegraded":"2023-11-24T06:29:53.035086+0000","last_fullsized":"2023-11-24T06:29:53.035086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:05.510729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.046437+0000","last_change":"2023-11-24T06:29:53.039678+0000","last_active":"2023-11-24T06:29:53.046437+0000","last_peered":"2023-11-24T06:29:53.046437+0000","last_clean":"2023-11-24T06:29:53.046437+0000","last_became_active":"2023-11-24T06:29:53.039382+0000","last_became_peered":"2023-11-24T06:29:53.039382+0000","last_unstale":"2023-11-24T06:29:53.046437+0000","last_undegraded":"2023-11-24T06:29:53.046437+0000","last_fullsized":"2023-11-24T06:29:53.046437+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:25.146749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.018477+0000","last_change":"2023-11-24T06:29:52.218380+0000","last_active":"2023-11-24T06:29:53.018477+0000","last_peered":"2023-11-24T06:29:53.018477+0000","last_clean":"2023-11-24T06:29:53.018477+0000","last_became_active":"2023-11-24T06:29:52.217778+0000","last_became_peered":"2023-11-24T06:29:52.217778+0000","last_unstale":"2023-11-24T06:29:53.018477+0000","last_undegraded":"2023-11-24T06:29:53.018477+0000","last_fullsized":"2023-11-24T06:29:53.018477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:39.097647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053057+0000","last_change":"2023-11-24T06:29:53.045611+0000","last_active":"2023-11-24T06:29:53.053057+0000","last_peered":"2023-11-24T06:29:53.053057+0000","last_clean":"2023-11-24T06:29:53.053057+0000","last_became_active":"2023-11-24T06:29:53.044985+0000","last_became_peered":"2023-11-24T06:29:53.044985+0000","last_unstale":"2023-11-24T06:29:53.053057+0000","last_undegraded":"2023-11-24T06:29:53.053057+0000","last_fullsized":"2023-11-24T06:29:53.053057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:58.716734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.037224+0000","last_change":"2023-11-24T06:29:52.215591+0000","last_active":"2023-11-24T06:29:53.037224+0000","last_peered":"2023-11-24T06:29:53.037224+0000","last_clean":"2023-11-24T06:29:53.037224+0000","last_became_active":"2023-11-24T06:29:52.214821+0000","last_became_peered":"2023-11-24T06:29:52.214821+0000","last_unstale":"2023-11-24T06:29:53.037224+0000","last_undegraded":"2023-11-24T06:29:53.037224+0000","last_fullsized":"2023-11-24T06:29:53.037224+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:45.825749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017332+0000","last_change":"2023-11-24T06:29:53.038284+0000","last_active":"2023-11-24T06:29:54.017332+0000","last_peered":"2023-11-24T06:29:54.017332+0000","last_clean":"2023-11-24T06:29:54.017332+0000","last_became_active":"2023-11-24T06:29:53.037590+0000","last_became_peered":"2023-11-24T06:29:53.037590+0000","last_unstale":"2023-11-24T06:29:54.017332+0000","last_undegraded":"2023-11-24T06:29:54.017332+0000","last_fullsized":"2023-11-24T06:29:54.017332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:11:49.701095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.021007+0000","last_change":"2023-11-24T06:29:52.208050+0000","last_active":"2023-11-24T06:29:53.021007+0000","last_peered":"2023-11-24T06:29:53.021007+0000","last_clean":"2023-11-24T06:29:53.021007+0000","last_became_active":"2023-11-24T06:29:52.207718+0000","last_became_peered":"2023-11-24T06:29:52.207718+0000","last_unstale":"2023-11-24T06:29:53.021007+0000","last_undegraded":"2023-11-24T06:29:53.021007+0000","last_fullsized":"2023-11-24T06:29:53.021007+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:52.559537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017618+0000","last_change":"2023-11-24T06:29:53.042687+0000","last_active":"2023-11-24T06:29:54.017618+0000","last_peered":"2023-11-24T06:29:54.017618+0000","last_clean":"2023-11-24T06:29:54.017618+0000","last_became_active":"2023-11-24T06:29:53.042383+0000","last_became_peered":"2023-11-24T06:29:53.042383+0000","last_unstale":"2023-11-24T06:29:54.017618+0000","last_undegraded":"2023-11-24T06:29:54.017618+0000","last_fullsized":"2023-11-24T06:29:54.017618+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:04.164163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.031634+0000","last_change":"2023-11-24T06:29:52.213841+0000","last_active":"2023-11-24T06:29:53.031634+0000","last_peered":"2023-11-24T06:29:53.031634+0000","last_clean":"2023-11-24T06:29:53.031634+0000","last_became_active":"2023-11-24T06:29:52.213503+0000","last_became_peered":"2023-11-24T06:29:52.213503+0000","last_unstale":"2023-11-24T06:29:53.031634+0000","last_undegraded":"2023-11-24T06:29:53.031634+0000","last_fullsized":"2023-11-24T06:29:53.031634+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:23:34.198375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.038300+0000","last_change":"2023-11-24T06:29:53.035185+0000","last_active":"2023-11-24T06:29:53.038300+0000","last_peered":"2023-11-24T06:29:53.038300+0000","last_clean":"2023-11-24T06:29:53.038300+0000","last_became_active":"2023-11-24T06:29:53.034893+0000","last_became_peered":"2023-11-24T06:29:53.034893+0000","last_unstale":"2023-11-24T06:29:53.038300+0000","last_undegraded":"2023-11-24T06:29:53.038300+0000","last_fullsized":"2023-11-24T06:29:53.038300+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:56.568611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.021194+0000","last_change":"2023-11-24T06:29:52.210848+0000","last_active":"2023-11-24T06:29:53.021194+0000","last_peered":"2023-11-24T06:29:53.021194+0000","last_clean":"2023-11-24T06:29:53.021194+0000","last_became_active":"2023-11-24T06:29:52.210472+0000","last_became_peered":"2023-11-24T06:29:52.210472+0000","last_unstale":"2023-11-24T06:29:53.021194+0000","last_undegraded":"2023-11-24T06:29:53.021194+0000","last_fullsized":"2023-11-24T06:29:53.021194+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:17.527323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016577+0000","last_change":"2023-11-24T06:29:53.039669+0000","last_active":"2023-11-24T06:29:54.016577+0000","last_peered":"2023-11-24T06:29:54.016577+0000","last_clean":"2023-11-24T06:29:54.016577+0000","last_became_active":"2023-11-24T06:29:53.039024+0000","last_became_peered":"2023-11-24T06:29:53.039024+0000","last_unstale":"2023-11-24T06:29:54.016577+0000","last_undegraded":"2023-11-24T06:29:54.016577+0000","last_fullsized":"2023-11-24T06:29:54.016577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:37.120835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016055+0000","last_change":"2023-11-24T06:29:52.204624+0000","last_active":"2023-11-24T06:29:54.016055+0000","last_peered":"2023-11-24T06:29:54.016055+0000","last_clean":"2023-11-24T06:29:54.016055+0000","last_became_active":"2023-11-24T06:29:52.204334+0000","last_became_peered":"2023-11-24T06:29:52.204334+0000","last_unstale":"2023-11-24T06:29:54.016055+0000","last_undegraded":"2023-11-24T06:29:54.016055+0000","last_fullsized":"2023-11-24T06:29:54.016055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:19.058926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016236+0000","last_change":"2023-11-24T06:29:53.041234+0000","last_active":"2023-11-24T06:29:54.016236+0000","last_peered":"2023-11-24T06:29:54.016236+0000","last_clean":"2023-11-24T06:29:54.016236+0000","last_became_active":"2023-11-24T06:29:53.040942+0000","last_became_peered":"2023-11-24T06:29:53.040942+0000","last_unstale":"2023-11-24T06:29:54.016236+0000","last_undegraded":"2023-11-24T06:29:54.016236+0000","last_fullsized":"2023-11-24T06:29:54.016236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:40.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.014493+0000","last_change":"2023-11-24T06:29:52.212278+0000","last_active":"2023-11-24T06:29:53.014493+0000","last_peered":"2023-11-24T06:29:53.014493+0000","last_clean":"2023-11-24T06:29:53.014493+0000","last_became_active":"2023-11-24T06:29:52.211944+0000","last_became_peered":"2023-11-24T06:29:52.211944+0000","last_unstale":"2023-11-24T06:29:53.014493+0000","last_undegraded":"2023-11-24T06:29:53.014493+0000","last_fullsized":"2023-11-24T06:29:53.014493+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:52.298030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.013941+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:29:53.013941+0000","last_peered":"2023-11-24T06:29:53.013941+0000","last_clean":"2023-11-24T06:29:53.013941+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:29:53.013941+0000","last_undegraded":"2023-11-24T06:29:53.013941+0000","last_fullsized":"2023-11-24T06:29:53.013941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1298432,"data_stored":879677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.29,"15min":1.3280000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5780000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":4.0599999999999996},{"interface":"front","average":{"1min":1.181,"5min":1.2789999999999999,"15min":1.2949999999999999},"min":{"1min":0.75700000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.76,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":4.556}]},{"osd":1,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.65100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":2.456},{"interface":"front","average":{"1min":1.218,"5min":1.359,"15min":1.383},"min":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.585,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":3.944}]},{"osd":2,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.2649999999999999,"15min":1.2849999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6479999999999999,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":5.4589999999999996},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1919999999999999,"15min":1.1990000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.754,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":4.3799999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.365,"15min":1.403},"min":{"1min":0.80900000000000005,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.637,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":8.3539999999999992},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2050000000000001,"15min":1.2030000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.8200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.542,"15min":1.585},"min":{"1min":0.72099999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7170000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":2.1059999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.5569999999999999,"15min":1.585},"min":{"1min":1.0589999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.962,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":5.3449999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.395,"15min":1.411},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.921,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":2.3639999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.7170000000000001,"15min":1.758},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.845,"5min":6.7359999999999998,"15min":6.7359999999999998},"last":17.582999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.825,"15min":1.8779999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.093,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":2.8140000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.659,"15min":1.706},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.881,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":6.2210000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":289149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0980000000000001,"15min":1.1020000000000001},"min":{"1min":0.73499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7390000000000001,"5min":1.829,"15min":1.829},"last":1.151},{"interface":"front","average":{"1min":1.155,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.76800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.623,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.81200000000000006}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2210000000000001,"15min":1.2130000000000001},"min":{"1min":0.71099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6299999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.464},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.2010000000000001,"15min":1.2070000000000001},"min":{"1min":0.88900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.494,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.2}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.159},"min":{"1min":0.75700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.609,"5min":3.597,"15min":3.597},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.304,"5min":1.292,"15min":1.3140000000000001},"min":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6359999999999999,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.5529999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3129999999999999,"15min":1.3180000000000001},"min":{"1min":0.78100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5620000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.506},{"interface":"front","average":{"1min":1.25,"5min":1.2969999999999999,"15min":1.325},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5960000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.234}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.8919999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.623},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.385,"15min":1.397},"min":{"1min":0.79600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7729999999999999,"5min":3.04,"15min":3.04},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3720000000000001,"15min":1.377},"min":{"1min":0.97699999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7330000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.552,"15min":1.6020000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.851,"5min":3.774,"15min":3.774},"last":1.319}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5449999999999999,"15min":1.5589999999999999},"min":{"1min":1.0029999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7689999999999999,"5min":3.403,"15min":3.403},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.512},"min":{"1min":0.875,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.792,"5min":3.649,"15min":3.649},"last":1.5960000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":704512,"data_stored":289801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.516,"5min":1.903,"15min":1.903},"last":1.125},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.288,"15min":1.3140000000000001},"min":{"1min":0.91700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.915,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.4490000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.145,"15min":1.1399999999999999},"min":{"1min":0.78900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.327,"5min":1.147,"15min":1.1140000000000001},"min":{"1min":0.89500000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.206}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3109999999999999,"15min":1.2969999999999999},"min":{"1min":0.99199999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1669999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.077}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3360000000000001,"15min":1.375},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.952,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3899999999999999},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.3460000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3180000000000001,"15min":1.3340000000000001},"min":{"1min":0.878,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.839,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.472,"5min":1.5029999999999999,"15min":1.5369999999999999},"min":{"1min":0.82299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0099999999999998,"5min":3.085,"15min":3.085},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4690000000000001,"15min":1.4450000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.474,"15min":1.468},"min":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0270000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.3700000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.651,"5min":1.6140000000000001,"15min":1.6240000000000001},"min":{"1min":1.046,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.141,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.28},{"interface":"front","average":{"1min":1.714,"5min":1.6919999999999999,"15min":1.716},"min":{"1min":1.0129999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3159999999999998,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.407}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":290761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.073,"15min":1.073},"min":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.982,"15min":1.982},"last":0.752},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.089,"15min":1.1120000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.468,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.016}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.147,"15min":1.113},"min":{"1min":0.84199999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.349},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.26,"15min":1.3049999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5840000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8129999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.244},{"interface":"front","average":{"1min":1.27,"5min":1.3140000000000001,"15min":1.3320000000000001},"min":{"1min":0.79800000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7250000000000001,"5min":2.254,"15min":2.254},"last":0.83899999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.294,"15min":1.333},"min":{"1min":0.76100000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.833,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.343,"5min":1.3779999999999999,"15min":1.3720000000000001},"min":{"1min":0.753,"5min":0.746,"15min":0.746},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4079999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.482,"15min":1.5149999999999999},"min":{"1min":0.874,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8109999999999999,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.389,"5min":1.522,"15min":1.5609999999999999},"min":{"1min":0.85799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.923,"5min":3.52,"15min":3.52},"last":1.1890000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.474,"15min":1.482},"min":{"1min":0.79200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.542,"15min":1.5740000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.859,"5min":2.452,"15min":2.452},"last":1.079}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.595,"15min":1.6559999999999999},"min":{"1min":0.64000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.7729999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.615,"15min":1.613},"min":{"1min":0.84799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.0099999999999998,"5min":2.363,"15min":2.363},"last":1.5129999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":704512,"data_stored":289205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.40200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.90700000000000003,"5min":1.355,"15min":1.355},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.92700000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.41199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4690000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1160000000000001},"min":{"1min":0.51000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.4950000000000001,"5min":1.786,"15min":1.786},"last":1.1830000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.499,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4710000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.3},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4450000000000001,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.2470000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.0299999999999998,"5min":3.081,"15min":3.081},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3300000000000001,"15min":1.3380000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0910000000000002,"5min":2.758,"15min":2.758},"last":1.5309999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.377,"15min":1.4039999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8200000000000001,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.722,"15min":1.782},"min":{"1min":0.88600000000000001,"5min":0.745,"15min":0.745},"max":{"1min":2.149,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.4099999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3819999999999999,"15min":1.409},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.6919999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.5349999999999999,"15min":1.5449999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.1030000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.732}]},{"osd":7,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.5960000000000001,"15min":1.6160000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.1829999999999998,"5min":4.641,"15min":4.641},"last":1.611},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.55,"15min":1.5760000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.9750000000000001,"5min":3.798,"15min":3.798},"last":1.4690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":704512,"data_stored":289205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2330000000000001,"5min":1.427,"15min":1.427},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.432,"5min":0.31,"15min":0.31},"max":{"1min":1.161,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.2609999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.96299999999999997},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2410000000000001,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":2.1389999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0449999999999999,"15min":1.0509999999999999},"min":{"1min":0.56899999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.512,"5min":1.55,"15min":1.55},"last":1.379}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.101,"15min":1.1100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6850000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.115},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.103,"15min":1.119},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.597,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.1479999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.492,"15min":1.52},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.79,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":2.6869999999999998},{"interface":"front","average":{"1min":1.343,"5min":1.4690000000000001,"15min":1.49},"min":{"1min":0.94999999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6899999999999999,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":2.5800000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4590000000000001,"15min":1.466},"min":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.911,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.776},{"interface":"front","average":{"1min":1.385,"5min":1.49,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.9410000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.6499999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4279999999999999,"15min":1.4379999999999999},"min":{"1min":0.78200000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.6899999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.863},{"interface":"front","average":{"1min":1.419,"5min":1.5229999999999999,"15min":1.5409999999999999},"min":{"1min":0.85199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.0720000000000001,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.9390000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.5369999999999999,"15min":1.5569999999999999},"min":{"1min":0.83099999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.974,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.714},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.5309999999999999,"15min":1.5389999999999999},"min":{"1min":0.90300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1699999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.829}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":700416,"data_stored":289239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.002,"15min":1.032},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.696,"5min":2.593,"15min":2.593},"last":1.212},{"interface":"front","average":{"1min":1.008,"5min":1.02,"15min":1.032},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5980000000000001,"5min":1.702,"15min":1.702},"last":1.135}]},{"osd":2,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.212,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7669999999999999,"5min":1.829,"15min":1.829},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1040000000000001,"15min":1.1200000000000001},"min":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.603,"5min":1.766,"15min":1.766},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0289999999999999,"15min":1.069},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2250000000000001,"5min":1.798,"15min":1.798},"last":1.036}]},{"osd":4,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.246,"15min":1.2529999999999999},"min":{"1min":0.89000000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.621,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.077},{"interface":"front","average":{"1min":1.359,"5min":1.417,"15min":1.4450000000000001},"min":{"1min":1.0509999999999999,"5min":1.032,"15min":1.032},"max":{"1min":1.837,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.534}]},{"osd":5,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.294,"15min":1.302},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.514,"5min":2.016,"15min":2.016},"last":1.365},{"interface":"front","average":{"1min":1.391,"5min":1.389,"15min":1.399},"min":{"1min":0.91700000000000004,"5min":0.877,"15min":0.877},"max":{"1min":1.7410000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.4079999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.391,"15min":1.393},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.847,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4339999999999999,"15min":1.45},"min":{"1min":0.96399999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.6499999999999999,"5min":2.371,"15min":2.371},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.444,"15min":1.448},"min":{"1min":0.89300000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9730000000000001,"5min":2.246,"15min":2.246},"last":1.633},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4950000000000001,"15min":1.5189999999999999},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.7629999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.587}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1306624,"data_stored":881711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.48999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.607,"5min":1.722,"15min":1.722},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.5,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.999,"15min":0.998},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.119,"15min":1.137},"min":{"1min":0.63400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.538,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.93100000000000005,"15min":0.94899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5229999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.085,"15min":1.0900000000000001},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.639,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2090000000000001,"15min":1.2190000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6659999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.093},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3100000000000001,"15min":1.3080000000000001},"min":{"1min":0.91400000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.986,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.145}]},{"osd":5,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.321,"15min":1.333},"min":{"1min":0.71799999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.704,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.2509999999999999,"15min":1.2709999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6000000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4710000000000001,"15min":1.4870000000000001},"min":{"1min":0.95499999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7230000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.492,"15min":1.518},"min":{"1min":0.91300000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.8069999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.123}]},{"osd":7,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.446,"15min":1.456},"min":{"1min":0.86499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9059999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.056},{"interface":"front","average":{"1min":1.278,"5min":1.399,"15min":1.419},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.853,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.89800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:57.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:29:58.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.001+0000 7f15ae55c640 1 Processor -- start 2023-11-24T06:29:58.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.001+0000 7f15ae55c640 1 -- start start 2023-11-24T06:29:58.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.003+0000 7f15ae55c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f15a8160890 0x7f15a80d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.003+0000 7f15ae55c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15a80c6be0 0x7f15a80cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.003+0000 7f15ae55c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a80cf6a0 con 0x7f15a8160c70 2023-11-24T06:29:58.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.003+0000 7f15ae55c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80d1430 con 0x7f15a80c6be0 2023-11-24T06:29:58.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.003+0000 7f15ae55c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80b4370 con 0x7f15a8160890 2023-11-24T06:29:58.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.003+0000 7f15add5b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15a80c6be0 0x7f15a80cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.004+0000 7f15add5b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15a80c6be0 0x7f15a80cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43310/0 (socket says 172.21.15.16:43310) 2023-11-24T06:29:58.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.004+0000 7f15add5b640 1 -- 172.21.15.16:0/1519092757 learned_addr learned my addr 172.21.15.16:0/1519092757 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:58.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.004+0000 7f15ad55a640 1 --2- 172.21.15.16:0/1519092757 >> v2:172.21.15.173:3300/0 conn(0x7f15a8160890 0x7f15a80d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.004+0000 7f15add5b640 1 -- 172.21.15.16:0/1519092757 >> v2:172.21.15.173:3300/0 conn(0x7f15a8160890 msgr2=0x7f15a80d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:58.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.004+0000 7f15add5b640 1 --2- 172.21.15.16:0/1519092757 >> v2:172.21.15.173:3300/0 conn(0x7f15a8160890 0x7f15a80d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.004+0000 7f15add5b640 1 -- 172.21.15.16:0/1519092757 >> v1:172.21.15.173:6789/0 conn(0x7f15a8160c70 legacy=0x7f15a80d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:58.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.004+0000 7f15add5b640 1 -- 172.21.15.16:0/1519092757 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80ce7c0 con 0x7f15a80c6be0 2023-11-24T06:29:58.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.005+0000 7f15add5b640 1 --2- 172.21.15.16:0/1519092757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a80c6be0 0x7f15a80cfb30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f15a80b5c10 tx=0x7f15a002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08d292724e092be server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:58.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.006+0000 7f159ffff640 1 -- 172.21.15.16:0/1519092757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a002c960 con 0x7f15a80c6be0 2023-11-24T06:29:58.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.006+0000 7f159ffff640 1 -- 172.21.15.16:0/1519092757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a002cac0 con 0x7f15a80c6be0 2023-11-24T06:29:58.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.006+0000 7f159ffff640 1 -- 172.21.15.16:0/1519092757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a002ce50 con 0x7f15a80c6be0 2023-11-24T06:29:58.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.006+0000 7f15ae55c640 1 -- 172.21.15.16:0/1519092757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a80c6be0 msgr2=0x7f15a80cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.006+0000 7f15ae55c640 1 --2- 172.21.15.16:0/1519092757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a80c6be0 0x7f15a80cfb30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f15a80b5c10 tx=0x7f15a002bdc0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.006+0000 7f15ae55c640 1 -- 172.21.15.16:0/1519092757 shutdown_connections 2023-11-24T06:29:58.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.006+0000 7f15ae55c640 1 --2- 172.21.15.16:0/1519092757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a80c6be0 0x7f15a80cfb30 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.006+0000 7f15ae55c640 1 --2- 172.21.15.16:0/1519092757 >> v2:172.21.15.173:3300/0 conn(0x7f15a8160890 0x7f15a80d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.006+0000 7f15ae55c640 1 -- 172.21.15.16:0/1519092757 >> 172.21.15.16:0/1519092757 conn(0x7f15a80d3090 msgr2=0x7f15a80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:58.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.007+0000 7f15ae55c640 1 -- 172.21.15.16:0/1519092757 shutdown_connections 2023-11-24T06:29:58.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.007+0000 7f15ae55c640 1 -- 172.21.15.16:0/1519092757 wait complete. 2023-11-24T06:29:58.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.008+0000 7f15ae55c640 1 Processor -- start 2023-11-24T06:29:58.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.008+0000 7f15ae55c640 1 -- start start 2023-11-24T06:29:58.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.009+0000 7f15ae55c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15a80c6be0 0x7f15a81f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.009+0000 7f15ae55c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a8160890 0x7f15a80b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.009+0000 7f15ae55c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15a8160c70 0x7f15a806b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.009+0000 7f15ae55c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81f3ea0 con 0x7f15a8160890 2023-11-24T06:29:58.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.009+0000 7f15ae55c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81f3600 con 0x7f15a8160c70 2023-11-24T06:29:58.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.009+0000 7f15ae55c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81f38e0 con 0x7f15a80c6be0 2023-11-24T06:29:58.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15acd59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a8160890 0x7f15a80b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15acd59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a8160890 0x7f15a80b66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43320/0 (socket says 172.21.15.16:43320) 2023-11-24T06:29:58.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15acd59640 1 -- 172.21.15.16:0/3357578624 learned_addr learned my addr 172.21.15.16:0/3357578624 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:58.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15acd59640 1 -- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15a80c6be0 msgr2=0x7f15a81f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15add5b640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15a8160c70 0x7f15a806b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15acd59640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15a80c6be0 0x7f15a81f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15acd59640 1 -- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15a8160c70 msgr2=0x7f15a806b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15acd59640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15a8160c70 0x7f15a806b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15acd59640 1 -- 172.21.15.16:0/3357578624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0009b30 con 0x7f15a8160890 2023-11-24T06:29:58.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.010+0000 7f15acd59640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a8160890 0x7f15a80b66e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f159800c430 tx=0x7f159800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:58.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.011+0000 7f159dffb640 1 -- 172.21.15.16:0/3357578624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598011790 con 0x7f15a8160890 2023-11-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.011+0000 7f159dffb640 1 -- 172.21.15.16:0/3357578624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15980118f0 con 0x7f15a8160890 2023-11-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.011+0000 7f159dffb640 1 -- 172.21.15.16:0/3357578624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598011ce0 con 0x7f15a8160890 2023-11-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.011+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a81f3bc0 con 0x7f15a8160890 2023-11-24T06:29:58.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.011+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a80b6c20 con 0x7f15a8160890 2023-11-24T06:29:58.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.012+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a81f0b60 con 0x7f15a8160890 2023-11-24T06:29:58.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.013+0000 7f159dffb640 1 -- 172.21.15.16:0/3357578624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f159801c460 con 0x7f15a8160890 2023-11-24T06:29:58.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.014+0000 7f159dffb640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f158c06b080 0x7f158c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.014+0000 7f159dffb640 1 -- 172.21.15.16:0/3357578624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1598089940 con 0x7f15a8160890 2023-11-24T06:29:58.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.015+0000 7f15ad55a640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f158c06b080 0x7f158c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.016+0000 7f15ad55a640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f158c06b080 0x7f158c06d540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f15a4006fd0 tx=0x7f15a4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:58.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.018+0000 7f159dffb640 1 -- 172.21.15.16:0/3357578624 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15a81f0b60 con 0x7f15a8160890 2023-11-24T06:29:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.211+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15a80ced80 con 0x7f158c06b080 2023-11-24T06:29:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.234+0000 7f159dffb640 1 -- 172.21.15.16:0/3357578624 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286452 (secure 0 0 0) 0x7f15a400be20 con 0x7f158c06b080 2023-11-24T06:29:58.240 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:58.243 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.242+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f158c06b080 msgr2=0x7f158c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.242+0000 7f15ae55c640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f158c06b080 0x7f158c06d540 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f15a4006fd0 tx=0x7f15a4008040 comp rx=0 tx=0).stop 2023-11-24T06:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.242+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a8160890 msgr2=0x7f15a80b66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.242+0000 7f15ae55c640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a8160890 0x7f15a80b66e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f159800c430 tx=0x7f159800c8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.242+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 shutdown_connections 2023-11-24T06:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.242+0000 7f15ae55c640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f158c06b080 0x7f158c06d540 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.242+0000 7f15ae55c640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15a8160c70 0x7f15a806b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.242+0000 7f15ae55c640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15a8160890 0x7f15a80b66e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.243+0000 7f15ae55c640 1 --2- 172.21.15.16:0/3357578624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15a80c6be0 0x7f15a81f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.243+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 >> 172.21.15.16:0/3357578624 conn(0x7f15a80d3090 msgr2=0x7f15a80c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.243+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 shutdown_connections 2023-11-24T06:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.243+0000 7f15ae55c640 1 -- 172.21.15.16:0/3357578624 wait complete. 2023-11-24T06:29:58.265 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2023-11-24T06:29:58.164861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243676,"kb_used_data":6664,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731044,"statfs":{"total":773094113280,"available":772844589056,"internally_reserved":0,"allocated":6823936,"data_stored":3498748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002105"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.015038+0000","last_change":"2023-11-24T06:29:53.046793+0000","last_active":"2023-11-24T06:29:54.015038+0000","last_peered":"2023-11-24T06:29:54.015038+0000","last_clean":"2023-11-24T06:29:54.015038+0000","last_became_active":"2023-11-24T06:29:53.042557+0000","last_became_peered":"2023-11-24T06:29:53.042557+0000","last_unstale":"2023-11-24T06:29:54.015038+0000","last_undegraded":"2023-11-24T06:29:54.015038+0000","last_fullsized":"2023-11-24T06:29:54.015038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:02.210505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.018049+0000","last_change":"2023-11-24T06:29:53.040014+0000","last_active":"2023-11-24T06:29:54.018049+0000","last_peered":"2023-11-24T06:29:54.018049+0000","last_clean":"2023-11-24T06:29:54.018049+0000","last_became_active":"2023-11-24T06:29:53.039690+0000","last_became_peered":"2023-11-24T06:29:53.039690+0000","last_unstale":"2023-11-24T06:29:54.018049+0000","last_undegraded":"2023-11-24T06:29:54.018049+0000","last_fullsized":"2023-11-24T06:29:54.018049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:13.571691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016653+0000","last_change":"2023-11-24T06:29:53.036294+0000","last_active":"2023-11-24T06:29:54.016653+0000","last_peered":"2023-11-24T06:29:54.016653+0000","last_clean":"2023-11-24T06:29:54.016653+0000","last_became_active":"2023-11-24T06:29:53.035467+0000","last_became_peered":"2023-11-24T06:29:53.035467+0000","last_unstale":"2023-11-24T06:29:54.016653+0000","last_undegraded":"2023-11-24T06:29:54.016653+0000","last_fullsized":"2023-11-24T06:29:54.016653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:51.921598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053540+0000","last_change":"2023-11-24T06:29:53.049704+0000","last_active":"2023-11-24T06:29:53.053540+0000","last_peered":"2023-11-24T06:29:53.053540+0000","last_clean":"2023-11-24T06:29:53.053540+0000","last_became_active":"2023-11-24T06:29:53.049405+0000","last_became_peered":"2023-11-24T06:29:53.049405+0000","last_unstale":"2023-11-24T06:29:53.053540+0000","last_undegraded":"2023-11-24T06:29:53.053540+0000","last_fullsized":"2023-11-24T06:29:53.053540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:02.389910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.014079+0000","last_change":"2023-11-24T06:29:53.037591+0000","last_active":"2023-11-24T06:29:54.014079+0000","last_peered":"2023-11-24T06:29:54.014079+0000","last_clean":"2023-11-24T06:29:54.014079+0000","last_became_active":"2023-11-24T06:29:53.037251+0000","last_became_peered":"2023-11-24T06:29:53.037251+0000","last_unstale":"2023-11-24T06:29:54.014079+0000","last_undegraded":"2023-11-24T06:29:54.014079+0000","last_fullsized":"2023-11-24T06:29:54.014079+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:48.817335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.052862+0000","last_change":"2023-11-24T06:29:53.043815+0000","last_active":"2023-11-24T06:29:53.052862+0000","last_peered":"2023-11-24T06:29:53.052862+0000","last_clean":"2023-11-24T06:29:53.052862+0000","last_became_active":"2023-11-24T06:29:53.043488+0000","last_became_peered":"2023-11-24T06:29:53.043488+0000","last_unstale":"2023-11-24T06:29:53.052862+0000","last_undegraded":"2023-11-24T06:29:53.052862+0000","last_fullsized":"2023-11-24T06:29:53.052862+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:04:32.346084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051029+0000","last_change":"2023-11-24T06:29:53.046134+0000","last_active":"2023-11-24T06:29:53.051029+0000","last_peered":"2023-11-24T06:29:53.051029+0000","last_clean":"2023-11-24T06:29:53.051029+0000","last_became_active":"2023-11-24T06:29:53.045723+0000","last_became_peered":"2023-11-24T06:29:53.045723+0000","last_unstale":"2023-11-24T06:29:53.051029+0000","last_undegraded":"2023-11-24T06:29:53.051029+0000","last_fullsized":"2023-11-24T06:29:53.051029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:40.111154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.015963+0000","last_change":"2023-11-24T06:29:53.046696+0000","last_active":"2023-11-24T06:29:54.015963+0000","last_peered":"2023-11-24T06:29:54.015963+0000","last_clean":"2023-11-24T06:29:54.015963+0000","last_became_active":"2023-11-24T06:29:53.045921+0000","last_became_peered":"2023-11-24T06:29:53.045921+0000","last_unstale":"2023-11-24T06:29:54.015963+0000","last_undegraded":"2023-11-24T06:29:54.015963+0000","last_fullsized":"2023-11-24T06:29:54.015963+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:16.766027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.049219+0000","last_change":"2023-11-24T06:29:53.042523+0000","last_active":"2023-11-24T06:29:53.049219+0000","last_peered":"2023-11-24T06:29:53.049219+0000","last_clean":"2023-11-24T06:29:53.049219+0000","last_became_active":"2023-11-24T06:29:53.042246+0000","last_became_peered":"2023-11-24T06:29:53.042246+0000","last_unstale":"2023-11-24T06:29:53.049219+0000","last_undegraded":"2023-11-24T06:29:53.049219+0000","last_fullsized":"2023-11-24T06:29:53.049219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:05.258049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.045787+0000","last_change":"2023-11-24T06:29:53.038488+0000","last_active":"2023-11-24T06:29:53.045787+0000","last_peered":"2023-11-24T06:29:53.045787+0000","last_clean":"2023-11-24T06:29:53.045787+0000","last_became_active":"2023-11-24T06:29:53.038122+0000","last_became_peered":"2023-11-24T06:29:53.038122+0000","last_unstale":"2023-11-24T06:29:53.045787+0000","last_undegraded":"2023-11-24T06:29:53.045787+0000","last_fullsized":"2023-11-24T06:29:53.045787+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:53.958643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.050166+0000","last_change":"2023-11-24T06:29:53.044081+0000","last_active":"2023-11-24T06:29:53.050166+0000","last_peered":"2023-11-24T06:29:53.050166+0000","last_clean":"2023-11-24T06:29:53.050166+0000","last_became_active":"2023-11-24T06:29:53.043733+0000","last_became_peered":"2023-11-24T06:29:53.043733+0000","last_unstale":"2023-11-24T06:29:53.050166+0000","last_undegraded":"2023-11-24T06:29:53.050166+0000","last_fullsized":"2023-11-24T06:29:53.050166+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:13:54.758822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051441+0000","last_change":"2023-11-24T06:29:53.047989+0000","last_active":"2023-11-24T06:29:53.051441+0000","last_peered":"2023-11-24T06:29:53.051441+0000","last_clean":"2023-11-24T06:29:53.051441+0000","last_became_active":"2023-11-24T06:29:53.047632+0000","last_became_peered":"2023-11-24T06:29:53.047632+0000","last_unstale":"2023-11-24T06:29:53.051441+0000","last_undegraded":"2023-11-24T06:29:53.051441+0000","last_fullsized":"2023-11-24T06:29:53.051441+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:45.864860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.043661+0000","last_change":"2023-11-24T06:29:53.038620+0000","last_active":"2023-11-24T06:29:53.043661+0000","last_peered":"2023-11-24T06:29:53.043661+0000","last_clean":"2023-11-24T06:29:53.043661+0000","last_became_active":"2023-11-24T06:29:53.038279+0000","last_became_peered":"2023-11-24T06:29:53.038279+0000","last_unstale":"2023-11-24T06:29:53.043661+0000","last_undegraded":"2023-11-24T06:29:53.043661+0000","last_fullsized":"2023-11-24T06:29:53.043661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:17.383196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.049652+0000","last_change":"2023-11-24T06:29:53.043353+0000","last_active":"2023-11-24T06:29:53.049652+0000","last_peered":"2023-11-24T06:29:53.049652+0000","last_clean":"2023-11-24T06:29:53.049652+0000","last_became_active":"2023-11-24T06:29:53.043066+0000","last_became_peered":"2023-11-24T06:29:53.043066+0000","last_unstale":"2023-11-24T06:29:53.049652+0000","last_undegraded":"2023-11-24T06:29:53.049652+0000","last_fullsized":"2023-11-24T06:29:53.049652+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:14.238180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.048149+0000","last_change":"2023-11-24T06:29:53.040251+0000","last_active":"2023-11-24T06:29:53.048149+0000","last_peered":"2023-11-24T06:29:53.048149+0000","last_clean":"2023-11-24T06:29:53.048149+0000","last_became_active":"2023-11-24T06:29:53.039911+0000","last_became_peered":"2023-11-24T06:29:53.039911+0000","last_unstale":"2023-11-24T06:29:53.048149+0000","last_undegraded":"2023-11-24T06:29:53.048149+0000","last_fullsized":"2023-11-24T06:29:53.048149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:08:55.106042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.052458+0000","last_change":"2023-11-24T06:29:53.035847+0000","last_active":"2023-11-24T06:29:53.052458+0000","last_peered":"2023-11-24T06:29:53.052458+0000","last_clean":"2023-11-24T06:29:53.052458+0000","last_became_active":"2023-11-24T06:29:53.035556+0000","last_became_peered":"2023-11-24T06:29:53.035556+0000","last_unstale":"2023-11-24T06:29:53.052458+0000","last_undegraded":"2023-11-24T06:29:53.052458+0000","last_fullsized":"2023-11-24T06:29:53.052458+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:46.839257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017942+0000","last_change":"2023-11-24T06:29:53.042369+0000","last_active":"2023-11-24T06:29:54.017942+0000","last_peered":"2023-11-24T06:29:54.017942+0000","last_clean":"2023-11-24T06:29:54.017942+0000","last_became_active":"2023-11-24T06:29:53.042006+0000","last_became_peered":"2023-11-24T06:29:53.042006+0000","last_unstale":"2023-11-24T06:29:54.017942+0000","last_undegraded":"2023-11-24T06:29:54.017942+0000","last_fullsized":"2023-11-24T06:29:54.017942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:06:48.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016934+0000","last_change":"2023-11-24T06:29:53.044752+0000","last_active":"2023-11-24T06:29:54.016934+0000","last_peered":"2023-11-24T06:29:54.016934+0000","last_clean":"2023-11-24T06:29:54.016934+0000","last_became_active":"2023-11-24T06:29:53.042188+0000","last_became_peered":"2023-11-24T06:29:53.042188+0000","last_unstale":"2023-11-24T06:29:54.016934+0000","last_undegraded":"2023-11-24T06:29:54.016934+0000","last_fullsized":"2023-11-24T06:29:54.016934+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:50.660874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.013731+0000","last_change":"2023-11-24T06:29:53.044120+0000","last_active":"2023-11-24T06:29:54.013731+0000","last_peered":"2023-11-24T06:29:54.013731+0000","last_clean":"2023-11-24T06:29:54.013731+0000","last_became_active":"2023-11-24T06:29:53.043786+0000","last_became_peered":"2023-11-24T06:29:53.043786+0000","last_unstale":"2023-11-24T06:29:54.013731+0000","last_undegraded":"2023-11-24T06:29:54.013731+0000","last_fullsized":"2023-11-24T06:29:54.013731+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:19.348209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017106+0000","last_change":"2023-11-24T06:29:53.050639+0000","last_active":"2023-11-24T06:29:54.017106+0000","last_peered":"2023-11-24T06:29:54.017106+0000","last_clean":"2023-11-24T06:29:54.017106+0000","last_became_active":"2023-11-24T06:29:53.048846+0000","last_became_peered":"2023-11-24T06:29:53.048846+0000","last_unstale":"2023-11-24T06:29:54.017106+0000","last_undegraded":"2023-11-24T06:29:54.017106+0000","last_fullsized":"2023-11-24T06:29:54.017106+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:30.452190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016973+0000","last_change":"2023-11-24T06:29:53.037352+0000","last_active":"2023-11-24T06:29:54.016973+0000","last_peered":"2023-11-24T06:29:54.016973+0000","last_clean":"2023-11-24T06:29:54.016973+0000","last_became_active":"2023-11-24T06:29:53.036524+0000","last_became_peered":"2023-11-24T06:29:53.036524+0000","last_unstale":"2023-11-24T06:29:54.016973+0000","last_undegraded":"2023-11-24T06:29:54.016973+0000","last_fullsized":"2023-11-24T06:29:54.016973+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:06.524237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017050+0000","last_change":"2023-11-24T06:29:53.046624+0000","last_active":"2023-11-24T06:29:54.017050+0000","last_peered":"2023-11-24T06:29:54.017050+0000","last_clean":"2023-11-24T06:29:54.017050+0000","last_became_active":"2023-11-24T06:29:53.046396+0000","last_became_peered":"2023-11-24T06:29:53.046396+0000","last_unstale":"2023-11-24T06:29:54.017050+0000","last_undegraded":"2023-11-24T06:29:54.017050+0000","last_fullsized":"2023-11-24T06:29:54.017050+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:23:25.711085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016205+0000","last_change":"2023-11-24T06:29:53.048944+0000","last_active":"2023-11-24T06:29:54.016205+0000","last_peered":"2023-11-24T06:29:54.016205+0000","last_clean":"2023-11-24T06:29:54.016205+0000","last_became_active":"2023-11-24T06:29:53.048577+0000","last_became_peered":"2023-11-24T06:29:53.048577+0000","last_unstale":"2023-11-24T06:29:54.016205+0000","last_undegraded":"2023-11-24T06:29:54.016205+0000","last_fullsized":"2023-11-24T06:29:54.016205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:24:50.830782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.046010+0000","last_change":"2023-11-24T06:29:53.039295+0000","last_active":"2023-11-24T06:29:53.046010+0000","last_peered":"2023-11-24T06:29:53.046010+0000","last_clean":"2023-11-24T06:29:53.046010+0000","last_became_active":"2023-11-24T06:29:53.037574+0000","last_became_peered":"2023-11-24T06:29:53.037574+0000","last_unstale":"2023-11-24T06:29:53.046010+0000","last_undegraded":"2023-11-24T06:29:53.046010+0000","last_fullsized":"2023-11-24T06:29:53.046010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:22:38.460485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.015417+0000","last_change":"2023-11-24T06:29:53.034475+0000","last_active":"2023-11-24T06:29:54.015417+0000","last_peered":"2023-11-24T06:29:54.015417+0000","last_clean":"2023-11-24T06:29:54.015417+0000","last_became_active":"2023-11-24T06:29:53.034154+0000","last_became_peered":"2023-11-24T06:29:53.034154+0000","last_unstale":"2023-11-24T06:29:54.015417+0000","last_undegraded":"2023-11-24T06:29:54.015417+0000","last_fullsized":"2023-11-24T06:29:54.015417+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:52.569269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053565+0000","last_change":"2023-11-24T06:29:53.049910+0000","last_active":"2023-11-24T06:29:53.053565+0000","last_peered":"2023-11-24T06:29:53.053565+0000","last_clean":"2023-11-24T06:29:53.053565+0000","last_became_active":"2023-11-24T06:29:53.049577+0000","last_became_peered":"2023-11-24T06:29:53.049577+0000","last_unstale":"2023-11-24T06:29:53.053565+0000","last_undegraded":"2023-11-24T06:29:53.053565+0000","last_fullsized":"2023-11-24T06:29:53.053565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:16:49.009307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053724+0000","last_change":"2023-11-24T06:29:53.052592+0000","last_active":"2023-11-24T06:29:53.053724+0000","last_peered":"2023-11-24T06:29:53.053724+0000","last_clean":"2023-11-24T06:29:53.053724+0000","last_became_active":"2023-11-24T06:29:53.052246+0000","last_became_peered":"2023-11-24T06:29:53.052246+0000","last_unstale":"2023-11-24T06:29:53.053724+0000","last_undegraded":"2023-11-24T06:29:53.053724+0000","last_fullsized":"2023-11-24T06:29:53.053724+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:04.370530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.014655+0000","last_change":"2023-11-24T06:29:53.038466+0000","last_active":"2023-11-24T06:29:54.014655+0000","last_peered":"2023-11-24T06:29:54.014655+0000","last_clean":"2023-11-24T06:29:54.014655+0000","last_became_active":"2023-11-24T06:29:53.038188+0000","last_became_peered":"2023-11-24T06:29:53.038188+0000","last_unstale":"2023-11-24T06:29:54.014655+0000","last_undegraded":"2023-11-24T06:29:54.014655+0000","last_fullsized":"2023-11-24T06:29:54.014655+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:55.314608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017399+0000","last_change":"2023-11-24T06:29:53.047642+0000","last_active":"2023-11-24T06:29:54.017399+0000","last_peered":"2023-11-24T06:29:54.017399+0000","last_clean":"2023-11-24T06:29:54.017399+0000","last_became_active":"2023-11-24T06:29:53.046864+0000","last_became_peered":"2023-11-24T06:29:53.046864+0000","last_unstale":"2023-11-24T06:29:54.017399+0000","last_undegraded":"2023-11-24T06:29:54.017399+0000","last_fullsized":"2023-11-24T06:29:54.017399+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:25.572027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016734+0000","last_change":"2023-11-24T06:29:53.037294+0000","last_active":"2023-11-24T06:29:54.016734+0000","last_peered":"2023-11-24T06:29:54.016734+0000","last_clean":"2023-11-24T06:29:54.016734+0000","last_became_active":"2023-11-24T06:29:53.035756+0000","last_became_peered":"2023-11-24T06:29:53.035756+0000","last_unstale":"2023-11-24T06:29:54.016734+0000","last_undegraded":"2023-11-24T06:29:54.016734+0000","last_fullsized":"2023-11-24T06:29:54.016734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:41.311201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.013225+0000","last_change":"2023-11-24T06:29:53.034068+0000","last_active":"2023-11-24T06:29:54.013225+0000","last_peered":"2023-11-24T06:29:54.013225+0000","last_clean":"2023-11-24T06:29:54.013225+0000","last_became_active":"2023-11-24T06:29:53.033563+0000","last_became_peered":"2023-11-24T06:29:53.033563+0000","last_unstale":"2023-11-24T06:29:54.013225+0000","last_undegraded":"2023-11-24T06:29:54.013225+0000","last_fullsized":"2023-11-24T06:29:54.013225+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:23.796861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053150+0000","last_change":"2023-11-24T06:29:53.046246+0000","last_active":"2023-11-24T06:29:53.053150+0000","last_peered":"2023-11-24T06:29:53.053150+0000","last_clean":"2023-11-24T06:29:53.053150+0000","last_became_active":"2023-11-24T06:29:53.045979+0000","last_became_peered":"2023-11-24T06:29:53.045979+0000","last_unstale":"2023-11-24T06:29:53.053150+0000","last_undegraded":"2023-11-24T06:29:53.053150+0000","last_fullsized":"2023-11-24T06:29:53.053150+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:37.548739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053431+0000","last_change":"2023-11-24T06:29:53.052059+0000","last_active":"2023-11-24T06:29:53.053431+0000","last_peered":"2023-11-24T06:29:53.053431+0000","last_clean":"2023-11-24T06:29:53.053431+0000","last_became_active":"2023-11-24T06:29:53.051735+0000","last_became_peered":"2023-11-24T06:29:53.051735+0000","last_unstale":"2023-11-24T06:29:53.053431+0000","last_undegraded":"2023-11-24T06:29:53.053431+0000","last_fullsized":"2023-11-24T06:29:53.053431+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:41.311201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.230833+0000","last_change":"2023-11-24T06:29:52.216329+0000","last_active":"2023-11-24T06:29:53.230833+0000","last_peered":"2023-11-24T06:29:53.230833+0000","last_clean":"2023-11-24T06:29:53.230833+0000","last_became_active":"2023-11-24T06:29:52.215999+0000","last_became_peered":"2023-11-24T06:29:52.215999+0000","last_unstale":"2023-11-24T06:29:53.230833+0000","last_undegraded":"2023-11-24T06:29:53.230833+0000","last_fullsized":"2023-11-24T06:29:53.230833+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:11.511440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.045657+0000","last_change":"2023-11-24T06:29:53.039140+0000","last_active":"2023-11-24T06:29:53.045657+0000","last_peered":"2023-11-24T06:29:53.045657+0000","last_clean":"2023-11-24T06:29:53.045657+0000","last_became_active":"2023-11-24T06:29:53.038799+0000","last_became_peered":"2023-11-24T06:29:53.038799+0000","last_unstale":"2023-11-24T06:29:53.045657+0000","last_undegraded":"2023-11-24T06:29:53.045657+0000","last_fullsized":"2023-11-24T06:29:53.045657+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:45.375004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017729+0000","last_change":"2023-11-24T06:29:52.208760+0000","last_active":"2023-11-24T06:29:54.017729+0000","last_peered":"2023-11-24T06:29:54.017729+0000","last_clean":"2023-11-24T06:29:54.017729+0000","last_became_active":"2023-11-24T06:29:52.208452+0000","last_became_peered":"2023-11-24T06:29:52.208452+0000","last_unstale":"2023-11-24T06:29:54.017729+0000","last_undegraded":"2023-11-24T06:29:54.017729+0000","last_fullsized":"2023-11-24T06:29:54.017729+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:37.588360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.052881+0000","last_change":"2023-11-24T06:29:53.044787+0000","last_active":"2023-11-24T06:29:53.052881+0000","last_peered":"2023-11-24T06:29:53.052881+0000","last_clean":"2023-11-24T06:29:53.052881+0000","last_became_active":"2023-11-24T06:29:53.039939+0000","last_became_peered":"2023-11-24T06:29:53.039939+0000","last_unstale":"2023-11-24T06:29:53.052881+0000","last_undegraded":"2023-11-24T06:29:53.052881+0000","last_fullsized":"2023-11-24T06:29:53.052881+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:24:50.830782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.220944+0000","last_change":"2023-11-24T06:29:52.214623+0000","last_active":"2023-11-24T06:29:53.220944+0000","last_peered":"2023-11-24T06:29:53.220944+0000","last_clean":"2023-11-24T06:29:53.220944+0000","last_became_active":"2023-11-24T06:29:52.214328+0000","last_became_peered":"2023-11-24T06:29:52.214328+0000","last_unstale":"2023-11-24T06:29:53.220944+0000","last_undegraded":"2023-11-24T06:29:53.220944+0000","last_fullsized":"2023-11-24T06:29:53.220944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:07.906349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053082+0000","last_change":"2023-11-24T06:29:53.045869+0000","last_active":"2023-11-24T06:29:53.053082+0000","last_peered":"2023-11-24T06:29:53.053082+0000","last_clean":"2023-11-24T06:29:53.053082+0000","last_became_active":"2023-11-24T06:29:53.045330+0000","last_became_peered":"2023-11-24T06:29:53.045330+0000","last_unstale":"2023-11-24T06:29:53.053082+0000","last_undegraded":"2023-11-24T06:29:53.053082+0000","last_fullsized":"2023-11-24T06:29:53.053082+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:48:40.169825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.227585+0000","last_change":"2023-11-24T06:29:52.215836+0000","last_active":"2023-11-24T06:29:53.227585+0000","last_peered":"2023-11-24T06:29:53.227585+0000","last_clean":"2023-11-24T06:29:53.227585+0000","last_became_active":"2023-11-24T06:29:52.214260+0000","last_became_peered":"2023-11-24T06:29:52.214260+0000","last_unstale":"2023-11-24T06:29:53.227585+0000","last_undegraded":"2023-11-24T06:29:53.227585+0000","last_fullsized":"2023-11-24T06:29:53.227585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:11.511440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017097+0000","last_change":"2023-11-24T06:29:53.042286+0000","last_active":"2023-11-24T06:29:54.017097+0000","last_peered":"2023-11-24T06:29:54.017097+0000","last_clean":"2023-11-24T06:29:54.017097+0000","last_became_active":"2023-11-24T06:29:53.041991+0000","last_became_peered":"2023-11-24T06:29:53.041991+0000","last_unstale":"2023-11-24T06:29:54.017097+0000","last_undegraded":"2023-11-24T06:29:54.017097+0000","last_fullsized":"2023-11-24T06:29:54.017097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:53.629136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.220811+0000","last_change":"2023-11-24T06:29:52.214566+0000","last_active":"2023-11-24T06:29:53.220811+0000","last_peered":"2023-11-24T06:29:53.220811+0000","last_clean":"2023-11-24T06:29:53.220811+0000","last_became_active":"2023-11-24T06:29:52.214269+0000","last_became_peered":"2023-11-24T06:29:52.214269+0000","last_unstale":"2023-11-24T06:29:53.220811+0000","last_undegraded":"2023-11-24T06:29:53.220811+0000","last_fullsized":"2023-11-24T06:29:53.220811+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:43.465119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017248+0000","last_change":"2023-11-24T06:29:53.040441+0000","last_active":"2023-11-24T06:29:54.017248+0000","last_peered":"2023-11-24T06:29:54.017248+0000","last_clean":"2023-11-24T06:29:54.017248+0000","last_became_active":"2023-11-24T06:29:53.040066+0000","last_became_peered":"2023-11-24T06:29:53.040066+0000","last_unstale":"2023-11-24T06:29:54.017248+0000","last_undegraded":"2023-11-24T06:29:54.017248+0000","last_fullsized":"2023-11-24T06:29:54.017248+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:05.887029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.223721+0000","last_change":"2023-11-24T06:29:52.217828+0000","last_active":"2023-11-24T06:29:53.223721+0000","last_peered":"2023-11-24T06:29:53.223721+0000","last_clean":"2023-11-24T06:29:53.223721+0000","last_became_active":"2023-11-24T06:29:52.217458+0000","last_became_peered":"2023-11-24T06:29:52.217458+0000","last_unstale":"2023-11-24T06:29:53.223721+0000","last_undegraded":"2023-11-24T06:29:53.223721+0000","last_fullsized":"2023-11-24T06:29:53.223721+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:58:46.276511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.043648+0000","last_change":"2023-11-24T06:29:53.034170+0000","last_active":"2023-11-24T06:29:53.043648+0000","last_peered":"2023-11-24T06:29:53.043648+0000","last_clean":"2023-11-24T06:29:53.043648+0000","last_became_active":"2023-11-24T06:29:53.033402+0000","last_became_peered":"2023-11-24T06:29:53.033402+0000","last_unstale":"2023-11-24T06:29:53.043648+0000","last_undegraded":"2023-11-24T06:29:53.043648+0000","last_fullsized":"2023-11-24T06:29:53.043648+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:37:09.168419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.036802+0000","last_change":"2023-11-24T06:29:52.213281+0000","last_active":"2023-11-24T06:29:53.036802+0000","last_peered":"2023-11-24T06:29:53.036802+0000","last_clean":"2023-11-24T06:29:53.036802+0000","last_became_active":"2023-11-24T06:29:52.210667+0000","last_became_peered":"2023-11-24T06:29:52.210667+0000","last_unstale":"2023-11-24T06:29:53.036802+0000","last_undegraded":"2023-11-24T06:29:53.036802+0000","last_fullsized":"2023-11-24T06:29:53.036802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:34:11.416727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016467+0000","last_change":"2023-11-24T06:29:53.039889+0000","last_active":"2023-11-24T06:29:54.016467+0000","last_peered":"2023-11-24T06:29:54.016467+0000","last_clean":"2023-11-24T06:29:54.016467+0000","last_became_active":"2023-11-24T06:29:53.039385+0000","last_became_peered":"2023-11-24T06:29:53.039385+0000","last_unstale":"2023-11-24T06:29:54.016467+0000","last_undegraded":"2023-11-24T06:29:54.016467+0000","last_fullsized":"2023-11-24T06:29:54.016467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:59:58.960040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.028958+0000","last_change":"2023-11-24T06:29:52.212440+0000","last_active":"2023-11-24T06:29:53.028958+0000","last_peered":"2023-11-24T06:29:53.028958+0000","last_clean":"2023-11-24T06:29:53.028958+0000","last_became_active":"2023-11-24T06:29:52.212149+0000","last_became_peered":"2023-11-24T06:29:52.212149+0000","last_unstale":"2023-11-24T06:29:53.028958+0000","last_undegraded":"2023-11-24T06:29:53.028958+0000","last_fullsized":"2023-11-24T06:29:53.028958+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:57.138021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.048550+0000","last_change":"2023-11-24T06:29:53.040815+0000","last_active":"2023-11-24T06:29:53.048550+0000","last_peered":"2023-11-24T06:29:53.048550+0000","last_clean":"2023-11-24T06:29:53.048550+0000","last_became_active":"2023-11-24T06:29:53.040451+0000","last_became_peered":"2023-11-24T06:29:53.040451+0000","last_unstale":"2023-11-24T06:29:53.048550+0000","last_undegraded":"2023-11-24T06:29:53.048550+0000","last_fullsized":"2023-11-24T06:29:53.048550+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:22:38.460485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016782+0000","last_change":"2023-11-24T06:29:52.211167+0000","last_active":"2023-11-24T06:29:54.016782+0000","last_peered":"2023-11-24T06:29:54.016782+0000","last_clean":"2023-11-24T06:29:54.016782+0000","last_became_active":"2023-11-24T06:29:52.210900+0000","last_became_peered":"2023-11-24T06:29:52.210900+0000","last_unstale":"2023-11-24T06:29:54.016782+0000","last_undegraded":"2023-11-24T06:29:54.016782+0000","last_fullsized":"2023-11-24T06:29:54.016782+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:04:31.332577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016831+0000","last_change":"2023-11-24T06:29:53.041045+0000","last_active":"2023-11-24T06:29:54.016831+0000","last_peered":"2023-11-24T06:29:54.016831+0000","last_clean":"2023-11-24T06:29:54.016831+0000","last_became_active":"2023-11-24T06:29:53.040725+0000","last_became_peered":"2023-11-24T06:29:53.040725+0000","last_unstale":"2023-11-24T06:29:54.016831+0000","last_undegraded":"2023-11-24T06:29:54.016831+0000","last_fullsized":"2023-11-24T06:29:54.016831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:08.919856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017822+0000","last_change":"2023-11-24T06:29:52.206226+0000","last_active":"2023-11-24T06:29:54.017822+0000","last_peered":"2023-11-24T06:29:54.017822+0000","last_clean":"2023-11-24T06:29:54.017822+0000","last_became_active":"2023-11-24T06:29:52.205678+0000","last_became_peered":"2023-11-24T06:29:52.205678+0000","last_unstale":"2023-11-24T06:29:54.017822+0000","last_undegraded":"2023-11-24T06:29:54.017822+0000","last_fullsized":"2023-11-24T06:29:54.017822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:11.492077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.046412+0000","last_change":"2023-11-24T06:29:53.039401+0000","last_active":"2023-11-24T06:29:53.046412+0000","last_peered":"2023-11-24T06:29:53.046412+0000","last_clean":"2023-11-24T06:29:53.046412+0000","last_became_active":"2023-11-24T06:29:53.039082+0000","last_became_peered":"2023-11-24T06:29:53.039082+0000","last_unstale":"2023-11-24T06:29:53.046412+0000","last_undegraded":"2023-11-24T06:29:53.046412+0000","last_fullsized":"2023-11-24T06:29:53.046412+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:19:45.049987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016955+0000","last_change":"2023-11-24T06:29:52.204934+0000","last_active":"2023-11-24T06:29:54.016955+0000","last_peered":"2023-11-24T06:29:54.016955+0000","last_clean":"2023-11-24T06:29:54.016955+0000","last_became_active":"2023-11-24T06:29:52.204625+0000","last_became_peered":"2023-11-24T06:29:52.204625+0000","last_unstale":"2023-11-24T06:29:54.016955+0000","last_undegraded":"2023-11-24T06:29:54.016955+0000","last_fullsized":"2023-11-24T06:29:54.016955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:36.535231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016660+0000","last_change":"2023-11-24T06:29:53.040364+0000","last_active":"2023-11-24T06:29:54.016660+0000","last_peered":"2023-11-24T06:29:54.016660+0000","last_clean":"2023-11-24T06:29:54.016660+0000","last_became_active":"2023-11-24T06:29:53.039736+0000","last_became_peered":"2023-11-24T06:29:53.039736+0000","last_unstale":"2023-11-24T06:29:54.016660+0000","last_undegraded":"2023-11-24T06:29:54.016660+0000","last_fullsized":"2023-11-24T06:29:54.016660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:04.370530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.234436+0000","last_change":"2023-11-24T06:29:52.213776+0000","last_active":"2023-11-24T06:29:53.234436+0000","last_peered":"2023-11-24T06:29:53.234436+0000","last_clean":"2023-11-24T06:29:53.234436+0000","last_became_active":"2023-11-24T06:29:52.213484+0000","last_became_peered":"2023-11-24T06:29:52.213484+0000","last_unstale":"2023-11-24T06:29:53.234436+0000","last_undegraded":"2023-11-24T06:29:53.234436+0000","last_fullsized":"2023-11-24T06:29:53.234436+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:39.365789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051409+0000","last_change":"2023-11-24T06:29:53.047505+0000","last_active":"2023-11-24T06:29:53.051409+0000","last_peered":"2023-11-24T06:29:53.051409+0000","last_clean":"2023-11-24T06:29:53.051409+0000","last_became_active":"2023-11-24T06:29:53.044791+0000","last_became_peered":"2023-11-24T06:29:53.044791+0000","last_unstale":"2023-11-24T06:29:53.051409+0000","last_undegraded":"2023-11-24T06:29:53.051409+0000","last_fullsized":"2023-11-24T06:29:53.051409+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:14.238180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017330+0000","last_change":"2023-11-24T06:29:52.205001+0000","last_active":"2023-11-24T06:29:54.017330+0000","last_peered":"2023-11-24T06:29:54.017330+0000","last_clean":"2023-11-24T06:29:54.017330+0000","last_became_active":"2023-11-24T06:29:52.204804+0000","last_became_peered":"2023-11-24T06:29:52.204804+0000","last_unstale":"2023-11-24T06:29:54.017330+0000","last_undegraded":"2023-11-24T06:29:54.017330+0000","last_fullsized":"2023-11-24T06:29:54.017330+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:18.913373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017306+0000","last_change":"2023-11-24T06:29:53.038626+0000","last_active":"2023-11-24T06:29:54.017306+0000","last_peered":"2023-11-24T06:29:54.017306+0000","last_clean":"2023-11-24T06:29:54.017306+0000","last_became_active":"2023-11-24T06:29:53.038334+0000","last_became_peered":"2023-11-24T06:29:53.038334+0000","last_unstale":"2023-11-24T06:29:54.017306+0000","last_undegraded":"2023-11-24T06:29:54.017306+0000","last_fullsized":"2023-11-24T06:29:54.017306+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:38.601868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.017786+0000","last_change":"2023-11-24T06:29:52.212030+0000","last_active":"2023-11-24T06:29:53.017786+0000","last_peered":"2023-11-24T06:29:53.017786+0000","last_clean":"2023-11-24T06:29:53.017786+0000","last_became_active":"2023-11-24T06:29:52.211740+0000","last_became_peered":"2023-11-24T06:29:52.211740+0000","last_unstale":"2023-11-24T06:29:53.017786+0000","last_undegraded":"2023-11-24T06:29:53.017786+0000","last_fullsized":"2023-11-24T06:29:53.017786+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:08:54.092535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.047896+0000","last_change":"2023-11-24T06:29:53.039526+0000","last_active":"2023-11-24T06:29:53.047896+0000","last_peered":"2023-11-24T06:29:53.047896+0000","last_clean":"2023-11-24T06:29:53.047896+0000","last_became_active":"2023-11-24T06:29:53.039282+0000","last_became_peered":"2023-11-24T06:29:53.039282+0000","last_unstale":"2023-11-24T06:29:53.047896+0000","last_undegraded":"2023-11-24T06:29:53.047896+0000","last_fullsized":"2023-11-24T06:29:53.047896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:17.383196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.036820+0000","last_change":"2023-11-24T06:29:52.208985+0000","last_active":"2023-11-24T06:29:53.036820+0000","last_peered":"2023-11-24T06:29:53.036820+0000","last_clean":"2023-11-24T06:29:53.036820+0000","last_became_active":"2023-11-24T06:29:52.208478+0000","last_became_peered":"2023-11-24T06:29:52.208478+0000","last_unstale":"2023-11-24T06:29:53.036820+0000","last_undegraded":"2023-11-24T06:29:53.036820+0000","last_fullsized":"2023-11-24T06:29:53.036820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:41.427319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016565+0000","last_change":"2023-11-24T06:29:53.045873+0000","last_active":"2023-11-24T06:29:54.016565+0000","last_peered":"2023-11-24T06:29:54.016565+0000","last_clean":"2023-11-24T06:29:54.016565+0000","last_became_active":"2023-11-24T06:29:53.045549+0000","last_became_peered":"2023-11-24T06:29:53.045549+0000","last_unstale":"2023-11-24T06:29:54.016565+0000","last_undegraded":"2023-11-24T06:29:54.016565+0000","last_fullsized":"2023-11-24T06:29:54.016565+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:48:40.169825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016095+0000","last_change":"2023-11-24T06:29:52.206665+0000","last_active":"2023-11-24T06:29:54.016095+0000","last_peered":"2023-11-24T06:29:54.016095+0000","last_clean":"2023-11-24T06:29:54.016095+0000","last_became_active":"2023-11-24T06:29:52.206385+0000","last_became_peered":"2023-11-24T06:29:52.206385+0000","last_unstale":"2023-11-24T06:29:54.016095+0000","last_undegraded":"2023-11-24T06:29:54.016095+0000","last_fullsized":"2023-11-24T06:29:54.016095+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:23:34.198375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017208+0000","last_change":"2023-11-24T06:29:53.036064+0000","last_active":"2023-11-24T06:29:54.017208+0000","last_peered":"2023-11-24T06:29:54.017208+0000","last_clean":"2023-11-24T06:29:54.017208+0000","last_became_active":"2023-11-24T06:29:53.035634+0000","last_became_peered":"2023-11-24T06:29:53.035634+0000","last_unstale":"2023-11-24T06:29:54.017208+0000","last_undegraded":"2023-11-24T06:29:54.017208+0000","last_fullsized":"2023-11-24T06:29:54.017208+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:11.855717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.018125+0000","last_change":"2023-11-24T06:29:52.201939+0000","last_active":"2023-11-24T06:29:54.018125+0000","last_peered":"2023-11-24T06:29:54.018125+0000","last_clean":"2023-11-24T06:29:54.018125+0000","last_became_active":"2023-11-24T06:29:52.201661+0000","last_became_peered":"2023-11-24T06:29:52.201661+0000","last_unstale":"2023-11-24T06:29:54.018125+0000","last_undegraded":"2023-11-24T06:29:54.018125+0000","last_fullsized":"2023-11-24T06:29:54.018125+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:07.675887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.048080+0000","last_change":"2023-11-24T06:29:53.043592+0000","last_active":"2023-11-24T06:29:53.048080+0000","last_peered":"2023-11-24T06:29:53.048080+0000","last_clean":"2023-11-24T06:29:53.048080+0000","last_became_active":"2023-11-24T06:29:53.043335+0000","last_became_peered":"2023-11-24T06:29:53.043335+0000","last_unstale":"2023-11-24T06:29:53.048080+0000","last_undegraded":"2023-11-24T06:29:53.048080+0000","last_fullsized":"2023-11-24T06:29:53.048080+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:45.394544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.221249+0000","last_change":"2023-11-24T06:29:52.211084+0000","last_active":"2023-11-24T06:29:53.221249+0000","last_peered":"2023-11-24T06:29:53.221249+0000","last_clean":"2023-11-24T06:29:53.221249+0000","last_became_active":"2023-11-24T06:29:52.207269+0000","last_became_peered":"2023-11-24T06:29:52.207269+0000","last_unstale":"2023-11-24T06:29:53.221249+0000","last_undegraded":"2023-11-24T06:29:53.221249+0000","last_fullsized":"2023-11-24T06:29:53.221249+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:49.647367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017424+0000","last_change":"2023-11-24T06:29:53.040165+0000","last_active":"2023-11-24T06:29:54.017424+0000","last_peered":"2023-11-24T06:29:54.017424+0000","last_clean":"2023-11-24T06:29:54.017424+0000","last_became_active":"2023-11-24T06:29:53.037665+0000","last_became_peered":"2023-11-24T06:29:53.037665+0000","last_unstale":"2023-11-24T06:29:54.017424+0000","last_undegraded":"2023-11-24T06:29:54.017424+0000","last_fullsized":"2023-11-24T06:29:54.017424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:12.433603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.229382+0000","last_change":"2023-11-24T06:29:52.215444+0000","last_active":"2023-11-24T06:29:53.229382+0000","last_peered":"2023-11-24T06:29:53.229382+0000","last_clean":"2023-11-24T06:29:53.229382+0000","last_became_active":"2023-11-24T06:29:52.213594+0000","last_became_peered":"2023-11-24T06:29:52.213594+0000","last_unstale":"2023-11-24T06:29:53.229382+0000","last_undegraded":"2023-11-24T06:29:53.229382+0000","last_fullsized":"2023-11-24T06:29:53.229382+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:44.488029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017398+0000","last_change":"2023-11-24T06:29:53.040048+0000","last_active":"2023-11-24T06:29:54.017398+0000","last_peered":"2023-11-24T06:29:54.017398+0000","last_clean":"2023-11-24T06:29:54.017398+0000","last_became_active":"2023-11-24T06:29:53.039299+0000","last_became_peered":"2023-11-24T06:29:53.039299+0000","last_unstale":"2023-11-24T06:29:54.017398+0000","last_undegraded":"2023-11-24T06:29:54.017398+0000","last_fullsized":"2023-11-24T06:29:54.017398+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:12.505585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.031583+0000","last_change":"2023-11-24T06:29:52.215132+0000","last_active":"2023-11-24T06:29:53.031583+0000","last_peered":"2023-11-24T06:29:53.031583+0000","last_clean":"2023-11-24T06:29:53.031583+0000","last_became_active":"2023-11-24T06:29:52.212194+0000","last_became_peered":"2023-11-24T06:29:52.212194+0000","last_unstale":"2023-11-24T06:29:53.031583+0000","last_undegraded":"2023-11-24T06:29:53.031583+0000","last_fullsized":"2023-11-24T06:29:53.031583+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:50.908090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.047753+0000","last_change":"2023-11-24T06:29:53.040431+0000","last_active":"2023-11-24T06:29:53.047753+0000","last_peered":"2023-11-24T06:29:53.047753+0000","last_clean":"2023-11-24T06:29:53.047753+0000","last_became_active":"2023-11-24T06:29:53.040085+0000","last_became_peered":"2023-11-24T06:29:53.040085+0000","last_unstale":"2023-11-24T06:29:53.047753+0000","last_undegraded":"2023-11-24T06:29:53.047753+0000","last_fullsized":"2023-11-24T06:29:53.047753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:05.258049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017576+0000","last_change":"2023-11-24T06:29:52.211045+0000","last_active":"2023-11-24T06:29:54.017576+0000","last_peered":"2023-11-24T06:29:54.017576+0000","last_clean":"2023-11-24T06:29:54.017576+0000","last_became_active":"2023-11-24T06:29:52.210783+0000","last_became_peered":"2023-11-24T06:29:52.210783+0000","last_unstale":"2023-11-24T06:29:54.017576+0000","last_undegraded":"2023-11-24T06:29:54.017576+0000","last_fullsized":"2023-11-24T06:29:54.017576+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:08.839177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053346+0000","last_change":"2023-11-24T06:29:53.051186+0000","last_active":"2023-11-24T06:29:53.053346+0000","last_peered":"2023-11-24T06:29:53.053346+0000","last_clean":"2023-11-24T06:29:53.053346+0000","last_became_active":"2023-11-24T06:29:53.049439+0000","last_became_peered":"2023-11-24T06:29:53.049439+0000","last_unstale":"2023-11-24T06:29:53.053346+0000","last_undegraded":"2023-11-24T06:29:53.053346+0000","last_fullsized":"2023-11-24T06:29:53.053346+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:40:48.242120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.020785+0000","last_change":"2023-11-24T06:29:52.202046+0000","last_active":"2023-11-24T06:29:53.020785+0000","last_peered":"2023-11-24T06:29:53.020785+0000","last_clean":"2023-11-24T06:29:53.020785+0000","last_became_active":"2023-11-24T06:29:52.201750+0000","last_became_peered":"2023-11-24T06:29:52.201750+0000","last_unstale":"2023-11-24T06:29:53.020785+0000","last_undegraded":"2023-11-24T06:29:53.020785+0000","last_fullsized":"2023-11-24T06:29:53.020785+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:45.954019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.052660+0000","last_change":"2023-11-24T06:29:53.038612+0000","last_active":"2023-11-24T06:29:53.052660+0000","last_peered":"2023-11-24T06:29:53.052660+0000","last_clean":"2023-11-24T06:29:53.052660+0000","last_became_active":"2023-11-24T06:29:53.038309+0000","last_became_peered":"2023-11-24T06:29:53.038309+0000","last_unstale":"2023-11-24T06:29:53.052660+0000","last_undegraded":"2023-11-24T06:29:53.052660+0000","last_fullsized":"2023-11-24T06:29:53.052660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:45.501536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.037094+0000","last_change":"2023-11-24T06:29:52.212924+0000","last_active":"2023-11-24T06:29:53.037094+0000","last_peered":"2023-11-24T06:29:53.037094+0000","last_clean":"2023-11-24T06:29:53.037094+0000","last_became_active":"2023-11-24T06:29:52.212374+0000","last_became_peered":"2023-11-24T06:29:52.212374+0000","last_unstale":"2023-11-24T06:29:53.037094+0000","last_undegraded":"2023-11-24T06:29:53.037094+0000","last_fullsized":"2023-11-24T06:29:53.037094+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:36.535231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051532+0000","last_change":"2023-11-24T06:29:53.049139+0000","last_active":"2023-11-24T06:29:53.051532+0000","last_peered":"2023-11-24T06:29:53.051532+0000","last_clean":"2023-11-24T06:29:53.051532+0000","last_became_active":"2023-11-24T06:29:53.048815+0000","last_became_peered":"2023-11-24T06:29:53.048815+0000","last_unstale":"2023-11-24T06:29:53.051532+0000","last_undegraded":"2023-11-24T06:29:53.051532+0000","last_fullsized":"2023-11-24T06:29:53.051532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:42.041698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.220194+0000","last_change":"2023-11-24T06:29:52.197996+0000","last_active":"2023-11-24T06:29:53.220194+0000","last_peered":"2023-11-24T06:29:53.220194+0000","last_clean":"2023-11-24T06:29:53.220194+0000","last_became_active":"2023-11-24T06:29:52.197655+0000","last_became_peered":"2023-11-24T06:29:52.197655+0000","last_unstale":"2023-11-24T06:29:53.220194+0000","last_undegraded":"2023-11-24T06:29:53.220194+0000","last_fullsized":"2023-11-24T06:29:53.220194+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:04.696421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.051428+0000","last_change":"2023-11-24T06:29:53.047961+0000","last_active":"2023-11-24T06:29:53.051428+0000","last_peered":"2023-11-24T06:29:53.051428+0000","last_clean":"2023-11-24T06:29:53.051428+0000","last_became_active":"2023-11-24T06:29:53.046978+0000","last_became_peered":"2023-11-24T06:29:53.046978+0000","last_unstale":"2023-11-24T06:29:53.051428+0000","last_undegraded":"2023-11-24T06:29:53.051428+0000","last_fullsized":"2023-11-24T06:29:53.051428+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:23:35.211882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.035086+0000","last_change":"2023-11-24T06:29:52.218275+0000","last_active":"2023-11-24T06:29:53.035086+0000","last_peered":"2023-11-24T06:29:53.035086+0000","last_clean":"2023-11-24T06:29:53.035086+0000","last_became_active":"2023-11-24T06:29:52.217960+0000","last_became_peered":"2023-11-24T06:29:52.217960+0000","last_unstale":"2023-11-24T06:29:53.035086+0000","last_undegraded":"2023-11-24T06:29:53.035086+0000","last_fullsized":"2023-11-24T06:29:53.035086+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:05.510729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.046437+0000","last_change":"2023-11-24T06:29:53.039678+0000","last_active":"2023-11-24T06:29:53.046437+0000","last_peered":"2023-11-24T06:29:53.046437+0000","last_clean":"2023-11-24T06:29:53.046437+0000","last_became_active":"2023-11-24T06:29:53.039382+0000","last_became_peered":"2023-11-24T06:29:53.039382+0000","last_unstale":"2023-11-24T06:29:53.046437+0000","last_undegraded":"2023-11-24T06:29:53.046437+0000","last_fullsized":"2023-11-24T06:29:53.046437+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:25.146749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.018477+0000","last_change":"2023-11-24T06:29:52.218380+0000","last_active":"2023-11-24T06:29:53.018477+0000","last_peered":"2023-11-24T06:29:53.018477+0000","last_clean":"2023-11-24T06:29:53.018477+0000","last_became_active":"2023-11-24T06:29:52.217778+0000","last_became_peered":"2023-11-24T06:29:52.217778+0000","last_unstale":"2023-11-24T06:29:53.018477+0000","last_undegraded":"2023-11-24T06:29:53.018477+0000","last_fullsized":"2023-11-24T06:29:53.018477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:39.097647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.053057+0000","last_change":"2023-11-24T06:29:53.045611+0000","last_active":"2023-11-24T06:29:53.053057+0000","last_peered":"2023-11-24T06:29:53.053057+0000","last_clean":"2023-11-24T06:29:53.053057+0000","last_became_active":"2023-11-24T06:29:53.044985+0000","last_became_peered":"2023-11-24T06:29:53.044985+0000","last_unstale":"2023-11-24T06:29:53.053057+0000","last_undegraded":"2023-11-24T06:29:53.053057+0000","last_fullsized":"2023-11-24T06:29:53.053057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:58.716734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.037224+0000","last_change":"2023-11-24T06:29:52.215591+0000","last_active":"2023-11-24T06:29:53.037224+0000","last_peered":"2023-11-24T06:29:53.037224+0000","last_clean":"2023-11-24T06:29:53.037224+0000","last_became_active":"2023-11-24T06:29:52.214821+0000","last_became_peered":"2023-11-24T06:29:52.214821+0000","last_unstale":"2023-11-24T06:29:53.037224+0000","last_undegraded":"2023-11-24T06:29:53.037224+0000","last_fullsized":"2023-11-24T06:29:53.037224+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:45.825749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017332+0000","last_change":"2023-11-24T06:29:53.038284+0000","last_active":"2023-11-24T06:29:54.017332+0000","last_peered":"2023-11-24T06:29:54.017332+0000","last_clean":"2023-11-24T06:29:54.017332+0000","last_became_active":"2023-11-24T06:29:53.037590+0000","last_became_peered":"2023-11-24T06:29:53.037590+0000","last_unstale":"2023-11-24T06:29:54.017332+0000","last_undegraded":"2023-11-24T06:29:54.017332+0000","last_fullsized":"2023-11-24T06:29:54.017332+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:11:49.701095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.021007+0000","last_change":"2023-11-24T06:29:52.208050+0000","last_active":"2023-11-24T06:29:53.021007+0000","last_peered":"2023-11-24T06:29:53.021007+0000","last_clean":"2023-11-24T06:29:53.021007+0000","last_became_active":"2023-11-24T06:29:52.207718+0000","last_became_peered":"2023-11-24T06:29:52.207718+0000","last_unstale":"2023-11-24T06:29:53.021007+0000","last_undegraded":"2023-11-24T06:29:53.021007+0000","last_fullsized":"2023-11-24T06:29:53.021007+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:52.559537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.017618+0000","last_change":"2023-11-24T06:29:53.042687+0000","last_active":"2023-11-24T06:29:54.017618+0000","last_peered":"2023-11-24T06:29:54.017618+0000","last_clean":"2023-11-24T06:29:54.017618+0000","last_became_active":"2023-11-24T06:29:53.042383+0000","last_became_peered":"2023-11-24T06:29:53.042383+0000","last_unstale":"2023-11-24T06:29:54.017618+0000","last_undegraded":"2023-11-24T06:29:54.017618+0000","last_fullsized":"2023-11-24T06:29:54.017618+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:04.164163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.031634+0000","last_change":"2023-11-24T06:29:52.213841+0000","last_active":"2023-11-24T06:29:53.031634+0000","last_peered":"2023-11-24T06:29:53.031634+0000","last_clean":"2023-11-24T06:29:53.031634+0000","last_became_active":"2023-11-24T06:29:52.213503+0000","last_became_peered":"2023-11-24T06:29:52.213503+0000","last_unstale":"2023-11-24T06:29:53.031634+0000","last_undegraded":"2023-11-24T06:29:53.031634+0000","last_fullsized":"2023-11-24T06:29:53.031634+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:23:34.198375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.038300+0000","last_change":"2023-11-24T06:29:53.035185+0000","last_active":"2023-11-24T06:29:53.038300+0000","last_peered":"2023-11-24T06:29:53.038300+0000","last_clean":"2023-11-24T06:29:53.038300+0000","last_became_active":"2023-11-24T06:29:53.034893+0000","last_became_peered":"2023-11-24T06:29:53.034893+0000","last_unstale":"2023-11-24T06:29:53.038300+0000","last_undegraded":"2023-11-24T06:29:53.038300+0000","last_fullsized":"2023-11-24T06:29:53.038300+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:56.568611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.021194+0000","last_change":"2023-11-24T06:29:52.210848+0000","last_active":"2023-11-24T06:29:53.021194+0000","last_peered":"2023-11-24T06:29:53.021194+0000","last_clean":"2023-11-24T06:29:53.021194+0000","last_became_active":"2023-11-24T06:29:52.210472+0000","last_became_peered":"2023-11-24T06:29:52.210472+0000","last_unstale":"2023-11-24T06:29:53.021194+0000","last_undegraded":"2023-11-24T06:29:53.021194+0000","last_fullsized":"2023-11-24T06:29:53.021194+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:18:17.527323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016577+0000","last_change":"2023-11-24T06:29:53.039669+0000","last_active":"2023-11-24T06:29:54.016577+0000","last_peered":"2023-11-24T06:29:54.016577+0000","last_clean":"2023-11-24T06:29:54.016577+0000","last_became_active":"2023-11-24T06:29:53.039024+0000","last_became_peered":"2023-11-24T06:29:53.039024+0000","last_unstale":"2023-11-24T06:29:54.016577+0000","last_undegraded":"2023-11-24T06:29:54.016577+0000","last_fullsized":"2023-11-24T06:29:54.016577+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:37.120835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016055+0000","last_change":"2023-11-24T06:29:52.204624+0000","last_active":"2023-11-24T06:29:54.016055+0000","last_peered":"2023-11-24T06:29:54.016055+0000","last_clean":"2023-11-24T06:29:54.016055+0000","last_became_active":"2023-11-24T06:29:52.204334+0000","last_became_peered":"2023-11-24T06:29:52.204334+0000","last_unstale":"2023-11-24T06:29:54.016055+0000","last_undegraded":"2023-11-24T06:29:54.016055+0000","last_fullsized":"2023-11-24T06:29:54.016055+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:19.058926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:29:54.016236+0000","last_change":"2023-11-24T06:29:53.041234+0000","last_active":"2023-11-24T06:29:54.016236+0000","last_peered":"2023-11-24T06:29:54.016236+0000","last_clean":"2023-11-24T06:29:54.016236+0000","last_became_active":"2023-11-24T06:29:53.040942+0000","last_became_peered":"2023-11-24T06:29:53.040942+0000","last_unstale":"2023-11-24T06:29:54.016236+0000","last_undegraded":"2023-11-24T06:29:54.016236+0000","last_fullsized":"2023-11-24T06:29:54.016236+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:52.170161+0000","last_clean_scrub_stamp":"2023-11-24T06:29:52.170161+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:40.332527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.014493+0000","last_change":"2023-11-24T06:29:52.212278+0000","last_active":"2023-11-24T06:29:53.014493+0000","last_peered":"2023-11-24T06:29:53.014493+0000","last_clean":"2023-11-24T06:29:53.014493+0000","last_became_active":"2023-11-24T06:29:52.211944+0000","last_became_peered":"2023-11-24T06:29:52.211944+0000","last_unstale":"2023-11-24T06:29:53.014493+0000","last_undegraded":"2023-11-24T06:29:53.014493+0000","last_fullsized":"2023-11-24T06:29:53.014493+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:29:51.156653+0000","last_clean_scrub_stamp":"2023-11-24T06:29:51.156653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:40:52.298030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":205,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:29:53.013941+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:29:53.013941+0000","last_peered":"2023-11-24T06:29:53.013941+0000","last_clean":"2023-11-24T06:29:53.013941+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:29:53.013941+0000","last_undegraded":"2023-11-24T06:29:53.013941+0000","last_fullsized":"2023-11-24T06:29:53.013941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1298432,"data_stored":879677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.29,"15min":1.3280000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5780000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":4.0599999999999996},{"interface":"front","average":{"1min":1.181,"5min":1.2789999999999999,"15min":1.2949999999999999},"min":{"1min":0.75700000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.76,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":4.556}]},{"osd":1,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.65100000000000002,"5min":0.629,"15min":0.629},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":2.456},{"interface":"front","average":{"1min":1.218,"5min":1.359,"15min":1.383},"min":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.585,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":3.944}]},{"osd":2,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.2649999999999999,"15min":1.2849999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6479999999999999,"5min":5.3380000000000001,"15min":5.3380000000000001},"last":5.4589999999999996},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1919999999999999,"15min":1.1990000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.754,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":4.3799999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.365,"15min":1.403},"min":{"1min":0.80900000000000005,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.637,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":8.3539999999999992},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2050000000000001,"15min":1.2030000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.8200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.542,"15min":1.585},"min":{"1min":0.72099999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7170000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":2.1059999999999999},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.5569999999999999,"15min":1.585},"min":{"1min":1.0589999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.962,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":5.3449999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.395,"15min":1.411},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.921,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":2.3639999999999999},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.7170000000000001,"15min":1.758},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.845,"5min":6.7359999999999998,"15min":6.7359999999999998},"last":17.582999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.825,"15min":1.8779999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.093,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":2.8140000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.659,"15min":1.706},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.881,"5min":5.7270000000000003,"15min":5.7270000000000003},"last":6.2210000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":289149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0980000000000001,"15min":1.1020000000000001},"min":{"1min":0.73499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7390000000000001,"5min":1.829,"15min":1.829},"last":1.151},{"interface":"front","average":{"1min":1.155,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.76800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.623,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.81200000000000006}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2210000000000001,"15min":1.2130000000000001},"min":{"1min":0.71099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6299999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.464},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.2010000000000001,"15min":1.2070000000000001},"min":{"1min":0.88900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.494,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.2}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.159},"min":{"1min":0.75700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.609,"5min":3.597,"15min":3.597},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.304,"5min":1.292,"15min":1.3140000000000001},"min":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6359999999999999,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.5529999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3129999999999999,"15min":1.3180000000000001},"min":{"1min":0.78100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5620000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.506},{"interface":"front","average":{"1min":1.25,"5min":1.2969999999999999,"15min":1.325},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5960000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.234}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.8919999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.623},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.385,"15min":1.397},"min":{"1min":0.79600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7729999999999999,"5min":3.04,"15min":3.04},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3720000000000001,"15min":1.377},"min":{"1min":0.97699999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7330000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.552,"15min":1.6020000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.851,"5min":3.774,"15min":3.774},"last":1.319}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5449999999999999,"15min":1.5589999999999999},"min":{"1min":1.0029999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7689999999999999,"5min":3.403,"15min":3.403},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.512},"min":{"1min":0.875,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.792,"5min":3.649,"15min":3.649},"last":1.5960000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":704512,"data_stored":289801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.516,"5min":1.903,"15min":1.903},"last":1.125},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.288,"15min":1.3140000000000001},"min":{"1min":0.91700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.915,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.4490000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.145,"15min":1.1399999999999999},"min":{"1min":0.78900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.327,"5min":1.147,"15min":1.1140000000000001},"min":{"1min":0.89500000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.206}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3109999999999999,"15min":1.2969999999999999},"min":{"1min":0.99199999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1669999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.077}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3360000000000001,"15min":1.375},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.952,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3899999999999999},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.3460000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3180000000000001,"15min":1.3340000000000001},"min":{"1min":0.878,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.839,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.472,"5min":1.5029999999999999,"15min":1.5369999999999999},"min":{"1min":0.82299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0099999999999998,"5min":3.085,"15min":3.085},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4690000000000001,"15min":1.4450000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.474,"15min":1.468},"min":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0270000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.3700000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.651,"5min":1.6140000000000001,"15min":1.6240000000000001},"min":{"1min":1.046,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.141,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.28},{"interface":"front","average":{"1min":1.714,"5min":1.6919999999999999,"15min":1.716},"min":{"1min":1.0129999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3159999999999998,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.407}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":708608,"data_stored":290761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.073,"15min":1.073},"min":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.982,"15min":1.982},"last":0.752},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.089,"15min":1.1120000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.468,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.016}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.147,"15min":1.113},"min":{"1min":0.84199999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.349},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.26,"15min":1.3049999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5840000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8129999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.244},{"interface":"front","average":{"1min":1.27,"5min":1.3140000000000001,"15min":1.3320000000000001},"min":{"1min":0.79800000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7250000000000001,"5min":2.254,"15min":2.254},"last":0.83899999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.294,"15min":1.333},"min":{"1min":0.76100000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.833,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.343,"5min":1.3779999999999999,"15min":1.3720000000000001},"min":{"1min":0.753,"5min":0.746,"15min":0.746},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4079999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.482,"15min":1.5149999999999999},"min":{"1min":0.874,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8109999999999999,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.389,"5min":1.522,"15min":1.5609999999999999},"min":{"1min":0.85799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.923,"5min":3.52,"15min":3.52},"last":1.1890000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.474,"15min":1.482},"min":{"1min":0.79200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.542,"15min":1.5740000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.859,"5min":2.452,"15min":2.452},"last":1.079}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.595,"15min":1.6559999999999999},"min":{"1min":0.64000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.7729999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.615,"15min":1.613},"min":{"1min":0.84799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.0099999999999998,"5min":2.363,"15min":2.363},"last":1.5129999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":704512,"data_stored":289205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.40200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.90700000000000003,"5min":1.355,"15min":1.355},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.45900000000000002,"5min":0.372,"15min":0.372},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.92700000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.41199999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4690000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.1000000000000001,"15min":1.1160000000000001},"min":{"1min":0.51000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.4950000000000001,"5min":1.786,"15min":1.786},"last":1.1830000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.499,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4710000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.3},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.087,"15min":1.1060000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4450000000000001,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.2470000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.367,"15min":1.3959999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.0299999999999998,"5min":3.081,"15min":3.081},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3300000000000001,"15min":1.3380000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.0910000000000002,"5min":2.758,"15min":2.758},"last":1.5309999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.377,"15min":1.4039999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8200000000000001,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.722,"15min":1.782},"min":{"1min":0.88600000000000001,"5min":0.745,"15min":0.745},"max":{"1min":2.149,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.4099999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.3819999999999999,"15min":1.409},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.6919999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.5349999999999999,"15min":1.5449999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.1030000000000002,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.732}]},{"osd":7,"last update":"Fri Nov 24 06:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.5960000000000001,"15min":1.6160000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.1829999999999998,"5min":4.641,"15min":4.641},"last":1.611},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.55,"15min":1.5760000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.9750000000000001,"5min":3.798,"15min":3.798},"last":1.4690000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":704512,"data_stored":289205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2330000000000001,"5min":1.427,"15min":1.427},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.97899999999999998,"15min":1.0109999999999999},"min":{"1min":0.432,"5min":0.31,"15min":0.31},"max":{"1min":1.161,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.2609999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.94099999999999995,"15min":0.96299999999999997},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2410000000000001,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":2.1389999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0449999999999999,"15min":1.0509999999999999},"min":{"1min":0.56899999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.512,"5min":1.55,"15min":1.55},"last":1.379}]},{"osd":3,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.101,"15min":1.1100000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6850000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.115},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.103,"15min":1.119},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.597,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.1479999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.492,"15min":1.52},"min":{"1min":0.871,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.79,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":2.6869999999999998},{"interface":"front","average":{"1min":1.343,"5min":1.4690000000000001,"15min":1.49},"min":{"1min":0.94999999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6899999999999999,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":2.5800000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4590000000000001,"15min":1.466},"min":{"1min":0.84999999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.911,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.776},{"interface":"front","average":{"1min":1.385,"5min":1.49,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.9410000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.6499999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4279999999999999,"15min":1.4379999999999999},"min":{"1min":0.78200000000000003,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.6899999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.863},{"interface":"front","average":{"1min":1.419,"5min":1.5229999999999999,"15min":1.5409999999999999},"min":{"1min":0.85199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.0720000000000001,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.9390000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.5369999999999999,"15min":1.5569999999999999},"min":{"1min":0.83099999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.974,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.714},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.5309999999999999,"15min":1.5389999999999999},"min":{"1min":0.90300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1699999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.829}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31872,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":700416,"data_stored":289239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.002,"15min":1.032},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.696,"5min":2.593,"15min":2.593},"last":1.212},{"interface":"front","average":{"1min":1.008,"5min":1.02,"15min":1.032},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5980000000000001,"5min":1.702,"15min":1.702},"last":1.135}]},{"osd":2,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.212,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7669999999999999,"5min":1.829,"15min":1.829},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1040000000000001,"15min":1.1200000000000001},"min":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.603,"5min":1.766,"15min":1.766},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0289999999999999,"15min":1.069},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2250000000000001,"5min":1.798,"15min":1.798},"last":1.036}]},{"osd":4,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.246,"15min":1.2529999999999999},"min":{"1min":0.89000000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.621,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.077},{"interface":"front","average":{"1min":1.359,"5min":1.417,"15min":1.4450000000000001},"min":{"1min":1.0509999999999999,"5min":1.032,"15min":1.032},"max":{"1min":1.837,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.534}]},{"osd":5,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.294,"15min":1.302},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.514,"5min":2.016,"15min":2.016},"last":1.365},{"interface":"front","average":{"1min":1.391,"5min":1.389,"15min":1.399},"min":{"1min":0.91700000000000004,"5min":0.877,"15min":0.877},"max":{"1min":1.7410000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.4079999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.391,"15min":1.393},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.847,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4339999999999999,"15min":1.45},"min":{"1min":0.96399999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.6499999999999999,"5min":2.371,"15min":2.371},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.444,"15min":1.448},"min":{"1min":0.89300000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9730000000000001,"5min":2.246,"15min":2.246},"last":1.633},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4950000000000001,"15min":1.5189999999999999},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.7629999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.587}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1306624,"data_stored":881711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.48999999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.607,"5min":1.722,"15min":1.722},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.42499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.5,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.999,"15min":0.998},"min":{"1min":0.65200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.59399999999999997},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.119,"15min":1.137},"min":{"1min":0.63400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.538,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.93100000000000005,"15min":0.94899999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5229999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.085,"15min":1.0900000000000001},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.639,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2090000000000001,"15min":1.2190000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6659999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.093},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3100000000000001,"15min":1.3080000000000001},"min":{"1min":0.91400000000000003,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.986,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.145}]},{"osd":5,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.321,"15min":1.333},"min":{"1min":0.71799999999999997,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.704,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.2509999999999999,"15min":1.2709999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6000000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.4710000000000001,"15min":1.4870000000000001},"min":{"1min":0.95499999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.7230000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.492,"15min":1.518},"min":{"1min":0.91300000000000003,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.8069999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.123}]},{"osd":7,"last update":"Fri Nov 24 06:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.446,"15min":1.456},"min":{"1min":0.86499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9059999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.056},{"interface":"front","average":{"1min":1.278,"5min":1.399,"15min":1.419},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.853,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.89800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:29:58.271 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:29:58.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:29:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.542+0000 7f64de5e9640 1 Processor -- start 2023-11-24T06:29:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.542+0000 7f64de5e9640 1 -- start start 2023-11-24T06:29:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.544+0000 7f64de5e9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d815aba0 0x7f64d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.544+0000 7f64de5e9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64d806d900 0x7f64d815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.544+0000 7f64de5e9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d806dd60 con 0x7f64d815df30 2023-11-24T06:29:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.544+0000 7f64de5e9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d80ad810 con 0x7f64d806d900 2023-11-24T06:29:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.544+0000 7f64de5e9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80ad380 con 0x7f64d815aba0 2023-11-24T06:29:58.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.545+0000 7f64dcde6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64d806d900 0x7f64d815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.545+0000 7f64ddde8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f64d815df30 0x7f64d8160d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43072/0 (socket says 172.21.15.16:43072) 2023-11-24T06:29:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.545+0000 7f64ddde8640 1 -- 172.21.15.16:0/3286431239 learned_addr learned my addr 172.21.15.16:0/3286431239 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.545+0000 7f64dd5e7640 1 --2- 172.21.15.16:0/3286431239 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d815aba0 0x7f64d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.545+0000 7f64dcde6640 1 -- 172.21.15.16:0/3286431239 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d815aba0 msgr2=0x7f64d806d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.545+0000 7f64dcde6640 1 --2- 172.21.15.16:0/3286431239 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d815aba0 0x7f64d806d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.545+0000 7f64dcde6640 1 -- 172.21.15.16:0/3286431239 >> v1:172.21.15.16:6789/0 conn(0x7f64d815df30 legacy=0x7f64d8160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.545+0000 7f64dcde6640 1 -- 172.21.15.16:0/3286431239 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80aca50 con 0x7f64d806d900 2023-11-24T06:29:58.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.546+0000 7f64dcde6640 1 --2- 172.21.15.16:0/3286431239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d806d900 0x7f64d815d9f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f64c80097a0 tx=0x7f64c802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6d640bafd181c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:58.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64cffff640 1 -- 172.21.15.16:0/3286431239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c802d040 con 0x7f64d806d900 2023-11-24T06:29:58.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64cffff640 1 -- 172.21.15.16:0/3286431239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8031810 con 0x7f64d806d900 2023-11-24T06:29:58.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64cffff640 1 -- 172.21.15.16:0/3286431239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8031ba0 con 0x7f64d806d900 2023-11-24T06:29:58.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64de5e9640 1 -- 172.21.15.16:0/3286431239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d806d900 msgr2=0x7f64d815d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64de5e9640 1 --2- 172.21.15.16:0/3286431239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d806d900 0x7f64d815d9f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f64c80097a0 tx=0x7f64c802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64de5e9640 1 -- 172.21.15.16:0/3286431239 shutdown_connections 2023-11-24T06:29:58.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64de5e9640 1 --2- 172.21.15.16:0/3286431239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d806d900 0x7f64d815d9f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64de5e9640 1 --2- 172.21.15.16:0/3286431239 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d815aba0 0x7f64d806d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64de5e9640 1 -- 172.21.15.16:0/3286431239 >> 172.21.15.16:0/3286431239 conn(0x7f64d80b0b70 msgr2=0x7f64d80b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:58.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64de5e9640 1 -- 172.21.15.16:0/3286431239 shutdown_connections 2023-11-24T06:29:58.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.547+0000 7f64de5e9640 1 -- 172.21.15.16:0/3286431239 wait complete. 2023-11-24T06:29:58.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.549+0000 7f64de5e9640 1 Processor -- start 2023-11-24T06:29:58.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.549+0000 7f64de5e9640 1 -- start start 2023-11-24T06:29:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.550+0000 7f64de5e9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d806d900 0x7f64d8143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.550+0000 7f64de5e9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64d815aba0 0x7f64d8144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.550+0000 7f64de5e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815df30 0x7f64d81454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.550+0000 7f64de5e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8144240 con 0x7f64d815df30 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.550+0000 7f64de5e9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d81431b0 con 0x7f64d815aba0 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.550+0000 7f64de5e9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80cc3f0 con 0x7f64d806d900 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.551+0000 7f64ddde8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815df30 0x7f64d81454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.551+0000 7f64ddde8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815df30 0x7f64d81454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43342/0 (socket says 172.21.15.16:43342) 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.551+0000 7f64ddde8640 1 -- 172.21.15.16:0/4138587536 learned_addr learned my addr 172.21.15.16:0/4138587536 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.551+0000 7f64dd5e7640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d806d900 0x7f64d8143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.551+0000 7f64ddde8640 1 -- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d806d900 msgr2=0x7f64d8143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.551+0000 7f64ddde8640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d806d900 0x7f64d8143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.551+0000 7f64ddde8640 1 -- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64d815aba0 msgr2=0x7f64d8144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.551+0000 7f64ddde8640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64d815aba0 0x7f64d8144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.551+0000 7f64ddde8640 1 -- 172.21.15.16:0/4138587536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009400 con 0x7f64d815df30 2023-11-24T06:29:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.552+0000 7f64ddde8640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815df30 0x7f64d81454d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ca80 tx=0x7f64d000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.552+0000 7f64cdffb640 1 -- 172.21.15.16:0/4138587536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d000ac30 con 0x7f64d815df30 2023-11-24T06:29:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.552+0000 7f64cdffb640 1 -- 172.21.15.16:0/4138587536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d000ad90 con 0x7f64d815df30 2023-11-24T06:29:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.552+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d80cc6d0 con 0x7f64d815df30 2023-11-24T06:29:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.552+0000 7f64cdffb640 1 -- 172.21.15.16:0/4138587536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0012600 con 0x7f64d815df30 2023-11-24T06:29:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.552+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d80ccc90 con 0x7f64d815df30 2023-11-24T06:29:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.553+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d8067450 con 0x7f64d815df30 2023-11-24T06:29:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.554+0000 7f64cdffb640 1 -- 172.21.15.16:0/4138587536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f64d0007070 con 0x7f64d815df30 2023-11-24T06:29:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.555+0000 7f64cdffb640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64bc06b0b0 0x7f64bc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.555+0000 7f64cdffb640 1 -- 172.21.15.16:0/4138587536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64d0089850 con 0x7f64d815df30 2023-11-24T06:29:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.556+0000 7f64dd5e7640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64bc06b0b0 0x7f64bc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.557+0000 7f64dd5e7640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64bc06b0b0 0x7f64bc06d570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f64d4006fd0 tx=0x7f64d4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:58.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.559+0000 7f64cdffb640 1 -- 172.21.15.16:0/4138587536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64d00545b0 con 0x7f64d815df30 2023-11-24T06:29:58.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.823+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f64d815edc0 con 0x7f64d815df30 2023-11-24T06:29:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.827+0000 7f64cdffb640 1 -- 172.21.15.16:0/4138587536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f64d00170e0 con 0x7f64d815df30 2023-11-24T06:29:58.829 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-24T06:29:58.830 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA== 2023-11-24T06:29:58.830 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2023-11-24T06:29:58.830 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2023-11-24T06:29:58.830 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:29:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-11-24T06:29:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64bc06b0b0 msgr2=0x7f64bc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64bc06b0b0 0x7f64bc06d570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f64d4006fd0 tx=0x7f64d4008040 comp rx=0 tx=0).stop 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815df30 msgr2=0x7f64d81454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815df30 0x7f64d81454d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ca80 tx=0x7f64d000cf40 comp rx=0 tx=0).stop 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 shutdown_connections 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64bc06b0b0 0x7f64bc06d570 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815df30 0x7f64d81454d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64d815aba0 0x7f64d8144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 --2- 172.21.15.16:0/4138587536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64d806d900 0x7f64d8143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 >> 172.21.15.16:0/4138587536 conn(0x7f64d80b0b70 msgr2=0x7f64d8146630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 shutdown_connections 2023-11-24T06:29:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:58.830+0000 7f64de5e9640 1 -- 172.21.15.16:0/4138587536 wait complete. 2023-11-24T06:29:58.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:29:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.067+0000 7fdf69f3a640 1 Processor -- start 2023-11-24T06:29:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.067+0000 7fdf69f3a640 1 -- start start 2023-11-24T06:29:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.068+0000 7fdf69f3a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf64143ae0 0x7fdf640cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.068+0000 7fdf69f3a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf640cf7d0 0x7fdf6415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.068+0000 7fdf69f3a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf640cfc30 con 0x7fdf6415e1e0 2023-11-24T06:29:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.068+0000 7fdf69f3a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640b1ee0 con 0x7fdf64143ae0 2023-11-24T06:29:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.068+0000 7fdf69f3a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640b2780 con 0x7fdf640cf7d0 2023-11-24T06:29:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.069+0000 7fdf63fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdf6415e1e0 0x7fdf6415a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43078/0 (socket says 172.21.15.16:43078) 2023-11-24T06:29:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.069+0000 7fdf63fff640 1 -- 172.21.15.16:0/3429564901 learned_addr learned my addr 172.21.15.16:0/3429564901 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:29:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.069+0000 7fdf637fe640 1 --2- 172.21.15.16:0/3429564901 >> v2:172.21.15.16:3300/0 conn(0x7fdf64143ae0 0x7fdf640cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.069+0000 7fdf5bfff640 1 --2- 172.21.15.16:0/3429564901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf640cf7d0 0x7fdf6415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.069+0000 7fdf637fe640 1 -- 172.21.15.16:0/3429564901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf640cf7d0 msgr2=0x7fdf6415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.069+0000 7fdf637fe640 1 --2- 172.21.15.16:0/3429564901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf640cf7d0 0x7fdf6415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.069+0000 7fdf637fe640 1 -- 172.21.15.16:0/3429564901 >> v1:172.21.15.16:6789/0 conn(0x7fdf6415e1e0 legacy=0x7fdf6415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.069+0000 7fdf637fe640 1 -- 172.21.15.16:0/3429564901 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6415e990 con 0x7fdf64143ae0 2023-11-24T06:29:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.070+0000 7fdf637fe640 1 --2- 172.21.15.16:0/3429564901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf64143ae0 0x7fdf640cf290 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdf540097a0 tx=0x7fdf5402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62229bd793c1ffb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf62ffd640 1 -- 172.21.15.16:0/3429564901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5402d040 con 0x7fdf64143ae0 2023-11-24T06:29:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf62ffd640 1 -- 172.21.15.16:0/3429564901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf54031810 con 0x7fdf64143ae0 2023-11-24T06:29:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf62ffd640 1 -- 172.21.15.16:0/3429564901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54031ba0 con 0x7fdf64143ae0 2023-11-24T06:29:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf69f3a640 1 -- 172.21.15.16:0/3429564901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf64143ae0 msgr2=0x7fdf640cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf69f3a640 1 --2- 172.21.15.16:0/3429564901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf64143ae0 0x7fdf640cf290 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdf540097a0 tx=0x7fdf5402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf69f3a640 1 -- 172.21.15.16:0/3429564901 shutdown_connections 2023-11-24T06:29:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf69f3a640 1 --2- 172.21.15.16:0/3429564901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf640cf7d0 0x7fdf6415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf69f3a640 1 --2- 172.21.15.16:0/3429564901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf64143ae0 0x7fdf640cf290 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf69f3a640 1 -- 172.21.15.16:0/3429564901 >> 172.21.15.16:0/3429564901 conn(0x7fdf6406a800 msgr2=0x7fdf6406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf69f3a640 1 -- 172.21.15.16:0/3429564901 shutdown_connections 2023-11-24T06:29:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.071+0000 7fdf69f3a640 1 -- 172.21.15.16:0/3429564901 wait complete. 2023-11-24T06:29:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.073+0000 7fdf69f3a640 1 Processor -- start 2023-11-24T06:29:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.073+0000 7fdf69f3a640 1 -- start start 2023-11-24T06:29:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.074+0000 7fdf69f3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf640cf7d0 0x7fdf641e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.074+0000 7fdf69f3a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf64143ae0 0x7fdf640cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.074+0000 7fdf69f3a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf6415e1e0 0x7fdf640cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.074+0000 7fdf69f3a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6415a920 con 0x7fdf640cf7d0 2023-11-24T06:29:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.074+0000 7fdf69f3a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6415b780 con 0x7fdf64143ae0 2023-11-24T06:29:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.074+0000 7fdf69f3a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6415ab70 con 0x7fdf6415e1e0 2023-11-24T06:29:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf640cf7d0 0x7fdf641e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf640cf7d0 0x7fdf641e3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43368/0 (socket says 172.21.15.16:43368) 2023-11-24T06:29:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf637fe640 1 -- 172.21.15.16:0/4265892404 learned_addr learned my addr 172.21.15.16:0/4265892404 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf637fe640 1 -- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf6415e1e0 msgr2=0x7fdf640cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf637fe640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf6415e1e0 0x7fdf640cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf637fe640 1 -- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf64143ae0 msgr2=0x7fdf640cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf637fe640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf64143ae0 0x7fdf640cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf637fe640 1 -- 172.21.15.16:0/4265892404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54009400 con 0x7fdf640cf7d0 2023-11-24T06:29:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf637fe640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf640cf7d0 0x7fdf641e3b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdf540098c0 tx=0x7fdf540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.075+0000 7fdf60ff9640 1 -- 172.21.15.16:0/4265892404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5403ad60 con 0x7fdf640cf7d0 2023-11-24T06:29:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.076+0000 7fdf60ff9640 1 -- 172.21.15.16:0/4265892404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf54044400 con 0x7fdf640cf7d0 2023-11-24T06:29:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.076+0000 7fdf60ff9640 1 -- 172.21.15.16:0/4265892404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540447b0 con 0x7fdf640cf7d0 2023-11-24T06:29:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.076+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6415ae50 con 0x7fdf640cf7d0 2023-11-24T06:29:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.076+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6415b410 con 0x7fdf640cf7d0 2023-11-24T06:29:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.077+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf640675d0 con 0x7fdf640cf7d0 2023-11-24T06:29:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.078+0000 7fdf60ff9640 1 -- 172.21.15.16:0/4265892404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fdf5404d440 con 0x7fdf640cf7d0 2023-11-24T06:29:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.079+0000 7fdf60ff9640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4406b080 0x7fdf4406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.079+0000 7fdf60ff9640 1 -- 172.21.15.16:0/4265892404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf540abfe0 con 0x7fdf640cf7d0 2023-11-24T06:29:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.079+0000 7fdf5bfff640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4406b080 0x7fdf4406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.080+0000 7fdf5bfff640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4406b080 0x7fdf4406d540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c006fd0 tx=0x7fdf4c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.083+0000 7fdf60ff9640 1 -- 172.21.15.16:0/4265892404 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf54076be0 con 0x7fdf640cf7d0 2023-11-24T06:29:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.304+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf640af700 con 0x7fdf640cf7d0 2023-11-24T06:29:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.305+0000 7fdf60ff9640 1 -- 172.21.15.16:0/4265892404 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fdf54035090 con 0x7fdf640cf7d0 2023-11-24T06:29:59.308 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:29:59.308 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3782726360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3782726360},{"type":"v1","addr":"172.21.15.173:6835","nonce":3782726360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1586323927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1586323927},{"type":"v1","addr":"172.21.15.16:6835","nonce":1586323927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1730788036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1730788036},{"type":"v1","addr":"172.21.15.173:6837","nonce":1730788036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:52.667087+0000","modified":"2023-11-24T06:29:55.722711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/515502120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":515502120},{"type":"v1","addr":"172.21.15.16:6833","nonce":515502120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:29:59.308 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 34 2023-11-24T06:29:59.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4406b080 msgr2=0x7fdf4406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4406b080 0x7fdf4406d540 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c006fd0 tx=0x7fdf4c008040 comp rx=0 tx=0).stop 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf640cf7d0 msgr2=0x7fdf641e3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf640cf7d0 0x7fdf641e3b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdf540098c0 tx=0x7fdf540046c0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 shutdown_connections 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4406b080 0x7fdf4406d540 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf6415e1e0 0x7fdf640cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf64143ae0 0x7fdf640cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 --2- 172.21.15.16:0/4265892404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf640cf7d0 0x7fdf641e3b50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 >> 172.21.15.16:0/4265892404 conn(0x7fdf6406a800 msgr2=0x7fdf6406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 shutdown_connections 2023-11-24T06:29:59.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.308+0000 7fdf69f3a640 1 -- 172.21.15.16:0/4265892404 wait complete. 2023-11-24T06:29:59.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:29:52.667087+0000', 'modified': '2023-11-24T06:29:55.722711+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6201}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6201': {'gid': 6201, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6833/515502120', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 515502120}, {'type': 'v1', 'addr': '172.21.15.16:6833', 'nonce': 515502120}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:29:59.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:29:59.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:29:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.965+0000 7f76f3090640 1 Processor -- start 2023-11-24T06:29:59.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.965+0000 7f76f3090640 1 -- start start 2023-11-24T06:29:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.967+0000 7f76f3090640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76ec13df80 0x7f76ec15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.967+0000 7f76f3090640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76ec13e360 0x7f76ec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.967+0000 7f76f3090640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ec06c470 con 0x7f76ec15f9c0 2023-11-24T06:29:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.967+0000 7f76f3090640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ec06c750 con 0x7f76ec13df80 2023-11-24T06:29:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.967+0000 7f76f3090640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec06ca30 con 0x7f76ec13e360 2023-11-24T06:29:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.967+0000 7f76f0e2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76ec13df80 0x7f76ec15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.967+0000 7f76f0e2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76ec13df80 0x7f76ec15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43372/0 (socket says 172.21.15.16:43372) 2023-11-24T06:29:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.967+0000 7f76f0e2c640 1 -- 172.21.15.16:0/1564870711 learned_addr learned my addr 172.21.15.16:0/1564870711 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.968+0000 7f76f0e2c640 1 -- 172.21.15.16:0/1564870711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76ec13e360 msgr2=0x7f76ec0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.968+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1564870711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76ec13e360 0x7f76ec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.968+0000 7f76f0e2c640 1 -- 172.21.15.16:0/1564870711 >> v1:172.21.15.173:6789/0 conn(0x7f76ec15f9c0 legacy=0x7f76ec1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.968+0000 7f76f0e2c640 1 -- 172.21.15.16:0/1564870711 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec06cd10 con 0x7f76ec13df80 2023-11-24T06:29:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.969+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1564870711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13df80 0x7f76ec15bee0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009870 tx=0x7f76e002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b0224e7c7a69d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.969+0000 7f76eb7fe640 1 -- 172.21.15.16:0/1564870711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0031640 con 0x7f76ec13df80 2023-11-24T06:29:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.969+0000 7f76eb7fe640 1 -- 172.21.15.16:0/1564870711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e00317a0 con 0x7f76ec13df80 2023-11-24T06:29:59.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.969+0000 7f76eb7fe640 1 -- 172.21.15.16:0/1564870711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0031b30 con 0x7f76ec13df80 2023-11-24T06:29:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.970+0000 7f76f3090640 1 -- 172.21.15.16:0/1564870711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13df80 msgr2=0x7f76ec15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.970+0000 7f76f3090640 1 --2- 172.21.15.16:0/1564870711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13df80 0x7f76ec15bee0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009870 tx=0x7f76e002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.970+0000 7f76f3090640 1 -- 172.21.15.16:0/1564870711 shutdown_connections 2023-11-24T06:29:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.970+0000 7f76f3090640 1 --2- 172.21.15.16:0/1564870711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76ec13e360 0x7f76ec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.970+0000 7f76f3090640 1 --2- 172.21.15.16:0/1564870711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13df80 0x7f76ec15bee0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.970+0000 7f76f3090640 1 -- 172.21.15.16:0/1564870711 >> 172.21.15.16:0/1564870711 conn(0x7f76ec06a800 msgr2=0x7f76ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.970+0000 7f76f3090640 1 -- 172.21.15.16:0/1564870711 shutdown_connections 2023-11-24T06:29:59.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.970+0000 7f76f3090640 1 -- 172.21.15.16:0/1564870711 wait complete. 2023-11-24T06:29:59.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.973+0000 7f76f3090640 1 Processor -- start 2023-11-24T06:29:59.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.973+0000 7f76f3090640 1 -- start start 2023-11-24T06:29:59.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.974+0000 7f76f3090640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76ec13df80 0x7f76ec0b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.974+0000 7f76f3090640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13e360 0x7f76ec1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.974+0000 7f76f3090640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76ec15f9c0 0x7f76ec157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.974+0000 7f76f3090640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec0ad7a0 con 0x7f76ec13e360 2023-11-24T06:29:59.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.974+0000 7f76f3090640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec1f3c20 con 0x7f76ec15f9c0 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.974+0000 7f76f3090640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec0ac8d0 con 0x7f76ec13df80 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.974+0000 7f76ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13e360 0x7f76ec1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13e360 0x7f76ec1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43386/0 (socket says 172.21.15.16:43386) 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76ebfff640 1 -- 172.21.15.16:0/1051168502 learned_addr learned my addr 172.21.15.16:0/1051168502 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76ec13df80 0x7f76ec0b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76f162d640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76ec15f9c0 0x7f76ec157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76ebfff640 1 -- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76ec13df80 msgr2=0x7f76ec0b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76ebfff640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76ec13df80 0x7f76ec0b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76ebfff640 1 -- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76ec15f9c0 msgr2=0x7f76ec157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76ebfff640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76ec15f9c0 0x7f76ec157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76ebfff640 1 -- 172.21.15.16:0/1051168502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e00094d0 con 0x7f76ec13e360 2023-11-24T06:29:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.975+0000 7f76ebfff640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13e360 0x7f76ec1576e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f76d800b4a0 tx=0x7f76d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.976+0000 7f76e97fa640 1 -- 172.21.15.16:0/1051168502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8010610 con 0x7f76ec13e360 2023-11-24T06:29:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.976+0000 7f76e97fa640 1 -- 172.21.15.16:0/1051168502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d8010770 con 0x7f76ec13e360 2023-11-24T06:29:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.976+0000 7f76e97fa640 1 -- 172.21.15.16:0/1051168502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8010b20 con 0x7f76ec13e360 2023-11-24T06:29:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.976+0000 7f76f3090640 1 -- 172.21.15.16:0/1051168502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ec0acb20 con 0x7f76ec13e360 2023-11-24T06:29:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.976+0000 7f76f3090640 1 -- 172.21.15.16:0/1051168502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec0ad0e0 con 0x7f76ec13e360 2023-11-24T06:29:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.979+0000 7f76e97fa640 1 -- 172.21.15.16:0/1051168502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f76d8010c80 con 0x7f76ec13e360 2023-11-24T06:29:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.980+0000 7f76e97fa640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f76d006b080 0x7f76d006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.980+0000 7f76e97fa640 1 -- 172.21.15.16:0/1051168502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76d80896e0 con 0x7f76ec13e360 2023-11-24T06:29:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.981+0000 7f76f5c1f740 1 Processor -- start 2023-11-24T06:29:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.981+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f76d006b080 0x7f76d006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.981+0000 7f76f5c1f740 1 -- start start 2023-11-24T06:29:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a952b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a952a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a956e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a9489880 con 0x55c7a952aea0 2023-11-24T06:29:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a9498ae0 con 0x55c7a952a5e0 2023-11-24T06:29:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7a94961d0 con 0x55c7a9525ba0 2023-11-24T06:29:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f0e2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a952b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f0e2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a952b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43396/0 (socket says 172.21.15.16:43396) 2023-11-24T06:29:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f0e2c640 1 -- 172.21.15.16:0/1606808355 learned_addr learned my addr 172.21.15.16:0/1606808355 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f76d006b080 0x7f76d006d540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f76e0036000 tx=0x7f76e003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76ebfff640 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a952a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f162d640 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a956e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f0e2c640 1 -- 172.21.15.16:0/1606808355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 msgr2=0x55c7a952a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.982+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a952a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.983+0000 7f76f0e2c640 1 -- 172.21.15.16:0/1606808355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 msgr2=0x55c7a956e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.983+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a956e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.983+0000 7f76f0e2c640 1 -- 172.21.15.16:0/1606808355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7a9459860 con 0x55c7a952aea0 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.983+0000 7f76f162d640 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a956e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.983+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a952b280 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009960 tx=0x7f76e0031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3164b3e34f2a203a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.983+0000 7f76d67fc640 1 -- 172.21.15.16:0/1606808355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e003a9a0 con 0x55c7a952aea0 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.983+0000 7f76d67fc640 1 -- 172.21.15.16:0/1606808355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e003ab00 con 0x55c7a952aea0 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.983+0000 7f76d67fc640 1 -- 172.21.15.16:0/1606808355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e004f480 con 0x55c7a952aea0 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.984+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1606808355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 msgr2=0x55c7a952b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.984+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a952b280 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009960 tx=0x7f76e0031000 comp rx=0 tx=0).stop 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.984+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1606808355 shutdown_connections 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.984+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a956e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.984+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a952a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.984+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/1606808355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a952b280 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.984+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1606808355 >> 172.21.15.16:0/1606808355 conn(0x55c7a94ecc70 msgr2=0x55c7a9570990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.984+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1606808355 shutdown_connections 2023-11-24T06:29:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.984+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1606808355 wait complete. 2023-11-24T06:29:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.985+0000 7f76f5c1f740 1 -- start start 2023-11-24T06:29:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.985+0000 7f76f5c1f740 1 Processor -- start 2023-11-24T06:29:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.986+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9577590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.986+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9573ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.986+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a95743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.986+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a9491ac0 con 0x55c7a952aea0 2023-11-24T06:29:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.986+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a942e2f0 con 0x55c7a952a5e0 2023-11-24T06:29:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.986+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7a944bc70 con 0x55c7a9525ba0 2023-11-24T06:29:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.986+0000 7f76f5c1f740 10 client.0 _ll_register_callbacks cb 0x55c7a94503b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:29:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.986+0000 7f76f162d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a95743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.986+0000 7f76f162d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a95743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43398/0 (socket says 172.21.15.16:43398) 2023-11-24T06:29:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76f162d640 1 -- 172.21.15.16:0/444468597 learned_addr learned my addr 172.21.15.16:0/444468597 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76ebfff640 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9573ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9577590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76f162d640 1 -- 172.21.15.16:0/444468597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 msgr2=0x55c7a9577590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76f162d640 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9577590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76f162d640 1 -- 172.21.15.16:0/444468597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 msgr2=0x55c7a9573ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76f162d640 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9573ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76f162d640 1 -- 172.21.15.16:0/444468597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec06cd10 con 0x55c7a952aea0 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9577590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76f162d640 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a95743e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f76e400f500 tx=0x7f76e400f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f0db769fee6632 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:29:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.987+0000 7f76cb7fe640 1 -- 172.21.15.16:0/444468597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003b80 con 0x55c7a952aea0 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76cb7fe640 1 -- 172.21.15.16:0/444468597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76e4003ce0 con 0x55c7a952aea0 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76f5c1f740 1 -- 172.21.15.16:0/444468597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7a948aaa0 con 0x55c7a952aea0 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76cb7fe640 1 -- 172.21.15.16:0/444468597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76e4003ea0 con 0x55c7a952aea0 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76f5c1f740 10 client.6357 fetch_fsmap learned FSMap version 34 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76f5c1f740 1 -- 172.21.15.16:0/444468597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c7a9490140 con 0x55c7a952aea0 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76cb7fe640 1 -- 172.21.15.16:0/444468597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7a948aaa0 con 0x55c7a952aea0 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76cb7fe640 1 -- 172.21.15.16:0/444468597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f76e4012070 con 0x55c7a952aea0 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76f5c1f740 10 client.6357 fetch_fsmap finished waiting for FSMap version 34 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76f5c1f740 10 client.6357 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76f5c1f740 20 client.6357 populate_metadata read hostname 'smithi016' 2023-11-24T06:29:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] conn(0x55c7a954ffd0 0x55c7a9575040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:29:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76f5c1f740 4 client.6357 mds_command: new command op to 6201 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:29:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.988+0000 7f76f5c1f740 1 -- 172.21.15.16:0/444468597 --> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7a948a7c0 con 0x55c7a954ffd0 2023-11-24T06:29:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.989+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] conn(0x55c7a954ffd0 0x55c7a9575040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:29:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.989+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] conn(0x55c7a954ffd0 0x55c7a9575040 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e3ea46eac4c713 server_cookie=5d96cd6e38b8540a in_seq=0 out_seq=0 2023-11-24T06:29:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.989+0000 7f76cb7fe640 10 client.6357 ms_handle_connect on v2:172.21.15.16:6832/515502120 2023-11-24T06:29:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76cb7fe640 1 -- 172.21.15.16:0/444468597 <== mds.0 v2:172.21.15.16:6832/515502120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7a948a7c0 con 0x55c7a954ffd0 2023-11-24T06:29:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76cb7fe640 10 client.6357 handle_command_reply: tid=0 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76f5c1f740 1 client.6357 shutdown 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76f5c1f740 1 -- 172.21.15.16:0/444468597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 msgr2=0x55c7a95743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a95743e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f76e400f500 tx=0x7f76e400f9c0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76f5c1f740 1 -- 172.21.15.16:0/444468597 shutdown_connections 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] conn(0x55c7a954ffd0 0x55c7a9575040 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a95743e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9573ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/444468597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9577590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76f5c1f740 1 -- 172.21.15.16:0/444468597 >> 172.21.15.16:0/444468597 conn(0x55c7a94ecc70 msgr2=0x55c7a9524b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.990+0000 7f76cb7fe640 0 client.6357 ms_handle_reset on v2:172.21.15.16:6832/515502120 2023-11-24T06:29:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.991+0000 7f76f5c1f740 1 -- 172.21.15.16:0/444468597 shutdown_connections 2023-11-24T06:29:59.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.991+0000 7f76f5c1f740 1 -- 172.21.15.16:0/444468597 wait complete. 2023-11-24T06:29:59.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:29:59.991+0000 7f76f5c1f740 20 client.6357 trim_cache size 0 max 16384 2023-11-24T06:30:00.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.008+0000 7f76f5c1f740 1 Processor -- start 2023-11-24T06:30:00.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.008+0000 7f76f5c1f740 1 -- start start 2023-11-24T06:30:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.009+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9571160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.009+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9582a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.009+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9582f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.009+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a91f4f20 con 0x55c7a952aea0 2023-11-24T06:30:00.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.009+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a95716a0 con 0x55c7a952a5e0 2023-11-24T06:30:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.009+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7a9527b70 con 0x55c7a9525ba0 2023-11-24T06:30:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.009+0000 7f76f162d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9582f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.009+0000 7f76f162d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9582f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43408/0 (socket says 172.21.15.16:43408) 2023-11-24T06:30:00.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.009+0000 7f76f162d640 1 -- 172.21.15.16:0/1998338200 learned_addr learned my addr 172.21.15.16:0/1998338200 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:00.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76ebfff640 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9582a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9571160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76f162d640 1 -- 172.21.15.16:0/1998338200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 msgr2=0x55c7a9571160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76f162d640 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9571160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76f162d640 1 -- 172.21.15.16:0/1998338200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 msgr2=0x55c7a9582a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76f162d640 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9582a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76f162d640 1 -- 172.21.15.16:0/1998338200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7a942db80 con 0x55c7a952aea0 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9571160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76f162d640 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9582f90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f76e4001220 tx=0x7f76e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ada02dd32a3b59 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76d6ffd640 1 -- 172.21.15.16:0/1998338200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e400b070 con 0x55c7a952aea0 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76d6ffd640 1 -- 172.21.15.16:0/1998338200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e4029070 con 0x55c7a952aea0 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.010+0000 7f76d6ffd640 1 -- 172.21.15.16:0/1998338200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003b80 con 0x55c7a952aea0 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.011+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1998338200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 msgr2=0x55c7a9582f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.011+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9582f90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f76e4001220 tx=0x7f76e4013000 comp rx=0 tx=0).stop 2023-11-24T06:30:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.011+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1998338200 shutdown_connections 2023-11-24T06:30:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.011+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9582f90 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.011+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9582a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.011+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/1998338200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a9571160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.011+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1998338200 >> 172.21.15.16:0/1998338200 conn(0x55c7a94ecc70 msgr2=0x55c7a95212a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.011+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1998338200 shutdown_connections 2023-11-24T06:30:00.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.011+0000 7f76f5c1f740 1 -- 172.21.15.16:0/1998338200 wait complete. 2023-11-24T06:30:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.012+0000 7f76f5c1f740 1 -- start start 2023-11-24T06:30:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.012+0000 7f76f5c1f740 1 Processor -- start 2023-11-24T06:30:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.013+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a95dd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.013+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9573ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.013+0000 7f76f5c1f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9574010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.013+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a9555680 con 0x55c7a952aea0 2023-11-24T06:30:00.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.013+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a9583370 con 0x55c7a952a5e0 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.013+0000 7f76f5c1f740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7a9570c70 con 0x55c7a9525ba0 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.013+0000 7f76f5c1f740 10 client.0 _ll_register_callbacks cb 0x7f76e40015a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f162d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9574010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f162d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9574010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43410/0 (socket says 172.21.15.16:43410) 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f162d640 1 -- 172.21.15.16:0/4100812296 learned_addr learned my addr 172.21.15.16:0/4100812296 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a95dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76ebfff640 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9573ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f162d640 1 -- 172.21.15.16:0/4100812296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 msgr2=0x55c7a95dd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f162d640 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a95dd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f162d640 1 -- 172.21.15.16:0/4100812296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 msgr2=0x55c7a9573ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f162d640 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9573ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f162d640 1 -- 172.21.15.16:0/4100812296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7a9490140 con 0x55c7a952aea0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76f162d640 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9574010 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f76e400f4d0 tx=0x7f76e40013a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711ce0777937c550 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.014+0000 7f76cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76cb7fe640 1 -- 172.21.15.16:0/4100812296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40056e0 con 0x55c7a952aea0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76cb7fe640 1 -- 172.21.15.16:0/4100812296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76e4006b10 con 0x55c7a952aea0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76f5c1f740 1 -- 172.21.15.16:0/4100812296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76e003ab00 con 0x55c7a952aea0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76cb7fe640 1 -- 172.21.15.16:0/4100812296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76e4006cf0 con 0x55c7a952aea0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76f5c1f740 10 client.6363 fetch_fsmap learned FSMap version 34 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76f5c1f740 1 -- 172.21.15.16:0/4100812296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55c7a9453170 con 0x55c7a952aea0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76cb7fe640 1 -- 172.21.15.16:0/4100812296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e003ab00 con 0x55c7a952aea0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76cb7fe640 1 -- 172.21.15.16:0/4100812296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f76e4007030 con 0x55c7a952aea0 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76f5c1f740 10 client.6363 fetch_fsmap finished waiting for FSMap version 34 2023-11-24T06:30:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76f5c1f740 10 client.6363 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76f5c1f740 20 client.6363 populate_metadata read hostname 'smithi016' 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] conn(0x55c7a9551d00 0x55c7a95520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76f5c1f740 4 client.6363 mds_command: new command op to 6201 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.015+0000 7f76f5c1f740 1 -- 172.21.15.16:0/4100812296 --> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c7a91926c0 con 0x55c7a9551d00 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.016+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] conn(0x55c7a9551d00 0x55c7a95520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.016+0000 7f76f0e2c640 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] conn(0x55c7a9551d00 0x55c7a95520e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1808cc0272f09a87 server_cookie=78fea068c5b1ddb5 in_seq=0 out_seq=0 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.016+0000 7f76cb7fe640 10 client.6363 ms_handle_connect on v2:172.21.15.16:6832/515502120 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.016+0000 7f76cb7fe640 1 -- 172.21.15.16:0/4100812296 <== mds.0 v2:172.21.15.16:6832/515502120 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55c7a91926c0 con 0x55c7a9551d00 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.016+0000 7f76cb7fe640 10 client.6363 handle_command_reply: tid=0 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.016+0000 7f76f5c1f740 1 client.6363 shutdown 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 -- 172.21.15.16:0/4100812296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 msgr2=0x55c7a9574010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9574010 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f76e400f4d0 tx=0x7f76e40013a0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 -- 172.21.15.16:0/4100812296 shutdown_connections 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.16:6832/515502120,v1:172.21.15.16:6833/515502120] conn(0x55c7a9551d00 0x55c7a95520e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7a952aea0 0x55c7a9574010 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c7a952a5e0 0x55c7a9573ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 --2- 172.21.15.16:0/4100812296 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55c7a9525ba0 0x55c7a95dd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76cb7fe640 0 client.6363 ms_handle_reset on v2:172.21.15.16:6832/515502120 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 -- 172.21.15.16:0/4100812296 >> 172.21.15.16:0/4100812296 conn(0x55c7a94ecc70 msgr2=0x55c7a9567900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 -- 172.21.15.16:0/4100812296 shutdown_connections 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 1 -- 172.21.15.16:0/4100812296 wait complete. 2023-11-24T06:30:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.017+0000 7f76f5c1f740 20 client.6363 trim_cache size 0 max 16384 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6dc1668b-58b0-47dd-b8dc-4164e5a228ab", 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6201, 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.8118740669999998, 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 34, 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 59, 2023-11-24T06:30:00.024 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 59, 2023-11-24T06:30:00.025 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 13.159669888 2023-11-24T06:30:00.025 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:30:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 -- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f76d006b080 msgr2=0x7f76d006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f76d006b080 0x7f76d006d540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f76e0036000 tx=0x7f76e003c040 comp rx=0 tx=0).stop 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 -- 172.21.15.16:0/1051168502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13e360 msgr2=0x7f76ec1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13e360 0x7f76ec1576e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f76d800b4a0 tx=0x7f76d800b960 comp rx=0 tx=0).stop 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 -- 172.21.15.16:0/1051168502 shutdown_connections 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f76d006b080 0x7f76d006d540 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76ec15f9c0 0x7f76ec157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec13e360 0x7f76ec1576e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 --2- 172.21.15.16:0/1051168502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76ec13df80 0x7f76ec0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 -- 172.21.15.16:0/1051168502 >> 172.21.15.16:0/1051168502 conn(0x7f76ec06a800 msgr2=0x7f76ec06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 -- 172.21.15.16:0/1051168502 shutdown_connections 2023-11-24T06:30:00.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.018+0000 7f76f3090640 1 -- 172.21.15.16:0/1051168502 wait complete. 2023-11-24T06:30:00.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:30:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.264+0000 7f8f8ce71640 1 Processor -- start 2023-11-24T06:30:00.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.264+0000 7f8f8ce71640 1 -- start start 2023-11-24T06:30:00.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.266+0000 7f8f8ce71640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f88071410 0x7f8f88146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.266+0000 7f8f8ce71640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8f88071f00 0x7f8f88166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.266+0000 7f8f8ce71640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f880b2e10 con 0x7f8f8806c7c0 2023-11-24T06:30:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.266+0000 7f8f8ce71640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8806cc50 con 0x7f8f88071410 2023-11-24T06:30:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.266+0000 7f8f8ce71640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8806cec0 con 0x7f8f88071f00 2023-11-24T06:30:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.266+0000 7f8f86ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8f8806c7c0 0x7f8f880717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43094/0 (socket says 172.21.15.16:43094) 2023-11-24T06:30:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.266+0000 7f8f86ffd640 1 -- 172.21.15.16:0/2813992885 learned_addr learned my addr 172.21.15.16:0/2813992885 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:00.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.267+0000 7f8f877fe640 1 --2- 172.21.15.16:0/2813992885 >> v2:172.21.15.16:3300/0 conn(0x7f8f88071410 0x7f8f88146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.267+0000 7f8f867fc640 1 -- 172.21.15.16:0/2813992885 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053751185 0 0) 0x7f8f880b2e10 con 0x7f8f8806c7c0 2023-11-24T06:30:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.267+0000 7f8f867fc640 1 -- 172.21.15.16:0/2813992885 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f70003520 con 0x7f8f8806c7c0 2023-11-24T06:30:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.267+0000 7f8f87fff640 1 --2- 172.21.15.16:0/2813992885 >> v2:172.21.15.173:3300/0 conn(0x7f8f88071f00 0x7f8f88166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.267+0000 7f8f877fe640 1 -- 172.21.15.16:0/2813992885 >> v2:172.21.15.173:3300/0 conn(0x7f8f88071f00 msgr2=0x7f8f88166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.267+0000 7f8f877fe640 1 --2- 172.21.15.16:0/2813992885 >> v2:172.21.15.173:3300/0 conn(0x7f8f88071f00 0x7f8f88166cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.267+0000 7f8f867fc640 1 -- 172.21.15.16:0/2813992885 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 638779465 0 0) 0x7f8f70003520 con 0x7f8f8806c7c0 2023-11-24T06:30:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.267+0000 7f8f877fe640 1 -- 172.21.15.16:0/2813992885 >> v1:172.21.15.16:6789/0 conn(0x7f8f8806c7c0 legacy=0x7f8f880717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.267+0000 7f8f877fe640 1 -- 172.21.15.16:0/2813992885 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8806d1a0 con 0x7f8f88071410 2023-11-24T06:30:00.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.269+0000 7f8f877fe640 1 --2- 172.21.15.16:0/2813992885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071410 0x7f8f88146770 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8f880ad5c0 tx=0x7f8f7802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cd82a1b7416351 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:00.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.269+0000 7f8f867fc640 1 -- 172.21.15.16:0/2813992885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c001670 con 0x7f8f88071410 2023-11-24T06:30:00.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.269+0000 7f8f867fc640 1 -- 172.21.15.16:0/2813992885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f78033440 con 0x7f8f88071410 2023-11-24T06:30:00.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.269+0000 7f8f867fc640 1 -- 172.21.15.16:0/2813992885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f780337d0 con 0x7f8f88071410 2023-11-24T06:30:00.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.269+0000 7f8f8ce71640 1 -- 172.21.15.16:0/2813992885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071410 msgr2=0x7f8f88146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.269+0000 7f8f8ce71640 1 --2- 172.21.15.16:0/2813992885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071410 0x7f8f88146770 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8f880ad5c0 tx=0x7f8f7802b7a0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.270+0000 7f8f8ce71640 1 -- 172.21.15.16:0/2813992885 shutdown_connections 2023-11-24T06:30:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.270+0000 7f8f8ce71640 1 --2- 172.21.15.16:0/2813992885 >> v2:172.21.15.173:3300/0 conn(0x7f8f88071f00 0x7f8f88166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.270+0000 7f8f8ce71640 1 --2- 172.21.15.16:0/2813992885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071410 0x7f8f88146770 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.270+0000 7f8f8ce71640 1 -- 172.21.15.16:0/2813992885 >> 172.21.15.16:0/2813992885 conn(0x7f8f8806a800 msgr2=0x7f8f8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.270+0000 7f8f8ce71640 1 -- 172.21.15.16:0/2813992885 shutdown_connections 2023-11-24T06:30:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.270+0000 7f8f8ce71640 1 -- 172.21.15.16:0/2813992885 wait complete. 2023-11-24T06:30:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.272+0000 7f8f8ce71640 1 Processor -- start 2023-11-24T06:30:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.272+0000 7f8f8ce71640 1 -- start start 2023-11-24T06:30:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f8ce71640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f8806c7c0 0x7f8f8815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f8ce71640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f88071410 0x7f8f881422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f8ce71640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071f00 0x7f8f88142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f8ce71640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8815d0c0 con 0x7f8f88071f00 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f8ce71640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f88142e60 con 0x7f8f8806c7c0 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f8ce71640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f88143140 con 0x7f8f88071410 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071f00 0x7f8f88142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071f00 0x7f8f88142830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43434/0 (socket says 172.21.15.16:43434) 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f87fff640 1 -- 172.21.15.16:0/29518108 learned_addr learned my addr 172.21.15.16:0/29518108 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f877fe640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f8806c7c0 0x7f8f8815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f87fff640 1 -- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f88071410 msgr2=0x7f8f881422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f86ffd640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f88071410 0x7f8f881422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f87fff640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f88071410 0x7f8f881422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f87fff640 1 -- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f8806c7c0 msgr2=0x7f8f8815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f87fff640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f8806c7c0 0x7f8f8815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.273+0000 7f8f87fff640 1 -- 172.21.15.16:0/29518108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78009400 con 0x7f8f88071f00 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.274+0000 7f8f877fe640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f8806c7c0 0x7f8f8815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.274+0000 7f8f86ffd640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f88071410 0x7f8f881422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.274+0000 7f8f87fff640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071f00 0x7f8f88142830 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000cc30 tx=0x7f8f8000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.274+0000 7f8f6ffff640 1 -- 172.21.15.16:0/29518108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8000ce40 con 0x7f8f88071f00 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.274+0000 7f8f6ffff640 1 -- 172.21.15.16:0/29518108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f80003990 con 0x7f8f88071f00 2023-11-24T06:30:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.274+0000 7f8f6ffff640 1 -- 172.21.15.16:0/29518108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80003d20 con 0x7f8f88071f00 2023-11-24T06:30:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.274+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8813d310 con 0x7f8f88071f00 2023-11-24T06:30:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.274+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8813d8d0 con 0x7f8f88071f00 2023-11-24T06:30:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.276+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8813daf0 con 0x7f8f88071f00 2023-11-24T06:30:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.277+0000 7f8f6ffff640 1 -- 172.21.15.16:0/29518108 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8f80003e80 con 0x7f8f88071f00 2023-11-24T06:30:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.277+0000 7f8f6ffff640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f7406b080 0x7f8f7406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.278+0000 7f8f6ffff640 1 -- 172.21.15.16:0/29518108 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f80089e90 con 0x7f8f88071f00 2023-11-24T06:30:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.278+0000 7f8f877fe640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f7406b080 0x7f8f7406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.279+0000 7f8f877fe640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f7406b080 0x7f8f7406d540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f78035000 tx=0x7f8f7802f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.282+0000 7f8f6ffff640 1 -- 172.21.15.16:0/29518108 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f80054b60 con 0x7f8f88071f00 2023-11-24T06:30:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.515+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8f88067450 con 0x7f8f88071f00 2023-11-24T06:30:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.515+0000 7f8f6ffff640 1 -- 172.21.15.16:0/29518108 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8f8001d070 con 0x7f8f88071f00 2023-11-24T06:30:00.518 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:30:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f7406b080 msgr2=0x7f8f7406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f7406b080 0x7f8f7406d540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f78035000 tx=0x7f8f7802f000 comp rx=0 tx=0).stop 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071f00 msgr2=0x7f8f88142830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071f00 0x7f8f88142830 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000cc30 tx=0x7f8f8000a5c0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 shutdown_connections 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8f7406b080 0x7f8f7406d540 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f88071f00 0x7f8f88142830 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8f88071410 0x7f8f881422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 --2- 172.21.15.16:0/29518108 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f8806c7c0 0x7f8f8815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 >> 172.21.15.16:0/29518108 conn(0x7f8f8806a800 msgr2=0x7f8f880b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 shutdown_connections 2023-11-24T06:30:00.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:00.518+0000 7f8f8ce71640 1 -- 172.21.15.16:0/29518108 wait complete. 2023-11-24T06:30:00.540 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:30:00.540 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:30:00.540 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-24T06:30:00.541 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:30:00.541 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:00.541 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:30:00.541 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:30:00.541 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:30:00.542 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:30:00.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:00.542 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-24T06:30:00.596 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:30:00.597 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:30:00.597 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:30:00.597 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:00.597 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-24T06:30:00.598 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:00.598 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:00.598 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:00.598 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-24T06:30:00.598 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:00.598 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:00.599 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-24T06:30:00.599 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:30:00.599 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:00.599 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-24T06:30:00.599 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:30:00.599 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42143sec preferred_lft 42143sec 2023-11-24T06:30:00.600 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-24T06:30:00.600 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:00.600 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:00.600 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:00.600 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-24T06:30:00.600 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:30:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:30:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8cfd:d2ff:fee1:5b18/64 scope link 2023-11-24T06:30:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:00.601 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:30:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8c89:e0ff:fefc:a3fb/64 scope link 2023-11-24T06:30:00.602 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:00.602 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:30:00.602 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether ea:e3:8b:37:17:ee brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:00.602 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e8e3:8bff:fe37:17ee/64 scope link 2023-11-24T06:30:00.602 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:00.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:00.603 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:30:00.702 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-24T06:30:00.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:00.707 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-24T06:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-24T06:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:30:00.768 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:30:00.769 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:30:00.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:00.769 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:30:00.769 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:30:00.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:30:00.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:30:00.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:30:00.769 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:30:00.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:00.962 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-24T06:30:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:30:01.018 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:30:01.019 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:30:01.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:01.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-24T06:30:01.115 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:30:01.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:30:01.116 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:01.170 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:30:01.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:30:01.171 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:01.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:30:01.328 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:30:01.388 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:30:01.388 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:30:01.388 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:30:01.388 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:30:01.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:30:01.390 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:30:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:30:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:30:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:30:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:30:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:01.391 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:01.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:01.392 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:30:01.450 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:30:01.450 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:30:01.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:30:01.539 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:30:01.582 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:30:01.583 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:30:01.583 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:30:01.583 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:30:01.583 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:30:01.584 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:30:01.585 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:30:01.585 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:30:01.585 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:30:01.585 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:30:01.585 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:30:01.585 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:30:01.585 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:30:01.585 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:30:01.585 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:30:01.586 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:30:01.586 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:30:01.586 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:30:01.586 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:30:01.586 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:30:01.586 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:30:01.587 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:01.587 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:01.587 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:01.587 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:01.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:01.588 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:30:01.654 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-24T06:30:01.652+0000 7f8679a43480 -1 init, newargv = 0x55f425100fc0 newargc=15 2023-11-24T06:30:01.655 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[108126]: starting ceph client 2023-11-24T06:30:01.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[108126]: starting fuse 2023-11-24T06:30:01.669 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-24T06:30:01.670 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:30:01.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:30:01.670 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-24T06:30:01.670 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-24T06:30:01.670 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-24T06:30:01.670 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-24T06:30:01.670 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:30:01.671 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:30:01.672 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:30:01.672 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-24T06:30:01.672 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-24T06:30:01.672 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-24T06:30:01.672 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:30:01.672 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:30:01.672 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-24T06:30:01.672 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-24T06:30:01.871 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.108126.asok 2023-11-24T06:30:01.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:30:01.889 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.108126.asok 2023-11-24T06:30:01.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:01.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.108126.asok status 2023-11-24T06:30:02.142 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:30:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-24T06:30:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:30:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:30:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-24T06:30:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-24T06:30:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:30:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "108126", 2023-11-24T06:30:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6378, 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 6378 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:30:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1436573085 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1436573085 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.6378 192.168.0.1:0/1436573085", 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1436573085", 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 34, 2023-11-24T06:30:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 59, 2023-11-24T06:30:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:30:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-24T06:30:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-24T06:30:02.145 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:30:02.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:02.158 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-24T06:30:02.158 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:02.215 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-24T06:30:02.216 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:02.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:02.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:02.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:30:02.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.540+0000 7f5f7111d640 1 Processor -- start 2023-11-24T06:30:02.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.540+0000 7f5f7111d640 1 -- start start 2023-11-24T06:30:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.541+0000 7f5f7111d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c15d3a0 0x7f5f6c06d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.541+0000 7f5f7111d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5f6c15d780 0x7f5f6c0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.541+0000 7f5f7111d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6c161430 con 0x7f5f6c160f90 2023-11-24T06:30:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.541+0000 7f5f7111d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c1616f0 con 0x7f5f6c15d780 2023-11-24T06:30:02.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.541+0000 7f5f7111d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c1619d0 con 0x7f5f6c15d3a0 2023-11-24T06:30:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.541+0000 7f5f6a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5f6c160f90 0x7f5f6c0d0a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43098/0 (socket says 172.21.15.16:43098) 2023-11-24T06:30:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.541+0000 7f5f6a59c640 1 -- 172.21.15.16:0/1577987732 learned_addr learned my addr 172.21.15.16:0/1577987732 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f6ad9d640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c15d3a0 0x7f5f6c06d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:02.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f6b59e640 1 --2- 172.21.15.16:0/1577987732 >> v2:172.21.15.173:3300/0 conn(0x7f5f6c15d780 0x7f5f6c0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293006104 0 0) 0x7f5f6c161430 con 0x7f5f6c160f90 2023-11-24T06:30:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f54003520 con 0x7f5f6c160f90 2023-11-24T06:30:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2231215272 0 0) 0x7f5f54003520 con 0x7f5f6c160f90 2023-11-24T06:30:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c15d3a0 msgr2=0x7f5f6c06d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:02.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c15d3a0 0x7f5f6c06d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 >> v2:172.21.15.173:3300/0 conn(0x7f5f6c15d780 msgr2=0x7f5f6c0b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 --2- 172.21.15.16:0/1577987732 >> v2:172.21.15.173:3300/0 conn(0x7f5f6c15d780 0x7f5f6c0b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c161cb0 con 0x7f5f6c160f90 2023-11-24T06:30:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f5f600019f0 con 0x7f5f6c160f90 2023-11-24T06:30:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 >> v1:172.21.15.16:6789/0 conn(0x7f5f6c160f90 legacy=0x7f5f6c0d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f54003970 0x7f5f54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f540047c0 0x7f5f54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c15d3a0 0x7f5f54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c1616f0 con 0x7f5f540047c0 2023-11-24T06:30:02.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c1619d0 con 0x7f5f54003970 2023-11-24T06:30:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5400b980 con 0x7f5f6c15d3a0 2023-11-24T06:30:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.542+0000 7f5f6b59e640 1 --2- 172.21.15.16:0/1577987732 >> v2:172.21.15.173:3300/0 conn(0x7f5f6c15d780 0x7f5f6c0b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:30:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.543+0000 7f5f6a59c640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f540047c0 0x7f5f54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.543+0000 7f5f6ad9d640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f54003970 0x7f5f54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:02.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.543+0000 7f5f6a59c640 1 -- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c15d3a0 msgr2=0x7f5f54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.543+0000 7f5f6a59c640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c15d3a0 0x7f5f54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.543+0000 7f5f6a59c640 1 -- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f54003970 msgr2=0x7f5f54003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.543+0000 7f5f6a59c640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f54003970 0x7f5f54003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.543+0000 7f5f6a59c640 1 -- 172.21.15.16:0/1577987732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f54003520 con 0x7f5f540047c0 2023-11-24T06:30:02.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.543+0000 7f5f6ad9d640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f54003970 0x7f5f54003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.544+0000 7f5f6a59c640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f540047c0 0x7f5f54004c40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000a830 tx=0x7f5f540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305f68bbce2d2f43 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60034630 con 0x7f5f540047c0 2023-11-24T06:30:02.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6002f070 con 0x7f5f540047c0 2023-11-24T06:30:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f69d9b640 1 -- 172.21.15.16:0/1577987732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600349f0 con 0x7f5f540047c0 2023-11-24T06:30:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f7111d640 1 -- 172.21.15.16:0/1577987732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f540047c0 msgr2=0x7f5f54004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f7111d640 1 --2- 172.21.15.16:0/1577987732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f540047c0 0x7f5f54004c40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000a830 tx=0x7f5f540057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f6b59e640 1 -- 172.21.15.16:0/1577987732 reap_dead start 2023-11-24T06:30:02.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f7111d640 1 -- 172.21.15.16:0/1577987732 shutdown_connections 2023-11-24T06:30:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f7111d640 1 -- 172.21.15.16:0/1577987732 >> 172.21.15.16:0/1577987732 conn(0x7f5f6c06e7a0 msgr2=0x7f5f6c06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f7111d640 1 -- 172.21.15.16:0/1577987732 shutdown_connections 2023-11-24T06:30:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.545+0000 7f5f7111d640 1 -- 172.21.15.16:0/1577987732 wait complete. 2023-11-24T06:30:02.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.547+0000 7f5f7111d640 1 Processor -- start 2023-11-24T06:30:02.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.547+0000 7f5f7111d640 1 -- start start 2023-11-24T06:30:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.548+0000 7f5f7111d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f540047c0 0x7f5f6c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.548+0000 7f5f7111d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c159c80 0x7f5f6c14cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.548+0000 7f5f7111d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c14d4e0 0x7f5f6c1f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.548+0000 7f5f7111d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c14e570 con 0x7f5f6c159c80 2023-11-24T06:30:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.548+0000 7f5f7111d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c14d960 con 0x7f5f540047c0 2023-11-24T06:30:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.548+0000 7f5f7111d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c14dc40 con 0x7f5f6c14d4e0 2023-11-24T06:30:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.548+0000 7f5f6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c159c80 0x7f5f6c14cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c159c80 0x7f5f6c14cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36298/0 (socket says 172.21.15.16:36298) 2023-11-24T06:30:02.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6a59c640 1 -- 172.21.15.16:0/2842387699 learned_addr learned my addr 172.21.15.16:0/2842387699 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6ad9d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f540047c0 0x7f5f6c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6b59e640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c14d4e0 0x7f5f6c1f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6a59c640 1 -- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c14d4e0 msgr2=0x7f5f6c1f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6a59c640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c14d4e0 0x7f5f6c1f6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6a59c640 1 -- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f540047c0 msgr2=0x7f5f6c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6a59c640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f540047c0 0x7f5f6c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6a59c640 1 -- 172.21.15.16:0/2842387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c14df20 con 0x7f5f6c159c80 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6ad9d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f540047c0 0x7f5f6c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f6a59c640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c159c80 0x7f5f6c14cfa0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000a830 tx=0x7f5f6c161f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.549+0000 7f5f537fe640 1 -- 172.21.15.16:0/2842387699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600346b0 con 0x7f5f6c159c80 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.550+0000 7f5f537fe640 1 -- 172.21.15.16:0/2842387699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60034810 con 0x7f5f6c159c80 2023-11-24T06:30:02.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.550+0000 7f5f537fe640 1 -- 172.21.15.16:0/2842387699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60034bc0 con 0x7f5f6c159c80 2023-11-24T06:30:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.550+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c14e200 con 0x7f5f6c159c80 2023-11-24T06:30:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.550+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c1f70a0 con 0x7f5f6c159c80 2023-11-24T06:30:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.551+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6c14e6e0 con 0x7f5f6c159c80 2023-11-24T06:30:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.552+0000 7f5f537fe640 1 -- 172.21.15.16:0/2842387699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5f60031030 con 0x7f5f6c159c80 2023-11-24T06:30:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.553+0000 7f5f537fe640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5f5806b080 0x7f5f5806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.553+0000 7f5f537fe640 1 -- 172.21.15.16:0/2842387699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f600ab490 con 0x7f5f6c159c80 2023-11-24T06:30:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.554+0000 7f5f6ad9d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5f5806b080 0x7f5f5806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.555+0000 7f5f6ad9d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5f5806b080 0x7f5f5806d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0099c0 tx=0x7f5f5c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:02.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.558+0000 7f5f537fe640 1 -- 172.21.15.16:0/2842387699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f60076110 con 0x7f5f6c159c80 2023-11-24T06:30:02.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.791+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5f6c0675d0 con 0x7f5f6c159c80 2023-11-24T06:30:02.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.792+0000 7f5f537fe640 1 -- 172.21.15.16:0/2842387699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5f60079730 con 0x7f5f6c159c80 2023-11-24T06:30:02.794 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:30:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5f5806b080 msgr2=0x7f5f5806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5f5806b080 0x7f5f5806d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0099c0 tx=0x7f5f5c008040 comp rx=0 tx=0).stop 2023-11-24T06:30:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c159c80 msgr2=0x7f5f6c14cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c159c80 0x7f5f6c14cfa0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000a830 tx=0x7f5f6c161f90 comp rx=0 tx=0).stop 2023-11-24T06:30:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 shutdown_connections 2023-11-24T06:30:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5f5806b080 0x7f5f5806d540 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f6c14d4e0 0x7f5f6c1f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c159c80 0x7f5f6c14cfa0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 --2- 172.21.15.16:0/2842387699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f540047c0 0x7f5f6c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 >> 172.21.15.16:0/2842387699 conn(0x7f5f6c06e7a0 msgr2=0x7f5f6c0cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 shutdown_connections 2023-11-24T06:30:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:02.795+0000 7f5f7111d640 1 -- 172.21.15.16:0/2842387699 wait complete. 2023-11-24T06:30:02.823 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:30:02.823 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:30:02.823 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2023-11-24T06:30:02.824 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:30:02.824 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:02.824 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:30:02.824 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:30:02.824 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:30:02.824 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:30:02.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:02.824 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2023-11-24T06:30:02.886 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42201sec preferred_lft 42201sec 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:02.887 INFO:teuthology.orchestra.run.smithi173.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::c4e1:59ff:feae:2f56/64 scope link 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ea:3b:9f:a2:9b:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::e83b:9fff:fea2:9ba6/64 scope link 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::64de:a3ff:fee0:a827/64 scope link 2023-11-24T06:30:02.888 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:02.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:02.890 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:30:02.981 INFO:teuthology.orchestra.run.smithi173.stdout:1 2023-11-24T06:30:02.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:02.985 DEBUG:teuthology.orchestra.run.smithi173:> route 2023-11-24T06:30:03.038 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2023-11-24T06:30:03.039 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:30:03.039 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:30:03.039 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:30:03.039 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:30:03.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:03.040 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:30:03.040 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:30:03.040 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:30:03.040 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:30:03.040 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:30:03.040 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:30:03.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:03.214 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2023-11-24T06:30:03.264 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:30:03.264 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:30:03.265 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:30:03.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:03.265 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2023-11-24T06:30:03.360 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:30:03.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:30:03.360 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:03.415 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:30:03.415 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:30:03.415 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:03.466 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:30:03.557 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:30:03.616 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:30:03.617 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:30:03.617 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:30:03.617 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:30:03.617 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:30:03.617 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:30:03.617 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:30:03.617 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:30:03.618 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:30:03.618 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:30:03.618 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:30:03.618 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:30:03.618 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:30:03.618 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:30:03.618 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:30:03.619 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:30:03.619 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:30:03.619 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:30:03.619 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:30:03.619 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:30:03.619 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:03.620 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:03.621 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:03.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:03.622 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:30:03.674 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:30:03.674 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:30:03.680 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:30:03.758 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:30:03.805 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:30:03.806 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:30:03.806 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:30:03.806 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:30:03.806 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:30:03.806 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:30:03.806 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:30:03.806 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:30:03.806 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:30:03.806 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:03.807 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:03.808 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:03.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:03.809 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:30:03.874 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:2023-11-24T06:30:03.871+0000 7f3500e3e480 -1 init, newargv = 0x559e2a597800 newargc=15 2023-11-24T06:30:03.875 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[96883]: starting ceph client 2023-11-24T06:30:03.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[96883]: starting fuse 2023-11-24T06:30:03.896 INFO:teuthology.orchestra.run.smithi173.stdout:39 2023-11-24T06:30:03.897 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-11-24T06:30:03.897 DEBUG:teuthology.orchestra.run.smithi173:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:30:03.897 DEBUG:teuthology.orchestra.run.smithi173:> import glob 2023-11-24T06:30:03.897 DEBUG:teuthology.orchestra.run.smithi173:> import re 2023-11-24T06:30:03.897 DEBUG:teuthology.orchestra.run.smithi173:> import os 2023-11-24T06:30:03.897 DEBUG:teuthology.orchestra.run.smithi173:> import subprocess 2023-11-24T06:30:03.897 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:30:03.897 DEBUG:teuthology.orchestra.run.smithi173:> def _find_admin_socket(client_name): 2023-11-24T06:30:03.898 DEBUG:teuthology.orchestra.run.smithi173:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:30:03.898 DEBUG:teuthology.orchestra.run.smithi173:> files = glob.glob(asok_path) 2023-11-24T06:30:03.898 DEBUG:teuthology.orchestra.run.smithi173:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:30:03.898 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:30:03.898 DEBUG:teuthology.orchestra.run.smithi173:> # Given a non-glob path, it better be there 2023-11-24T06:30:03.899 DEBUG:teuthology.orchestra.run.smithi173:> if "*" not in asok_path: 2023-11-24T06:30:03.899 DEBUG:teuthology.orchestra.run.smithi173:> assert(len(files) == 1) 2023-11-24T06:30:03.899 DEBUG:teuthology.orchestra.run.smithi173:> return files[0] 2023-11-24T06:30:03.899 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:30:03.899 DEBUG:teuthology.orchestra.run.smithi173:> for f in files: 2023-11-24T06:30:03.900 DEBUG:teuthology.orchestra.run.smithi173:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:30:03.900 DEBUG:teuthology.orchestra.run.smithi173:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:30:03.900 DEBUG:teuthology.orchestra.run.smithi173:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:30:03.900 DEBUG:teuthology.orchestra.run.smithi173:> contents = proc_f.read() 2023-11-24T06:30:03.900 DEBUG:teuthology.orchestra.run.smithi173:> if mountpoint in contents: 2023-11-24T06:30:03.900 DEBUG:teuthology.orchestra.run.smithi173:> return f 2023-11-24T06:30:03.900 DEBUG:teuthology.orchestra.run.smithi173:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:30:03.900 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:30:03.901 DEBUG:teuthology.orchestra.run.smithi173:> print(_find_admin_socket("client.1")) 2023-11-24T06:30:03.901 DEBUG:teuthology.orchestra.run.smithi173:> ' 2023-11-24T06:30:04.095 INFO:teuthology.orchestra.run.smithi173.stdout:/var/run/ceph/ceph-client.1.96883.asok 2023-11-24T06:30:04.099 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:30:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:30:04.112 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.96883.asok 2023-11-24T06:30:04.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:04.113 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.96883.asok status 2023-11-24T06:30:04.350 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-11-24T06:30:04.350 INFO:teuthology.orchestra.run.smithi173.stdout: "metadata": { 2023-11-24T06:30:04.350 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: "entity_id": "1", 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: "hostname": "smithi173", 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: "pid": "96883", 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: "root": "/" 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_count": 0, 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_pinned_count": 0, 2023-11-24T06:30:04.351 INFO:teuthology.orchestra.run.smithi173.stdout: "id": 5099, 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: "inst": { 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: "name": { 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "client", 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: "num": 5099 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 1726285761 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:30:04.352 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 1726285761 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "inst_str": "client.5099 192.168.0.1:0/1726285761", 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "addr_str": "192.168.0.1:0/1726285761", 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "inode_count": 1, 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_epoch": 34, 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch": 59, 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "blocklisted": false, 2023-11-24T06:30:04.353 INFO:teuthology.orchestra.run.smithi173.stdout: "fs_name": "cephfs" 2023-11-24T06:30:04.354 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-11-24T06:30:04.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:04.365 DEBUG:teuthology.orchestra.run.smithi173:> stat --file-system '--printf=%T 2023-11-24T06:30:04.365 DEBUG:teuthology.orchestra.run.smithi173:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:04.420 INFO:teuthology.orchestra.run.smithi173.stdout:fuseblk 2023-11-24T06:30:04.421 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:04.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:04.421 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:04.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:30:04.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.748+0000 7f855f37c640 1 Processor -- start 2023-11-24T06:30:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.748+0000 7f855f37c640 1 -- start start 2023-11-24T06:30:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.750+0000 7f855f37c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f855813df80 0x7f855815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.750+0000 7f855f37c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855806c470 con 0x7f85580cf9a0 2023-11-24T06:30:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.750+0000 7f855f37c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855806c750 con 0x7f855813e360 2023-11-24T06:30:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.750+0000 7f855f37c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855806ca30 con 0x7f855813df80 2023-11-24T06:30:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.750+0000 7f855d919640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f85580cf9a0 0x7f85580cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35452/0 (socket says 172.21.15.16:35452) 2023-11-24T06:30:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.750+0000 7f855d919640 1 -- 172.21.15.16:0/788320044 learned_addr learned my addr 172.21.15.16:0/788320044 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f855d118640 1 --2- 172.21.15.16:0/788320044 >> v2:172.21.15.173:3300/0 conn(0x7f855813df80 0x7f855815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426632440 0 0) 0x7f855806c470 con 0x7f85580cf9a0 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8544003520 con 0x7f85580cf9a0 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1169860054 0 0) 0x7f8544003520 con 0x7f85580cf9a0 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 >> v2:172.21.15.173:3300/0 conn(0x7f855813df80 msgr2=0x7f855815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 --2- 172.21.15.16:0/788320044 >> v2:172.21.15.173:3300/0 conn(0x7f855813df80 0x7f855815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 >> v1:172.21.15.173:6789/0 conn(0x7f855813e360 legacy=0x7f85580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855806cd10 con 0x7f85580cf9a0 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f85500019f0 con 0x7f85580cf9a0 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 >> v1:172.21.15.16:6789/0 conn(0x7f85580cf9a0 legacy=0x7f85580cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 --2- 172.21.15.16:0/788320044 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85440039c0 0x7f8544003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 --2- 172.21.15.16:0/788320044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f8544004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:04.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 --2- 172.21.15.16:0/788320044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544005180 0x7f854400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855806ca30 con 0x7f85440047c0 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854400c080 con 0x7f8544005180 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.751+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854400c240 con 0x7f85440039c0 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.752+0000 7f855c917640 1 --2- 172.21.15.16:0/788320044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f8544004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.752+0000 7f855c917640 1 -- 172.21.15.16:0/788320044 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85440039c0 msgr2=0x7f8544003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.752+0000 7f855c917640 1 --2- 172.21.15.16:0/788320044 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85440039c0 0x7f8544003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.752+0000 7f855c917640 1 -- 172.21.15.16:0/788320044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544005180 msgr2=0x7f854400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.752+0000 7f855c917640 1 --2- 172.21.15.16:0/788320044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8544005180 0x7f854400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.752+0000 7f855c917640 1 -- 172.21.15.16:0/788320044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8544003520 con 0x7f85440047c0 2023-11-24T06:30:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.752+0000 7f855d919640 1 -- 172.21.15.16:0/788320044 reap_dead start 2023-11-24T06:30:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.753+0000 7f855c917640 1 --2- 172.21.15.16:0/788320044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f8544004c40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8548001720 tx=0x7f85440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea42e01bf016832 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.753+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480052d0 con 0x7f85440047c0 2023-11-24T06:30:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.753+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854802e050 con 0x7f85440047c0 2023-11-24T06:30:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.753+0000 7f854ffff640 1 -- 172.21.15.16:0/788320044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548003830 con 0x7f85440047c0 2023-11-24T06:30:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.754+0000 7f855f37c640 1 -- 172.21.15.16:0/788320044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 msgr2=0x7f8544004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.754+0000 7f855f37c640 1 --2- 172.21.15.16:0/788320044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f8544004c40 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8548001720 tx=0x7f85440057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.754+0000 7f855f37c640 1 -- 172.21.15.16:0/788320044 shutdown_connections 2023-11-24T06:30:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.754+0000 7f855f37c640 1 --2- 172.21.15.16:0/788320044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f8544004c40 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.754+0000 7f855f37c640 1 -- 172.21.15.16:0/788320044 >> 172.21.15.16:0/788320044 conn(0x7f855806a800 msgr2=0x7f855806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.754+0000 7f855f37c640 1 -- 172.21.15.16:0/788320044 shutdown_connections 2023-11-24T06:30:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.754+0000 7f855f37c640 1 -- 172.21.15.16:0/788320044 wait complete. 2023-11-24T06:30:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.756+0000 7f855f37c640 1 Processor -- start 2023-11-24T06:30:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.756+0000 7f855f37c640 1 -- start start 2023-11-24T06:30:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.757+0000 7f855f37c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f85581e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.757+0000 7f855f37c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85580ade70 0x7f855813d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.757+0000 7f855f37c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85580b2cf0 0x7f85580b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.757+0000 7f855f37c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85580b36b0 con 0x7f85440047c0 2023-11-24T06:30:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.757+0000 7f855f37c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85581e7790 con 0x7f85580b2cf0 2023-11-24T06:30:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.757+0000 7f855f37c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85581e5030 con 0x7f85580ade70 2023-11-24T06:30:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.757+0000 7f855d118640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f85581e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855d118640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f85581e4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36304/0 (socket says 172.21.15.16:36304) 2023-11-24T06:30:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855d118640 1 -- 172.21.15.16:0/1526532981 learned_addr learned my addr 172.21.15.16:0/1526532981 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855c917640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85580ade70 0x7f855813d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855d919640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85580b2cf0 0x7f85580b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855d118640 1 -- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85580ade70 msgr2=0x7f855813d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855d118640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85580ade70 0x7f855813d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855d118640 1 -- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85580b2cf0 msgr2=0x7f85580b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855d118640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85580b2cf0 0x7f85580b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855d118640 1 -- 172.21.15.16:0/1526532981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85581e52d0 con 0x7f85440047c0 2023-11-24T06:30:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855c917640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85580ade70 0x7f855813d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.758+0000 7f855d118640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f85581e4af0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f855400e770 tx=0x7f855400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.759+0000 7f854dffb640 1 -- 172.21.15.16:0/1526532981 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400c730 con 0x7f85440047c0 2023-11-24T06:30:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.759+0000 7f854dffb640 1 -- 172.21.15.16:0/1526532981 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855400c890 con 0x7f85440047c0 2023-11-24T06:30:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.759+0000 7f854dffb640 1 -- 172.21.15.16:0/1526532981 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400cc40 con 0x7f85440047c0 2023-11-24T06:30:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.759+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85581e6cb0 con 0x7f85440047c0 2023-11-24T06:30:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.759+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85581e7270 con 0x7f85440047c0 2023-11-24T06:30:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.760+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85581e6b10 con 0x7f85440047c0 2023-11-24T06:30:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.761+0000 7f854dffb640 1 -- 172.21.15.16:0/1526532981 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8554013470 con 0x7f85440047c0 2023-11-24T06:30:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.763+0000 7f854dffb640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f853c06b0b0 0x7f853c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.763+0000 7f854dffb640 1 -- 172.21.15.16:0/1526532981 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8554089b10 con 0x7f85440047c0 2023-11-24T06:30:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.763+0000 7f855c917640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f853c06b0b0 0x7f853c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.765+0000 7f855c917640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f853c06b0b0 0x7f853c06d570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8548033000 tx=0x7f8548009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.768+0000 7f854dffb640 1 -- 172.21.15.16:0/1526532981 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85581e6b10 con 0x7f85440047c0 2023-11-24T06:30:04.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:04.996+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f85581e6b10 con 0x7f85440047c0 2023-11-24T06:30:05.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.239+0000 7f854dffb640 1 -- 172.21.15.16:0/1526532981 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 136+0+0 (secure 0 0 0) 0x7f85581e6b10 con 0x7f85440047c0 2023-11-24T06:30:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f853c06b0b0 msgr2=0x7f853c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f853c06b0b0 0x7f853c06d570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8548033000 tx=0x7f8548009030 comp rx=0 tx=0).stop 2023-11-24T06:30:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 msgr2=0x7f85581e4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f85581e4af0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f855400e770 tx=0x7f855400ec30 comp rx=0 tx=0).stop 2023-11-24T06:30:05.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 shutdown_connections 2023-11-24T06:30:05.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f853c06b0b0 0x7f853c06d570 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85580b2cf0 0x7f85580b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85580ade70 0x7f855813d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 --2- 172.21.15.16:0/1526532981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440047c0 0x7f85581e4af0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 >> 172.21.15.16:0/1526532981 conn(0x7f855806a800 msgr2=0x7f85580cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:05.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 shutdown_connections 2023-11-24T06:30:05.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.243+0000 7f855f37c640 1 -- 172.21.15.16:0/1526532981 wait complete. 2023-11-24T06:30:05.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:30:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.478+0000 7f532f133640 1 Processor -- start 2023-11-24T06:30:05.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.478+0000 7f532f133640 1 -- start start 2023-11-24T06:30:05.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.479+0000 7f532f133640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53280ae5a0 0x7f532815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532f133640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f53280cf290 0x7f53280cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532f133640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5328070c40 con 0x7f53280b3ae0 2023-11-24T06:30:05.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532f133640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5328071450 con 0x7f53280ae5a0 2023-11-24T06:30:05.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532f133640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53280cfc60 con 0x7f53280cf290 2023-11-24T06:30:05.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532cecf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53280ae5a0 0x7f532815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532cecf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53280ae5a0 0x7f532815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36306/0 (socket says 172.21.15.16:36306) 2023-11-24T06:30:05.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532cecf640 1 -- 172.21.15.16:0/2773458462 learned_addr learned my addr 172.21.15.16:0/2773458462 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:05.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532cecf640 1 -- 172.21.15.16:0/2773458462 >> v2:172.21.15.173:3300/0 conn(0x7f53280cf290 msgr2=0x7f53280cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:05.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532d6d0640 1 --2- 172.21.15.16:0/2773458462 >> v2:172.21.15.173:3300/0 conn(0x7f53280cf290 0x7f53280cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532cecf640 1 --2- 172.21.15.16:0/2773458462 >> v2:172.21.15.173:3300/0 conn(0x7f53280cf290 0x7f53280cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532cecf640 1 -- 172.21.15.16:0/2773458462 >> v1:172.21.15.173:6789/0 conn(0x7f53280b3ae0 legacy=0x7f532806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:05.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532cecf640 1 -- 172.21.15.16:0/2773458462 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532806c470 con 0x7f53280ae5a0 2023-11-24T06:30:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.480+0000 7f532d6d0640 1 --2- 172.21.15.16:0/2773458462 >> v2:172.21.15.173:3300/0 conn(0x7f53280cf290 0x7f53280cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.482+0000 7f532cecf640 1 --2- 172.21.15.16:0/2773458462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 0x7f532815e430 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f53280b3f70 tx=0x7f531802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfa5f107508c0df server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.482+0000 7f53277fe640 1 -- 172.21.15.16:0/2773458462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531802cb20 con 0x7f53280ae5a0 2023-11-24T06:30:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.482+0000 7f53277fe640 1 -- 172.21.15.16:0/2773458462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531802cc80 con 0x7f53280ae5a0 2023-11-24T06:30:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.482+0000 7f53277fe640 1 -- 172.21.15.16:0/2773458462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180343f0 con 0x7f53280ae5a0 2023-11-24T06:30:05.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.483+0000 7f532f133640 1 -- 172.21.15.16:0/2773458462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 msgr2=0x7f532815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.483+0000 7f532f133640 1 --2- 172.21.15.16:0/2773458462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 0x7f532815e430 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f53280b3f70 tx=0x7f531802bf80 comp rx=0 tx=0).stop 2023-11-24T06:30:05.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.483+0000 7f532f133640 1 -- 172.21.15.16:0/2773458462 shutdown_connections 2023-11-24T06:30:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.483+0000 7f532f133640 1 --2- 172.21.15.16:0/2773458462 >> v2:172.21.15.173:3300/0 conn(0x7f53280cf290 0x7f53280cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.483+0000 7f532f133640 1 --2- 172.21.15.16:0/2773458462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 0x7f532815e430 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.483+0000 7f532f133640 1 -- 172.21.15.16:0/2773458462 >> 172.21.15.16:0/2773458462 conn(0x7f532806a800 msgr2=0x7f532806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.483+0000 7f532f133640 1 -- 172.21.15.16:0/2773458462 shutdown_connections 2023-11-24T06:30:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.483+0000 7f532f133640 1 -- 172.21.15.16:0/2773458462 wait complete. 2023-11-24T06:30:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.485+0000 7f532f133640 1 Processor -- start 2023-11-24T06:30:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.485+0000 7f532f133640 1 -- start start 2023-11-24T06:30:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532f133640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 0x7f5328153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532f133640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53280b3ae0 0x7f532814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532f133640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53280cf290 0x7f5328151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532f133640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5328150a10 con 0x7f53280ae5a0 2023-11-24T06:30:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532f133640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532815bfb0 con 0x7f53280b3ae0 2023-11-24T06:30:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532f133640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53281608c0 con 0x7f53280cf290 2023-11-24T06:30:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532cecf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 0x7f5328153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532cecf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 0x7f5328153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36312/0 (socket says 172.21.15.16:36312) 2023-11-24T06:30:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532cecf640 1 -- 172.21.15.16:0/1089279632 learned_addr learned my addr 172.21.15.16:0/1089279632 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532cecf640 1 -- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53280cf290 msgr2=0x7f5328151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532cecf640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53280cf290 0x7f5328151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532cecf640 1 -- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53280b3ae0 msgr2=0x7f532814f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532cecf640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53280b3ae0 0x7f532814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.486+0000 7f532cecf640 1 -- 172.21.15.16:0/1089279632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318009b30 con 0x7f53280ae5a0 2023-11-24T06:30:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.487+0000 7f532cecf640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 0x7f5328153660 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5318036040 tx=0x7f5318004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.487+0000 7f53257fa640 1 -- 172.21.15.16:0/1089279632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318009510 con 0x7f53280ae5a0 2023-11-24T06:30:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.487+0000 7f53257fa640 1 -- 172.21.15.16:0/1089279632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531803e450 con 0x7f53280ae5a0 2023-11-24T06:30:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.487+0000 7f53257fa640 1 -- 172.21.15.16:0/1089279632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531803e7e0 con 0x7f53280ae5a0 2023-11-24T06:30:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.487+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532815c200 con 0x7f53280ae5a0 2023-11-24T06:30:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.487+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532815c7c0 con 0x7f53280ae5a0 2023-11-24T06:30:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.488+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53280675d0 con 0x7f53280ae5a0 2023-11-24T06:30:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.489+0000 7f53257fa640 1 -- 172.21.15.16:0/1089279632 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f531803e940 con 0x7f53280ae5a0 2023-11-24T06:30:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.490+0000 7f53257fa640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f531406b080 0x7f531406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.491+0000 7f53257fa640 1 -- 172.21.15.16:0/1089279632 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53180ac700 con 0x7f53280ae5a0 2023-11-24T06:30:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.491+0000 7f5327fff640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f531406b080 0x7f531406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.492+0000 7f5327fff640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f531406b080 0x7f531406d540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5328159530 tx=0x7f531c007990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:05.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.494+0000 7f53257fa640 1 -- 172.21.15.16:0/1089279632 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5318077440 con 0x7f53280ae5a0 2023-11-24T06:30:05.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.711+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5328071170 con 0x7f531406b080 2023-11-24T06:30:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.715+0000 7f53257fa640 1 -- 172.21.15.16:0/1089279632 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f531c0050d0 con 0x7f531406b080 2023-11-24T06:30:05.718 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:30:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephfs" 2023-11-24T06:30:05.718 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:05.718 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:30:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f531406b080 msgr2=0x7f531406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f531406b080 0x7f531406d540 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5328159530 tx=0x7f531c007990 comp rx=0 tx=0).stop 2023-11-24T06:30:05.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 msgr2=0x7f5328153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 0x7f5328153660 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5318036040 tx=0x7f5318004a40 comp rx=0 tx=0).stop 2023-11-24T06:30:05.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 shutdown_connections 2023-11-24T06:30:05.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f531406b080 0x7f531406d540 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f53280cf290 0x7f5328151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53280b3ae0 0x7f532814f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 --2- 172.21.15.16:0/1089279632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53280ae5a0 0x7f5328153660 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 >> 172.21.15.16:0/1089279632 conn(0x7f532806a800 msgr2=0x7f53280b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:05.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 shutdown_connections 2023-11-24T06:30:05.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.717+0000 7f532f133640 1 -- 172.21.15.16:0/1089279632 wait complete. 2023-11-24T06:30:05.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:30:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.964+0000 7fac18bfb640 1 Processor -- start 2023-11-24T06:30:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.965+0000 7fac18bfb640 1 -- start start 2023-11-24T06:30:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.966+0000 7fac18bfb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac14160920 0x7fac140b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.966+0000 7fac18bfb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fac140cfe70 0x7fac14161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.966+0000 7fac18bfb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac140d0a10 con 0x7fac140cf9f0 2023-11-24T06:30:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.966+0000 7fac18bfb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac140d1f50 con 0x7fac140cfe70 2023-11-24T06:30:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.966+0000 7fac18bfb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac140cf080 con 0x7fac14160920 2023-11-24T06:30:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.967+0000 7fac12ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fac140cf9f0 0x7fac14161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35466/0 (socket says 172.21.15.16:35466) 2023-11-24T06:30:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.967+0000 7fac12ffd640 1 -- 172.21.15.16:0/1238540526 learned_addr learned my addr 172.21.15.16:0/1238540526 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.967+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244894396 0 0) 0x7fac140d0a10 con 0x7fac140cf9f0 2023-11-24T06:30:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.967+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac00003520 con 0x7fac140cf9f0 2023-11-24T06:30:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.967+0000 7fac137fe640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac14160920 0x7fac140b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.967+0000 7fac13fff640 1 --2- 172.21.15.16:0/1238540526 >> v2:172.21.15.173:3300/0 conn(0x7fac140cfe70 0x7fac14161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.967+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1739238933 0 0) 0x7fac00003520 con 0x7fac140cf9f0 2023-11-24T06:30:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac14160920 msgr2=0x7fac140b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac14160920 0x7fac140b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 >> v2:172.21.15.173:3300/0 conn(0x7fac140cfe70 msgr2=0x7fac14161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 --2- 172.21.15.16:0/1238540526 >> v2:172.21.15.173:3300/0 conn(0x7fac140cfe70 0x7fac14161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac140c6be0 con 0x7fac140cf9f0 2023-11-24T06:30:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fabfc001670 con 0x7fac140cf9f0 2023-11-24T06:30:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 >> v1:172.21.15.16:6789/0 conn(0x7fac140cf9f0 legacy=0x7fac14161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac14160920 0x7fac00003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac00003ff0 0x7fac000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac00004d00 0x7fac00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac140d1f50 con 0x7fac00004d00 2023-11-24T06:30:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac140cf080 con 0x7fac00003ff0 2023-11-24T06:30:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0000bae0 con 0x7fac14160920 2023-11-24T06:30:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac13fff640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac00004d00 0x7fac00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac12ffd640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac00003ff0 0x7fac000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac137fe640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac14160920 0x7fac00003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac13fff640 1 -- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac14160920 msgr2=0x7fac00003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac13fff640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac14160920 0x7fac00003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac13fff640 1 -- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac00003ff0 msgr2=0x7fac000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac13fff640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac00003ff0 0x7fac000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac13fff640 1 -- 172.21.15.16:0/1238540526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac00003520 con 0x7fac00004d00 2023-11-24T06:30:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac137fe640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac14160920 0x7fac00003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:05.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.968+0000 7fac12ffd640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac00003ff0 0x7fac000047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:05.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.969+0000 7fac13fff640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac00004d00 0x7fac00005180 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00ba30 tx=0x7fac000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7869bbfae9c38b50 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:05.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.970+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c032680 con 0x7fac00004d00 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.970+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c0327e0 con 0x7fac00004d00 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.970+0000 7fac127fc640 1 -- 172.21.15.16:0/1238540526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c032b70 con 0x7fac00004d00 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.970+0000 7fac18bfb640 1 -- 172.21.15.16:0/1238540526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac00004d00 msgr2=0x7fac00005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.970+0000 7fac18bfb640 1 --2- 172.21.15.16:0/1238540526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac00004d00 0x7fac00005180 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00ba30 tx=0x7fac000057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.970+0000 7fac13fff640 1 -- 172.21.15.16:0/1238540526 reap_dead start 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.970+0000 7fac18bfb640 1 -- 172.21.15.16:0/1238540526 shutdown_connections 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.970+0000 7fac18bfb640 1 -- 172.21.15.16:0/1238540526 >> 172.21.15.16:0/1238540526 conn(0x7fac140d3090 msgr2=0x7fac140d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.970+0000 7fac18bfb640 1 -- 172.21.15.16:0/1238540526 shutdown_connections 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.971+0000 7fac18bfb640 1 -- 172.21.15.16:0/1238540526 wait complete. 2023-11-24T06:30:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.972+0000 7fac18bfb640 1 Processor -- start 2023-11-24T06:30:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.972+0000 7fac18bfb640 1 -- start start 2023-11-24T06:30:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.973+0000 7fac18bfb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac00004d00 0x7fac141f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.973+0000 7fac18bfb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1414bd00 0x7fac140b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.973+0000 7fac18bfb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac1414b040 0x7fac1414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.973+0000 7fac18bfb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac140b6780 con 0x7fac1414bd00 2023-11-24T06:30:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.973+0000 7fac18bfb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1414ba00 con 0x7fac1414b040 2023-11-24T06:30:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.973+0000 7fac18bfb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac14148ff0 con 0x7fac00004d00 2023-11-24T06:30:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1414bd00 0x7fac140b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1414bd00 0x7fac140b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36324/0 (socket says 172.21.15.16:36324) 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac12ffd640 1 -- 172.21.15.16:0/1907346414 learned_addr learned my addr 172.21.15.16:0/1907346414 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac137fe640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac00004d00 0x7fac141f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac12ffd640 1 -- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac00004d00 msgr2=0x7fac141f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac12ffd640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac00004d00 0x7fac141f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac12ffd640 1 -- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac1414b040 msgr2=0x7fac1414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac12ffd640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac1414b040 0x7fac1414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac12ffd640 1 -- 172.21.15.16:0/1907346414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac140cc540 con 0x7fac1414bd00 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.974+0000 7fac12ffd640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1414bd00 0x7fac140b6240 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00d8e0 tx=0x7fabfc00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.975+0000 7fac0bfff640 1 -- 172.21.15.16:0/1907346414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0074e0 con 0x7fac1414bd00 2023-11-24T06:30:05.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.975+0000 7fac0bfff640 1 -- 172.21.15.16:0/1907346414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc007640 con 0x7fac1414bd00 2023-11-24T06:30:05.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.975+0000 7fac0bfff640 1 -- 172.21.15.16:0/1907346414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0049e0 con 0x7fac1414bd00 2023-11-24T06:30:05.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.975+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac140cc820 con 0x7fac1414bd00 2023-11-24T06:30:05.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.975+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac14149290 con 0x7fac1414bd00 2023-11-24T06:30:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.976+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac14149450 con 0x7fac1414bd00 2023-11-24T06:30:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.977+0000 7fac0bfff640 1 -- 172.21.15.16:0/1907346414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fabfc004b40 con 0x7fac1414bd00 2023-11-24T06:30:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.978+0000 7fac0bfff640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabf406b0b0 0x7fabf406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.978+0000 7fac0bfff640 1 -- 172.21.15.16:0/1907346414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabfc0895a0 con 0x7fac1414bd00 2023-11-24T06:30:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.979+0000 7fac137fe640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabf406b0b0 0x7fabf406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:05.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.980+0000 7fac137fe640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabf406b0b0 0x7fabf406d570 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fac04009fd0 tx=0x7fac0400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:05.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:05.982+0000 7fac0bfff640 1 -- 172.21.15.16:0/1907346414 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fabfc054210 con 0x7fac1414bd00 2023-11-24T06:30:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.175+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fac14067450 con 0x7fac1414bd00 2023-11-24T06:30:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.176+0000 7fac0bfff640 1 -- 172.21.15.16:0/1907346414 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fabfc057830 con 0x7fac1414bd00 2023-11-24T06:30:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.178+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabf406b0b0 msgr2=0x7fabf406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.178+0000 7fac18bfb640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabf406b0b0 0x7fabf406d570 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fac04009fd0 tx=0x7fac0400b040 comp rx=0 tx=0).stop 2023-11-24T06:30:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.178+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1414bd00 msgr2=0x7fac140b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.178+0000 7fac18bfb640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1414bd00 0x7fac140b6240 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00d8e0 tx=0x7fabfc00dda0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.179+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 shutdown_connections 2023-11-24T06:30:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.179+0000 7fac18bfb640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fabf406b0b0 0x7fabf406d570 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.179+0000 7fac18bfb640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac1414b040 0x7fac1414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.179+0000 7fac18bfb640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1414bd00 0x7fac140b6240 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.179+0000 7fac18bfb640 1 --2- 172.21.15.16:0/1907346414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac00004d00 0x7fac141f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.179+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 >> 172.21.15.16:0/1907346414 conn(0x7fac140d3090 msgr2=0x7fac140c92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.179+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 shutdown_connections 2023-11-24T06:30:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.179+0000 7fac18bfb640 1 -- 172.21.15.16:0/1907346414 wait complete. 2023-11-24T06:30:06.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol8627 2023-11-24T06:30:06.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.417+0000 7f5b3e62e640 1 Processor -- start 2023-11-24T06:30:06.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.418+0000 7f5b3e62e640 1 -- start start 2023-11-24T06:30:06.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.419+0000 7f5b3e62e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b380cfde0 0x7f5b380b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.419+0000 7f5b3e62e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b380d1c60 con 0x7f5b3815d4b0 2023-11-24T06:30:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.419+0000 7f5b3e62e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b380b2730 con 0x7f5b380cf9a0 2023-11-24T06:30:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.419+0000 7f5b3e62e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b380d1ec0 con 0x7f5b380cfde0 2023-11-24T06:30:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.420+0000 7f5b3d62c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5b3815d4b0 0x7f5b380d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35468/0 (socket says 172.21.15.16:35468) 2023-11-24T06:30:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.420+0000 7f5b3d62c640 1 -- 172.21.15.16:0/1191922030 learned_addr learned my addr 172.21.15.16:0/1191922030 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.420+0000 7f5b3de2d640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b380cfde0 0x7f5b380b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.420+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708094996 0 0) 0x7f5b380d1c60 con 0x7f5b3815d4b0 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.420+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b24003520 con 0x7f5b3815d4b0 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 931139784 0 0) 0x7f5b24003520 con 0x7f5b3815d4b0 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b380cfde0 msgr2=0x7f5b380b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b380cfde0 0x7f5b380b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 >> v1:172.21.15.173:6789/0 conn(0x7f5b380cf9a0 legacy=0x7f5b380b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b380b29d0 con 0x7f5b3815d4b0 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f5b340019f0 con 0x7f5b3815d4b0 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 >> v1:172.21.15.16:6789/0 conn(0x7f5b3815d4b0 legacy=0x7f5b380d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b240039a0 0x7f5b24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b380cfde0 0x7f5b240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b24004d00 0x7f5b24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b380d1ec0 con 0x7f5b24004d00 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2400ba60 con 0x7f5b240039a0 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2400bc20 con 0x7f5b380cfde0 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b3de2d640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b24004d00 0x7f5b24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b3ce2b640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b380cfde0 0x7f5b240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.421+0000 7f5b3d62c640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b240039a0 0x7f5b24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.422+0000 7f5b3de2d640 1 -- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b380cfde0 msgr2=0x7f5b240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.422+0000 7f5b3de2d640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b380cfde0 0x7f5b240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.422+0000 7f5b3de2d640 1 -- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b240039a0 msgr2=0x7f5b24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.422+0000 7f5b3de2d640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b240039a0 0x7f5b24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.422+0000 7f5b3de2d640 1 -- 172.21.15.16:0/1191922030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24003520 con 0x7f5b24004d00 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.423+0000 7f5b3de2d640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b24004d00 0x7f5b24005180 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009a90 tx=0x7f5b240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9300369199fe9ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.423+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3002c680 con 0x7f5b24004d00 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.423+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3002c7e0 con 0x7f5b24004d00 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.423+0000 7f5b2ffff640 1 -- 172.21.15.16:0/1191922030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3002cb70 con 0x7f5b24004d00 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.424+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1191922030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b24004d00 msgr2=0x7f5b24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.424+0000 7f5b3e62e640 1 --2- 172.21.15.16:0/1191922030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b24004d00 0x7f5b24005180 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009a90 tx=0x7f5b240057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.424+0000 7f5b3de2d640 1 -- 172.21.15.16:0/1191922030 reap_dead start 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.424+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1191922030 shutdown_connections 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.424+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1191922030 >> 172.21.15.16:0/1191922030 conn(0x7f5b380d3000 msgr2=0x7f5b380d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.424+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1191922030 shutdown_connections 2023-11-24T06:30:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.424+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1191922030 wait complete. 2023-11-24T06:30:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.426+0000 7f5b3e62e640 1 Processor -- start 2023-11-24T06:30:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.426+0000 7f5b3e62e640 1 -- start start 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3e62e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b3814d3d0 0x7f5b380b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3e62e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3809dd30 0x7f5b380ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3e62e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b380ced60 0x7f5b381f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3e62e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b380cf1e0 con 0x7f5b3809dd30 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3e62e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3809f390 con 0x7f5b380ced60 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3e62e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3809f020 con 0x7f5b3814d3d0 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3ce2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3809dd30 0x7f5b380ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3ce2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3809dd30 0x7f5b380ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36348/0 (socket says 172.21.15.16:36348) 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3ce2b640 1 -- 172.21.15.16:0/1731155956 learned_addr learned my addr 172.21.15.16:0/1731155956 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3d62c640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b3814d3d0 0x7f5b380b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.427+0000 7f5b3de2d640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b380ced60 0x7f5b381f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b3ce2b640 1 -- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b3814d3d0 msgr2=0x7f5b380b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b3ce2b640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b3814d3d0 0x7f5b380b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b3ce2b640 1 -- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b380ced60 msgr2=0x7f5b381f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b3ce2b640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b380ced60 0x7f5b381f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b3ce2b640 1 -- 172.21.15.16:0/1731155956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b380b7800 con 0x7f5b3809dd30 2023-11-24T06:30:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b3de2d640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b380ced60 0x7f5b381f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b3ce2b640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3809dd30 0x7f5b380ce820 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800e580 tx=0x7f5b2800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b2dffb640 1 -- 172.21.15.16:0/1731155956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b280052e0 con 0x7f5b3809dd30 2023-11-24T06:30:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b2dffb640 1 -- 172.21.15.16:0/1731155956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b28005440 con 0x7f5b3809dd30 2023-11-24T06:30:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b2dffb640 1 -- 172.21.15.16:0/1731155956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28003810 con 0x7f5b3809dd30 2023-11-24T06:30:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3809e4a0 con 0x7f5b3809dd30 2023-11-24T06:30:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.428+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3809ea60 con 0x7f5b3809dd30 2023-11-24T06:30:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.430+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b38067450 con 0x7f5b3809dd30 2023-11-24T06:30:06.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.431+0000 7f5b2dffb640 1 -- 172.21.15.16:0/1731155956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5b28003970 con 0x7f5b3809dd30 2023-11-24T06:30:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.432+0000 7f5b2dffb640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1c06b0b0 0x7f5b1c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.432+0000 7f5b2dffb640 1 -- 172.21.15.16:0/1731155956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b28089850 con 0x7f5b3809dd30 2023-11-24T06:30:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.433+0000 7f5b3d62c640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1c06b0b0 0x7f5b1c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:06.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.434+0000 7f5b3d62c640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1c06b0b0 0x7f5b1c06d570 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b340013d0 tx=0x7f5b34009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:06.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.436+0000 7f5b2dffb640 1 -- 172.21.15.16:0/1731155956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b28054510 con 0x7f5b3809dd30 2023-11-24T06:30:06.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.658+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol8627", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b380d21a0 con 0x7f5b1c06b0b0 2023-11-24T06:30:06.676 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:06.673+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:06.676 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:06.673+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:06.676 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:06.673+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:06.676 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:06.673+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:06.676 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:06.673+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:06.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.749+0000 7f5b2dffb640 1 -- 172.21.15.16:0/1731155956 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5b34001950 con 0x7f5b1c06b0b0 2023-11-24T06:30:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.752+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1c06b0b0 msgr2=0x7f5b1c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.752+0000 7f5b3e62e640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1c06b0b0 0x7f5b1c06d570 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b340013d0 tx=0x7f5b34009040 comp rx=0 tx=0).stop 2023-11-24T06:30:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.752+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3809dd30 msgr2=0x7f5b380ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:06.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.752+0000 7f5b3e62e640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3809dd30 0x7f5b380ce820 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800e580 tx=0x7f5b2800ea40 comp rx=0 tx=0).stop 2023-11-24T06:30:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.753+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 shutdown_connections 2023-11-24T06:30:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.753+0000 7f5b3e62e640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5b1c06b0b0 0x7f5b1c06d570 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.753+0000 7f5b3e62e640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b380ced60 0x7f5b381f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.753+0000 7f5b3e62e640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b3809dd30 0x7f5b380ce820 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.753+0000 7f5b3e62e640 1 --2- 172.21.15.16:0/1731155956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b3814d3d0 0x7f5b380b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.753+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 >> 172.21.15.16:0/1731155956 conn(0x7f5b380d3000 msgr2=0x7f5b380cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.753+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 shutdown_connections 2023-11-24T06:30:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.753+0000 7f5b3e62e640 1 -- 172.21.15.16:0/1731155956 wait complete. 2023-11-24T06:30:06.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol8627 subvol_snap7164 2023-11-24T06:30:06.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.994+0000 7febf5fba640 1 Processor -- start 2023-11-24T06:30:06.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.994+0000 7febf5fba640 1 -- start start 2023-11-24T06:30:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.996+0000 7febf5fba640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7febf0162430 0x7febf00b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.996+0000 7febf5fba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.996+0000 7febf5fba640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf00702e0 con 0x7febf0162810 2023-11-24T06:30:07.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.996+0000 7febf5fba640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf00705c0 con 0x7febf0162430 2023-11-24T06:30:07.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.996+0000 7febf5fba640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf00708a0 con 0x7febf00b3650 2023-11-24T06:30:07.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.997+0000 7febf57b9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.997+0000 7febf57b9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf006c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:48194/0 (socket says 172.21.15.16:48194) 2023-11-24T06:30:07.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.997+0000 7febf57b9640 1 -- 172.21.15.16:0/2607888026 learned_addr learned my addr 172.21.15.16:0/2607888026 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:07.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.997+0000 7febf4fb8640 1 --2- 172.21.15.16:0/2607888026 >> v2:172.21.15.173:3300/0 conn(0x7febf0162430 0x7febf00b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.998+0000 7febf57b9640 1 -- 172.21.15.16:0/2607888026 >> v2:172.21.15.173:3300/0 conn(0x7febf0162430 msgr2=0x7febf00b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:07.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.998+0000 7febf57b9640 1 --2- 172.21.15.16:0/2607888026 >> v2:172.21.15.173:3300/0 conn(0x7febf0162430 0x7febf00b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.998+0000 7febf57b9640 1 -- 172.21.15.16:0/2607888026 >> v1:172.21.15.173:6789/0 conn(0x7febf0162810 legacy=0x7febf0071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:06.998+0000 7febf57b9640 1 -- 172.21.15.16:0/2607888026 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf0070b80 con 0x7febf00b3650 2023-11-24T06:30:07.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.000+0000 7febf57b9640 1 --2- 172.21.15.16:0/2607888026 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf006c8c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7febe8009870 tx=0x7febe802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485513a1f1a4e39f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.000+0000 7febe77fe640 1 -- 172.21.15.16:0/2607888026 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8030840 con 0x7febf00b3650 2023-11-24T06:30:07.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.000+0000 7febe77fe640 1 -- 172.21.15.16:0/2607888026 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe80309a0 con 0x7febf00b3650 2023-11-24T06:30:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.000+0000 7febe77fe640 1 -- 172.21.15.16:0/2607888026 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8030d30 con 0x7febf00b3650 2023-11-24T06:30:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.001+0000 7febf5fba640 1 -- 172.21.15.16:0/2607888026 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 msgr2=0x7febf006c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.001+0000 7febf5fba640 1 --2- 172.21.15.16:0/2607888026 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf006c8c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7febe8009870 tx=0x7febe802b980 comp rx=0 tx=0).stop 2023-11-24T06:30:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.001+0000 7febf5fba640 1 -- 172.21.15.16:0/2607888026 shutdown_connections 2023-11-24T06:30:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.001+0000 7febf5fba640 1 --2- 172.21.15.16:0/2607888026 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf006c8c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.001+0000 7febf5fba640 1 --2- 172.21.15.16:0/2607888026 >> v2:172.21.15.173:3300/0 conn(0x7febf0162430 0x7febf00b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.001+0000 7febf5fba640 1 -- 172.21.15.16:0/2607888026 >> 172.21.15.16:0/2607888026 conn(0x7febf006a800 msgr2=0x7febf006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.001+0000 7febf5fba640 1 -- 172.21.15.16:0/2607888026 shutdown_connections 2023-11-24T06:30:07.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.001+0000 7febf5fba640 1 -- 172.21.15.16:0/2607888026 wait complete. 2023-11-24T06:30:07.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.004+0000 7febf5fba640 1 Processor -- start 2023-11-24T06:30:07.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.004+0000 7febf5fba640 1 -- start start 2023-11-24T06:30:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.005+0000 7febf5fba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf0164010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.005+0000 7febf5fba640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf0162430 0x7febf0148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.005+0000 7febf5fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0165130 0x7febf01655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.005+0000 7febf5fba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf0165b20 con 0x7febf0165130 2023-11-24T06:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.005+0000 7febf5fba640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf0162f70 con 0x7febf0162430 2023-11-24T06:30:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.005+0000 7febf5fba640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf0163250 con 0x7febf00b3650 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.005+0000 7febf57b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0165130 0x7febf01655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febf57b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0165130 0x7febf01655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36352/0 (socket says 172.21.15.16:36352) 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febf57b9640 1 -- 172.21.15.16:0/3962240336 learned_addr learned my addr 172.21.15.16:0/3962240336 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febf4fb8640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf0164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febe7fff640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf0162430 0x7febf0148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febf57b9640 1 -- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 msgr2=0x7febf0164010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febf57b9640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf0164010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febf57b9640 1 -- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf0162430 msgr2=0x7febf0148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febf57b9640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf0162430 0x7febf0148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febf57b9640 1 -- 172.21.15.16:0/3962240336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe80094d0 con 0x7febf0165130 2023-11-24T06:30:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.006+0000 7febf57b9640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0165130 0x7febf01655e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7febe8009870 tx=0x7febe8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.007+0000 7febe57fa640 1 -- 172.21.15.16:0/3962240336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8039c90 con 0x7febf0165130 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.007+0000 7febe57fa640 1 -- 172.21.15.16:0/3962240336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe8039e10 con 0x7febf0165130 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.007+0000 7febe57fa640 1 -- 172.21.15.16:0/3962240336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8043640 con 0x7febf0165130 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.007+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf0163530 con 0x7febf0165130 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.007+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf0163af0 con 0x7febf0165130 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.008+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf0067450 con 0x7febf0165130 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.009+0000 7febe57fa640 1 -- 172.21.15.16:0/3962240336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7febe80437a0 con 0x7febf0165130 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.010+0000 7febe57fa640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febdc06b080 0x7febdc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.010+0000 7febe57fa640 1 -- 172.21.15.16:0/3962240336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febe803e070 con 0x7febf0165130 2023-11-24T06:30:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.011+0000 7febf4fb8640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febdc06b080 0x7febdc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.012+0000 7febf4fb8640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febdc06b080 0x7febdc06d540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7febec0074a0 tx=0x7febec00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.015+0000 7febe57fa640 1 -- 172.21.15.16:0/3962240336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febe8077780 con 0x7febf0165130 2023-11-24T06:30:07.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.241+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol8627", "snap_name": "subvol_snap7164", "target": ["mon-mgr", ""]}) v1 -- 0x7febf00b27c0 con 0x7febdc06b080 2023-11-24T06:30:07.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.268+0000 7febe57fa640 1 -- 172.21.15.16:0/3962240336 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7febf00b27c0 con 0x7febdc06b080 2023-11-24T06:30:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.271+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febdc06b080 msgr2=0x7febdc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.271+0000 7febf5fba640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febdc06b080 0x7febdc06d540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7febec0074a0 tx=0x7febec00b040 comp rx=0 tx=0).stop 2023-11-24T06:30:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.271+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0165130 msgr2=0x7febf01655e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.271+0000 7febf5fba640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0165130 0x7febf01655e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7febe8009870 tx=0x7febe8004730 comp rx=0 tx=0).stop 2023-11-24T06:30:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.271+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 shutdown_connections 2023-11-24T06:30:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.271+0000 7febf5fba640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febdc06b080 0x7febdc06d540 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.272+0000 7febf5fba640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0165130 0x7febf01655e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.272+0000 7febf5fba640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf0162430 0x7febf0148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.272+0000 7febf5fba640 1 --2- 172.21.15.16:0/3962240336 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febf00b3650 0x7febf0164010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.272+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 >> 172.21.15.16:0/3962240336 conn(0x7febf006a800 msgr2=0x7febf00af300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:07.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.272+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 shutdown_connections 2023-11-24T06:30:07.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.272+0000 7febf5fba640 1 -- 172.21.15.16:0/3962240336 wait complete. 2023-11-24T06:30:07.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol8627 subvol_snap7164 2023-11-24T06:30:07.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.507+0000 7f457fbb0640 1 Processor -- start 2023-11-24T06:30:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.508+0000 7f457fbb0640 1 -- start start 2023-11-24T06:30:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.509+0000 7f457fbb0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 0x7f457806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.509+0000 7f457fbb0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4578160f90 0x7f45780d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.509+0000 7f457fbb0640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f457815d780 0x7f45780b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.509+0000 7f457fbb0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4578161430 con 0x7f4578160f90 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.509+0000 7f457fbb0640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45781616f0 con 0x7f457815d780 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.509+0000 7f457fbb0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45781619d0 con 0x7f457815d3a0 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457d14b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4578160f90 0x7f45780d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457d14b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4578160f90 0x7f45780d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36360/0 (socket says 172.21.15.16:36360) 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457d14b640 1 -- 172.21.15.16:0/1497297195 learned_addr learned my addr 172.21.15.16:0/1497297195 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457d94c640 1 --2- 172.21.15.16:0/1497297195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 0x7f457806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457d14b640 1 -- 172.21.15.16:0/1497297195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 msgr2=0x7f457806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:07.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457e14d640 1 --2- 172.21.15.16:0/1497297195 >> v2:172.21.15.173:3300/0 conn(0x7f457815d780 0x7f45780b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457d14b640 1 --2- 172.21.15.16:0/1497297195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 0x7f457806d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457d14b640 1 -- 172.21.15.16:0/1497297195 >> v2:172.21.15.173:3300/0 conn(0x7f457815d780 msgr2=0x7f45780b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457d14b640 1 --2- 172.21.15.16:0/1497297195 >> v2:172.21.15.173:3300/0 conn(0x7f457815d780 0x7f45780b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.510+0000 7f457d14b640 1 -- 172.21.15.16:0/1497297195 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4578161cb0 con 0x7f4578160f90 2023-11-24T06:30:07.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.511+0000 7f457d14b640 1 --2- 172.21.15.16:0/1497297195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 0x7f45780d0a60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f45780d20e0 tx=0x7f456c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f0b6dc8d2903ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:07.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.512+0000 7f457c94a640 1 -- 172.21.15.16:0/1497297195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c031860 con 0x7f4578160f90 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.512+0000 7f457c94a640 1 -- 172.21.15.16:0/1497297195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c0319c0 con 0x7f4578160f90 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.512+0000 7f457c94a640 1 -- 172.21.15.16:0/1497297195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c031d50 con 0x7f4578160f90 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.512+0000 7f457fbb0640 1 -- 172.21.15.16:0/1497297195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 msgr2=0x7f45780d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.512+0000 7f457fbb0640 1 --2- 172.21.15.16:0/1497297195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 0x7f45780d0a60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f45780d20e0 tx=0x7f456c02b900 comp rx=0 tx=0).stop 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.513+0000 7f457fbb0640 1 -- 172.21.15.16:0/1497297195 shutdown_connections 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.513+0000 7f457fbb0640 1 --2- 172.21.15.16:0/1497297195 >> v2:172.21.15.173:3300/0 conn(0x7f457815d780 0x7f45780b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.513+0000 7f457fbb0640 1 --2- 172.21.15.16:0/1497297195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 0x7f45780d0a60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.513+0000 7f457fbb0640 1 --2- 172.21.15.16:0/1497297195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 0x7f457806d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.513+0000 7f457fbb0640 1 -- 172.21.15.16:0/1497297195 >> 172.21.15.16:0/1497297195 conn(0x7f457806e7a0 msgr2=0x7f457806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:07.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.513+0000 7f457fbb0640 1 -- 172.21.15.16:0/1497297195 shutdown_connections 2023-11-24T06:30:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.513+0000 7f457fbb0640 1 -- 172.21.15.16:0/1497297195 wait complete. 2023-11-24T06:30:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.515+0000 7f457fbb0640 1 Processor -- start 2023-11-24T06:30:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.515+0000 7f457fbb0640 1 -- start start 2023-11-24T06:30:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.516+0000 7f457fbb0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 0x7f45780af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.516+0000 7f457fbb0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f457815d780 0x7f45780ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.516+0000 7f457fbb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 0x7f45780c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457fbb0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45780ad330 con 0x7f4578160f90 2023-11-24T06:30:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457fbb0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45780ad610 con 0x7f457815d780 2023-11-24T06:30:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457fbb0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45780ad8f0 con 0x7f457815d3a0 2023-11-24T06:30:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457e14d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 0x7f45780c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457e14d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 0x7f45780c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36370/0 (socket says 172.21.15.16:36370) 2023-11-24T06:30:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457e14d640 1 -- 172.21.15.16:0/4227400191 learned_addr learned my addr 172.21.15.16:0/4227400191 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457d94c640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 0x7f45780af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457d14b640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f457815d780 0x7f45780ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457e14d640 1 -- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 msgr2=0x7f45780af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457e14d640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 0x7f45780af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457e14d640 1 -- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f457815d780 msgr2=0x7f45780ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457e14d640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f457815d780 0x7f45780ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457e14d640 1 -- 172.21.15.16:0/4227400191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c0094d0 con 0x7f4578160f90 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457d94c640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 0x7f45780af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.517+0000 7f457d14b640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f457815d780 0x7f45780ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.518+0000 7f457e14d640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 0x7f45780c9880 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f457000c590 tx=0x7f457000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.518+0000 7f45767fc640 1 -- 172.21.15.16:0/4227400191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570004220 con 0x7f4578160f90 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.518+0000 7f45767fc640 1 -- 172.21.15.16:0/4227400191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45700043a0 con 0x7f4578160f90 2023-11-24T06:30:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.518+0000 7f45767fc640 1 -- 172.21.15.16:0/4227400191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570003850 con 0x7f4578160f90 2023-11-24T06:30:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.518+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45780adbd0 con 0x7f4578160f90 2023-11-24T06:30:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.518+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45780afe90 con 0x7f4578160f90 2023-11-24T06:30:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.519+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45780675d0 con 0x7f4578160f90 2023-11-24T06:30:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.520+0000 7f45767fc640 1 -- 172.21.15.16:0/4227400191 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f45700039b0 con 0x7f4578160f90 2023-11-24T06:30:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.521+0000 7f45767fc640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f456406b0b0 0x7f456406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.521+0000 7f45767fc640 1 -- 172.21.15.16:0/4227400191 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4570089770 con 0x7f4578160f90 2023-11-24T06:30:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.522+0000 7f457d94c640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f456406b0b0 0x7f456406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.523+0000 7f457d94c640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f456406b0b0 0x7f456406d570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4568008f10 tx=0x7f4568008460 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.526+0000 7f45767fc640 1 -- 172.21.15.16:0/4227400191 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4570054440 con 0x7f4578160f90 2023-11-24T06:30:07.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.753+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol8627", "snap_name": "subvol_snap7164", "target": ["mon-mgr", ""]}) v1 -- 0x7f45780ad780 con 0x7f456406b0b0 2023-11-24T06:30:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.806+0000 7f45767fc640 1 -- 172.21.15.16:0/4227400191 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f45780ad780 con 0x7f456406b0b0 2023-11-24T06:30:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.809+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f456406b0b0 msgr2=0x7f456406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.809+0000 7f457fbb0640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f456406b0b0 0x7f456406d570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4568008f10 tx=0x7f4568008460 comp rx=0 tx=0).stop 2023-11-24T06:30:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.809+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 msgr2=0x7f45780c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.809+0000 7f457fbb0640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 0x7f45780c9880 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f457000c590 tx=0x7f457000ca50 comp rx=0 tx=0).stop 2023-11-24T06:30:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.809+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 shutdown_connections 2023-11-24T06:30:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.809+0000 7f457fbb0640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f456406b0b0 0x7f456406d570 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.809+0000 7f457fbb0640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4578160f90 0x7f45780c9880 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.809+0000 7f457fbb0640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f457815d780 0x7f45780ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.810+0000 7f457fbb0640 1 --2- 172.21.15.16:0/4227400191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f457815d3a0 0x7f45780af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.810+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 >> 172.21.15.16:0/4227400191 conn(0x7f457806e7a0 msgr2=0x7f45780b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.810+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 shutdown_connections 2023-11-24T06:30:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:07.810+0000 7f457fbb0640 1 -- 172.21.15.16:0/4227400191 wait complete. 2023-11-24T06:30:07.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol8627 subvol_snap7164 2023-11-24T06:30:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.064+0000 7fe68a9d0640 1 Processor -- start 2023-11-24T06:30:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.065+0000 7fe68a9d0640 1 -- start start 2023-11-24T06:30:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.066+0000 7fe68a9d0640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe684160920 0x7fe6840b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.066+0000 7fe68a9d0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6840cfea0 0x7fe684161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.066+0000 7fe68a9d0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6840d0a10 con 0x7fe6840cf9f0 2023-11-24T06:30:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.066+0000 7fe68a9d0640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6840d0330 con 0x7fe684160920 2023-11-24T06:30:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.066+0000 7fe68a9d0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840d1f50 con 0x7fe6840cfea0 2023-11-24T06:30:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.067+0000 7fe6891cd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe6840cf9f0 0x7fe684161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35476/0 (socket says 172.21.15.16:35476) 2023-11-24T06:30:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.067+0000 7fe6891cd640 1 -- 172.21.15.16:0/912987605 learned_addr learned my addr 172.21.15.16:0/912987605 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.067+0000 7fe68a1cf640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6840cfea0 0x7fe684161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.067+0000 7fe6899ce640 1 --2- 172.21.15.16:0/912987605 >> v2:172.21.15.173:3300/0 conn(0x7fe684160920 0x7fe6840b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.067+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307781724 0 0) 0x7fe6840d0a10 con 0x7fe6840cf9f0 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.067+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe66c003520 con 0x7fe6840cf9f0 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1158640099 0 0) 0x7fe66c003520 con 0x7fe6840cf9f0 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6840cfea0 msgr2=0x7fe684161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6840cfea0 0x7fe684161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 >> v2:172.21.15.173:3300/0 conn(0x7fe684160920 msgr2=0x7fe6840b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 --2- 172.21.15.16:0/912987605 >> v2:172.21.15.173:3300/0 conn(0x7fe684160920 0x7fe6840b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840cf080 con 0x7fe6840cf9f0 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fe678001670 con 0x7fe6840cf9f0 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 >> v1:172.21.15.16:6789/0 conn(0x7fe6840cf9f0 legacy=0x7fe684161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe66c003970 0x7fe66c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6840cfea0 0x7fe66c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c004d00 0x7fe66c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840d0330 con 0x7fe66c004d00 2023-11-24T06:30:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840d1f50 con 0x7fe66c003970 2023-11-24T06:30:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c00b980 con 0x7fe6840cfea0 2023-11-24T06:30:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.068+0000 7fe68a1cf640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c004d00 0x7fe66c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.069+0000 7fe6899ce640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe66c003970 0x7fe66c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.069+0000 7fe6891cd640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6840cfea0 0x7fe66c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.069+0000 7fe68a1cf640 1 -- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6840cfea0 msgr2=0x7fe66c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.069+0000 7fe68a1cf640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe6840cfea0 0x7fe66c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.069+0000 7fe68a1cf640 1 -- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe66c003970 msgr2=0x7fe66c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.069+0000 7fe68a1cf640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe66c003970 0x7fe66c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.069+0000 7fe68a1cf640 1 -- 172.21.15.16:0/912987605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c003520 con 0x7fe66c004d00 2023-11-24T06:30:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.069+0000 7fe6899ce640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe66c003970 0x7fe66c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:30:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.071+0000 7fe68a1cf640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c004d00 0x7fe66c005180 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009fc0 tx=0x7fe66c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeafd415785e15af server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.071+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c02cc20 con 0x7fe66c004d00 2023-11-24T06:30:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.071+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c00f040 con 0x7fe66c004d00 2023-11-24T06:30:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.071+0000 7fe6889cc640 1 -- 172.21.15.16:0/912987605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c034440 con 0x7fe66c004d00 2023-11-24T06:30:08.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.071+0000 7fe68a9d0640 1 -- 172.21.15.16:0/912987605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c004d00 msgr2=0x7fe66c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.071+0000 7fe68a9d0640 1 --2- 172.21.15.16:0/912987605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe66c004d00 0x7fe66c005180 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009fc0 tx=0x7fe66c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.071+0000 7fe68a1cf640 1 -- 172.21.15.16:0/912987605 reap_dead start 2023-11-24T06:30:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.072+0000 7fe68a9d0640 1 -- 172.21.15.16:0/912987605 shutdown_connections 2023-11-24T06:30:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.072+0000 7fe68a9d0640 1 -- 172.21.15.16:0/912987605 >> 172.21.15.16:0/912987605 conn(0x7fe6840d3090 msgr2=0x7fe6840d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.072+0000 7fe68a9d0640 1 -- 172.21.15.16:0/912987605 shutdown_connections 2023-11-24T06:30:08.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.072+0000 7fe68a9d0640 1 -- 172.21.15.16:0/912987605 wait complete. 2023-11-24T06:30:08.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.074+0000 7fe68a9d0640 1 Processor -- start 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.074+0000 7fe68a9d0640 1 -- start start 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe68a9d0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe66c004d00 0x7fe6840cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe68a9d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe68414d5e0 0x7fe68414d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe68a9d0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe68415b060 0x7fe68415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe68a9d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68415ba50 con 0x7fe68414d5e0 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe68a9d0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841f0ab0 con 0x7fe68415b060 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe68a9d0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841f3100 con 0x7fe66c004d00 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe6891cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe68414d5e0 0x7fe68414d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe6891cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe68414d5e0 0x7fe68414d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36396/0 (socket says 172.21.15.16:36396) 2023-11-24T06:30:08.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe6891cd640 1 -- 172.21.15.16:0/3608827817 learned_addr learned my addr 172.21.15.16:0/3608827817 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.075+0000 7fe68a1cf640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe68415b060 0x7fe68415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6891cd640 1 -- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe66c004d00 msgr2=0x7fe6840cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6899ce640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe66c004d00 0x7fe6840cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6891cd640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe66c004d00 0x7fe6840cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6891cd640 1 -- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe68415b060 msgr2=0x7fe68415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6891cd640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe68415b060 0x7fe68415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6891cd640 1 -- 172.21.15.16:0/3608827817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6841f33e0 con 0x7fe68414d5e0 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe68a1cf640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe68415b060 0x7fe68415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6899ce640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe66c004d00 0x7fe6840cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6891cd640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe68414d5e0 0x7fe68414d080 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe67800e840 tx=0x7fe67800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6827fc640 1 -- 172.21.15.16:0/3608827817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800c870 con 0x7fe68414d5e0 2023-11-24T06:30:08.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6827fc640 1 -- 172.21.15.16:0/3608827817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67800c9d0 con 0x7fe68414d5e0 2023-11-24T06:30:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe6827fc640 1 -- 172.21.15.16:0/3608827817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800cd80 con 0x7fe68414d5e0 2023-11-24T06:30:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe68a9d0640 1 -- 172.21.15.16:0/3608827817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6841f36c0 con 0x7fe68414d5e0 2023-11-24T06:30:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.076+0000 7fe68a9d0640 1 -- 172.21.15.16:0/3608827817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6841f3ec0 con 0x7fe68414d5e0 2023-11-24T06:30:08.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.078+0000 7fe68a9d0640 1 -- 172.21.15.16:0/3608827817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6840d3fd0 con 0x7fe68414d5e0 2023-11-24T06:30:08.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.078+0000 7fe6827fc640 1 -- 172.21.15.16:0/3608827817 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fe67801d460 con 0x7fe68414d5e0 2023-11-24T06:30:08.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.079+0000 7fe6827fc640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe67006b080 0x7fe67006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.080+0000 7fe6827fc640 1 -- 172.21.15.16:0/3608827817 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6780899f0 con 0x7fe68414d5e0 2023-11-24T06:30:08.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.080+0000 7fe6899ce640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe67006b080 0x7fe67006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.082+0000 7fe6899ce640 1 --2- 172.21.15.16:0/3608827817 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe67006b080 0x7fe67006d540 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe6840c9be0 tx=0x7fe674008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:08.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.084+0000 7fe6827fc640 1 -- 172.21.15.16:0/3608827817 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe678054810 con 0x7fe68414d5e0 2023-11-24T06:30:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.319+0000 7fe68a9d0640 1 -- 172.21.15.16:0/3608827817 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol8627", "snap_name": "subvol_snap7164", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6840ceac0 con 0x7fe67006b080 2023-11-24T06:30:08.353 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.350+0000 7fd2b5afe640 -1 mgr.server reply reply (2) No such file or directory snapshot 'b'subvol_snap7164'' does not exist 2023-11-24T06:30:08.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.350+0000 7fe6827fc640 1 -- 172.21.15.16:0/3608827817 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: -2 snapshot 'b'subvol_snap7164'' does not exist) v1 ==== 52+0+0 (secure 0 0 0) 0x7fe674006dd0 con 0x7fe67006b080 2023-11-24T06:30:08.354 INFO:teuthology.orchestra.run.smithi016.stderr:Error ENOENT: snapshot 'b'subvol_snap7164'' does not exist 2023-11-24T06:30:08.368 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-24T06:30:08.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol8627 2023-11-24T06:30:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.595+0000 7faa84083640 1 Processor -- start 2023-11-24T06:30:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.595+0000 7faa84083640 1 -- start start 2023-11-24T06:30:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.597+0000 7faa84083640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7faa7c13e360 0x7faa7c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.597+0000 7faa84083640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa7c15f9c0 0x7faa7c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.597+0000 7faa84083640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c15c5f0 con 0x7faa7c13df80 2023-11-24T06:30:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.597+0000 7faa84083640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c06c470 con 0x7faa7c15f9c0 2023-11-24T06:30:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.597+0000 7faa84083640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c06c750 con 0x7faa7c13e360 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.598+0000 7faa81e1f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faa7c13df80 0x7faa7c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35478/0 (socket says 172.21.15.16:35478) 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.598+0000 7faa81e1f640 1 -- 172.21.15.16:0/1798938382 learned_addr learned my addr 172.21.15.16:0/1798938382 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.598+0000 7faa82620640 1 --2- 172.21.15.16:0/1798938382 >> v2:172.21.15.16:3300/0 conn(0x7faa7c15f9c0 0x7faa7c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.598+0000 7faa8161e640 1 --2- 172.21.15.16:0/1798938382 >> v2:172.21.15.173:3300/0 conn(0x7faa7c13e360 0x7faa7c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.598+0000 7faa80e1d640 1 -- 172.21.15.16:0/1798938382 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178150124 0 0) 0x7faa7c15c5f0 con 0x7faa7c13df80 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.599+0000 7faa80e1d640 1 -- 172.21.15.16:0/1798938382 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa68003520 con 0x7faa7c13df80 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.599+0000 7faa82620640 1 -- 172.21.15.16:0/1798938382 >> v2:172.21.15.173:3300/0 conn(0x7faa7c13e360 msgr2=0x7faa7c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.599+0000 7faa82620640 1 --2- 172.21.15.16:0/1798938382 >> v2:172.21.15.173:3300/0 conn(0x7faa7c13e360 0x7faa7c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.599+0000 7faa82620640 1 -- 172.21.15.16:0/1798938382 >> v1:172.21.15.16:6789/0 conn(0x7faa7c13df80 legacy=0x7faa7c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.599+0000 7faa82620640 1 -- 172.21.15.16:0/1798938382 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c06ca30 con 0x7faa7c15f9c0 2023-11-24T06:30:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.600+0000 7faa82620640 1 --2- 172.21.15.16:0/1798938382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c15f9c0 0x7faa7c0cf7d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7faa7c06ded0 tx=0x7faa7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837acb8cd7aa0e85 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.600+0000 7faa80e1d640 1 -- 172.21.15.16:0/1798938382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7402cc80 con 0x7faa7c15f9c0 2023-11-24T06:30:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.600+0000 7faa80e1d640 1 -- 172.21.15.16:0/1798938382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7402cde0 con 0x7faa7c15f9c0 2023-11-24T06:30:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.600+0000 7faa80e1d640 1 -- 172.21.15.16:0/1798938382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74034660 con 0x7faa7c15f9c0 2023-11-24T06:30:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.601+0000 7faa84083640 1 -- 172.21.15.16:0/1798938382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c15f9c0 msgr2=0x7faa7c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.601+0000 7faa84083640 1 --2- 172.21.15.16:0/1798938382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c15f9c0 0x7faa7c0cf7d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7faa7c06ded0 tx=0x7faa7402bed0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.601+0000 7faa84083640 1 -- 172.21.15.16:0/1798938382 shutdown_connections 2023-11-24T06:30:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.601+0000 7faa84083640 1 --2- 172.21.15.16:0/1798938382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c15f9c0 0x7faa7c0cf7d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.601+0000 7faa84083640 1 --2- 172.21.15.16:0/1798938382 >> v2:172.21.15.173:3300/0 conn(0x7faa7c13e360 0x7faa7c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.601+0000 7faa84083640 1 -- 172.21.15.16:0/1798938382 >> 172.21.15.16:0/1798938382 conn(0x7faa7c06a800 msgr2=0x7faa7c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.601+0000 7faa84083640 1 -- 172.21.15.16:0/1798938382 shutdown_connections 2023-11-24T06:30:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.601+0000 7faa84083640 1 -- 172.21.15.16:0/1798938382 wait complete. 2023-11-24T06:30:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.603+0000 7faa84083640 1 Processor -- start 2023-11-24T06:30:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.603+0000 7faa84083640 1 -- start start 2023-11-24T06:30:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.604+0000 7faa84083640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa7c13df80 0x7faa7c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.604+0000 7faa84083640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c13e360 0x7faa7c15a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.604+0000 7faa84083640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa7c15f9c0 0x7faa7c157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.604+0000 7faa84083640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c159e60 con 0x7faa7c13e360 2023-11-24T06:30:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.604+0000 7faa84083640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c159550 con 0x7faa7c15f9c0 2023-11-24T06:30:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.604+0000 7faa84083640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c156b40 con 0x7faa7c13df80 2023-11-24T06:30:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.604+0000 7faa8161e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c13e360 0x7faa7c15a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.604+0000 7faa8161e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c13e360 0x7faa7c15a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36420/0 (socket says 172.21.15.16:36420) 2023-11-24T06:30:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.604+0000 7faa8161e640 1 -- 172.21.15.16:0/3423338574 learned_addr learned my addr 172.21.15.16:0/3423338574 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa82620640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa7c15f9c0 0x7faa7c157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa81e1f640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa7c13df80 0x7faa7c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa8161e640 1 -- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa7c13df80 msgr2=0x7faa7c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa8161e640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa7c13df80 0x7faa7c15df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa8161e640 1 -- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa7c15f9c0 msgr2=0x7faa7c157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa8161e640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa7c15f9c0 0x7faa7c157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa8161e640 1 -- 172.21.15.16:0/3423338574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa74009a70 con 0x7faa7c13e360 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa82620640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa7c15f9c0 0x7faa7c157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa81e1f640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa7c13df80 0x7faa7c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa8161e640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c13e360 0x7faa7c15a280 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e980 tx=0x7faa6c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa727fc640 1 -- 172.21.15.16:0/3423338574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c009670 con 0x7faa7c13e360 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa727fc640 1 -- 172.21.15.16:0/3423338574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c00ca00 con 0x7faa7c13e360 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa727fc640 1 -- 172.21.15.16:0/3423338574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00cd90 con 0x7faa7c13e360 2023-11-24T06:30:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.605+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c1597a0 con 0x7faa7c13e360 2023-11-24T06:30:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.606+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c15e4b0 con 0x7faa7c13e360 2023-11-24T06:30:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.607+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c0675d0 con 0x7faa7c13e360 2023-11-24T06:30:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.608+0000 7faa727fc640 1 -- 172.21.15.16:0/3423338574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7faa6c013470 con 0x7faa7c13e360 2023-11-24T06:30:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.608+0000 7faa727fc640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa6006b080 0x7faa6006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.609+0000 7faa727fc640 1 -- 172.21.15.16:0/3423338574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa6c089cc0 con 0x7faa7c13e360 2023-11-24T06:30:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.609+0000 7faa81e1f640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa6006b080 0x7faa6006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.610+0000 7faa81e1f640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa6006b080 0x7faa6006d540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faa78005880 tx=0x7faa78009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.613+0000 7faa727fc640 1 -- 172.21.15.16:0/3423338574 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa6c054a40 con 0x7faa7c13e360 2023-11-24T06:30:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.832+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol8627", "target": ["mon-mgr", ""]}) v1 -- 0x7faa7c06cff0 con 0x7faa6006b080 2023-11-24T06:30:08.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.870+0000 7faa727fc640 1 -- 172.21.15.16:0/3423338574 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faa78001c80 con 0x7faa6006b080 2023-11-24T06:30:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.872+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa6006b080 msgr2=0x7faa6006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.872+0000 7faa84083640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa6006b080 0x7faa6006d540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faa78005880 tx=0x7faa78009040 comp rx=0 tx=0).stop 2023-11-24T06:30:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.872+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c13e360 msgr2=0x7faa7c15a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.872+0000 7faa84083640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c13e360 0x7faa7c15a280 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e980 tx=0x7faa6c00ee40 comp rx=0 tx=0).stop 2023-11-24T06:30:08.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.873+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 shutdown_connections 2023-11-24T06:30:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.873+0000 7faa84083640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7faa6006b080 0x7faa6006d540 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.873+0000 7faa84083640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa7c15f9c0 0x7faa7c157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.873+0000 7faa84083640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7c13e360 0x7faa7c15a280 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.873+0000 7faa84083640 1 --2- 172.21.15.16:0/3423338574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa7c13df80 0x7faa7c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.873+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 >> 172.21.15.16:0/3423338574 conn(0x7faa7c06a800 msgr2=0x7faa7c0b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.873+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 shutdown_connections 2023-11-24T06:30:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:08.873+0000 7faa84083640 1 -- 172.21.15.16:0/3423338574 wait complete. 2023-11-24T06:30:08.883 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.881+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.884 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.881+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.884 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.881+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.884 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.881+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.884 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.881+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:08.896 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T06:30:08.911 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.908+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.911 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.908+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.911 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.908+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.911 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.908+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.911 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:08.908+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:08.964 INFO:teuthology.orchestra.run.smithi016.stdout:2 2023-11-24T06:30:08.966 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:30:08.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:30:09.087 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:30:09.205 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:30:09.205 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:30:09.206 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:30:09.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:09.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.489+0000 7f7f0b59e640 1 Processor -- start 2023-11-24T06:30:09.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.489+0000 7f7f0b59e640 1 -- start start 2023-11-24T06:30:09.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.491+0000 7f7f0b59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7f0c0cc500 0x7f7f0c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.491+0000 7f7f0b59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f0c06d160 0x7f7f0c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.491+0000 7f7f0b59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f0c070e50 con 0x7f7f0c143c40 2023-11-24T06:30:09.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.491+0000 7f7f0b59e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0c0723c0 con 0x7f7f0c06d160 2023-11-24T06:30:09.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.491+0000 7f7f0b59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0c0cced0 con 0x7f7f0c0cc500 2023-11-24T06:30:09.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.491+0000 7f7f0ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f0c06d160 0x7f7f0c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.491+0000 7f7f0ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f0c06d160 0x7f7f0c06d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36436/0 (socket says 172.21.15.16:36436) 2023-11-24T06:30:09.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.491+0000 7f7f0ad9d640 1 -- 172.21.15.16:0/3910532110 learned_addr learned my addr 172.21.15.16:0/3910532110 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:09.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.492+0000 7f7f09d9b640 1 --2- 172.21.15.16:0/3910532110 >> v2:172.21.15.173:3300/0 conn(0x7f7f0c0cc500 0x7f7f0c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.492+0000 7f7f0ad9d640 1 -- 172.21.15.16:0/3910532110 >> v2:172.21.15.173:3300/0 conn(0x7f7f0c0cc500 msgr2=0x7f7f0c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:09.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.492+0000 7f7f0ad9d640 1 --2- 172.21.15.16:0/3910532110 >> v2:172.21.15.173:3300/0 conn(0x7f7f0c0cc500 0x7f7f0c0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.492+0000 7f7f0ad9d640 1 -- 172.21.15.16:0/3910532110 >> v1:172.21.15.173:6789/0 conn(0x7f7f0c143c40 legacy=0x7f7f0c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:09.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.492+0000 7f7f0ad9d640 1 -- 172.21.15.16:0/3910532110 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c144020 con 0x7f7f0c06d160 2023-11-24T06:30:09.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.492+0000 7f7f09d9b640 1 --2- 172.21.15.16:0/3910532110 >> v2:172.21.15.173:3300/0 conn(0x7f7f0c0cc500 0x7f7f0c0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:09.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.493+0000 7f7f0ad9d640 1 --2- 172.21.15.16:0/3910532110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c06d160 0x7f7f0c06d560 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0b44a0 tx=0x7f7f0402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea9ba32775a7adc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:09.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.493+0000 7f7f0959a640 1 -- 172.21.15.16:0/3910532110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0402c9f0 con 0x7f7f0c06d160 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.493+0000 7f7f0959a640 1 -- 172.21.15.16:0/3910532110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0402cb50 con 0x7f7f0c06d160 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.493+0000 7f7f0959a640 1 -- 172.21.15.16:0/3910532110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f040343f0 con 0x7f7f0c06d160 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.494+0000 7f7f0b59e640 1 -- 172.21.15.16:0/3910532110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c06d160 msgr2=0x7f7f0c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.494+0000 7f7f0b59e640 1 --2- 172.21.15.16:0/3910532110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c06d160 0x7f7f0c06d560 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0b44a0 tx=0x7f7f0402be50 comp rx=0 tx=0).stop 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.494+0000 7f7f0b59e640 1 -- 172.21.15.16:0/3910532110 shutdown_connections 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.494+0000 7f7f0b59e640 1 --2- 172.21.15.16:0/3910532110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c06d160 0x7f7f0c06d560 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.494+0000 7f7f0b59e640 1 --2- 172.21.15.16:0/3910532110 >> v2:172.21.15.173:3300/0 conn(0x7f7f0c0cc500 0x7f7f0c0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.494+0000 7f7f0b59e640 1 -- 172.21.15.16:0/3910532110 >> 172.21.15.16:0/3910532110 conn(0x7f7f0c06a800 msgr2=0x7f7f0c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.495+0000 7f7f0b59e640 1 -- 172.21.15.16:0/3910532110 shutdown_connections 2023-11-24T06:30:09.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.495+0000 7f7f0b59e640 1 -- 172.21.15.16:0/3910532110 wait complete. 2023-11-24T06:30:09.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.497+0000 7f7f0b59e640 1 Processor -- start 2023-11-24T06:30:09.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.497+0000 7f7f0b59e640 1 -- start start 2023-11-24T06:30:09.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.498+0000 7f7f0b59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f0c06d160 0x7f7f0c1f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.498+0000 7f7f0b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c0cc500 0x7f7f0c0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.498+0000 7f7f0b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f0c143c40 0x7f7f0c1f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.498+0000 7f7f0b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c13cec0 con 0x7f7f0c0cc500 2023-11-24T06:30:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.498+0000 7f7f0b59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c1e4420 con 0x7f7f0c06d160 2023-11-24T06:30:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.498+0000 7f7f0b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c1e4700 con 0x7f7f0c143c40 2023-11-24T06:30:09.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f09d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c0cc500 0x7f7f0c0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f09d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c0cc500 0x7f7f0c0b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36450/0 (socket says 172.21.15.16:36450) 2023-11-24T06:30:09.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f09d9b640 1 -- 172.21.15.16:0/756126002 learned_addr learned my addr 172.21.15.16:0/756126002 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:09.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f0a59c640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f0c06d160 0x7f7f0c1f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f09d9b640 1 -- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f0c143c40 msgr2=0x7f7f0c1f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:09.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f09d9b640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f0c143c40 0x7f7f0c1f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f09d9b640 1 -- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f0c06d160 msgr2=0x7f7f0c1f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:09.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f09d9b640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f0c06d160 0x7f7f0c1f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f09d9b640 1 -- 172.21.15.16:0/756126002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04009b30 con 0x7f7f0c0cc500 2023-11-24T06:30:09.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.499+0000 7f7f0a59c640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f0c06d160 0x7f7f0c1f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.500+0000 7f7f09d9b640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c0cc500 0x7f7f0c0b2c20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c7f0 tx=0x7f7f0000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.500+0000 7f7efaffd640 1 -- 172.21.15.16:0/756126002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00007a30 con 0x7f7f0c0cc500 2023-11-24T06:30:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.500+0000 7f7efaffd640 1 -- 172.21.15.16:0/756126002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f00007b90 con 0x7f7f0c0cc500 2023-11-24T06:30:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.500+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0c1e49e0 con 0x7f7f0c0cc500 2023-11-24T06:30:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.500+0000 7f7efaffd640 1 -- 172.21.15.16:0/756126002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000ce40 con 0x7f7f0c0cc500 2023-11-24T06:30:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.500+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c1e4fa0 con 0x7f7f0c0cc500 2023-11-24T06:30:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.501+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0c067450 con 0x7f7f0c0cc500 2023-11-24T06:30:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.502+0000 7f7efaffd640 1 -- 172.21.15.16:0/756126002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f7f000113f0 con 0x7f7f0c0cc500 2023-11-24T06:30:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.503+0000 7f7efaffd640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7eec06b110 0x7f7eec06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.503+0000 7f7efaffd640 1 -- 172.21.15.16:0/756126002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f00089d50 con 0x7f7f0c0cc500 2023-11-24T06:30:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.504+0000 7f7f0a59c640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7eec06b110 0x7f7eec06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.505+0000 7f7f0a59c640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7eec06b110 0x7f7eec06d5d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00a830 tx=0x7f7efc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:09.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.507+0000 7f7efaffd640 1 -- 172.21.15.16:0/756126002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f00054960 con 0x7f7f0c0cc500 2023-11-24T06:30:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.703+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f0c0ceb30 con 0x7f7f0c0cc500 2023-11-24T06:30:09.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.705+0000 7f7efaffd640 1 -- 172.21.15.16:0/756126002 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16565 (secure 0 0 0) 0x7f7f0001f090 con 0x7f7f0c0cc500 2023-11-24T06:30:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:09.709 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":59,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:29:54.005340+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:50.546182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:51.698824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7eec06b110 msgr2=0x7f7eec06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7eec06b110 0x7f7eec06d5d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00a830 tx=0x7f7efc008040 comp rx=0 tx=0).stop 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c0cc500 msgr2=0x7f7f0c0b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c0cc500 0x7f7f0c0b2c20 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c7f0 tx=0x7f7f0000cbf0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 shutdown_connections 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7eec06b110 0x7f7eec06d5d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f0c143c40 0x7f7f0c1f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f0c0cc500 0x7f7f0c0b2c20 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 --2- 172.21.15.16:0/756126002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f0c06d160 0x7f7f0c1f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.707+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 >> 172.21.15.16:0/756126002 conn(0x7f7f0c06a800 msgr2=0x7f7f0c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.708+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 shutdown_connections 2023-11-24T06:30:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.708+0000 7f7f0b59e640 1 -- 172.21.15.16:0/756126002 wait complete. 2023-11-24T06:30:09.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:29:50.546182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:29:51.698824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:30:09.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:30:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.956+0000 7f0bca9cf640 1 Processor -- start 2023-11-24T06:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.956+0000 7f0bca9cf640 1 -- start start 2023-11-24T06:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.958+0000 7f0bca9cf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0bc4071e40 0x7f0bc40ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.958+0000 7f0bca9cf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc415d770 0x7f0bc415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.958+0000 7f0bca9cf640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc415e140 con 0x7f0bc4143810 2023-11-24T06:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.958+0000 7f0bca9cf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc4143bf0 con 0x7f0bc4071e40 2023-11-24T06:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.958+0000 7f0bca9cf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40722f0 con 0x7f0bc415d770 2023-11-24T06:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.958+0000 7f0bc91cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0bc4071e40 0x7f0bc40ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.958+0000 7f0bc91cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0bc4071e40 0x7f0bc40ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36460/0 (socket says 172.21.15.16:36460) 2023-11-24T06:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.958+0000 7f0bc91cc640 1 -- 172.21.15.16:0/816488240 learned_addr learned my addr 172.21.15.16:0/816488240 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.959+0000 7f0bca1ce640 1 --2- 172.21.15.16:0/816488240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc415d770 0x7f0bc415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.959+0000 7f0bc91cc640 1 -- 172.21.15.16:0/816488240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc415d770 msgr2=0x7f0bc415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.959+0000 7f0bc91cc640 1 --2- 172.21.15.16:0/816488240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc415d770 0x7f0bc415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.959+0000 7f0bc91cc640 1 -- 172.21.15.16:0/816488240 >> v1:172.21.15.173:6789/0 conn(0x7f0bc4143810 legacy=0x7f0bc415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.959+0000 7f0bc91cc640 1 -- 172.21.15.16:0/816488240 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc406c730 con 0x7f0bc4071e40 2023-11-24T06:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.960+0000 7f0bc91cc640 1 --2- 172.21.15.16:0/816488240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc4071e40 0x7f0bc40ae170 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009870 tx=0x7f0bb402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2c4b4d6356cd81 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.960+0000 7f0bc89cb640 1 -- 172.21.15.16:0/816488240 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4031740 con 0x7f0bc4071e40 2023-11-24T06:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.960+0000 7f0bc89cb640 1 -- 172.21.15.16:0/816488240 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb40318a0 con 0x7f0bc4071e40 2023-11-24T06:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.960+0000 7f0bc89cb640 1 -- 172.21.15.16:0/816488240 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4031c30 con 0x7f0bc4071e40 2023-11-24T06:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.961+0000 7f0bca9cf640 1 -- 172.21.15.16:0/816488240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc4071e40 msgr2=0x7f0bc40ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.961+0000 7f0bca9cf640 1 --2- 172.21.15.16:0/816488240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc4071e40 0x7f0bc40ae170 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009870 tx=0x7f0bb402b7e0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.961+0000 7f0bca9cf640 1 -- 172.21.15.16:0/816488240 shutdown_connections 2023-11-24T06:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.961+0000 7f0bca9cf640 1 --2- 172.21.15.16:0/816488240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc415d770 0x7f0bc415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.961+0000 7f0bca9cf640 1 --2- 172.21.15.16:0/816488240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc4071e40 0x7f0bc40ae170 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.961+0000 7f0bca9cf640 1 -- 172.21.15.16:0/816488240 >> 172.21.15.16:0/816488240 conn(0x7f0bc406a7d0 msgr2=0x7f0bc406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.961+0000 7f0bca9cf640 1 -- 172.21.15.16:0/816488240 shutdown_connections 2023-11-24T06:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.961+0000 7f0bca9cf640 1 -- 172.21.15.16:0/816488240 wait complete. 2023-11-24T06:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.963+0000 7f0bca9cf640 1 Processor -- start 2023-11-24T06:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.963+0000 7f0bca9cf640 1 -- start start 2023-11-24T06:30:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.964+0000 7f0bca9cf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc4071e40 0x7f0bc415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.964+0000 7f0bca9cf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc4143810 0x7f0bc40cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.964+0000 7f0bca9cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc415d770 0x7f0bc40ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.964+0000 7f0bca9cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4145650 con 0x7f0bc415d770 2023-11-24T06:30:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.964+0000 7f0bca9cf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4144d80 con 0x7f0bc4071e40 2023-11-24T06:30:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.964+0000 7f0bca9cf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc415d040 con 0x7f0bc4143810 2023-11-24T06:30:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.964+0000 7f0bca1ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc415d770 0x7f0bc40ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.964+0000 7f0bca1ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc415d770 0x7f0bc40ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36476/0 (socket says 172.21.15.16:36476) 2023-11-24T06:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.964+0000 7f0bca1ce640 1 -- 172.21.15.16:0/1880398973 learned_addr learned my addr 172.21.15.16:0/1880398973 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.965+0000 7f0bc91cc640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc4143810 0x7f0bc40cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.965+0000 7f0bc99cd640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc4071e40 0x7f0bc415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.965+0000 7f0bca1ce640 1 -- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc4143810 msgr2=0x7f0bc40cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.965+0000 7f0bca1ce640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc4143810 0x7f0bc40cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.965+0000 7f0bca1ce640 1 -- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc4071e40 msgr2=0x7f0bc415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.965+0000 7f0bca1ce640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc4071e40 0x7f0bc415cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.965+0000 7f0bca1ce640 1 -- 172.21.15.16:0/1880398973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40094d0 con 0x7f0bc415d770 2023-11-24T06:30:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.965+0000 7f0bc91cc640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc4143810 0x7f0bc40cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.966+0000 7f0bca1ce640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc415d770 0x7f0bc40ccee0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc007e60 tx=0x7f0bbc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.966+0000 7f0bba7fc640 1 -- 172.21.15.16:0/1880398973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc014420 con 0x7f0bc415d770 2023-11-24T06:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.966+0000 7f0bba7fc640 1 -- 172.21.15.16:0/1880398973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc0145a0 con 0x7f0bc415d770 2023-11-24T06:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.966+0000 7f0bba7fc640 1 -- 172.21.15.16:0/1880398973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc014950 con 0x7f0bc415d770 2023-11-24T06:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.966+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc40cc960 con 0x7f0bc415d770 2023-11-24T06:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.966+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc4145270 con 0x7f0bc415d770 2023-11-24T06:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.968+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc4145790 con 0x7f0bc415d770 2023-11-24T06:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.968+0000 7f0bba7fc640 1 -- 172.21.15.16:0/1880398973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f0bbc014ab0 con 0x7f0bc415d770 2023-11-24T06:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.969+0000 7f0bba7fc640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba806b110 0x7f0ba806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.969+0000 7f0bba7fc640 1 -- 172.21.15.16:0/1880398973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bbc08a390 con 0x7f0bc415d770 2023-11-24T06:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.970+0000 7f0bc99cd640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba806b110 0x7f0ba806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.972+0000 7f0bc99cd640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba806b110 0x7f0ba806d5d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40b2340 tx=0x7f0bc0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:09.975+0000 7f0bba7fc640 1 -- 172.21.15.16:0/1880398973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bc4145790 con 0x7f0bc415d770 2023-11-24T06:30:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.169+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0bc4145790 con 0x7f0bc415d770 2023-11-24T06:30:10.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.171+0000 7f0bba7fc640 1 -- 172.21.15.16:0/1880398973 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0bc4145790 con 0x7f0bc415d770 2023-11-24T06:30:10.175 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:30:10.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba806b110 msgr2=0x7f0ba806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba806b110 0x7f0ba806d5d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40b2340 tx=0x7f0bc0009040 comp rx=0 tx=0).stop 2023-11-24T06:30:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc415d770 msgr2=0x7f0bc40ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc415d770 0x7f0bc40ccee0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc007e60 tx=0x7f0bbc00e4b0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 shutdown_connections 2023-11-24T06:30:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0ba806b110 0x7f0ba806d5d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc415d770 0x7f0bc40ccee0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bc4143810 0x7f0bc40cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 --2- 172.21.15.16:0/1880398973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bc4071e40 0x7f0bc415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 >> 172.21.15.16:0/1880398973 conn(0x7f0bc406a7d0 msgr2=0x7f0bc406abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 shutdown_connections 2023-11-24T06:30:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.174+0000 7f0bca9cf640 1 -- 172.21.15.16:0/1880398973 wait complete. 2023-11-24T06:30:10.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:30:10.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.423+0000 7f304bfff640 1 Processor -- start 2023-11-24T06:30:10.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.423+0000 7f304bfff640 1 -- start start 2023-11-24T06:30:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.426+0000 7f304bfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f304c0ccb50 0x7f304c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.426+0000 7f304bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c15bf10 0x7f304c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.426+0000 7f304bfff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c15f4e0 con 0x7f304c0adf20 2023-11-24T06:30:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.426+0000 7f304bfff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c15f970 con 0x7f304c0ccb50 2023-11-24T06:30:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.426+0000 7f304bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c15c3c0 con 0x7f304c15bf10 2023-11-24T06:30:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.426+0000 7f304affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f304c0adf20 0x7f304c0cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35482/0 (socket says 172.21.15.16:35482) 2023-11-24T06:30:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.426+0000 7f304affd640 1 -- 172.21.15.16:0/945365784 learned_addr learned my addr 172.21.15.16:0/945365784 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f3043fff640 1 --2- 172.21.15.16:0/945365784 >> v2:172.21.15.173:3300/0 conn(0x7f304c0ccb50 0x7f304c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304b7fe640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c15bf10 0x7f304c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027061527 0 0) 0x7f304c15f4e0 con 0x7f304c0adf20 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3038003520 con 0x7f304c0adf20 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2397342101 0 0) 0x7f3038003520 con 0x7f304c0adf20 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c15bf10 msgr2=0x7f304c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c15bf10 0x7f304c143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 >> v2:172.21.15.173:3300/0 conn(0x7f304c0ccb50 msgr2=0x7f304c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 --2- 172.21.15.16:0/945365784 >> v2:172.21.15.173:3300/0 conn(0x7f304c0ccb50 0x7f304c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c071130 con 0x7f304c0adf20 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f303c001670 con 0x7f304c0adf20 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 >> v1:172.21.15.16:6789/0 conn(0x7f304c0adf20 legacy=0x7f304c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3038003970 0x7f3038003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f3038004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c15bf10 0x7f3038005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c15f970 con 0x7f30380047c0 2023-11-24T06:30:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c15c3c0 con 0x7f3038003970 2023-11-24T06:30:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.427+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303800b9d0 con 0x7f304c15bf10 2023-11-24T06:30:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.428+0000 7f3043fff640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f3038004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.428+0000 7f3043fff640 1 -- 172.21.15.16:0/945365784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c15bf10 msgr2=0x7f3038005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.428+0000 7f3043fff640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c15bf10 0x7f3038005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.428+0000 7f3043fff640 1 -- 172.21.15.16:0/945365784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3038003970 msgr2=0x7f3038003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.428+0000 7f3043fff640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3038003970 0x7f3038003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.428+0000 7f3043fff640 1 -- 172.21.15.16:0/945365784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3038003520 con 0x7f30380047c0 2023-11-24T06:30:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.429+0000 7f3043fff640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f3038004c40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3034009ac0 tx=0x7f30380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de8941869041b62 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.429+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303402c6b0 con 0x7f30380047c0 2023-11-24T06:30:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.429+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303402c810 con 0x7f30380047c0 2023-11-24T06:30:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.429+0000 7f304a7fc640 1 -- 172.21.15.16:0/945365784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303402cba0 con 0x7f30380047c0 2023-11-24T06:30:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.430+0000 7f304bfff640 1 -- 172.21.15.16:0/945365784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 msgr2=0x7f3038004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.430+0000 7f304bfff640 1 --2- 172.21.15.16:0/945365784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f3038004c40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3034009ac0 tx=0x7f30380057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.430+0000 7f304b7fe640 1 -- 172.21.15.16:0/945365784 reap_dead start 2023-11-24T06:30:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.430+0000 7f304bfff640 1 -- 172.21.15.16:0/945365784 shutdown_connections 2023-11-24T06:30:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.430+0000 7f304bfff640 1 -- 172.21.15.16:0/945365784 >> 172.21.15.16:0/945365784 conn(0x7f304c06a7d0 msgr2=0x7f304c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.430+0000 7f304bfff640 1 -- 172.21.15.16:0/945365784 shutdown_connections 2023-11-24T06:30:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.430+0000 7f304bfff640 1 -- 172.21.15.16:0/945365784 wait complete. 2023-11-24T06:30:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.432+0000 7f304bfff640 1 Processor -- start 2023-11-24T06:30:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.432+0000 7f304bfff640 1 -- start start 2023-11-24T06:30:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.433+0000 7f304bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f304c13dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.433+0000 7f304bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c15a880 0x7f304c0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.433+0000 7f304bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0709f0 0x7f304c13f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.433+0000 7f304bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c070e70 con 0x7f30380047c0 2023-11-24T06:30:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.433+0000 7f304bfff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1f2a60 con 0x7f304c15a880 2023-11-24T06:30:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.433+0000 7f304bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1f2ce0 con 0x7f304c0709f0 2023-11-24T06:30:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.433+0000 7f304affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f304c13dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.433+0000 7f304affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f304c13dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36494/0 (socket says 172.21.15.16:36494) 2023-11-24T06:30:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f304affd640 1 -- 172.21.15.16:0/500375886 learned_addr learned my addr 172.21.15.16:0/500375886 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f3043fff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c15a880 0x7f304c0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f304b7fe640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0709f0 0x7f304c13f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f304affd640 1 -- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0709f0 msgr2=0x7f304c13f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f304affd640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0709f0 0x7f304c13f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f304affd640 1 -- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c15a880 msgr2=0x7f304c0704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f304affd640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c15a880 0x7f304c0704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f304affd640 1 -- 172.21.15.16:0/500375886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c1f2fc0 con 0x7f30380047c0 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f3043fff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c15a880 0x7f304c0704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f304affd640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f304c13dff0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c730 tx=0x7f303c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.434+0000 7f30437fe640 1 -- 172.21.15.16:0/500375886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c0074e0 con 0x7f30380047c0 2023-11-24T06:30:10.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.435+0000 7f30437fe640 1 -- 172.21.15.16:0/500375886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c007660 con 0x7f30380047c0 2023-11-24T06:30:10.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.435+0000 7f30437fe640 1 -- 172.21.15.16:0/500375886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c004a20 con 0x7f30380047c0 2023-11-24T06:30:10.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.435+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c1f32a0 con 0x7f30380047c0 2023-11-24T06:30:10.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.435+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c1f3860 con 0x7f30380047c0 2023-11-24T06:30:10.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.436+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c1f6030 con 0x7f30380047c0 2023-11-24T06:30:10.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.437+0000 7f30437fe640 1 -- 172.21.15.16:0/500375886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f303c004b80 con 0x7f30380047c0 2023-11-24T06:30:10.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.438+0000 7f30437fe640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f302c06b140 0x7f302c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.438+0000 7f30437fe640 1 -- 172.21.15.16:0/500375886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f303c088f40 con 0x7f30380047c0 2023-11-24T06:30:10.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.438+0000 7f3043fff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f302c06b140 0x7f302c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.440+0000 7f3043fff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f302c06b140 0x7f302c06d600 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3034036000 tx=0x7f3034030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:10.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.442+0000 7f30437fe640 1 -- 172.21.15.16:0/500375886 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f303c053b80 con 0x7f30380047c0 2023-11-24T06:30:10.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.637+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f304c067450 con 0x7f30380047c0 2023-11-24T06:30:10.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.638+0000 7f30437fe640 1 -- 172.21.15.16:0/500375886 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7f303c0180a0 con 0x7f30380047c0 2023-11-24T06:30:10.641 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:30:10.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.640+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f302c06b140 msgr2=0x7f302c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.640+0000 7f304bfff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f302c06b140 0x7f302c06d600 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3034036000 tx=0x7f3034030000 comp rx=0 tx=0).stop 2023-11-24T06:30:10.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 msgr2=0x7f304c13dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f304c13dff0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c730 tx=0x7f303c00cbf0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 shutdown_connections 2023-11-24T06:30:10.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f302c06b140 0x7f302c06d600 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0709f0 0x7f304c13f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c15a880 0x7f304c0704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 --2- 172.21.15.16:0/500375886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30380047c0 0x7f304c13dff0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 >> 172.21.15.16:0/500375886 conn(0x7f304c06a7d0 msgr2=0x7f304c06db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:10.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 shutdown_connections 2023-11-24T06:30:10.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.641+0000 7f304bfff640 1 -- 172.21.15.16:0/500375886 wait complete. 2023-11-24T06:30:10.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:30:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.885+0000 7f4d0b27c640 1 Processor -- start 2023-11-24T06:30:10.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.885+0000 7f4d0b27c640 1 -- start start 2023-11-24T06:30:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.887+0000 7f4d0b27c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d04143ae0 0x7f4d040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.887+0000 7f4d0b27c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d040cfc30 con 0x7f4d040cf7d0 2023-11-24T06:30:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.887+0000 7f4d0b27c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d040b1ee0 con 0x7f4d0415e3b0 2023-11-24T06:30:10.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.887+0000 7f4d0b27c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d040b2780 con 0x7f4d04143ae0 2023-11-24T06:30:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.887+0000 7f4d09018640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d04143ae0 0x7f4d040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.887+0000 7f4d08817640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4d040cf7d0 0x7f4d0415dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35484/0 (socket says 172.21.15.16:35484) 2023-11-24T06:30:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.887+0000 7f4d08817640 1 -- 172.21.15.16:0/3728588329 learned_addr learned my addr 172.21.15.16:0/3728588329 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.888+0000 7f4d09018640 1 -- 172.21.15.16:0/3728588329 >> v1:172.21.15.173:6789/0 conn(0x7f4d0415e3b0 legacy=0x7f4d040702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.888+0000 7f4d09018640 1 -- 172.21.15.16:0/3728588329 >> v1:172.21.15.16:6789/0 conn(0x7f4d040cf7d0 legacy=0x7f4d0415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.888+0000 7f4d09018640 1 -- 172.21.15.16:0/3728588329 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0415e990 con 0x7f4d04143ae0 2023-11-24T06:30:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.889+0000 7f4d09018640 1 --2- 172.21.15.16:0/3728588329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d04143ae0 0x7f4d040cf290 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4d000097a0 tx=0x7f4d0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829ecd4304f2efe5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.889+0000 7f4cfbfff640 1 -- 172.21.15.16:0/3728588329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00031830 con 0x7f4d04143ae0 2023-11-24T06:30:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.889+0000 7f4cfbfff640 1 -- 172.21.15.16:0/3728588329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d00031990 con 0x7f4d04143ae0 2023-11-24T06:30:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.889+0000 7f4cfbfff640 1 -- 172.21.15.16:0/3728588329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00031d20 con 0x7f4d04143ae0 2023-11-24T06:30:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.890+0000 7f4d0b27c640 1 -- 172.21.15.16:0/3728588329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d04143ae0 msgr2=0x7f4d040cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.890+0000 7f4d0b27c640 1 --2- 172.21.15.16:0/3728588329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d04143ae0 0x7f4d040cf290 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f4d000097a0 tx=0x7f4d0002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.890+0000 7f4d0b27c640 1 -- 172.21.15.16:0/3728588329 shutdown_connections 2023-11-24T06:30:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.890+0000 7f4d0b27c640 1 --2- 172.21.15.16:0/3728588329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d04143ae0 0x7f4d040cf290 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.890+0000 7f4d0b27c640 1 -- 172.21.15.16:0/3728588329 >> 172.21.15.16:0/3728588329 conn(0x7f4d0406a800 msgr2=0x7f4d0406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:10.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.890+0000 7f4d0b27c640 1 -- 172.21.15.16:0/3728588329 shutdown_connections 2023-11-24T06:30:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.890+0000 7f4d0b27c640 1 -- 172.21.15.16:0/3728588329 wait complete. 2023-11-24T06:30:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.891+0000 7f4d0b27c640 1 Processor -- start 2023-11-24T06:30:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.892+0000 7f4d0b27c640 1 -- start start 2023-11-24T06:30:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.892+0000 7f4d0b27c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d040cf7d0 0x7f4d041402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d0b27c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d04143ae0 0x7f4d041ef430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d0b27c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d0415e3b0 0x7f4d0413f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d0b27c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04140e60 con 0x7f4d040cf7d0 2023-11-24T06:30:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d0b27c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0413d800 con 0x7f4d04143ae0 2023-11-24T06:30:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d0b27c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d041f26d0 con 0x7f4d0415e3b0 2023-11-24T06:30:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d09018640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d040cf7d0 0x7f4d041402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d09018640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d040cf7d0 0x7f4d041402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36524/0 (socket says 172.21.15.16:36524) 2023-11-24T06:30:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d09018640 1 -- 172.21.15.16:0/1911871643 learned_addr learned my addr 172.21.15.16:0/1911871643 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d09018640 1 -- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d0415e3b0 msgr2=0x7f4d0413f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d09018640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d0415e3b0 0x7f4d0413f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d09018640 1 -- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d04143ae0 msgr2=0x7f4d041ef430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d08817640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d04143ae0 0x7f4d041ef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d09018640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d04143ae0 0x7f4d041ef430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d09018640 1 -- 172.21.15.16:0/1911871643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00009400 con 0x7f4d040cf7d0 2023-11-24T06:30:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.893+0000 7f4d08817640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d04143ae0 0x7f4d041ef430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.894+0000 7f4d09018640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d040cf7d0 0x7f4d041402f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4d000098c0 tx=0x7f4d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.894+0000 7f4cf9ffb640 1 -- 172.21.15.16:0/1911871643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0003ba70 con 0x7f4d040cf7d0 2023-11-24T06:30:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.894+0000 7f4cf9ffb640 1 -- 172.21.15.16:0/1911871643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0003bbf0 con 0x7f4d040cf7d0 2023-11-24T06:30:10.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.894+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d041f29b0 con 0x7f4d040cf7d0 2023-11-24T06:30:10.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.894+0000 7f4cf9ffb640 1 -- 172.21.15.16:0/1911871643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000453f0 con 0x7f4d040cf7d0 2023-11-24T06:30:10.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.894+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d041f2f70 con 0x7f4d040cf7d0 2023-11-24T06:30:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.896+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d040675d0 con 0x7f4d040cf7d0 2023-11-24T06:30:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.896+0000 7f4cf9ffb640 1 -- 172.21.15.16:0/1911871643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f4d00045550 con 0x7f4d040cf7d0 2023-11-24T06:30:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.897+0000 7f4cf9ffb640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ce806b110 0x7f4ce806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.897+0000 7f4cf9ffb640 1 -- 172.21.15.16:0/1911871643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d000abf80 con 0x7f4d040cf7d0 2023-11-24T06:30:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.898+0000 7f4d08817640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ce806b110 0x7f4ce806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:10.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.899+0000 7f4d08817640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ce806b110 0x7f4ce806d5d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4007fd0 tx=0x7f4cf4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:10.901+0000 7f4cf9ffb640 1 -- 172.21.15.16:0/1911871643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d00076cb0 con 0x7f4d040cf7d0 2023-11-24T06:30:11.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.036+0000 7f4cf9ffb640 1 -- 172.21.15.16:0/1911871643 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f4d00077080 con 0x7f4d040cf7d0 2023-11-24T06:30:11.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.102+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d0406ad10 con 0x7f4d040cf7d0 2023-11-24T06:30:11.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.103+0000 7f4cf9ffb640 1 -- 172.21.15.16:0/1911871643 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d0007d920 con 0x7f4d040cf7d0 2023-11-24T06:30:11.107 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:30:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ce806b110 msgr2=0x7f4ce806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ce806b110 0x7f4ce806d5d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4007fd0 tx=0x7f4cf4009040 comp rx=0 tx=0).stop 2023-11-24T06:30:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d040cf7d0 msgr2=0x7f4d041402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:11.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d040cf7d0 0x7f4d041402f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4d000098c0 tx=0x7f4d000046c0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 shutdown_connections 2023-11-24T06:30:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4ce806b110 0x7f4ce806d5d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d0415e3b0 0x7f4d0413f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d04143ae0 0x7f4d041ef430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 --2- 172.21.15.16:0/1911871643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d040cf7d0 0x7f4d041402f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 >> 172.21.15.16:0/1911871643 conn(0x7f4d0406a800 msgr2=0x7f4d0415dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 shutdown_connections 2023-11-24T06:30:11.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.106+0000 7f4d0b27c640 1 -- 172.21.15.16:0/1911871643 wait complete. 2023-11-24T06:30:11.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:30:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.354+0000 7fef2509b640 1 Processor -- start 2023-11-24T06:30:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.354+0000 7fef2509b640 1 -- start start 2023-11-24T06:30:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.356+0000 7fef2509b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2013e360 0x7fef200cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.356+0000 7fef2509b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef2015f9c0 0x7fef200cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.356+0000 7fef2509b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2015c5f0 con 0x7fef2013df80 2023-11-24T06:30:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.356+0000 7fef2509b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2006c470 con 0x7fef2015f9c0 2023-11-24T06:30:11.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.356+0000 7fef2509b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2006c750 con 0x7fef2013e360 2023-11-24T06:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.356+0000 7fef1f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef2015f9c0 0x7fef200cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.357+0000 7fef1f59e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef2015f9c0 0x7fef200cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36528/0 (socket says 172.21.15.16:36528) 2023-11-24T06:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.357+0000 7fef1f59e640 1 -- 172.21.15.16:0/1832082893 learned_addr learned my addr 172.21.15.16:0/1832082893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.357+0000 7fef1e59c640 1 --2- 172.21.15.16:0/1832082893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2013e360 0x7fef200cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.357+0000 7fef1f59e640 1 -- 172.21.15.16:0/1832082893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2013e360 msgr2=0x7fef200cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.357+0000 7fef1f59e640 1 --2- 172.21.15.16:0/1832082893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2013e360 0x7fef200cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.357+0000 7fef1f59e640 1 -- 172.21.15.16:0/1832082893 >> v1:172.21.15.173:6789/0 conn(0x7fef2013df80 legacy=0x7fef2015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.357+0000 7fef1f59e640 1 -- 172.21.15.16:0/1832082893 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2006ca30 con 0x7fef2015f9c0 2023-11-24T06:30:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.359+0000 7fef1f59e640 1 --2- 172.21.15.16:0/1832082893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2015f9c0 0x7fef200cf7d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fef18009e50 tx=0x7fef1802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f18ce914ee4b6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.359+0000 7fef1dd9b640 1 -- 172.21.15.16:0/1832082893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1802cb00 con 0x7fef2015f9c0 2023-11-24T06:30:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.359+0000 7fef1dd9b640 1 -- 172.21.15.16:0/1832082893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1802cc60 con 0x7fef2015f9c0 2023-11-24T06:30:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.359+0000 7fef1dd9b640 1 -- 172.21.15.16:0/1832082893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef180343f0 con 0x7fef2015f9c0 2023-11-24T06:30:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.360+0000 7fef2509b640 1 -- 172.21.15.16:0/1832082893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2015f9c0 msgr2=0x7fef200cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.360+0000 7fef2509b640 1 --2- 172.21.15.16:0/1832082893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2015f9c0 0x7fef200cf7d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fef18009e50 tx=0x7fef1802bf60 comp rx=0 tx=0).stop 2023-11-24T06:30:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.360+0000 7fef2509b640 1 -- 172.21.15.16:0/1832082893 shutdown_connections 2023-11-24T06:30:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.360+0000 7fef2509b640 1 --2- 172.21.15.16:0/1832082893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2015f9c0 0x7fef200cf7d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.360+0000 7fef2509b640 1 --2- 172.21.15.16:0/1832082893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2013e360 0x7fef200cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.360+0000 7fef2509b640 1 -- 172.21.15.16:0/1832082893 >> 172.21.15.16:0/1832082893 conn(0x7fef2006a800 msgr2=0x7fef2006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.360+0000 7fef2509b640 1 -- 172.21.15.16:0/1832082893 shutdown_connections 2023-11-24T06:30:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.360+0000 7fef2509b640 1 -- 172.21.15.16:0/1832082893 wait complete. 2023-11-24T06:30:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.362+0000 7fef2509b640 1 Processor -- start 2023-11-24T06:30:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.362+0000 7fef2509b640 1 -- start start 2023-11-24T06:30:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef2509b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2013df80 0x7fef2013d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef2509b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef2013e360 0x7fef200af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef2509b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2015f9c0 0x7fef200af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef2509b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2014e7b0 con 0x7fef2013df80 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef2509b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2013de30 con 0x7fef2013e360 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef2509b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2014dfa0 con 0x7fef2015f9c0 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef1ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2013df80 0x7fef2013d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef1ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2013df80 0x7fef2013d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36530/0 (socket says 172.21.15.16:36530) 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef1ed9d640 1 -- 172.21.15.16:0/208779141 learned_addr learned my addr 172.21.15.16:0/208779141 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef1f59e640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2015f9c0 0x7fef200af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.363+0000 7fef1e59c640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef2013e360 0x7fef200af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.364+0000 7fef1ed9d640 1 -- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2015f9c0 msgr2=0x7fef200af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.364+0000 7fef1ed9d640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2015f9c0 0x7fef200af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.364+0000 7fef1ed9d640 1 -- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef2013e360 msgr2=0x7fef200af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.364+0000 7fef1ed9d640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef2013e360 0x7fef200af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.364+0000 7fef1ed9d640 1 -- 172.21.15.16:0/208779141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef18009b30 con 0x7fef2013df80 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.364+0000 7fef1ed9d640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2013df80 0x7fef2013d8f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c400 tx=0x7fef1000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.364+0000 7fef077fe640 1 -- 172.21.15.16:0/208779141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef100052d0 con 0x7fef2013df80 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.364+0000 7fef077fe640 1 -- 172.21.15.16:0/208779141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef10005430 con 0x7fef2013df80 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.364+0000 7fef077fe640 1 -- 172.21.15.16:0/208779141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10003830 con 0x7fef2013df80 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.365+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2014e1f0 con 0x7fef2013df80 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.365+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2014bcf0 con 0x7fef2013df80 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.366+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2014bb50 con 0x7fef2013df80 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.367+0000 7fef077fe640 1 -- 172.21.15.16:0/208779141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fef10003990 con 0x7fef2013df80 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.368+0000 7fef077fe640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef0c06b260 0x7fef0c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.368+0000 7fef077fe640 1 -- 172.21.15.16:0/208779141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef10089280 con 0x7fef2013df80 2023-11-24T06:30:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.369+0000 7fef1e59c640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef0c06b260 0x7fef0c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.370+0000 7fef1e59c640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef0c06b260 0x7fef0c06d720 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fef2015a330 tx=0x7fef14008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.373+0000 7fef077fe640 1 -- 172.21.15.16:0/208779141 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef2014bb50 con 0x7fef2013df80 2023-11-24T06:30:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.580+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef2014bb50 con 0x7fef2013df80 2023-11-24T06:30:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.581+0000 7fef077fe640 1 -- 172.21.15.16:0/208779141 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) v1 ==== 80+0+0 (secure 0 0 0) 0x7fef2014bb50 con 0x7fef2013df80 2023-11-24T06:30:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:30:11.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef0c06b260 msgr2=0x7fef0c06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef0c06b260 0x7fef0c06d720 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fef2015a330 tx=0x7fef14008040 comp rx=0 tx=0).stop 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2013df80 msgr2=0x7fef2013d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2013df80 0x7fef2013d8f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c400 tx=0x7fef1000c8c0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 shutdown_connections 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fef0c06b260 0x7fef0c06d720 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fef2015f9c0 0x7fef200af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef2013e360 0x7fef200af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 --2- 172.21.15.16:0/208779141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef2013df80 0x7fef2013d8f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 >> 172.21.15.16:0/208779141 conn(0x7fef2006a800 msgr2=0x7fef200cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 shutdown_connections 2023-11-24T06:30:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:11.583+0000 7fef2509b640 1 -- 172.21.15.16:0/208779141 wait complete. 2023-11-24T06:30:11.605 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-24T06:30:11.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:11.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:11.701 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[108126]: fuse finished with error 0 and tester_r 0 2023-11-24T06:30:11.706 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:30:11.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:11.706 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:30:11.862 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:30:11.862 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:30:11.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.168+0000 7fae49815640 1 Processor -- start 2023-11-24T06:30:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.168+0000 7fae49815640 1 -- start start 2023-11-24T06:30:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.169+0000 7fae49815640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4415ae40 0x7fae440d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.170+0000 7fae49815640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fae440b3380 0x7fae4415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.170+0000 7fae49815640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4415e2c0 con 0x7fae4415d960 2023-11-24T06:30:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.170+0000 7fae49815640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae44070a40 con 0x7fae440b3380 2023-11-24T06:30:12.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.170+0000 7fae49815640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4415e520 con 0x7fae4415ae40 2023-11-24T06:30:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.170+0000 7fae42ffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4415ae40 0x7fae440d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.170+0000 7fae3a7fc640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fae4415d960 0x7fae440b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:55726/0 (socket says 172.21.15.16:55726) 2023-11-24T06:30:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.170+0000 7fae3a7fc640 1 -- 172.21.15.16:0/1620076472 learned_addr learned my addr 172.21.15.16:0/1620076472 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:12.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.170+0000 7fae437fe640 1 --2- 172.21.15.16:0/1620076472 >> v2:172.21.15.173:3300/0 conn(0x7fae440b3380 0x7fae4415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.171+0000 7fae427fc640 1 -- 172.21.15.16:0/1620076472 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470709491 0 0) 0x7fae4415e2c0 con 0x7fae4415d960 2023-11-24T06:30:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.171+0000 7fae427fc640 1 -- 172.21.15.16:0/1620076472 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae30003520 con 0x7fae4415d960 2023-11-24T06:30:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.171+0000 7fae42ffd640 1 -- 172.21.15.16:0/1620076472 >> v2:172.21.15.173:3300/0 conn(0x7fae440b3380 msgr2=0x7fae4415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.171+0000 7fae42ffd640 1 --2- 172.21.15.16:0/1620076472 >> v2:172.21.15.173:3300/0 conn(0x7fae440b3380 0x7fae4415dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.171+0000 7fae42ffd640 1 -- 172.21.15.16:0/1620076472 >> v1:172.21.15.173:6789/0 conn(0x7fae4415d960 legacy=0x7fae440b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.171+0000 7fae42ffd640 1 -- 172.21.15.16:0/1620076472 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4415e800 con 0x7fae4415ae40 2023-11-24T06:30:12.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.173+0000 7fae42ffd640 1 --2- 172.21.15.16:0/1620076472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4415ae40 0x7fae440d1a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae34009d90 tx=0x7fae3402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab414cafeb5a690 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.173+0000 7fae427fc640 1 -- 172.21.15.16:0/1620076472 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3402cc50 con 0x7fae4415ae40 2023-11-24T06:30:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.173+0000 7fae427fc640 1 -- 172.21.15.16:0/1620076472 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3402cdb0 con 0x7fae4415ae40 2023-11-24T06:30:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.173+0000 7fae427fc640 1 -- 172.21.15.16:0/1620076472 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34034660 con 0x7fae4415ae40 2023-11-24T06:30:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.174+0000 7fae49815640 1 -- 172.21.15.16:0/1620076472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4415ae40 msgr2=0x7fae440d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.174+0000 7fae49815640 1 --2- 172.21.15.16:0/1620076472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4415ae40 0x7fae440d1a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae34009d90 tx=0x7fae3402bea0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.174+0000 7fae49815640 1 -- 172.21.15.16:0/1620076472 shutdown_connections 2023-11-24T06:30:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.174+0000 7fae49815640 1 --2- 172.21.15.16:0/1620076472 >> v2:172.21.15.173:3300/0 conn(0x7fae440b3380 0x7fae4415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.174+0000 7fae49815640 1 --2- 172.21.15.16:0/1620076472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae4415ae40 0x7fae440d1a60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.174+0000 7fae49815640 1 -- 172.21.15.16:0/1620076472 >> 172.21.15.16:0/1620076472 conn(0x7fae4406a3e0 msgr2=0x7fae4406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.174+0000 7fae49815640 1 -- 172.21.15.16:0/1620076472 shutdown_connections 2023-11-24T06:30:12.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.174+0000 7fae49815640 1 -- 172.21.15.16:0/1620076472 wait complete. 2023-11-24T06:30:12.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.176+0000 7fae49815640 1 Processor -- start 2023-11-24T06:30:12.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.176+0000 7fae49815640 1 -- start start 2023-11-24T06:30:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.177+0000 7fae49815640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae440b3380 0x7fae440d0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.177+0000 7fae49815640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4415ae40 0x7fae44156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.177+0000 7fae49815640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae4415d960 0x7fae4415ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.177+0000 7fae49815640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440cfe80 con 0x7fae4415ae40 2023-11-24T06:30:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.177+0000 7fae49815640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440cf270 con 0x7fae4415d960 2023-11-24T06:30:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.177+0000 7fae49815640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440cf550 con 0x7fae440b3380 2023-11-24T06:30:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.177+0000 7fae3a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4415ae40 0x7fae44156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.177+0000 7fae3a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4415ae40 0x7fae44156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36536/0 (socket says 172.21.15.16:36536) 2023-11-24T06:30:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.177+0000 7fae3a7fc640 1 -- 172.21.15.16:0/2268217647 learned_addr learned my addr 172.21.15.16:0/2268217647 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae437fe640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae4415d960 0x7fae4415ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae42ffd640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae440b3380 0x7fae440d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae3a7fc640 1 -- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae440b3380 msgr2=0x7fae440d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae3a7fc640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae440b3380 0x7fae440d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae3a7fc640 1 -- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae4415d960 msgr2=0x7fae4415ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae3a7fc640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae4415d960 0x7fae4415ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae3a7fc640 1 -- 172.21.15.16:0/2268217647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34009a70 con 0x7fae4415ae40 2023-11-24T06:30:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae3a7fc640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4415ae40 0x7fae44156b60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00c390 tx=0x7fae2c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae40ff9640 1 -- 172.21.15.16:0/2268217647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c005ce0 con 0x7fae4415ae40 2023-11-24T06:30:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae40ff9640 1 -- 172.21.15.16:0/2268217647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c005e40 con 0x7fae4415ae40 2023-11-24T06:30:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae40ff9640 1 -- 172.21.15.16:0/2268217647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0072f0 con 0x7fae4415ae40 2023-11-24T06:30:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.178+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae440cf830 con 0x7fae4415ae40 2023-11-24T06:30:12.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.179+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae441570a0 con 0x7fae4415ae40 2023-11-24T06:30:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.180+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae44157260 con 0x7fae4415ae40 2023-11-24T06:30:12.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.181+0000 7fae40ff9640 1 -- 172.21.15.16:0/2268217647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fae2c007450 con 0x7fae4415ae40 2023-11-24T06:30:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.182+0000 7fae40ff9640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fae2406b230 0x7fae2406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.182+0000 7fae40ff9640 1 -- 172.21.15.16:0/2268217647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fae2c089370 con 0x7fae4415ae40 2023-11-24T06:30:12.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.182+0000 7fae42ffd640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fae2406b230 0x7fae2406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.183+0000 7fae42ffd640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fae2406b230 0x7fae2406d6f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae34036000 tx=0x7fae34030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:12.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.187+0000 7fae40ff9640 1 -- 172.21.15.16:0/2268217647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae2c053e50 con 0x7fae4415ae40 2023-11-24T06:30:12.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.385+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae440675d0 con 0x7fae4415ae40 2023-11-24T06:30:12.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.388+0000 7fae40ff9640 1 -- 172.21.15.16:0/2268217647 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16673 (secure 0 0 0) 0x7fae2c057470 con 0x7fae4415ae40 2023-11-24T06:30:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:12.392 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":60,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:12.048861+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:50.546182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:51.698824+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:12.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fae2406b230 msgr2=0x7fae2406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fae2406b230 0x7fae2406d6f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae34036000 tx=0x7fae34030000 comp rx=0 tx=0).stop 2023-11-24T06:30:12.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4415ae40 msgr2=0x7fae44156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4415ae40 0x7fae44156b60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00c390 tx=0x7fae2c00c850 comp rx=0 tx=0).stop 2023-11-24T06:30:12.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 shutdown_connections 2023-11-24T06:30:12.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fae2406b230 0x7fae2406d6f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae4415d960 0x7fae4415ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4415ae40 0x7fae44156b60 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 --2- 172.21.15.16:0/2268217647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae440b3380 0x7fae440d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 >> 172.21.15.16:0/2268217647 conn(0x7fae4406a3e0 msgr2=0x7fae4406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:12.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 shutdown_connections 2023-11-24T06:30:12.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.391+0000 7fae49815640 1 -- 172.21.15.16:0/2268217647 wait complete. 2023-11-24T06:30:12.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:29:50.546182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:29:51.698824+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:30:12.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:30:12.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.640+0000 7fcb1b116640 1 Processor -- start 2023-11-24T06:30:12.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.640+0000 7fcb1b116640 1 -- start start 2023-11-24T06:30:12.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.642+0000 7fcb1b116640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcb1413cf10 0x7fcb1413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.642+0000 7fcb1b116640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fcb140cddf0 0x7fcb140b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.642+0000 7fcb1b116640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb140ce440 con 0x7fcb1413fae0 2023-11-24T06:30:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.642+0000 7fcb1b116640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1413ff90 con 0x7fcb1413cf10 2023-11-24T06:30:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.642+0000 7fcb1b116640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb140ce220 con 0x7fcb140cddf0 2023-11-24T06:30:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.642+0000 7fcb1a114640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcb1413cf10 0x7fcb1413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.642+0000 7fcb19913640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcb1413fae0 0x7fcb140cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51660/0 (socket says 172.21.15.16:51660) 2023-11-24T06:30:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.643+0000 7fcb19913640 1 -- 172.21.15.16:0/804264878 learned_addr learned my addr 172.21.15.16:0/804264878 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:12.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.643+0000 7fcb1a915640 1 --2- 172.21.15.16:0/804264878 >> v2:172.21.15.173:3300/0 conn(0x7fcb140cddf0 0x7fcb140b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.643+0000 7fcb1a114640 1 -- 172.21.15.16:0/804264878 >> v2:172.21.15.173:3300/0 conn(0x7fcb140cddf0 msgr2=0x7fcb140b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:12.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.643+0000 7fcb1a114640 1 --2- 172.21.15.16:0/804264878 >> v2:172.21.15.173:3300/0 conn(0x7fcb140cddf0 0x7fcb140b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.643+0000 7fcb1a114640 1 -- 172.21.15.16:0/804264878 >> v1:172.21.15.16:6789/0 conn(0x7fcb1413fae0 legacy=0x7fcb140cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.643+0000 7fcb1a114640 1 -- 172.21.15.16:0/804264878 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140ce720 con 0x7fcb1413cf10 2023-11-24T06:30:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.644+0000 7fcb1a114640 1 --2- 172.21.15.16:0/804264878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 0x7fcb1413f5a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcb14157f90 tx=0x7fcb1002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bff0e17fc1bff8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb19112640 1 -- 172.21.15.16:0/804264878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10031830 con 0x7fcb1413cf10 2023-11-24T06:30:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb19112640 1 -- 172.21.15.16:0/804264878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10031990 con 0x7fcb1413cf10 2023-11-24T06:30:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb19112640 1 -- 172.21.15.16:0/804264878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10031d20 con 0x7fcb1413cf10 2023-11-24T06:30:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb1b116640 1 -- 172.21.15.16:0/804264878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 msgr2=0x7fcb1413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb1b116640 1 --2- 172.21.15.16:0/804264878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 0x7fcb1413f5a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcb14157f90 tx=0x7fcb1002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb1b116640 1 -- 172.21.15.16:0/804264878 shutdown_connections 2023-11-24T06:30:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb1b116640 1 --2- 172.21.15.16:0/804264878 >> v2:172.21.15.173:3300/0 conn(0x7fcb140cddf0 0x7fcb140b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb1b116640 1 --2- 172.21.15.16:0/804264878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 0x7fcb1413f5a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb1b116640 1 -- 172.21.15.16:0/804264878 >> 172.21.15.16:0/804264878 conn(0x7fcb1406a7d0 msgr2=0x7fcb1406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.645+0000 7fcb1b116640 1 -- 172.21.15.16:0/804264878 shutdown_connections 2023-11-24T06:30:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.646+0000 7fcb1b116640 1 -- 172.21.15.16:0/804264878 wait complete. 2023-11-24T06:30:12.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.647+0000 7fcb1b116640 1 Processor -- start 2023-11-24T06:30:12.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.647+0000 7fcb1b116640 1 -- start start 2023-11-24T06:30:12.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.648+0000 7fcb1b116640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb140cddf0 0x7fcb14138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.648+0000 7fcb1b116640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 0x7fcb14156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.648+0000 7fcb1b116640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb1413fae0 0x7fcb14153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.648+0000 7fcb1b116640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14155490 con 0x7fcb1413cf10 2023-11-24T06:30:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.648+0000 7fcb1b116640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14157630 con 0x7fcb140cddf0 2023-11-24T06:30:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.648+0000 7fcb1b116640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14157350 con 0x7fcb1413fae0 2023-11-24T06:30:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb19913640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 0x7fcb14156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb19913640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 0x7fcb14156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55152/0 (socket says 172.21.15.16:55152) 2023-11-24T06:30:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb19913640 1 -- 172.21.15.16:0/1536064709 learned_addr learned my addr 172.21.15.16:0/1536064709 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb1a915640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb1413fae0 0x7fcb14153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb1a114640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb140cddf0 0x7fcb14138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb19913640 1 -- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb1413fae0 msgr2=0x7fcb14153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb19913640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb1413fae0 0x7fcb14153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb19913640 1 -- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb140cddf0 msgr2=0x7fcb14138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb19913640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb140cddf0 0x7fcb14138110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb19913640 1 -- 172.21.15.16:0/1536064709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb10009400 con 0x7fcb1413cf10 2023-11-24T06:30:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.649+0000 7fcb19913640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 0x7fcb14156870 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c4b0 tx=0x7fcb0400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.650+0000 7fcb0affd640 1 -- 172.21.15.16:0/1536064709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04005ce0 con 0x7fcb1413cf10 2023-11-24T06:30:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.650+0000 7fcb0affd640 1 -- 172.21.15.16:0/1536064709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb04005e40 con 0x7fcb1413cf10 2023-11-24T06:30:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.650+0000 7fcb0affd640 1 -- 172.21.15.16:0/1536064709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04007270 con 0x7fcb1413cf10 2023-11-24T06:30:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.650+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1415b670 con 0x7fcb1413cf10 2023-11-24T06:30:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.650+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1415bc30 con 0x7fcb1413cf10 2023-11-24T06:30:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.651+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1415b4e0 con 0x7fcb1413cf10 2023-11-24T06:30:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.652+0000 7fcb0affd640 1 -- 172.21.15.16:0/1536064709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcb040073d0 con 0x7fcb1413cf10 2023-11-24T06:30:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.653+0000 7fcb0affd640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcaf806b260 0x7fcaf806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.653+0000 7fcb0affd640 1 -- 172.21.15.16:0/1536064709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcb04089130 con 0x7fcb1413cf10 2023-11-24T06:30:12.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.654+0000 7fcb1a114640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcaf806b260 0x7fcaf806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:12.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.655+0000 7fcb1a114640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcaf806b260 0x7fcaf806d720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcb10036000 tx=0x7fcb10008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:12.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.658+0000 7fcb0affd640 1 -- 172.21.15.16:0/1536064709 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb04053d00 con 0x7fcb1413cf10 2023-11-24T06:30:12.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.850+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb14067450 con 0x7fcb1413cf10 2023-11-24T06:30:12.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.850+0000 7fcb0affd640 1 -- 172.21.15.16:0/1536064709 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb04057320 con 0x7fcb1413cf10 2023-11-24T06:30:12.854 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:30:12.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcaf806b260 msgr2=0x7fcaf806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcaf806b260 0x7fcaf806d720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcb10036000 tx=0x7fcb10008030 comp rx=0 tx=0).stop 2023-11-24T06:30:12.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 msgr2=0x7fcb14156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 0x7fcb14156870 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c4b0 tx=0x7fcb0400c970 comp rx=0 tx=0).stop 2023-11-24T06:30:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 shutdown_connections 2023-11-24T06:30:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fcaf806b260 0x7fcaf806d720 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb1413fae0 0x7fcb14153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1413cf10 0x7fcb14156870 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 --2- 172.21.15.16:0/1536064709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb140cddf0 0x7fcb14138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 >> 172.21.15.16:0/1536064709 conn(0x7fcb1406a7d0 msgr2=0x7fcb140af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 shutdown_connections 2023-11-24T06:30:12.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:12.853+0000 7fcb1b116640 1 -- 172.21.15.16:0/1536064709 wait complete. 2023-11-24T06:30:12.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:30:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.109+0000 7f7db4f16640 1 Processor -- start 2023-11-24T06:30:13.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.110+0000 7f7db4f16640 1 -- start start 2023-11-24T06:30:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.111+0000 7f7db4f16640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7db015d1d0 0x7f7db00d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.111+0000 7f7db4f16640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7db00b11d0 0x7f7db00d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.111+0000 7f7db4f16640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db00d0090 con 0x7f7db00b1f70 2023-11-24T06:30:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.111+0000 7f7db4f16640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00d02e0 con 0x7f7db015d1d0 2023-11-24T06:30:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.111+0000 7f7db4f16640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00d05c0 con 0x7f7db00b11d0 2023-11-24T06:30:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.112+0000 7f7daf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7db015d1d0 0x7f7db00d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.112+0000 7f7daf7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7db015d1d0 0x7f7db00d1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55154/0 (socket says 172.21.15.16:55154) 2023-11-24T06:30:13.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.112+0000 7f7daf7fe640 1 -- 172.21.15.16:0/1339478015 learned_addr learned my addr 172.21.15.16:0/1339478015 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.112+0000 7f7daffff640 1 --2- 172.21.15.16:0/1339478015 >> v2:172.21.15.173:3300/0 conn(0x7f7db00b11d0 0x7f7db00d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.112+0000 7f7daf7fe640 1 -- 172.21.15.16:0/1339478015 >> v2:172.21.15.173:3300/0 conn(0x7f7db00b11d0 msgr2=0x7f7db00d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.112+0000 7f7daf7fe640 1 --2- 172.21.15.16:0/1339478015 >> v2:172.21.15.173:3300/0 conn(0x7f7db00b11d0 0x7f7db00d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.112+0000 7f7daf7fe640 1 -- 172.21.15.16:0/1339478015 >> v1:172.21.15.173:6789/0 conn(0x7f7db00b1f70 legacy=0x7f7db00cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.112+0000 7f7daf7fe640 1 -- 172.21.15.16:0/1339478015 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00d08a0 con 0x7f7db015d1d0 2023-11-24T06:30:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.113+0000 7f7daf7fe640 1 --2- 172.21.15.16:0/1339478015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 0x7f7db00d1630 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7db00d4110 tx=0x7f7da002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cb5178a1768f5d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.114+0000 7f7daeffd640 1 -- 172.21.15.16:0/1339478015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0031870 con 0x7f7db015d1d0 2023-11-24T06:30:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.114+0000 7f7daeffd640 1 -- 172.21.15.16:0/1339478015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da00319d0 con 0x7f7db015d1d0 2023-11-24T06:30:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.114+0000 7f7daeffd640 1 -- 172.21.15.16:0/1339478015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da0031d60 con 0x7f7db015d1d0 2023-11-24T06:30:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.114+0000 7f7db4f16640 1 -- 172.21.15.16:0/1339478015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 msgr2=0x7f7db00d1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.114+0000 7f7db4f16640 1 --2- 172.21.15.16:0/1339478015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 0x7f7db00d1630 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7db00d4110 tx=0x7f7da002b910 comp rx=0 tx=0).stop 2023-11-24T06:30:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.115+0000 7f7db4f16640 1 -- 172.21.15.16:0/1339478015 shutdown_connections 2023-11-24T06:30:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.115+0000 7f7db4f16640 1 --2- 172.21.15.16:0/1339478015 >> v2:172.21.15.173:3300/0 conn(0x7f7db00b11d0 0x7f7db00d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.115+0000 7f7db4f16640 1 --2- 172.21.15.16:0/1339478015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 0x7f7db00d1630 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.115+0000 7f7db4f16640 1 -- 172.21.15.16:0/1339478015 >> 172.21.15.16:0/1339478015 conn(0x7f7db00d3050 msgr2=0x7f7db00d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.115+0000 7f7db4f16640 1 -- 172.21.15.16:0/1339478015 shutdown_connections 2023-11-24T06:30:13.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.115+0000 7f7db4f16640 1 -- 172.21.15.16:0/1339478015 wait complete. 2023-11-24T06:30:13.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.117+0000 7f7db4f16640 1 Processor -- start 2023-11-24T06:30:13.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.118+0000 7f7db4f16640 1 -- start start 2023-11-24T06:30:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.119+0000 7f7db4f16640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db00b11d0 0x7f7db014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.119+0000 7f7db4f16640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00b1f70 0x7f7db009e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.119+0000 7f7db4f16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 0x7f7db009e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.119+0000 7f7db4f16640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b3f90 con 0x7f7db015d1d0 2023-11-24T06:30:13.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.119+0000 7f7db4f16640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b3310 con 0x7f7db00b1f70 2023-11-24T06:30:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.119+0000 7f7db4f16640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db00b35f0 con 0x7f7db00b11d0 2023-11-24T06:30:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.119+0000 7f7daffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 0x7f7db009e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.119+0000 7f7daffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 0x7f7db009e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55164/0 (socket says 172.21.15.16:55164) 2023-11-24T06:30:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.119+0000 7f7daffff640 1 -- 172.21.15.16:0/733651784 learned_addr learned my addr 172.21.15.16:0/733651784 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7daf7fe640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db00b11d0 0x7f7db014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7da7fff640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00b1f70 0x7f7db009e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7daffff640 1 -- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db00b11d0 msgr2=0x7f7db014c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7daffff640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db00b11d0 0x7f7db014c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7daffff640 1 -- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00b1f70 msgr2=0x7f7db009e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7daffff640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00b1f70 0x7f7db009e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7daffff640 1 -- 172.21.15.16:0/733651784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da00094d0 con 0x7f7db015d1d0 2023-11-24T06:30:13.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7daffff640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 0x7f7db009e640 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7da800cc30 tx=0x7f7da800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7dacff9640 1 -- 172.21.15.16:0/733651784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da800ce40 con 0x7f7db015d1d0 2023-11-24T06:30:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7dacff9640 1 -- 172.21.15.16:0/733651784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da8003990 con 0x7f7db015d1d0 2023-11-24T06:30:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.120+0000 7f7dacff9640 1 -- 172.21.15.16:0/733651784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8003d20 con 0x7f7db015d1d0 2023-11-24T06:30:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.121+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db00b38d0 con 0x7f7db015d1d0 2023-11-24T06:30:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.121+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db014c890 con 0x7f7db015d1d0 2023-11-24T06:30:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.122+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db00b40d0 con 0x7f7db015d1d0 2023-11-24T06:30:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.123+0000 7f7dacff9640 1 -- 172.21.15.16:0/733651784 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f7da8003e80 con 0x7f7db015d1d0 2023-11-24T06:30:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.124+0000 7f7dacff9640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7d9006b230 0x7f7d9006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.124+0000 7f7dacff9640 1 -- 172.21.15.16:0/733651784 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7da808a2c0 con 0x7f7db015d1d0 2023-11-24T06:30:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.125+0000 7f7daf7fe640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7d9006b230 0x7f7d9006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.126+0000 7f7daf7fe640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7d9006b230 0x7f7d9006d6f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7da0036000 tx=0x7f7da003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.129+0000 7f7dacff9640 1 -- 172.21.15.16:0/733651784 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7db00b40d0 con 0x7f7db015d1d0 2023-11-24T06:30:13.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.328+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7db00b40d0 con 0x7f7db015d1d0 2023-11-24T06:30:13.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.329+0000 7f7dacff9640 1 -- 172.21.15.16:0/733651784 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7da801d020 con 0x7f7db015d1d0 2023-11-24T06:30:13.332 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:30:13.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7d9006b230 msgr2=0x7f7d9006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7d9006b230 0x7f7d9006d6f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7da0036000 tx=0x7f7da003c040 comp rx=0 tx=0).stop 2023-11-24T06:30:13.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 msgr2=0x7f7db009e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 0x7f7db009e640 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7da800cc30 tx=0x7f7da800a5c0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 shutdown_connections 2023-11-24T06:30:13.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7d9006b230 0x7f7d9006d6f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db015d1d0 0x7f7db009e640 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00b1f70 0x7f7db009e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 --2- 172.21.15.16:0/733651784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db00b11d0 0x7f7db014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 >> 172.21.15.16:0/733651784 conn(0x7f7db00d3050 msgr2=0x7f7db00cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 shutdown_connections 2023-11-24T06:30:13.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.332+0000 7f7db4f16640 1 -- 172.21.15.16:0/733651784 wait complete. 2023-11-24T06:30:13.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:30:13.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.584+0000 7f023a8e1640 1 Processor -- start 2023-11-24T06:30:13.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.584+0000 7f023a8e1640 1 -- start start 2023-11-24T06:30:13.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.586+0000 7f023a8e1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02340adf20 0x7f0234070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.586+0000 7f023a8e1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f02340ccb50 0x7f0234143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.586+0000 7f023a8e1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023415f4e0 con 0x7f023415bf10 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.586+0000 7f023a8e1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023415f970 con 0x7f02340adf20 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.586+0000 7f023a8e1640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0234071130 con 0x7f02340ccb50 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.586+0000 7f02398df640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02340adf20 0x7f0234070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.586+0000 7f02390de640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f023415bf10 0x7f02340cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51672/0 (socket says 172.21.15.16:51672) 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.586+0000 7f02390de640 1 -- 172.21.15.16:0/2848650289 learned_addr learned my addr 172.21.15.16:0/2848650289 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.587+0000 7f023a0e0640 1 --2- 172.21.15.16:0/2848650289 >> v2:172.21.15.173:3300/0 conn(0x7f02340ccb50 0x7f0234143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.587+0000 7f02398df640 1 -- 172.21.15.16:0/2848650289 >> v2:172.21.15.173:3300/0 conn(0x7f02340ccb50 msgr2=0x7f0234143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.587+0000 7f02398df640 1 --2- 172.21.15.16:0/2848650289 >> v2:172.21.15.173:3300/0 conn(0x7f02340ccb50 0x7f0234143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.587+0000 7f02398df640 1 -- 172.21.15.16:0/2848650289 >> v1:172.21.15.16:6789/0 conn(0x7f023415bf10 legacy=0x7f02340cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.587+0000 7f02398df640 1 -- 172.21.15.16:0/2848650289 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023415c390 con 0x7f02340adf20 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.587+0000 7f02388dd640 1 -- 172.21.15.16:0/2848650289 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251317017 0 0) 0x7f023415f4e0 con 0x7f023415bf10 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.587+0000 7f023a0e0640 1 --2- 172.21.15.16:0/2848650289 >> v2:172.21.15.173:3300/0 conn(0x7f02340ccb50 0x7f0234143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.588+0000 7f02398df640 1 --2- 172.21.15.16:0/2848650289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02340adf20 0x7f0234070a30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f023406ff20 tx=0x7f023002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0fdd1518b84a3b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.589+0000 7f02388dd640 1 -- 172.21.15.16:0/2848650289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023002e070 con 0x7f02340adf20 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.589+0000 7f02388dd640 1 -- 172.21.15.16:0/2848650289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02300315a0 con 0x7f02340adf20 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.589+0000 7f02388dd640 1 -- 172.21.15.16:0/2848650289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230031930 con 0x7f02340adf20 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.589+0000 7f023a8e1640 1 -- 172.21.15.16:0/2848650289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02340adf20 msgr2=0x7f0234070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.589+0000 7f023a8e1640 1 --2- 172.21.15.16:0/2848650289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02340adf20 0x7f0234070a30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f023406ff20 tx=0x7f023002b840 comp rx=0 tx=0).stop 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.589+0000 7f023a8e1640 1 -- 172.21.15.16:0/2848650289 shutdown_connections 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.589+0000 7f023a8e1640 1 --2- 172.21.15.16:0/2848650289 >> v2:172.21.15.173:3300/0 conn(0x7f02340ccb50 0x7f0234143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.589+0000 7f023a8e1640 1 --2- 172.21.15.16:0/2848650289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02340adf20 0x7f0234070a30 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.589+0000 7f023a8e1640 1 -- 172.21.15.16:0/2848650289 >> 172.21.15.16:0/2848650289 conn(0x7f023406a7d0 msgr2=0x7f023406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:13.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.590+0000 7f023a8e1640 1 -- 172.21.15.16:0/2848650289 shutdown_connections 2023-11-24T06:30:13.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.590+0000 7f023a8e1640 1 -- 172.21.15.16:0/2848650289 wait complete. 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.592+0000 7f023a8e1640 1 Processor -- start 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.592+0000 7f023a8e1640 1 -- start start 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.593+0000 7f023a8e1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02340adf20 0x7f02340acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.593+0000 7f023a8e1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02340ccb50 0x7f023413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.593+0000 7f023a8e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f023415bf10 0x7f023413b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.593+0000 7f023a8e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023413b3a0 con 0x7f023415bf10 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.593+0000 7f023a8e1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02340b2b10 con 0x7f02340adf20 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.593+0000 7f023a8e1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02340b2df0 con 0x7f02340ccb50 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f023a0e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f023415bf10 0x7f023413b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f023a0e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f023415bf10 0x7f023413b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55178/0 (socket says 172.21.15.16:55178) 2023-11-24T06:30:13.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f023a0e0640 1 -- 172.21.15.16:0/2631968309 learned_addr learned my addr 172.21.15.16:0/2631968309 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:13.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f02390de640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02340ccb50 0x7f023413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f02398df640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02340adf20 0x7f02340acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f023a0e0640 1 -- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02340ccb50 msgr2=0x7f023413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f023a0e0640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02340ccb50 0x7f023413fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f023a0e0640 1 -- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02340adf20 msgr2=0x7f02340acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f023a0e0640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02340adf20 0x7f02340acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f023a0e0640 1 -- 172.21.15.16:0/2631968309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230009400 con 0x7f023415bf10 2023-11-24T06:30:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f02398df640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02340adf20 0x7f02340acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.594+0000 7f023a0e0640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f023415bf10 0x7f023413b7c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f022c00cef0 tx=0x7f022c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:13.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.595+0000 7f022a7fc640 1 -- 172.21.15.16:0/2631968309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c00d840 con 0x7f023415bf10 2023-11-24T06:30:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.595+0000 7f022a7fc640 1 -- 172.21.15.16:0/2631968309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c00d9c0 con 0x7f023415bf10 2023-11-24T06:30:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.595+0000 7f022a7fc640 1 -- 172.21.15.16:0/2631968309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c00dd70 con 0x7f023415bf10 2023-11-24T06:30:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.595+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023413afc0 con 0x7f023415bf10 2023-11-24T06:30:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.595+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02340ad260 con 0x7f023415bf10 2023-11-24T06:30:13.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.597+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023413ae00 con 0x7f023415bf10 2023-11-24T06:30:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.597+0000 7f022a7fc640 1 -- 172.21.15.16:0/2631968309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f022c01f440 con 0x7f023415bf10 2023-11-24T06:30:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.599+0000 7f022a7fc640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f021806b230 0x7f021806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.599+0000 7f022a7fc640 1 -- 172.21.15.16:0/2631968309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f022c08a630 con 0x7f023415bf10 2023-11-24T06:30:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.600+0000 7f02398df640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f021806b230 0x7f021806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.601+0000 7f02398df640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f021806b230 0x7f021806d6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0230036000 tx=0x7f023003d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.603+0000 7f022a7fc640 1 -- 172.21.15.16:0/2631968309 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f022c0551d0 con 0x7f023415bf10 2023-11-24T06:30:13.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.799+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02340ad260 con 0x7f023415bf10 2023-11-24T06:30:13.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.799+0000 7f022a7fc640 1 -- 172.21.15.16:0/2631968309 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02340ad260 con 0x7f023415bf10 2023-11-24T06:30:13.803 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:30:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.802+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f021806b230 msgr2=0x7f021806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.802+0000 7f023a8e1640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f021806b230 0x7f021806d6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0230036000 tx=0x7f023003d040 comp rx=0 tx=0).stop 2023-11-24T06:30:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.802+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f023415bf10 msgr2=0x7f023413b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.802+0000 7f023a8e1640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f023415bf10 0x7f023413b7c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f022c00cef0 tx=0x7f022c00a710 comp rx=0 tx=0).stop 2023-11-24T06:30:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.803+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 shutdown_connections 2023-11-24T06:30:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.803+0000 7f023a8e1640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f021806b230 0x7f021806d6f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.803+0000 7f023a8e1640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f023415bf10 0x7f023413b7c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.803+0000 7f023a8e1640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f02340ccb50 0x7f023413fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.803+0000 7f023a8e1640 1 --2- 172.21.15.16:0/2631968309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02340adf20 0x7f02340acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.803+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 >> 172.21.15.16:0/2631968309 conn(0x7f023406a7d0 msgr2=0x7f023406abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.803+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 shutdown_connections 2023-11-24T06:30:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:13.803+0000 7f023a8e1640 1 -- 172.21.15.16:0/2631968309 wait complete. 2023-11-24T06:30:13.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:30:14.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.055+0000 7f1d93ad0640 1 Processor -- start 2023-11-24T06:30:14.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.055+0000 7f1d93ad0640 1 -- start start 2023-11-24T06:30:14.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.056+0000 7f1d93ad0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c13e360 0x7f1d8c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.057+0000 7f1d93ad0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8c15c5f0 con 0x7f1d8c13df80 2023-11-24T06:30:14.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.057+0000 7f1d93ad0640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8c06c470 con 0x7f1d8c15f9c0 2023-11-24T06:30:14.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.057+0000 7f1d93ad0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c06c750 con 0x7f1d8c13e360 2023-11-24T06:30:14.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.057+0000 7f1d9186c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1d8c13df80 0x7f1d8c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51680/0 (socket says 172.21.15.16:51680) 2023-11-24T06:30:14.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.057+0000 7f1d9186c640 1 -- 172.21.15.16:0/207204046 learned_addr learned my addr 172.21.15.16:0/207204046 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:14.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.057+0000 7f1d9106b640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c13e360 0x7f1d8c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368347131 0 0) 0x7f1d8c15c5f0 con 0x7f1d8c13df80 2023-11-24T06:30:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d78003520 con 0x7f1d8c13df80 2023-11-24T06:30:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891529262 0 0) 0x7f1d78003520 con 0x7f1d8c13df80 2023-11-24T06:30:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c13e360 msgr2=0x7f1d8c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c13e360 0x7f1d8c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 >> v1:172.21.15.173:6789/0 conn(0x7f1d8c15f9c0 legacy=0x7f1d8c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c06ca30 con 0x7f1d8c13df80 2023-11-24T06:30:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f1d88001670 con 0x7f1d8c13df80 2023-11-24T06:30:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 >> v1:172.21.15.16:6789/0 conn(0x7f1d8c13df80 legacy=0x7f1d8c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c13e360 0x7f1d780039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d78003ee0 0x7f1d780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d78004d00 0x7f1d78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c06c750 con 0x7f1d78003ee0 2023-11-24T06:30:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7800ba30 con 0x7f1d78004d00 2023-11-24T06:30:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.058+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7800bbf0 con 0x7f1d8c13e360 2023-11-24T06:30:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.059+0000 7f1d9106b640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d78003ee0 0x7f1d780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.059+0000 7f1d9186c640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c13e360 0x7f1d780039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.059+0000 7f1d9206d640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d78004d00 0x7f1d78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.059+0000 7f1d9106b640 1 -- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c13e360 msgr2=0x7f1d780039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.059+0000 7f1d9106b640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c13e360 0x7f1d780039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.059+0000 7f1d9106b640 1 -- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d78004d00 msgr2=0x7f1d78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.059+0000 7f1d9106b640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d78004d00 0x7f1d78005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.059+0000 7f1d9106b640 1 -- 172.21.15.16:0/207204046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d78003520 con 0x7f1d78003ee0 2023-11-24T06:30:14.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.060+0000 7f1d9106b640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d78003ee0 0x7f1d780047c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009ac0 tx=0x7f1d780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c975fbe99c28452 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.061+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8002c6b0 con 0x7f1d78003ee0 2023-11-24T06:30:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.061+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8002c810 con 0x7f1d78003ee0 2023-11-24T06:30:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.061+0000 7f1d9086a640 1 -- 172.21.15.16:0/207204046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8002cba0 con 0x7f1d78003ee0 2023-11-24T06:30:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.061+0000 7f1d93ad0640 1 -- 172.21.15.16:0/207204046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d78003ee0 msgr2=0x7f1d780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.061+0000 7f1d93ad0640 1 --2- 172.21.15.16:0/207204046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d78003ee0 0x7f1d780047c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009ac0 tx=0x7f1d780057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.061+0000 7f1d9206d640 1 -- 172.21.15.16:0/207204046 reap_dead start 2023-11-24T06:30:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.061+0000 7f1d93ad0640 1 -- 172.21.15.16:0/207204046 shutdown_connections 2023-11-24T06:30:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.061+0000 7f1d93ad0640 1 -- 172.21.15.16:0/207204046 >> 172.21.15.16:0/207204046 conn(0x7f1d8c06a800 msgr2=0x7f1d8c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.062+0000 7f1d93ad0640 1 -- 172.21.15.16:0/207204046 shutdown_connections 2023-11-24T06:30:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.062+0000 7f1d93ad0640 1 -- 172.21.15.16:0/207204046 wait complete. 2023-11-24T06:30:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.064+0000 7f1d93ad0640 1 Processor -- start 2023-11-24T06:30:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.064+0000 7f1d93ad0640 1 -- start start 2023-11-24T06:30:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.065+0000 7f1d93ad0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d78003ee0 0x7f1d8c13d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.065+0000 7f1d93ad0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c152a60 0x7f1d8c152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.065+0000 7f1d93ad0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c14bf80 0x7f1d8c14c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.065+0000 7f1d93ad0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c153420 con 0x7f1d8c152a60 2023-11-24T06:30:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.065+0000 7f1d93ad0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c14e2b0 con 0x7f1d78003ee0 2023-11-24T06:30:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.065+0000 7f1d93ad0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c14c940 con 0x7f1d8c14bf80 2023-11-24T06:30:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.065+0000 7f1d9106b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c152a60 0x7f1d8c152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.065+0000 7f1d9106b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c152a60 0x7f1d8c152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55200/0 (socket says 172.21.15.16:55200) 2023-11-24T06:30:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.065+0000 7f1d9106b640 1 -- 172.21.15.16:0/343421030 learned_addr learned my addr 172.21.15.16:0/343421030 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d9206d640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c14bf80 0x7f1d8c14c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d9186c640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d78003ee0 0x7f1d8c13d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d9106b640 1 -- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c14bf80 msgr2=0x7f1d8c14c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d9106b640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c14bf80 0x7f1d8c14c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d9106b640 1 -- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d78003ee0 msgr2=0x7f1d8c13d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d9106b640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d78003ee0 0x7f1d8c13d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d9106b640 1 -- 172.21.15.16:0/343421030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c14d750 con 0x7f1d8c152a60 2023-11-24T06:30:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d9106b640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c152a60 0x7f1d8c152ee0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1d80037040 tx=0x7f1d80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d7e7fc640 1 -- 172.21.15.16:0/343421030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80003810 con 0x7f1d8c152a60 2023-11-24T06:30:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d7e7fc640 1 -- 172.21.15.16:0/343421030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d80003970 con 0x7f1d8c152a60 2023-11-24T06:30:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d7e7fc640 1 -- 172.21.15.16:0/343421030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80003d00 con 0x7f1d8c152a60 2023-11-24T06:30:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.066+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8c14cbd0 con 0x7f1d8c152a60 2023-11-24T06:30:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.067+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c14d190 con 0x7f1d8c152a60 2023-11-24T06:30:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.068+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c0675d0 con 0x7f1d8c152a60 2023-11-24T06:30:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.069+0000 7f1d7e7fc640 1 -- 172.21.15.16:0/343421030 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1d80034440 con 0x7f1d8c152a60 2023-11-24T06:30:14.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.071+0000 7f1d7e7fc640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1d7006b230 0x7f1d7006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.071+0000 7f1d7e7fc640 1 -- 172.21.15.16:0/343421030 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1d800ac050 con 0x7f1d8c152a60 2023-11-24T06:30:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.071+0000 7f1d9186c640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1d7006b230 0x7f1d7006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.072+0000 7f1d9186c640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1d7006b230 0x7f1d7006d6f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1d88001050 tx=0x7f1d88009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.075+0000 7f1d7e7fc640 1 -- 172.21.15.16:0/343421030 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d80076be0 con 0x7f1d8c152a60 2023-11-24T06:30:14.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.284+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1d8c06df00 con 0x7f1d8c152a60 2023-11-24T06:30:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.285+0000 7f1d7e7fc640 1 -- 172.21.15.16:0/343421030 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1d8004c080 con 0x7f1d8c152a60 2023-11-24T06:30:14.288 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:30:14.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.287+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1d7006b230 msgr2=0x7f1d7006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.287+0000 7f1d93ad0640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1d7006b230 0x7f1d7006d6f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1d88001050 tx=0x7f1d88009040 comp rx=0 tx=0).stop 2023-11-24T06:30:14.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.287+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c152a60 msgr2=0x7f1d8c152ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.287+0000 7f1d93ad0640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c152a60 0x7f1d8c152ee0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1d80037040 tx=0x7f1d80004730 comp rx=0 tx=0).stop 2023-11-24T06:30:14.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.288+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 shutdown_connections 2023-11-24T06:30:14.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.288+0000 7f1d93ad0640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1d7006b230 0x7f1d7006d6f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.288+0000 7f1d93ad0640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d8c14bf80 0x7f1d8c14c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.288+0000 7f1d93ad0640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c152a60 0x7f1d8c152ee0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.288+0000 7f1d93ad0640 1 --2- 172.21.15.16:0/343421030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d78003ee0 0x7f1d8c13d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.288+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 >> 172.21.15.16:0/343421030 conn(0x7f1d8c06a800 msgr2=0x7f1d8c15dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.288+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 shutdown_connections 2023-11-24T06:30:14.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.288+0000 7f1d93ad0640 1 -- 172.21.15.16:0/343421030 wait complete. 2023-11-24T06:30:14.311 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi173.front.sepia.ceph.com... 2023-11-24T06:30:14.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:14.311 DEBUG:teuthology.orchestra.run.smithi173:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:14.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[96883]: fuse finished with error 0 and tester_r 0 2023-11-24T06:30:14.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:30:14.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:14.415 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:30:14.471 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:30:14.471 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:30:14.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:30:14.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:30:14.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:14.536 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:30:14.594 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:30:14.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:30:14.595 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:30:14.596 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:30:14.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:30:14.655 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:30:14.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:14.656 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:30:14.710 INFO:teuthology.orchestra.run.smithi173.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:30:14.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:30:14.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:30:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.960+0000 7f6142768640 1 Processor -- start 2023-11-24T06:30:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.960+0000 7f6142768640 1 -- start start 2023-11-24T06:30:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.962+0000 7f6142768640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f613c13df80 0x7f613c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.962+0000 7f6142768640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 0x7f613c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.962+0000 7f6142768640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c06c470 con 0x7f613c15f9c0 2023-11-24T06:30:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.962+0000 7f6142768640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c06c750 con 0x7f613c13df80 2023-11-24T06:30:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.962+0000 7f6142768640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c06ca30 con 0x7f613c13e360 2023-11-24T06:30:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.962+0000 7f613bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f613c13df80 0x7f613c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.962+0000 7f6140d05640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f613c15f9c0 0x7f613c1408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51690/0 (socket says 172.21.15.16:51690) 2023-11-24T06:30:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.962+0000 7f6140d05640 1 -- 172.21.15.16:0/2417049074 learned_addr learned my addr 172.21.15.16:0/2417049074 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.963+0000 7f6133fff640 1 --2- 172.21.15.16:0/2417049074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 0x7f613c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.963+0000 7f613bfff640 1 -- 172.21.15.16:0/2417049074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 msgr2=0x7f613c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.963+0000 7f613bfff640 1 --2- 172.21.15.16:0/2417049074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 0x7f613c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.963+0000 7f613bfff640 1 -- 172.21.15.16:0/2417049074 >> v1:172.21.15.16:6789/0 conn(0x7f613c15f9c0 legacy=0x7f613c1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.963+0000 7f613bfff640 1 -- 172.21.15.16:0/2417049074 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c06cd10 con 0x7f613c13df80 2023-11-24T06:30:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.963+0000 7f613b7fe640 1 -- 172.21.15.16:0/2417049074 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669659320 0 0) 0x7f613c06c470 con 0x7f613c15f9c0 2023-11-24T06:30:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.964+0000 7f613bfff640 1 --2- 172.21.15.16:0/2417049074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c13df80 0x7f613c15bee0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f612c0097a0 tx=0x7f612c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a4a54ed032fed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.964+0000 7f613b7fe640 1 -- 172.21.15.16:0/2417049074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c031830 con 0x7f613c13df80 2023-11-24T06:30:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.964+0000 7f613b7fe640 1 -- 172.21.15.16:0/2417049074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612c031990 con 0x7f613c13df80 2023-11-24T06:30:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.964+0000 7f613b7fe640 1 -- 172.21.15.16:0/2417049074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c031d20 con 0x7f613c13df80 2023-11-24T06:30:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.965+0000 7f6142768640 1 -- 172.21.15.16:0/2417049074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c13df80 msgr2=0x7f613c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.965+0000 7f6142768640 1 --2- 172.21.15.16:0/2417049074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c13df80 0x7f613c15bee0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f612c0097a0 tx=0x7f612c02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.965+0000 7f6142768640 1 -- 172.21.15.16:0/2417049074 shutdown_connections 2023-11-24T06:30:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.965+0000 7f6142768640 1 --2- 172.21.15.16:0/2417049074 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 0x7f613c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.965+0000 7f6142768640 1 --2- 172.21.15.16:0/2417049074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c13df80 0x7f613c15bee0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.965+0000 7f6142768640 1 -- 172.21.15.16:0/2417049074 >> 172.21.15.16:0/2417049074 conn(0x7f613c06a800 msgr2=0x7f613c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.965+0000 7f6142768640 1 -- 172.21.15.16:0/2417049074 shutdown_connections 2023-11-24T06:30:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.965+0000 7f6142768640 1 -- 172.21.15.16:0/2417049074 wait complete. 2023-11-24T06:30:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.967+0000 7f6142768640 1 Processor -- start 2023-11-24T06:30:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.967+0000 7f6142768640 1 -- start start 2023-11-24T06:30:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.968+0000 7f6142768640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f613c13df80 0x7f613c1f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.968+0000 7f6142768640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 0x7f613c1f0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.968+0000 7f6142768640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15f9c0 0x7f613c0a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.968+0000 7f6142768640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c1f08b0 con 0x7f613c15f9c0 2023-11-24T06:30:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.968+0000 7f6142768640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c0a8e20 con 0x7f613c13df80 2023-11-24T06:30:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.968+0000 7f6142768640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c1f40f0 con 0x7f613c13e360 2023-11-24T06:30:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.968+0000 7f6140d05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15f9c0 0x7f613c0a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.968+0000 7f6140d05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15f9c0 0x7f613c0a85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55222/0 (socket says 172.21.15.16:55222) 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.968+0000 7f6140d05640 1 -- 172.21.15.16:0/2119344918 learned_addr learned my addr 172.21.15.16:0/2119344918 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f6133fff640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 0x7f613c1f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f6140d05640 1 -- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 msgr2=0x7f613c1f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f613bfff640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f613c13df80 0x7f613c1f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f6140d05640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 0x7f613c1f0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f6140d05640 1 -- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f613c13df80 msgr2=0x7f613c1f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f6140d05640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f613c13df80 0x7f613c1f3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f6140d05640 1 -- 172.21.15.16:0/2119344918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c009400 con 0x7f613c15f9c0 2023-11-24T06:30:14.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f6140d05640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15f9c0 0x7f613c0a85e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f613400c7a0 tx=0x7f613400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f61397fa640 1 -- 172.21.15.16:0/2119344918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613400a640 con 0x7f613c15f9c0 2023-11-24T06:30:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f61397fa640 1 -- 172.21.15.16:0/2119344918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613400a7a0 con 0x7f613c15f9c0 2023-11-24T06:30:14.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f61397fa640 1 -- 172.21.15.16:0/2119344918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613400abd0 con 0x7f613c15f9c0 2023-11-24T06:30:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.969+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c1f43d0 con 0x7f613c15f9c0 2023-11-24T06:30:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.970+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c0a8b20 con 0x7f613c15f9c0 2023-11-24T06:30:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.971+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c1f48f0 con 0x7f613c15f9c0 2023-11-24T06:30:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.972+0000 7f61397fa640 1 -- 172.21.15.16:0/2119344918 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f613400ad30 con 0x7f613c15f9c0 2023-11-24T06:30:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.972+0000 7f61397fa640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f611c06b230 0x7f611c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.973+0000 7f61397fa640 1 -- 172.21.15.16:0/2119344918 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6134089300 con 0x7f613c15f9c0 2023-11-24T06:30:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.973+0000 7f613bfff640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f611c06b230 0x7f611c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.974+0000 7f613bfff640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f611c06b230 0x7f611c06d6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f612c036000 tx=0x7f612c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:14.977+0000 7f61397fa640 1 -- 172.21.15.16:0/2119344918 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f613c1f48f0 con 0x7f613c15f9c0 2023-11-24T06:30:15.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.211+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f613c1f48f0 con 0x7f613c15f9c0 2023-11-24T06:30:15.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.212+0000 7f61397fa640 1 -- 172.21.15.16:0/2119344918 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f613c1f48f0 con 0x7f613c15f9c0 2023-11-24T06:30:15.215 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:15.215 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3782726360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3782726360},{"type":"v1","addr":"172.21.15.173:6835","nonce":3782726360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1586323927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1586323927},{"type":"v1","addr":"172.21.15.16:6835","nonce":1586323927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1730788036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1730788036},{"type":"v1","addr":"172.21.15.173:6837","nonce":1730788036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:52.667087+0000","modified":"2023-11-24T06:29:55.722711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/515502120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":515502120},{"type":"v1","addr":"172.21.15.16:6833","nonce":515502120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:30:15.216 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2023-11-24T06:30:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.214+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f611c06b230 msgr2=0x7f611c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.214+0000 7f6142768640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f611c06b230 0x7f611c06d6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f612c036000 tx=0x7f612c008030 comp rx=0 tx=0).stop 2023-11-24T06:30:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.214+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15f9c0 msgr2=0x7f613c0a85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.214+0000 7f6142768640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15f9c0 0x7f613c0a85e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f613400c7a0 tx=0x7f613400cc60 comp rx=0 tx=0).stop 2023-11-24T06:30:15.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.215+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 shutdown_connections 2023-11-24T06:30:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.215+0000 7f6142768640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f611c06b230 0x7f611c06d6f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.215+0000 7f6142768640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613c15f9c0 0x7f613c0a85e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.215+0000 7f6142768640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f613c13e360 0x7f613c1f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.215+0000 7f6142768640 1 --2- 172.21.15.16:0/2119344918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f613c13df80 0x7f613c1f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.215+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 >> 172.21.15.16:0/2119344918 conn(0x7f613c06a800 msgr2=0x7f613c070680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.215+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 shutdown_connections 2023-11-24T06:30:15.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.215+0000 7f6142768640 1 -- 172.21.15.16:0/2119344918 wait complete. 2023-11-24T06:30:15.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:15.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.459+0000 7fbd6ec78640 1 Processor -- start 2023-11-24T06:30:15.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.459+0000 7fbd6ec78640 1 -- start start 2023-11-24T06:30:15.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.460+0000 7fbd6ec78640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd6815d3c0 0x7fbd680d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.460+0000 7fbd6ec78640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 0x7fbd680afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.460+0000 7fbd6ec78640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbd680b01b0 0x7fbd680ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.460+0000 7fbd6ec78640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd680ce5d0 con 0x7fbd6815d3c0 2023-11-24T06:30:15.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.460+0000 7fbd6ec78640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd680d12f0 con 0x7fbd680b01b0 2023-11-24T06:30:15.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.460+0000 7fbd6ec78640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680b23c0 con 0x7fbd680d0e90 2023-11-24T06:30:15.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.461+0000 7fbd6dc76640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd6815d3c0 0x7fbd680d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.461+0000 7fbd6dc76640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd6815d3c0 0x7fbd680d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55228/0 (socket says 172.21.15.16:55228) 2023-11-24T06:30:15.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.461+0000 7fbd6dc76640 1 -- 172.21.15.16:0/1018477002 learned_addr learned my addr 172.21.15.16:0/1018477002 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:15.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.461+0000 7fbd6d475640 1 --2- 172.21.15.16:0/1018477002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 0x7fbd680afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.461+0000 7fbd6e477640 1 --2- 172.21.15.16:0/1018477002 >> v2:172.21.15.173:3300/0 conn(0x7fbd680b01b0 0x7fbd680ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.461+0000 7fbd6dc76640 1 -- 172.21.15.16:0/1018477002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 msgr2=0x7fbd680afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:15.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.461+0000 7fbd6dc76640 1 --2- 172.21.15.16:0/1018477002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 0x7fbd680afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.461+0000 7fbd6dc76640 1 -- 172.21.15.16:0/1018477002 >> v2:172.21.15.173:3300/0 conn(0x7fbd680b01b0 msgr2=0x7fbd680ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:15.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.461+0000 7fbd6dc76640 1 --2- 172.21.15.16:0/1018477002 >> v2:172.21.15.173:3300/0 conn(0x7fbd680b01b0 0x7fbd680ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.462+0000 7fbd6dc76640 1 -- 172.21.15.16:0/1018477002 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd680d0800 con 0x7fbd6815d3c0 2023-11-24T06:30:15.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.462+0000 7fbd6e477640 1 --2- 172.21.15.16:0/1018477002 >> v2:172.21.15.173:3300/0 conn(0x7fbd680b01b0 0x7fbd680ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:15.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.462+0000 7fbd6d475640 1 --2- 172.21.15.16:0/1018477002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 0x7fbd680afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.463+0000 7fbd6dc76640 1 --2- 172.21.15.16:0/1018477002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6815d3c0 0x7fbd680d0220 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbd6808d270 tx=0x7fbd5c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e183b4ab158973e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.463+0000 7fbd6cc74640 1 -- 172.21.15.16:0/1018477002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c02c9f0 con 0x7fbd6815d3c0 2023-11-24T06:30:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.463+0000 7fbd6cc74640 1 -- 172.21.15.16:0/1018477002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c02cb50 con 0x7fbd6815d3c0 2023-11-24T06:30:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.463+0000 7fbd6cc74640 1 -- 172.21.15.16:0/1018477002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c0343f0 con 0x7fbd6815d3c0 2023-11-24T06:30:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.464+0000 7fbd6ec78640 1 -- 172.21.15.16:0/1018477002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6815d3c0 msgr2=0x7fbd680d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.464+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/1018477002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6815d3c0 0x7fbd680d0220 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbd6808d270 tx=0x7fbd5c02be50 comp rx=0 tx=0).stop 2023-11-24T06:30:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.464+0000 7fbd6ec78640 1 -- 172.21.15.16:0/1018477002 shutdown_connections 2023-11-24T06:30:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.464+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/1018477002 >> v2:172.21.15.173:3300/0 conn(0x7fbd680b01b0 0x7fbd680ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.464+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/1018477002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 0x7fbd680afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.464+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/1018477002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd6815d3c0 0x7fbd680d0220 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.464+0000 7fbd6ec78640 1 -- 172.21.15.16:0/1018477002 >> 172.21.15.16:0/1018477002 conn(0x7fbd680d30b0 msgr2=0x7fbd680d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.464+0000 7fbd6ec78640 1 -- 172.21.15.16:0/1018477002 shutdown_connections 2023-11-24T06:30:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.464+0000 7fbd6ec78640 1 -- 172.21.15.16:0/1018477002 wait complete. 2023-11-24T06:30:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.466+0000 7fbd6ec78640 1 Processor -- start 2023-11-24T06:30:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.467+0000 7fbd6ec78640 1 -- start start 2023-11-24T06:30:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.468+0000 7fbd6ec78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd680b01b0 0x7fbd6815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.468+0000 7fbd6ec78640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 0x7fbd680b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.468+0000 7fbd6ec78640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd6815d3c0 0x7fbd680cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.468+0000 7fbd6ec78640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680b61f0 con 0x7fbd680b01b0 2023-11-24T06:30:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.468+0000 7fbd6ec78640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680cc490 con 0x7fbd6815d3c0 2023-11-24T06:30:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.468+0000 7fbd6ec78640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6815aad0 con 0x7fbd680d0e90 2023-11-24T06:30:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.468+0000 7fbd6dc76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd680b01b0 0x7fbd6815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.468+0000 7fbd6dc76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd680b01b0 0x7fbd6815acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55240/0 (socket says 172.21.15.16:55240) 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.468+0000 7fbd6dc76640 1 -- 172.21.15.16:0/4198769147 learned_addr learned my addr 172.21.15.16:0/4198769147 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.469+0000 7fbd6dc76640 1 -- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 msgr2=0x7fbd680b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.469+0000 7fbd6dc76640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 0x7fbd680b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.469+0000 7fbd6e477640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd6815d3c0 0x7fbd680cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.469+0000 7fbd6dc76640 1 -- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd6815d3c0 msgr2=0x7fbd680cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.469+0000 7fbd6dc76640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd6815d3c0 0x7fbd680cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.469+0000 7fbd6dc76640 1 -- 172.21.15.16:0/4198769147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c009b30 con 0x7fbd680b01b0 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.469+0000 7fbd6e477640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd6815d3c0 0x7fbd680cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.469+0000 7fbd6dc76640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd680b01b0 0x7fbd6815acd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c036040 tx=0x7fbd5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.470+0000 7fbd667fc640 1 -- 172.21.15.16:0/4198769147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c03e450 con 0x7fbd680b01b0 2023-11-24T06:30:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.470+0000 7fbd667fc640 1 -- 172.21.15.16:0/4198769147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c03e5f0 con 0x7fbd680b01b0 2023-11-24T06:30:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.470+0000 7fbd667fc640 1 -- 172.21.15.16:0/4198769147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c03e9a0 con 0x7fbd680b01b0 2023-11-24T06:30:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.470+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd680cb690 con 0x7fbd680b01b0 2023-11-24T06:30:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.470+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd680cc120 con 0x7fbd680b01b0 2023-11-24T06:30:15.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.471+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd680cbb70 con 0x7fbd680b01b0 2023-11-24T06:30:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.472+0000 7fbd667fc640 1 -- 172.21.15.16:0/4198769147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fbd5c03eb00 con 0x7fbd680b01b0 2023-11-24T06:30:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.473+0000 7fbd667fc640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd4c06b230 0x7fbd4c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.473+0000 7fbd667fc640 1 -- 172.21.15.16:0/4198769147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fbd5c0acd80 con 0x7fbd680b01b0 2023-11-24T06:30:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.474+0000 7fbd6d475640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd4c06b230 0x7fbd4c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.475+0000 7fbd6d475640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd4c06b230 0x7fbd4c06d6f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd54006fd0 tx=0x7fbd54008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:15.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.477+0000 7fbd667fc640 1 -- 172.21.15.16:0/4198769147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd680cbb70 con 0x7fbd680b01b0 2023-11-24T06:30:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.674+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd680cbb70 con 0x7fbd680b01b0 2023-11-24T06:30:15.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.677+0000 7fbd667fc640 1 -- 172.21.15.16:0/4198769147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16673 (secure 0 0 0) 0x7fbd680cbb70 con 0x7fbd680b01b0 2023-11-24T06:30:15.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:15.681 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":60,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:12.048861+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:50.546182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:51.698824+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd4c06b230 msgr2=0x7fbd4c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd4c06b230 0x7fbd4c06d6f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd54006fd0 tx=0x7fbd54008040 comp rx=0 tx=0).stop 2023-11-24T06:30:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd680b01b0 msgr2=0x7fbd6815acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd680b01b0 0x7fbd6815acd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c036040 tx=0x7fbd5c004a40 comp rx=0 tx=0).stop 2023-11-24T06:30:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 shutdown_connections 2023-11-24T06:30:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbd4c06b230 0x7fbd4c06d6f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd6815d3c0 0x7fbd680cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbd680d0e90 0x7fbd680b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 --2- 172.21.15.16:0/4198769147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd680b01b0 0x7fbd6815acd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 >> 172.21.15.16:0/4198769147 conn(0x7fbd680d30b0 msgr2=0x7fbd6806c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 shutdown_connections 2023-11-24T06:30:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.680+0000 7fbd6ec78640 1 -- 172.21.15.16:0/4198769147 wait complete. 2023-11-24T06:30:15.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:29:50.546182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:29:51.698824+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 60, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:30:15.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:30:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.912+0000 7fd088aad640 1 Processor -- start 2023-11-24T06:30:15.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.913+0000 7fd088aad640 1 -- start start 2023-11-24T06:30:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.914+0000 7fd088aad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0840ccb50 0x7fd084070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.914+0000 7fd088aad640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08415f4e0 con 0x7fd0840adf20 2023-11-24T06:30:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.914+0000 7fd088aad640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd084071130 con 0x7fd08415bf10 2023-11-24T06:30:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.914+0000 7fd088aad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08415f970 con 0x7fd0840ccb50 2023-11-24T06:30:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.914+0000 7fd082ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0840ccb50 0x7fd084070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.914+0000 7fd0837fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd0840adf20 0x7fd0840cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51706/0 (socket says 172.21.15.16:51706) 2023-11-24T06:30:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.914+0000 7fd0837fe640 1 -- 172.21.15.16:0/2775195778 learned_addr learned my addr 172.21.15.16:0/2775195778 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.915+0000 7fd082ffd640 1 -- 172.21.15.16:0/2775195778 >> v1:172.21.15.173:6789/0 conn(0x7fd08415bf10 legacy=0x7fd084143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.915+0000 7fd082ffd640 1 -- 172.21.15.16:0/2775195778 >> v1:172.21.15.16:6789/0 conn(0x7fd0840adf20 legacy=0x7fd0840cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.915+0000 7fd082ffd640 1 -- 172.21.15.16:0/2775195778 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08415ebb0 con 0x7fd0840ccb50 2023-11-24T06:30:15.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.915+0000 7fd0827fc640 1 -- 172.21.15.16:0/2775195778 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827968021 0 0) 0x7fd08415f4e0 con 0x7fd0840adf20 2023-11-24T06:30:15.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.916+0000 7fd082ffd640 1 --2- 172.21.15.16:0/2775195778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0840ccb50 0x7fd084070a30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd0780097a0 tx=0x7fd07802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5986c3e1d89592 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:15.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.916+0000 7fd0827fc640 1 -- 172.21.15.16:0/2775195778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07802e070 con 0x7fd0840ccb50 2023-11-24T06:30:15.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.916+0000 7fd0827fc640 1 -- 172.21.15.16:0/2775195778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0780315a0 con 0x7fd0840ccb50 2023-11-24T06:30:15.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.916+0000 7fd0827fc640 1 -- 172.21.15.16:0/2775195778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078031930 con 0x7fd0840ccb50 2023-11-24T06:30:15.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.917+0000 7fd088aad640 1 -- 172.21.15.16:0/2775195778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0840ccb50 msgr2=0x7fd084070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.917+0000 7fd088aad640 1 --2- 172.21.15.16:0/2775195778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0840ccb50 0x7fd084070a30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd0780097a0 tx=0x7fd07802b840 comp rx=0 tx=0).stop 2023-11-24T06:30:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.917+0000 7fd088aad640 1 -- 172.21.15.16:0/2775195778 shutdown_connections 2023-11-24T06:30:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.917+0000 7fd088aad640 1 --2- 172.21.15.16:0/2775195778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0840ccb50 0x7fd084070a30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.917+0000 7fd088aad640 1 -- 172.21.15.16:0/2775195778 >> 172.21.15.16:0/2775195778 conn(0x7fd08406a7d0 msgr2=0x7fd08406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.917+0000 7fd088aad640 1 -- 172.21.15.16:0/2775195778 shutdown_connections 2023-11-24T06:30:15.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.917+0000 7fd088aad640 1 -- 172.21.15.16:0/2775195778 wait complete. 2023-11-24T06:30:15.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.919+0000 7fd088aad640 1 Processor -- start 2023-11-24T06:30:15.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.919+0000 7fd088aad640 1 -- start start 2023-11-24T06:30:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.920+0000 7fd088aad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0840adf20 0x7fd08415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.920+0000 7fd088aad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0840ccb50 0x7fd0841f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.920+0000 7fd088aad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08415bf10 0x7fd0841f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.920+0000 7fd088aad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840b24c0 con 0x7fd08415bf10 2023-11-24T06:30:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.920+0000 7fd088aad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840b1e90 con 0x7fd0840adf20 2023-11-24T06:30:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.920+0000 7fd088aad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08415aeb0 con 0x7fd0840ccb50 2023-11-24T06:30:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.920+0000 7fd083fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08415bf10 0x7fd0841f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.920+0000 7fd083fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08415bf10 0x7fd0841f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55264/0 (socket says 172.21.15.16:55264) 2023-11-24T06:30:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.920+0000 7fd083fff640 1 -- 172.21.15.16:0/697232017 learned_addr learned my addr 172.21.15.16:0/697232017 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd0837fe640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0840adf20 0x7fd08415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd083fff640 1 -- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0840ccb50 msgr2=0x7fd0841f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd083fff640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0840ccb50 0x7fd0841f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd083fff640 1 -- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0840adf20 msgr2=0x7fd08415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd083fff640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0840adf20 0x7fd08415a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd083fff640 1 -- 172.21.15.16:0/697232017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078009400 con 0x7fd08415bf10 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd0837fe640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0840adf20 0x7fd08415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd083fff640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08415bf10 0x7fd0841f6550 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00c780 tx=0x7fd07c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd06bfff640 1 -- 172.21.15.16:0/697232017 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c004b60 con 0x7fd08415bf10 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd06bfff640 1 -- 172.21.15.16:0/697232017 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c00ce90 con 0x7fd08415bf10 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd06bfff640 1 -- 172.21.15.16:0/697232017 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00a850 con 0x7fd08415bf10 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd084071e60 con 0x7fd08415bf10 2023-11-24T06:30:15.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.921+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0840b2150 con 0x7fd08415bf10 2023-11-24T06:30:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.923+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0840b2630 con 0x7fd08415bf10 2023-11-24T06:30:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.923+0000 7fd06bfff640 1 -- 172.21.15.16:0/697232017 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd07c00a9b0 con 0x7fd08415bf10 2023-11-24T06:30:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.924+0000 7fd06bfff640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd07006b260 0x7fd07006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.925+0000 7fd06bfff640 1 -- 172.21.15.16:0/697232017 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd07c089e70 con 0x7fd08415bf10 2023-11-24T06:30:15.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.925+0000 7fd0837fe640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd07006b260 0x7fd07006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:15.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.926+0000 7fd0837fe640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd07006b260 0x7fd07006d720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd074000f60 tx=0x7fd074009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:15.929+0000 7fd06bfff640 1 -- 172.21.15.16:0/697232017 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd07c054950 con 0x7fd08415bf10 2023-11-24T06:30:16.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.125+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd084067450 con 0x7fd08415bf10 2023-11-24T06:30:16.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.125+0000 7fd06bfff640 1 -- 172.21.15.16:0/697232017 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd07c057f70 con 0x7fd08415bf10 2023-11-24T06:30:16.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd07006b260 msgr2=0x7fd07006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd07006b260 0x7fd07006d720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd074000f60 tx=0x7fd074009040 comp rx=0 tx=0).stop 2023-11-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08415bf10 msgr2=0x7fd0841f6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08415bf10 0x7fd0841f6550 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00c780 tx=0x7fd07c00cc40 comp rx=0 tx=0).stop 2023-11-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 shutdown_connections 2023-11-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd07006b260 0x7fd07006d720 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd08415bf10 0x7fd0841f6550 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0840ccb50 0x7fd0841f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 --2- 172.21.15.16:0/697232017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0840adf20 0x7fd08415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 >> 172.21.15.16:0/697232017 conn(0x7fd08406a7d0 msgr2=0x7fd08415b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 shutdown_connections 2023-11-24T06:30:16.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.128+0000 7fd088aad640 1 -- 172.21.15.16:0/697232017 wait complete. 2023-11-24T06:30:16.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:30:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.381+0000 7f6cde34e640 1 Processor -- start 2023-11-24T06:30:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.382+0000 7f6cde34e640 1 -- start start 2023-11-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.383+0000 7f6cde34e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6cd80b2e00 0x7f6cd8070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.383+0000 7f6cde34e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd80706a0 con 0x7f6cd806c730 2023-11-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.383+0000 7f6cde34e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd80708d0 con 0x7f6cd806cb10 2023-11-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.383+0000 7f6cde34e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd80b3290 con 0x7f6cd80b2e00 2023-11-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.384+0000 7f6cdd34c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6cd806c730 0x7f6cd8071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51720/0 (socket says 172.21.15.16:51720) 2023-11-24T06:30:16.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.384+0000 7f6cdd34c640 1 -- 172.21.15.16:0/820335955 learned_addr learned my addr 172.21.15.16:0/820335955 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.384+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263051337 0 0) 0x7f6cd80706a0 con 0x7f6cd806c730 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.384+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc4003520 con 0x7f6cd806c730 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.384+0000 7f6cdcb4b640 1 --2- 172.21.15.16:0/820335955 >> v2:172.21.15.173:3300/0 conn(0x7f6cd80b2e00 0x7f6cd8070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.384+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3693373669 0 0) 0x7f6cc4003520 con 0x7f6cd806c730 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.384+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 >> v2:172.21.15.173:3300/0 conn(0x7f6cd80b2e00 msgr2=0x7f6cd8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 --2- 172.21.15.16:0/820335955 >> v2:172.21.15.173:3300/0 conn(0x7f6cd80b2e00 0x7f6cd8070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 >> v1:172.21.15.173:6789/0 conn(0x7f6cd806cb10 legacy=0x7f6cd806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd8070bb0 con 0x7f6cd806c730 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f6cd4001670 con 0x7f6cd806c730 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 >> v1:172.21.15.16:6789/0 conn(0x7f6cd806c730 legacy=0x7f6cd8071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4003970 0x7f6cc4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc40047c0 0x7f6cc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4005180 0x7f6cc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc400c0a0 con 0x7f6cc4003970 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc400c210 con 0x7f6cc40047c0 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc400c3d0 con 0x7f6cc4005180 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6cdd34c640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4003970 0x7f6cc4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6cddb4d640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4005180 0x7f6cc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.385+0000 7f6cdd34c640 1 -- 172.21.15.16:0/820335955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4005180 msgr2=0x7f6cc400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.386+0000 7f6cdd34c640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4005180 0x7f6cc400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.386+0000 7f6cdd34c640 1 -- 172.21.15.16:0/820335955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc40047c0 msgr2=0x7f6cc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.386+0000 7f6cdd34c640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cc40047c0 0x7f6cc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.386+0000 7f6cdd34c640 1 -- 172.21.15.16:0/820335955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4003520 con 0x7f6cc4003970 2023-11-24T06:30:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.386+0000 7f6cddb4d640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4005180 0x7f6cc400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.386+0000 7f6cddb4d640 1 -- 172.21.15.16:0/820335955 reap_dead start 2023-11-24T06:30:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.387+0000 7f6cdd34c640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4003970 0x7f6cc4003d90 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400c240 tx=0x7f6cc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63907c75b775c8d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.387+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4005ad0 con 0x7f6cc4003970 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.387+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd4005c30 con 0x7f6cc4003970 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.387+0000 7f6ccffff640 1 -- 172.21.15.16:0/820335955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd402caf0 con 0x7f6cc4003970 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.387+0000 7f6cde34e640 1 -- 172.21.15.16:0/820335955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4003970 msgr2=0x7f6cc4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.387+0000 7f6cde34e640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4003970 0x7f6cc4003d90 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400c240 tx=0x7f6cc40057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.388+0000 7f6cde34e640 1 -- 172.21.15.16:0/820335955 shutdown_connections 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.388+0000 7f6cde34e640 1 --2- 172.21.15.16:0/820335955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cc4003970 0x7f6cc4003d90 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.388+0000 7f6cde34e640 1 -- 172.21.15.16:0/820335955 >> 172.21.15.16:0/820335955 conn(0x7f6cd806a7d0 msgr2=0x7f6cd806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.388+0000 7f6cde34e640 1 -- 172.21.15.16:0/820335955 shutdown_connections 2023-11-24T06:30:16.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.388+0000 7f6cde34e640 1 -- 172.21.15.16:0/820335955 wait complete. 2023-11-24T06:30:16.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.389+0000 7f6cde34e640 1 Processor -- start 2023-11-24T06:30:16.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.390+0000 7f6cde34e640 1 -- start start 2023-11-24T06:30:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.390+0000 7f6cde34e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4003970 0x7f6cd81f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.390+0000 7f6cde34e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd80a9c30 0x7f6cd80aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cde34e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cd80aadc0 0x7f6cd80a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cde34e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd81f7470 con 0x7f6cd80a9c30 2023-11-24T06:30:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cde34e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd80ab340 con 0x7f6cd80aadc0 2023-11-24T06:30:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cde34e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd80ab620 con 0x7f6cc4003970 2023-11-24T06:30:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cdcb4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd80a9c30 0x7f6cd80aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cdcb4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd80a9c30 0x7f6cd80aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55282/0 (socket says 172.21.15.16:55282) 2023-11-24T06:30:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cdcb4b640 1 -- 172.21.15.16:0/2259825708 learned_addr learned my addr 172.21.15.16:0/2259825708 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cdd34c640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4003970 0x7f6cd81f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cdcb4b640 1 -- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4003970 msgr2=0x7f6cd81f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cdcb4b640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4003970 0x7f6cd81f6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cdcb4b640 1 -- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cd80aadc0 msgr2=0x7f6cd80a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cdcb4b640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cd80aadc0 0x7f6cd80a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.391+0000 7f6cdcb4b640 1 -- 172.21.15.16:0/2259825708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd80ab900 con 0x7f6cd80a9c30 2023-11-24T06:30:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.392+0000 7f6cdcb4b640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd80a9c30 0x7f6cd80aa880 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800d530 tx=0x7f6cc800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.392+0000 7f6ccdffb640 1 -- 172.21.15.16:0/2259825708 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80125f0 con 0x7f6cd80a9c30 2023-11-24T06:30:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.392+0000 7f6ccdffb640 1 -- 172.21.15.16:0/2259825708 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc8012750 con 0x7f6cd80a9c30 2023-11-24T06:30:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.392+0000 7f6ccdffb640 1 -- 172.21.15.16:0/2259825708 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8012b00 con 0x7f6cd80a9c30 2023-11-24T06:30:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.392+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd80abbe0 con 0x7f6cd80a9c30 2023-11-24T06:30:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.392+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd81610e0 con 0x7f6cd80a9c30 2023-11-24T06:30:16.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.393+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd8067450 con 0x7f6cd80a9c30 2023-11-24T06:30:16.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.394+0000 7f6ccdffb640 1 -- 172.21.15.16:0/2259825708 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6cc8012c60 con 0x7f6cd80a9c30 2023-11-24T06:30:16.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.395+0000 7f6ccdffb640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6cbc06b260 0x7f6cbc06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.396+0000 7f6ccdffb640 1 -- 172.21.15.16:0/2259825708 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6cc808a130 con 0x7f6cd80a9c30 2023-11-24T06:30:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.396+0000 7f6cdd34c640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6cbc06b260 0x7f6cbc06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.397+0000 7f6cdd34c640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6cbc06b260 0x7f6cbc06d720 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4034000 tx=0x7f6cd4009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.400+0000 7f6ccdffb640 1 -- 172.21.15.16:0/2259825708 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6cc8054b80 con 0x7f6cd80a9c30 2023-11-24T06:30:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.590+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6cd80b2240 con 0x7f6cd80a9c30 2023-11-24T06:30:16.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.591+0000 7f6ccdffb640 1 -- 172.21.15.16:0/2259825708 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6cc80581a0 con 0x7f6cd80a9c30 2023-11-24T06:30:16.595 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:30:16.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6cbc06b260 msgr2=0x7f6cbc06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6cbc06b260 0x7f6cbc06d720 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4034000 tx=0x7f6cd4009030 comp rx=0 tx=0).stop 2023-11-24T06:30:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd80a9c30 msgr2=0x7f6cd80aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd80a9c30 0x7f6cd80aa880 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800d530 tx=0x7f6cc800d9f0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 shutdown_connections 2023-11-24T06:30:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6cbc06b260 0x7f6cbc06d720 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cd80aadc0 0x7f6cd80a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd80a9c30 0x7f6cd80aa880 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 --2- 172.21.15.16:0/2259825708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cc4003970 0x7f6cd81f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 >> 172.21.15.16:0/2259825708 conn(0x7f6cd806a7d0 msgr2=0x7f6cd80af850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 shutdown_connections 2023-11-24T06:30:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.594+0000 7f6cde34e640 1 -- 172.21.15.16:0/2259825708 wait complete. 2023-11-24T06:30:16.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.829+0000 7fbb7916f640 1 Processor -- start 2023-11-24T06:30:16.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.829+0000 7fbb7916f640 1 -- start start 2023-11-24T06:30:16.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.831+0000 7fbb7916f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbb740ae5a0 0x7fbb7415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.831+0000 7fbb7916f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb740cf290 0x7fbb740cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.831+0000 7fbb7916f640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb74070c40 con 0x7fbb740b3ae0 2023-11-24T06:30:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.831+0000 7fbb7916f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb74071450 con 0x7fbb740cf290 2023-11-24T06:30:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.831+0000 7fbb7916f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb740cfc60 con 0x7fbb740ae5a0 2023-11-24T06:30:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.832+0000 7fbb7359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb740cf290 0x7fbb740cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.832+0000 7fbb7359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb740cf290 0x7fbb740cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55286/0 (socket says 172.21.15.16:55286) 2023-11-24T06:30:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.832+0000 7fbb7359e640 1 -- 172.21.15.16:0/1545289020 learned_addr learned my addr 172.21.15.16:0/1545289020 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.832+0000 7fbb72d9d640 1 --2- 172.21.15.16:0/1545289020 >> v2:172.21.15.173:3300/0 conn(0x7fbb740ae5a0 0x7fbb7415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.832+0000 7fbb7359e640 1 -- 172.21.15.16:0/1545289020 >> v2:172.21.15.173:3300/0 conn(0x7fbb740ae5a0 msgr2=0x7fbb7415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.832+0000 7fbb7359e640 1 --2- 172.21.15.16:0/1545289020 >> v2:172.21.15.173:3300/0 conn(0x7fbb740ae5a0 0x7fbb7415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.832+0000 7fbb7359e640 1 -- 172.21.15.16:0/1545289020 >> v1:172.21.15.173:6789/0 conn(0x7fbb740b3ae0 legacy=0x7fbb7406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.832+0000 7fbb7359e640 1 -- 172.21.15.16:0/1545289020 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7406c470 con 0x7fbb740cf290 2023-11-24T06:30:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.833+0000 7fbb7359e640 1 --2- 172.21.15.16:0/1545289020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740cf290 0x7fbb740cf720 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbb740cfe20 tx=0x7fbb6c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46e73784d0722aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.834+0000 7fbb7259c640 1 -- 172.21.15.16:0/1545289020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c02cb20 con 0x7fbb740cf290 2023-11-24T06:30:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.834+0000 7fbb7259c640 1 -- 172.21.15.16:0/1545289020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c02cc80 con 0x7fbb740cf290 2023-11-24T06:30:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.834+0000 7fbb7259c640 1 -- 172.21.15.16:0/1545289020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0343f0 con 0x7fbb740cf290 2023-11-24T06:30:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.834+0000 7fbb7916f640 1 -- 172.21.15.16:0/1545289020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740cf290 msgr2=0x7fbb740cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.834+0000 7fbb7916f640 1 --2- 172.21.15.16:0/1545289020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740cf290 0x7fbb740cf720 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbb740cfe20 tx=0x7fbb6c02bf80 comp rx=0 tx=0).stop 2023-11-24T06:30:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.834+0000 7fbb7916f640 1 -- 172.21.15.16:0/1545289020 shutdown_connections 2023-11-24T06:30:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.835+0000 7fbb7916f640 1 --2- 172.21.15.16:0/1545289020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740cf290 0x7fbb740cf720 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.835+0000 7fbb7916f640 1 --2- 172.21.15.16:0/1545289020 >> v2:172.21.15.173:3300/0 conn(0x7fbb740ae5a0 0x7fbb7415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.835+0000 7fbb7916f640 1 -- 172.21.15.16:0/1545289020 >> 172.21.15.16:0/1545289020 conn(0x7fbb7406a800 msgr2=0x7fbb7406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:16.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.835+0000 7fbb7916f640 1 -- 172.21.15.16:0/1545289020 shutdown_connections 2023-11-24T06:30:16.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.835+0000 7fbb7916f640 1 -- 172.21.15.16:0/1545289020 wait complete. 2023-11-24T06:30:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.836+0000 7fbb7916f640 1 Processor -- start 2023-11-24T06:30:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.837+0000 7fbb7916f640 1 -- start start 2023-11-24T06:30:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb7916f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb740ae5a0 0x7fbb7414c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb7916f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740b3ae0 0x7fbb7415c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb7916f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb740cf290 0x7fbb7415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb7916f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb741f2ce0 con 0x7fbb740b3ae0 2023-11-24T06:30:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb7916f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7415fce0 con 0x7fbb740ae5a0 2023-11-24T06:30:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb7916f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7415f0d0 con 0x7fbb740cf290 2023-11-24T06:30:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740b3ae0 0x7fbb7415c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740b3ae0 0x7fbb7415c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55298/0 (socket says 172.21.15.16:55298) 2023-11-24T06:30:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb6a59c640 1 -- 172.21.15.16:0/1137927118 learned_addr learned my addr 172.21.15.16:0/1137927118 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb6a59c640 1 -- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb740cf290 msgr2=0x7fbb7415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb6a59c640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb740cf290 0x7fbb7415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb6a59c640 1 -- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb740ae5a0 msgr2=0x7fbb7414c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb6a59c640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb740ae5a0 0x7fbb7414c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.838+0000 7fbb6a59c640 1 -- 172.21.15.16:0/1137927118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c009b30 con 0x7fbb740b3ae0 2023-11-24T06:30:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.839+0000 7fbb6a59c640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740b3ae0 0x7fbb7415c720 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c430 tx=0x7fbb5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.839+0000 7fbb6bfff640 1 -- 172.21.15.16:0/1137927118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c011790 con 0x7fbb740b3ae0 2023-11-24T06:30:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.839+0000 7fbb6bfff640 1 -- 172.21.15.16:0/1137927118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c0118f0 con 0x7fbb740b3ae0 2023-11-24T06:30:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.839+0000 7fbb6bfff640 1 -- 172.21.15.16:0/1137927118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c011ce0 con 0x7fbb740b3ae0 2023-11-24T06:30:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.839+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7415f3b0 con 0x7fbb740b3ae0 2023-11-24T06:30:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.839+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7415f970 con 0x7fbb740b3ae0 2023-11-24T06:30:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.840+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb740675d0 con 0x7fbb740b3ae0 2023-11-24T06:30:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.841+0000 7fbb6bfff640 1 -- 172.21.15.16:0/1137927118 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fbb5c01c460 con 0x7fbb740b3ae0 2023-11-24T06:30:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.842+0000 7fbb6bfff640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb5406b260 0x7fbb5406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.842+0000 7fbb6bfff640 1 -- 172.21.15.16:0/1137927118 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fbb5c089cd0 con 0x7fbb740b3ae0 2023-11-24T06:30:16.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.843+0000 7fbb72d9d640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb5406b260 0x7fbb5406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:16.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.844+0000 7fbb72d9d640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb5406b260 0x7fbb5406d720 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbb640098f0 tx=0x7fbb64008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:16.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:16.847+0000 7fbb6bfff640 1 -- 172.21.15.16:0/1137927118 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb5c054760 con 0x7fbb740b3ae0 2023-11-24T06:30:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.046+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb7415fe20 con 0x7fbb740b3ae0 2023-11-24T06:30:17.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.047+0000 7fbb6bfff640 1 -- 172.21.15.16:0/1137927118 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb7415fe20 con 0x7fbb740b3ae0 2023-11-24T06:30:17.051 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:30:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.050+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb5406b260 msgr2=0x7fbb5406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.050+0000 7fbb7916f640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb5406b260 0x7fbb5406d720 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbb640098f0 tx=0x7fbb64008040 comp rx=0 tx=0).stop 2023-11-24T06:30:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.050+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740b3ae0 msgr2=0x7fbb7415c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.050+0000 7fbb7916f640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740b3ae0 0x7fbb7415c720 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c430 tx=0x7fbb5c00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.051+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 shutdown_connections 2023-11-24T06:30:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.051+0000 7fbb7916f640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbb5406b260 0x7fbb5406d720 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.051+0000 7fbb7916f640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb740cf290 0x7fbb7415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.051+0000 7fbb7916f640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb740b3ae0 0x7fbb7415c720 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.051+0000 7fbb7916f640 1 --2- 172.21.15.16:0/1137927118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb740ae5a0 0x7fbb7414c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.051+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 >> 172.21.15.16:0/1137927118 conn(0x7fbb7406a800 msgr2=0x7fbb740cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.051+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 shutdown_connections 2023-11-24T06:30:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.051+0000 7fbb7916f640 1 -- 172.21.15.16:0/1137927118 wait complete. 2023-11-24T06:30:17.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:30:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.293+0000 7fb892716640 1 Processor -- start 2023-11-24T06:30:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.293+0000 7fb892716640 1 -- start start 2023-11-24T06:30:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.294+0000 7fb892716640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb88c0cc500 0x7fb88c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.294+0000 7fb892716640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c06d160 0x7fb88c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.295+0000 7fb892716640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c070e50 con 0x7fb88c143c40 2023-11-24T06:30:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.295+0000 7fb892716640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88c0723c0 con 0x7fb88c0cc500 2023-11-24T06:30:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.295+0000 7fb892716640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0cced0 con 0x7fb88c06d160 2023-11-24T06:30:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.295+0000 7fb891714640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb88c143c40 0x7fb88c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51730/0 (socket says 172.21.15.16:51730) 2023-11-24T06:30:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.295+0000 7fb891714640 1 -- 172.21.15.16:0/1821005056 learned_addr learned my addr 172.21.15.16:0/1821005056 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.295+0000 7fb891f15640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c06d160 0x7fb88c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.295+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790691691 0 0) 0x7fb88c070e50 con 0x7fb88c143c40 2023-11-24T06:30:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.295+0000 7fb890f13640 1 --2- 172.21.15.16:0/1821005056 >> v2:172.21.15.173:3300/0 conn(0x7fb88c0cc500 0x7fb88c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.295+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb874003520 con 0x7fb88c143c40 2023-11-24T06:30:17.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2787639336 0 0) 0x7fb874003520 con 0x7fb88c143c40 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c06d160 msgr2=0x7fb88c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c06d160 0x7fb88c06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 >> v2:172.21.15.173:3300/0 conn(0x7fb88c0cc500 msgr2=0x7fb88c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 --2- 172.21.15.16:0/1821005056 >> v2:172.21.15.173:3300/0 conn(0x7fb88c0cc500 0x7fb88c0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c144020 con 0x7fb88c143c40 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fb87c001690 con 0x7fb88c143c40 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 >> v1:172.21.15.16:6789/0 conn(0x7fb88c143c40 legacy=0x7fb88c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb874003970 0x7fb874003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8740047c0 0x7fb874004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c06d160 0x7fb874005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0723c0 con 0x7fb8740047c0 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0cced0 con 0x7fb874003970 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87400bab0 con 0x7fb88c06d160 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb890f13640 1 --2- 172.21.15.16:0/1821005056 >> v2:172.21.15.173:3300/0 conn(0x7fb88c0cc500 0x7fb88c0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.296+0000 7fb890f13640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8740047c0 0x7fb874004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.297+0000 7fb891714640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb874003970 0x7fb874003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.297+0000 7fb891f15640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c06d160 0x7fb874005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.297+0000 7fb890f13640 1 -- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c06d160 msgr2=0x7fb874005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.297+0000 7fb890f13640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c06d160 0x7fb874005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.297+0000 7fb890f13640 1 -- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb874003970 msgr2=0x7fb874003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.297+0000 7fb890f13640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb874003970 0x7fb874003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.297+0000 7fb890f13640 1 -- 172.21.15.16:0/1821005056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb874003520 con 0x7fb8740047c0 2023-11-24T06:30:17.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.298+0000 7fb890f13640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8740047c0 0x7fb874004c40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb880006fd0 tx=0x7fb8740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd0c554a954ef60 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:17.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.298+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880030660 con 0x7fb8740047c0 2023-11-24T06:30:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.298+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8800307c0 con 0x7fb8740047c0 2023-11-24T06:30:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.298+0000 7fb88bfff640 1 -- 172.21.15.16:0/1821005056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880030b30 con 0x7fb8740047c0 2023-11-24T06:30:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.299+0000 7fb892716640 1 -- 172.21.15.16:0/1821005056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8740047c0 msgr2=0x7fb874004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.299+0000 7fb892716640 1 --2- 172.21.15.16:0/1821005056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8740047c0 0x7fb874004c40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb880006fd0 tx=0x7fb8740057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.299+0000 7fb891f15640 1 -- 172.21.15.16:0/1821005056 reap_dead start 2023-11-24T06:30:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.299+0000 7fb892716640 1 -- 172.21.15.16:0/1821005056 shutdown_connections 2023-11-24T06:30:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.299+0000 7fb892716640 1 -- 172.21.15.16:0/1821005056 >> 172.21.15.16:0/1821005056 conn(0x7fb88c06a800 msgr2=0x7fb88c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.299+0000 7fb892716640 1 -- 172.21.15.16:0/1821005056 shutdown_connections 2023-11-24T06:30:17.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.299+0000 7fb892716640 1 -- 172.21.15.16:0/1821005056 wait complete. 2023-11-24T06:30:17.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.301+0000 7fb892716640 1 Processor -- start 2023-11-24T06:30:17.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.301+0000 7fb892716640 1 -- start start 2023-11-24T06:30:17.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb892716640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8740047c0 0x7fb88c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb892716640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c13c390 0x7fb88c157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb892716640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88c13e110 0x7fb88c1f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb892716640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c157d60 con 0x7fb88c13e110 2023-11-24T06:30:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb892716640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c158040 con 0x7fb8740047c0 2023-11-24T06:30:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb892716640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c158320 con 0x7fb88c13c390 2023-11-24T06:30:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb891f15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88c13e110 0x7fb88c1f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb891f15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88c13e110 0x7fb88c1f35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55302/0 (socket says 172.21.15.16:55302) 2023-11-24T06:30:17.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb891f15640 1 -- 172.21.15.16:0/1077202076 learned_addr learned my addr 172.21.15.16:0/1077202076 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:17.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.302+0000 7fb891714640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8740047c0 0x7fb88c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb890f13640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c13c390 0x7fb88c157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb891f15640 1 -- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c13c390 msgr2=0x7fb88c157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb891f15640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c13c390 0x7fb88c157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb891f15640 1 -- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8740047c0 msgr2=0x7fb88c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb891f15640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8740047c0 0x7fb88c158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb891f15640 1 -- 172.21.15.16:0/1077202076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c158600 con 0x7fb88c13e110 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb891f15640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88c13e110 0x7fb88c1f35b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb88c144ca0 tx=0x7fb8840115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb889ffb640 1 -- 172.21.15.16:0/1077202076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400e070 con 0x7fb88c13e110 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb889ffb640 1 -- 172.21.15.16:0/1077202076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884011e10 con 0x7fb88c13e110 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb889ffb640 1 -- 172.21.15.16:0/1077202076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884003990 con 0x7fb88c13e110 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c1588e0 con 0x7fb88c13e110 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.303+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c13e590 con 0x7fb88c13e110 2023-11-24T06:30:17.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.305+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c067450 con 0x7fb88c13e110 2023-11-24T06:30:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.306+0000 7fb889ffb640 1 -- 172.21.15.16:0/1077202076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fb884003af0 con 0x7fb88c13e110 2023-11-24T06:30:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.307+0000 7fb889ffb640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb87806b230 0x7fb87806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.307+0000 7fb889ffb640 1 -- 172.21.15.16:0/1077202076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb88408a620 con 0x7fb88c13e110 2023-11-24T06:30:17.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.307+0000 7fb891714640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb87806b230 0x7fb87806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.308+0000 7fb891714640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb87806b230 0x7fb87806d6f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb87c001050 tx=0x7fb87c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.311+0000 7fb889ffb640 1 -- 172.21.15.16:0/1077202076 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb884054ff0 con 0x7fb88c13e110 2023-11-24T06:30:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.548+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb88c0ceb30 con 0x7fb88c13e110 2023-11-24T06:30:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.549+0000 7fb889ffb640 1 -- 172.21.15.16:0/1077202076 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fb884058610 con 0x7fb88c13e110 2023-11-24T06:30:17.552 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:17.552 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3782726360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3782726360},{"type":"v1","addr":"172.21.15.173:6835","nonce":3782726360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1586323927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1586323927},{"type":"v1","addr":"172.21.15.16:6835","nonce":1586323927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1730788036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1730788036},{"type":"v1","addr":"172.21.15.173:6837","nonce":1730788036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:52.667087+0000","modified":"2023-11-24T06:29:55.722711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/515502120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":515502120},{"type":"v1","addr":"172.21.15.16:6833","nonce":515502120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:30:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb87806b230 msgr2=0x7fb87806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb87806b230 0x7fb87806d6f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb87c001050 tx=0x7fb87c009040 comp rx=0 tx=0).stop 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88c13e110 msgr2=0x7fb88c1f35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88c13e110 0x7fb88c1f35b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb88c144ca0 tx=0x7fb8840115a0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 shutdown_connections 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb87806b230 0x7fb87806d6f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88c13e110 0x7fb88c1f35b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb88c13c390 0x7fb88c157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 --2- 172.21.15.16:0/1077202076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8740047c0 0x7fb88c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 >> 172.21.15.16:0/1077202076 conn(0x7fb88c06a800 msgr2=0x7fb88c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.551+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 shutdown_connections 2023-11-24T06:30:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.552+0000 7fb892716640 1 -- 172.21.15.16:0/1077202076 wait complete. 2023-11-24T06:30:17.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:17.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.798+0000 7f12bd143640 1 Processor -- start 2023-11-24T06:30:17.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.798+0000 7f12bd143640 1 -- start start 2023-11-24T06:30:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.799+0000 7f12bd143640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12b8160920 0x7f12b80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.799+0000 7f12bd143640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b80d0a10 con 0x7f12b80cf9f0 2023-11-24T06:30:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.800+0000 7f12bd143640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b8161e30 con 0x7f12b80cfea0 2023-11-24T06:30:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.800+0000 7f12bd143640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80d1f50 con 0x7f12b8160920 2023-11-24T06:30:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.800+0000 7f12b7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12b8160920 0x7f12b80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.800+0000 7f12b7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12b8160920 0x7f12b80b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55310/0 (socket says 172.21.15.16:55310) 2023-11-24T06:30:17.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.800+0000 7f12b7fff640 1 -- 172.21.15.16:0/1264613786 learned_addr learned my addr 172.21.15.16:0/1264613786 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.800+0000 7f12b7fff640 1 -- 172.21.15.16:0/1264613786 >> v1:172.21.15.173:6789/0 conn(0x7f12b80cfea0 legacy=0x7f12b8161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.800+0000 7f12b7fff640 1 -- 172.21.15.16:0/1264613786 >> v1:172.21.15.16:6789/0 conn(0x7f12b80cf9f0 legacy=0x7f12b8161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.800+0000 7f12b7fff640 1 -- 172.21.15.16:0/1264613786 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80cf080 con 0x7f12b8160920 2023-11-24T06:30:17.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.802+0000 7f12b7fff640 1 --2- 172.21.15.16:0/1264613786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b8160920 0x7f12b80b3f30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009e50 tx=0x7f12a802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4656170562ab8530 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.802+0000 7f12b6ffd640 1 -- 172.21.15.16:0/1264613786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a802cb20 con 0x7f12b8160920 2023-11-24T06:30:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.802+0000 7f12b6ffd640 1 -- 172.21.15.16:0/1264613786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a802cc80 con 0x7f12b8160920 2023-11-24T06:30:17.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.802+0000 7f12b6ffd640 1 -- 172.21.15.16:0/1264613786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80343f0 con 0x7f12b8160920 2023-11-24T06:30:17.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.802+0000 7f12bd143640 1 -- 172.21.15.16:0/1264613786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b8160920 msgr2=0x7f12b80b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.802+0000 7f12bd143640 1 --2- 172.21.15.16:0/1264613786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b8160920 0x7f12b80b3f30 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009e50 tx=0x7f12a802bf80 comp rx=0 tx=0).stop 2023-11-24T06:30:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.803+0000 7f12bd143640 1 -- 172.21.15.16:0/1264613786 shutdown_connections 2023-11-24T06:30:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.803+0000 7f12bd143640 1 --2- 172.21.15.16:0/1264613786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b8160920 0x7f12b80b3f30 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.803+0000 7f12bd143640 1 -- 172.21.15.16:0/1264613786 >> 172.21.15.16:0/1264613786 conn(0x7f12b80d3090 msgr2=0x7f12b80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.803+0000 7f12bd143640 1 -- 172.21.15.16:0/1264613786 shutdown_connections 2023-11-24T06:30:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.803+0000 7f12bd143640 1 -- 172.21.15.16:0/1264613786 wait complete. 2023-11-24T06:30:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.805+0000 7f12bd143640 1 Processor -- start 2023-11-24T06:30:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.805+0000 7f12bd143640 1 -- start start 2023-11-24T06:30:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.806+0000 7f12bd143640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b80cf9f0 0x7f12b8153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.806+0000 7f12bd143640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12b80cfea0 0x7f12b8151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.806+0000 7f12bd143640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12b8160920 0x7f12b80c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.806+0000 7f12bd143640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8152cb0 con 0x7f12b80cf9f0 2023-11-24T06:30:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.806+0000 7f12bd143640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8154470 con 0x7f12b80cfea0 2023-11-24T06:30:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.806+0000 7f12bd143640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8151890 con 0x7f12b8160920 2023-11-24T06:30:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.806+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b80cf9f0 0x7f12b8153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.806+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b80cf9f0 0x7f12b8153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55320/0 (socket says 172.21.15.16:55320) 2023-11-24T06:30:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.806+0000 7f12b7fff640 1 -- 172.21.15.16:0/3006351733 learned_addr learned my addr 172.21.15.16:0/3006351733 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12b7fff640 1 -- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12b8160920 msgr2=0x7f12b80c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12b7fff640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12b8160920 0x7f12b80c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12b7fff640 1 -- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12b80cfea0 msgr2=0x7f12b8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12b7fff640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12b80cfea0 0x7f12b8151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12b7fff640 1 -- 172.21.15.16:0/3006351733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8009b30 con 0x7f12b80cf9f0 2023-11-24T06:30:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12b7fff640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b80cf9f0 0x7f12b8153f30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009e50 tx=0x7f12a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12b4ff9640 1 -- 172.21.15.16:0/3006351733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8009510 con 0x7f12b80cf9f0 2023-11-24T06:30:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12b4ff9640 1 -- 172.21.15.16:0/3006351733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a803e450 con 0x7f12b80cf9f0 2023-11-24T06:30:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12b4ff9640 1 -- 172.21.15.16:0/3006351733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a803e800 con 0x7f12b80cf9f0 2023-11-24T06:30:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.807+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b8152e90 con 0x7f12b80cf9f0 2023-11-24T06:30:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.808+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b8153450 con 0x7f12b80cf9f0 2023-11-24T06:30:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.809+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b8067450 con 0x7f12b80cf9f0 2023-11-24T06:30:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.810+0000 7f12b4ff9640 1 -- 172.21.15.16:0/3006351733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f12a803e960 con 0x7f12b80cf9f0 2023-11-24T06:30:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.811+0000 7f12b4ff9640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f129806b230 0x7f129806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.811+0000 7f12b4ff9640 1 -- 172.21.15.16:0/3006351733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f12a80aca50 con 0x7f12b80cf9f0 2023-11-24T06:30:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.811+0000 7f12b77fe640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f129806b230 0x7f129806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.812+0000 7f12b77fe640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f129806b230 0x7f129806d6f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f12a0000f60 tx=0x7f12a00079d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:17.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:17.815+0000 7f12b4ff9640 1 -- 172.21.15.16:0/3006351733 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12a8077570 con 0x7f12b80cf9f0 2023-11-24T06:30:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.010+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12b808c7e0 con 0x7f12b80cf9f0 2023-11-24T06:30:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.011+0000 7f12b4ff9640 1 -- 172.21.15.16:0/3006351733 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16673 (secure 0 0 0) 0x7f12a807ab90 con 0x7f12b80cf9f0 2023-11-24T06:30:18.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:18.015 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":60,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:12.048861+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:50.546182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:51.698824+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f129806b230 msgr2=0x7f129806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f129806b230 0x7f129806d6f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f12a0000f60 tx=0x7f12a00079d0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b80cf9f0 msgr2=0x7f12b8153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b80cf9f0 0x7f12b8153f30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009e50 tx=0x7f12a8004a40 comp rx=0 tx=0).stop 2023-11-24T06:30:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 shutdown_connections 2023-11-24T06:30:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f129806b230 0x7f129806d6f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12b8160920 0x7f12b80c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12b80cfea0 0x7f12b8151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 --2- 172.21.15.16:0/3006351733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12b80cf9f0 0x7f12b8153f30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.014+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 >> 172.21.15.16:0/3006351733 conn(0x7f12b80d3090 msgr2=0x7f12b806aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:18.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.015+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 shutdown_connections 2023-11-24T06:30:18.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.015+0000 7f12bd143640 1 -- 172.21.15.16:0/3006351733 wait complete. 2023-11-24T06:30:18.037 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:30:18.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:30:18.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.259+0000 7fddb7105640 1 Processor -- start 2023-11-24T06:30:18.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.259+0000 7fddb7105640 1 -- start start 2023-11-24T06:30:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.261+0000 7fddb7105640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb013e360 0x7fddb00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.261+0000 7fddb7105640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fddb015f9c0 0x7fddb00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.261+0000 7fddb7105640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb015c5f0 con 0x7fddb013df80 2023-11-24T06:30:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.261+0000 7fddb7105640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb006c470 con 0x7fddb015f9c0 2023-11-24T06:30:18.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.261+0000 7fddb7105640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb006c750 con 0x7fddb013e360 2023-11-24T06:30:18.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.261+0000 7fddb4ea1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fddb013df80 0x7fddb015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51746/0 (socket says 172.21.15.16:51746) 2023-11-24T06:30:18.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.261+0000 7fddb4ea1640 1 -- 172.21.15.16:0/300887216 learned_addr learned my addr 172.21.15.16:0/300887216 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaffff640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb013e360 0x7fddb00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddb56a2640 1 --2- 172.21.15.16:0/300887216 >> v2:172.21.15.173:3300/0 conn(0x7fddb015f9c0 0x7fddb00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189746490 0 0) 0x7fddb015c5f0 con 0x7fddb013df80 2023-11-24T06:30:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdda0003520 con 0x7fddb013df80 2023-11-24T06:30:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782633676 0 0) 0x7fdda0003520 con 0x7fddb013df80 2023-11-24T06:30:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb013e360 msgr2=0x7fddb00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb013e360 0x7fddb00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 >> v2:172.21.15.173:3300/0 conn(0x7fddb015f9c0 msgr2=0x7fddb00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:18.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 --2- 172.21.15.16:0/300887216 >> v2:172.21.15.173:3300/0 conn(0x7fddb015f9c0 0x7fddb00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb006ca30 con 0x7fddb013df80 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fdda4001670 con 0x7fddb013df80 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 >> v1:172.21.15.16:6789/0 conn(0x7fddb013df80 legacy=0x7fddb015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb013e360 0x7fdda0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0003ef0 0x7fdda00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0004d00 0x7fdda0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb006c470 con 0x7fdda0004d00 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb006c750 con 0x7fdda0003ef0 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.262+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda000b980 con 0x7fddb013e360 2023-11-24T06:30:18.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.263+0000 7fddb56a2640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0004d00 0x7fdda0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.263+0000 7fddaffff640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0003ef0 0x7fdda00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.263+0000 7fddb4ea1640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb013e360 0x7fdda0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.263+0000 7fddb56a2640 1 -- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb013e360 msgr2=0x7fdda0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.263+0000 7fddb56a2640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb013e360 0x7fdda0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.263+0000 7fddb56a2640 1 -- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0003ef0 msgr2=0x7fdda00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.263+0000 7fddb56a2640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdda0003ef0 0x7fdda00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.263+0000 7fddb56a2640 1 -- 172.21.15.16:0/300887216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda0003520 con 0x7fdda0004d00 2023-11-24T06:30:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.264+0000 7fddb56a2640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0004d00 0x7fdda0005180 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009fc0 tx=0x7fdda00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c42437c73b18c4f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda802cc40 con 0x7fdda0004d00 2023-11-24T06:30:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda800f040 con 0x7fdda0004d00 2023-11-24T06:30:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddaf7fe640 1 -- 172.21.15.16:0/300887216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8034440 con 0x7fdda0004d00 2023-11-24T06:30:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddb7105640 1 -- 172.21.15.16:0/300887216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0004d00 msgr2=0x7fdda0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddb7105640 1 --2- 172.21.15.16:0/300887216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdda0004d00 0x7fdda0005180 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009fc0 tx=0x7fdda00057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddb56a2640 1 -- 172.21.15.16:0/300887216 reap_dead start 2023-11-24T06:30:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddb7105640 1 -- 172.21.15.16:0/300887216 shutdown_connections 2023-11-24T06:30:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddb7105640 1 -- 172.21.15.16:0/300887216 >> 172.21.15.16:0/300887216 conn(0x7fddb006a800 msgr2=0x7fddb006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddb7105640 1 -- 172.21.15.16:0/300887216 shutdown_connections 2023-11-24T06:30:18.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.265+0000 7fddb7105640 1 -- 172.21.15.16:0/300887216 wait complete. 2023-11-24T06:30:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.267+0000 7fddb7105640 1 Processor -- start 2023-11-24T06:30:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.267+0000 7fddb7105640 1 -- start start 2023-11-24T06:30:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.268+0000 7fddb7105640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdda0004d00 0x7fddb013a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.268+0000 7fddb7105640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb014a970 0x7fddb00b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.268+0000 7fddb7105640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb014adf0 0x7fddb00b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.268+0000 7fddb7105640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb014b980 con 0x7fddb014a970 2023-11-24T06:30:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.268+0000 7fddb7105640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb01f0ce0 con 0x7fddb014adf0 2023-11-24T06:30:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.268+0000 7fddb7105640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb014b3c0 con 0x7fdda0004d00 2023-11-24T06:30:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb014a970 0x7fddb00b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb014a970 0x7fddb00b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55330/0 (socket says 172.21.15.16:55330) 2023-11-24T06:30:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddaffff640 1 -- 172.21.15.16:0/73961998 learned_addr learned my addr 172.21.15.16:0/73961998 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddb56a2640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb014adf0 0x7fddb00b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddb4ea1640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdda0004d00 0x7fddb013a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddaffff640 1 -- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdda0004d00 msgr2=0x7fddb013a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddaffff640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdda0004d00 0x7fddb013a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddaffff640 1 -- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb014adf0 msgr2=0x7fddb00b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddaffff640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb014adf0 0x7fddb00b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddaffff640 1 -- 172.21.15.16:0/73961998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb014b6a0 con 0x7fddb014a970 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddb56a2640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb014adf0 0x7fddb00b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.269+0000 7fddaffff640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb014a970 0x7fddb00b2cf0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c011540 tx=0x7fdd9c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.270+0000 7fddad7fa640 1 -- 172.21.15.16:0/73961998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c015930 con 0x7fddb014a970 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.270+0000 7fddad7fa640 1 -- 172.21.15.16:0/73961998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9c015a90 con 0x7fddb014a970 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.270+0000 7fddad7fa640 1 -- 172.21.15.16:0/73961998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c015e60 con 0x7fddb014a970 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.270+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb014c360 con 0x7fddb014a970 2023-11-24T06:30:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.270+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb0150fb0 con 0x7fddb014a970 2023-11-24T06:30:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.271+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb00675d0 con 0x7fddb014a970 2023-11-24T06:30:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.272+0000 7fddad7fa640 1 -- 172.21.15.16:0/73961998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fdd9c00f460 con 0x7fddb014a970 2023-11-24T06:30:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.273+0000 7fddad7fa640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd9406f640 0x7fdd94071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.273+0000 7fddad7fa640 1 -- 172.21.15.16:0/73961998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fdd9c08a5e0 con 0x7fddb014a970 2023-11-24T06:30:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.274+0000 7fddb4ea1640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd9406f640 0x7fdd94071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.275+0000 7fddb4ea1640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd9406f640 0x7fdd94071b00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdda4001eb0 tx=0x7fdda4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.278+0000 7fddad7fa640 1 -- 172.21.15.16:0/73961998 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd9c055030 con 0x7fddb014a970 2023-11-24T06:30:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.511+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddb006df00 con 0x7fddb014a970 2023-11-24T06:30:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.513+0000 7fddad7fa640 1 -- 172.21.15.16:0/73961998 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fdd9c058650 con 0x7fddb014a970 2023-11-24T06:30:18.516 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:18.516 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3782726360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3782726360},{"type":"v1","addr":"172.21.15.173:6835","nonce":3782726360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1586323927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1586323927},{"type":"v1","addr":"172.21.15.16:6835","nonce":1586323927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1730788036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1730788036},{"type":"v1","addr":"172.21.15.173:6837","nonce":1730788036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:52.667087+0000","modified":"2023-11-24T06:29:55.722711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/515502120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":515502120},{"type":"v1","addr":"172.21.15.16:6833","nonce":515502120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:30:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2023-11-24T06:30:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd9406f640 msgr2=0x7fdd94071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd9406f640 0x7fdd94071b00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdda4001eb0 tx=0x7fdda4009040 comp rx=0 tx=0).stop 2023-11-24T06:30:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb014a970 msgr2=0x7fddb00b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb014a970 0x7fddb00b2cf0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c011540 tx=0x7fdd9c011a00 comp rx=0 tx=0).stop 2023-11-24T06:30:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 shutdown_connections 2023-11-24T06:30:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd9406f640 0x7fdd94071b00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb014adf0 0x7fddb00b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb014a970 0x7fddb00b2cf0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 --2- 172.21.15.16:0/73961998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdda0004d00 0x7fddb013a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.515+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 >> 172.21.15.16:0/73961998 conn(0x7fddb006a800 msgr2=0x7fddb01538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.516+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 shutdown_connections 2023-11-24T06:30:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.516+0000 7fddb7105640 1 -- 172.21.15.16:0/73961998 wait complete. 2023-11-24T06:30:18.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:30:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.765+0000 7fd59a783640 1 Processor -- start 2023-11-24T06:30:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.765+0000 7fd59a783640 1 -- start start 2023-11-24T06:30:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.767+0000 7fd59a783640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5940cc500 0x7fd5940cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.767+0000 7fd59a783640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd59406d160 0x7fd59406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.767+0000 7fd59a783640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd594070e50 con 0x7fd594143c40 2023-11-24T06:30:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.767+0000 7fd59a783640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5940723c0 con 0x7fd59406d160 2023-11-24T06:30:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.767+0000 7fd59a783640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd594144020 con 0x7fd5940cc500 2023-11-24T06:30:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.767+0000 7fd599f82640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd59406d160 0x7fd59406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.767+0000 7fd599f82640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd59406d160 0x7fd59406d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55334/0 (socket says 172.21.15.16:55334) 2023-11-24T06:30:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.767+0000 7fd599f82640 1 -- 172.21.15.16:0/2551191308 learned_addr learned my addr 172.21.15.16:0/2551191308 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.767+0000 7fd598f80640 1 --2- 172.21.15.16:0/2551191308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5940cc500 0x7fd5940cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.768+0000 7fd599f82640 1 -- 172.21.15.16:0/2551191308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5940cc500 msgr2=0x7fd5940cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:18.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.768+0000 7fd599f82640 1 --2- 172.21.15.16:0/2551191308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5940cc500 0x7fd5940cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.768+0000 7fd599f82640 1 -- 172.21.15.16:0/2551191308 >> v1:172.21.15.173:6789/0 conn(0x7fd594143c40 legacy=0x7fd594071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.768+0000 7fd599f82640 1 -- 172.21.15.16:0/2551191308 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5940ccea0 con 0x7fd59406d160 2023-11-24T06:30:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.769+0000 7fd599f82640 1 --2- 172.21.15.16:0/2551191308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 0x7fd59406d590 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009870 tx=0x7fd58c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668f1214db9d68de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.769+0000 7fd58bfff640 1 -- 172.21.15.16:0/2551191308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c031640 con 0x7fd59406d160 2023-11-24T06:30:18.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.769+0000 7fd58bfff640 1 -- 172.21.15.16:0/2551191308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c0317a0 con 0x7fd59406d160 2023-11-24T06:30:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.769+0000 7fd58bfff640 1 -- 172.21.15.16:0/2551191308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c031b30 con 0x7fd59406d160 2023-11-24T06:30:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.770+0000 7fd59a783640 1 -- 172.21.15.16:0/2551191308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 msgr2=0x7fd59406d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.770+0000 7fd59a783640 1 --2- 172.21.15.16:0/2551191308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 0x7fd59406d590 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009870 tx=0x7fd58c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.770+0000 7fd59a783640 1 -- 172.21.15.16:0/2551191308 shutdown_connections 2023-11-24T06:30:18.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.770+0000 7fd59a783640 1 --2- 172.21.15.16:0/2551191308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 0x7fd59406d590 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.770+0000 7fd59a783640 1 --2- 172.21.15.16:0/2551191308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5940cc500 0x7fd5940cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.770+0000 7fd59a783640 1 -- 172.21.15.16:0/2551191308 >> 172.21.15.16:0/2551191308 conn(0x7fd59406a800 msgr2=0x7fd59406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.770+0000 7fd59a783640 1 -- 172.21.15.16:0/2551191308 shutdown_connections 2023-11-24T06:30:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.770+0000 7fd59a783640 1 -- 172.21.15.16:0/2551191308 wait complete. 2023-11-24T06:30:18.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.772+0000 7fd59a783640 1 Processor -- start 2023-11-24T06:30:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.772+0000 7fd59a783640 1 -- start start 2023-11-24T06:30:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd59a783640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 0x7fd5940b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd59a783640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5940cc500 0x7fd5940b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd59a783640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd594143c40 0x7fd594138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd59a783640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd594139090 con 0x7fd59406d160 2023-11-24T06:30:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd59a783640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940b25a0 con 0x7fd5940cc500 2023-11-24T06:30:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd59a783640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940b2880 con 0x7fd594143c40 2023-11-24T06:30:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd599781640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 0x7fd5940b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd599781640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 0x7fd5940b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55348/0 (socket says 172.21.15.16:55348) 2023-11-24T06:30:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd599781640 1 -- 172.21.15.16:0/4189417857 learned_addr learned my addr 172.21.15.16:0/4189417857 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:18.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd599781640 1 -- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd594143c40 msgr2=0x7fd594138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd599781640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd594143c40 0x7fd594138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd599781640 1 -- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5940cc500 msgr2=0x7fd5940b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd599781640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5940cc500 0x7fd5940b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.773+0000 7fd599781640 1 -- 172.21.15.16:0/4189417857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c0094d0 con 0x7fd59406d160 2023-11-24T06:30:18.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.774+0000 7fd599781640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 0x7fd5940b3640 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c730 tx=0x7fd59000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.774+0000 7fd589ffb640 1 -- 172.21.15.16:0/4189417857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590003b70 con 0x7fd59406d160 2023-11-24T06:30:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.774+0000 7fd589ffb640 1 -- 172.21.15.16:0/4189417857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590003cf0 con 0x7fd59406d160 2023-11-24T06:30:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.774+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5940b2b60 con 0x7fd59406d160 2023-11-24T06:30:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.774+0000 7fd589ffb640 1 -- 172.21.15.16:0/4189417857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590003ea0 con 0x7fd59406d160 2023-11-24T06:30:18.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.774+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5940b3120 con 0x7fd59406d160 2023-11-24T06:30:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.775+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd594067450 con 0x7fd59406d160 2023-11-24T06:30:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.776+0000 7fd589ffb640 1 -- 172.21.15.16:0/4189417857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd59000aa90 con 0x7fd59406d160 2023-11-24T06:30:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.777+0000 7fd589ffb640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd57806b260 0x7fd57806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:18.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.777+0000 7fd589ffb640 1 -- 172.21.15.16:0/4189417857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd590089bc0 con 0x7fd59406d160 2023-11-24T06:30:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.778+0000 7fd598f80640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd57806b260 0x7fd57806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.779+0000 7fd598f80640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd57806b260 0x7fd57806d720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd5940713b0 tx=0x7fd584008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:18.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:18.782+0000 7fd589ffb640 1 -- 172.21.15.16:0/4189417857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd590054610 con 0x7fd59406d160 2023-11-24T06:30:19.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.005+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5940ceb30 con 0x7fd59406d160 2023-11-24T06:30:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.006+0000 7fd589ffb640 1 -- 172.21.15.16:0/4189417857 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fd590057c30 con 0x7fd59406d160 2023-11-24T06:30:19.010 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:19.010 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3782726360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3782726360},{"type":"v1","addr":"172.21.15.173:6835","nonce":3782726360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1586323927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1586323927},{"type":"v1","addr":"172.21.15.16:6835","nonce":1586323927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1730788036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1730788036},{"type":"v1","addr":"172.21.15.173:6837","nonce":1730788036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:29:52.667087+0000","modified":"2023-11-24T06:29:55.722711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6201},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6201":{"gid":6201,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6833/515502120","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":515502120},{"type":"v1","addr":"172.21.15.16:6833","nonce":515502120}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:30:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2023-11-24T06:30:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd57806b260 msgr2=0x7fd57806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd57806b260 0x7fd57806d720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd5940713b0 tx=0x7fd584008040 comp rx=0 tx=0).stop 2023-11-24T06:30:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 msgr2=0x7fd5940b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 0x7fd5940b3640 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c730 tx=0x7fd59000cbf0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 shutdown_connections 2023-11-24T06:30:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd57806b260 0x7fd57806d720 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd594143c40 0x7fd594138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5940cc500 0x7fd5940b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 --2- 172.21.15.16:0/4189417857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd59406d160 0x7fd5940b3640 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 >> 172.21.15.16:0/4189417857 conn(0x7fd59406a800 msgr2=0x7fd5940af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 shutdown_connections 2023-11-24T06:30:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.009+0000 7fd59a783640 1 -- 172.21.15.16:0/4189417857 wait complete. 2023-11-24T06:30:19.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.252+0000 7ff8af59e640 1 Processor -- start 2023-11-24T06:30:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.253+0000 7ff8af59e640 1 -- start start 2023-11-24T06:30:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.254+0000 7ff8af59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff8b0160920 0x7ff8b00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.254+0000 7ff8af59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 0x7ff8b00cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.254+0000 7ff8af59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b00d0a10 con 0x7ff8b0161010 2023-11-24T06:30:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.254+0000 7ff8af59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b00d03b0 con 0x7ff8b0160920 2023-11-24T06:30:19.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.254+0000 7ff8af59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b00d1f50 con 0x7ff8b00cf9f0 2023-11-24T06:30:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.255+0000 7ff8add9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 0x7ff8b00cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.255+0000 7ff8aed9d640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7ff8b0161010 0x7ff8b01614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:35762/0 (socket says 172.21.15.16:35762) 2023-11-24T06:30:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.255+0000 7ff8aed9d640 1 -- 172.21.15.16:0/3445707566 learned_addr learned my addr 172.21.15.16:0/3445707566 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.255+0000 7ff8ae59c640 1 --2- 172.21.15.16:0/3445707566 >> v2:172.21.15.173:3300/0 conn(0x7ff8b0160920 0x7ff8b00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.256+0000 7ff8ad59a640 1 -- 172.21.15.16:0/3445707566 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500033333 0 0) 0x7ff8b00d0a10 con 0x7ff8b0161010 2023-11-24T06:30:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.256+0000 7ff8ad59a640 1 -- 172.21.15.16:0/3445707566 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff89c003520 con 0x7ff8b0161010 2023-11-24T06:30:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.256+0000 7ff8ae59c640 1 -- 172.21.15.16:0/3445707566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 msgr2=0x7ff8b00cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.256+0000 7ff8ae59c640 1 --2- 172.21.15.16:0/3445707566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 0x7ff8b00cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.256+0000 7ff8ae59c640 1 -- 172.21.15.16:0/3445707566 >> v1:172.21.15.173:6789/0 conn(0x7ff8b0161010 legacy=0x7ff8b01614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.256+0000 7ff8ae59c640 1 -- 172.21.15.16:0/3445707566 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b00cf080 con 0x7ff8b0160920 2023-11-24T06:30:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.257+0000 7ff8ae59c640 1 --2- 172.21.15.16:0/3445707566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0160920 0x7ff8b00b3f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00097a0 tx=0x7ff8a002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58885e83dfca3f91 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.258+0000 7ff8ad59a640 1 -- 172.21.15.16:0/3445707566 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a002d040 con 0x7ff8b0160920 2023-11-24T06:30:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.258+0000 7ff8ad59a640 1 -- 172.21.15.16:0/3445707566 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a0031810 con 0x7ff8b0160920 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.258+0000 7ff8ad59a640 1 -- 172.21.15.16:0/3445707566 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0031b80 con 0x7ff8b0160920 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.258+0000 7ff8af59e640 1 -- 172.21.15.16:0/3445707566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0160920 msgr2=0x7ff8b00b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.258+0000 7ff8af59e640 1 --2- 172.21.15.16:0/3445707566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0160920 0x7ff8b00b3f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00097a0 tx=0x7ff8a002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.259+0000 7ff8af59e640 1 -- 172.21.15.16:0/3445707566 shutdown_connections 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.259+0000 7ff8af59e640 1 --2- 172.21.15.16:0/3445707566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 0x7ff8b00cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.259+0000 7ff8af59e640 1 --2- 172.21.15.16:0/3445707566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0160920 0x7ff8b00b3f30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.259+0000 7ff8af59e640 1 -- 172.21.15.16:0/3445707566 >> 172.21.15.16:0/3445707566 conn(0x7ff8b00d3090 msgr2=0x7ff8b00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.259+0000 7ff8af59e640 1 -- 172.21.15.16:0/3445707566 shutdown_connections 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.259+0000 7ff8af59e640 1 -- 172.21.15.16:0/3445707566 wait complete. 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.260+0000 7ff8af59e640 1 Processor -- start 2023-11-24T06:30:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.261+0000 7ff8af59e640 1 -- start start 2023-11-24T06:30:19.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.261+0000 7ff8af59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 0x7ff8b00ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8af59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8b0160920 0x7ff8b01f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8af59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0161010 0x7ff8b00b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8af59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b00cd0d0 con 0x7ff8b0160920 2023-11-24T06:30:19.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8af59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b01f34f0 con 0x7ff8b0161010 2023-11-24T06:30:19.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8af59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b00b3940 con 0x7ff8b00cf9f0 2023-11-24T06:30:19.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8b0160920 0x7ff8b01f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8add9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8b0160920 0x7ff8b01f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55350/0 (socket says 172.21.15.16:55350) 2023-11-24T06:30:19.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8add9b640 1 -- 172.21.15.16:0/2857179230 learned_addr learned my addr 172.21.15.16:0/2857179230 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8ae59c640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 0x7ff8b00ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8add9b640 1 -- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 msgr2=0x7ff8b00ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8add9b640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 0x7ff8b00ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8add9b640 1 -- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0161010 msgr2=0x7ff8b00b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8add9b640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0161010 0x7ff8b00b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8add9b640 1 -- 172.21.15.16:0/2857179230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a0009400 con 0x7ff8b0160920 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.262+0000 7ff8ae59c640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 0x7ff8b00ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.263+0000 7ff8add9b640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8b0160920 0x7ff8b01f2fb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e9b0 tx=0x7ff89800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.263+0000 7ff8a6ffd640 1 -- 172.21.15.16:0/2857179230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898009670 con 0x7ff8b0160920 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.263+0000 7ff8a6ffd640 1 -- 172.21.15.16:0/2857179230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89800c9a0 con 0x7ff8b0160920 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.263+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b00ca7f0 con 0x7ff8b0160920 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.263+0000 7ff8a6ffd640 1 -- 172.21.15.16:0/2857179230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800cd50 con 0x7ff8b0160920 2023-11-24T06:30:19.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.263+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b00cadb0 con 0x7ff8b0160920 2023-11-24T06:30:19.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.265+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b0067450 con 0x7ff8b0160920 2023-11-24T06:30:19.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.265+0000 7ff8a6ffd640 1 -- 172.21.15.16:0/2857179230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff898013470 con 0x7ff8b0160920 2023-11-24T06:30:19.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.266+0000 7ff8a6ffd640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff89006b230 0x7ff89006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.266+0000 7ff8a6ffd640 1 -- 172.21.15.16:0/2857179230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff898089f60 con 0x7ff8b0160920 2023-11-24T06:30:19.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.267+0000 7ff8ae59c640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff89006b230 0x7ff89006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.268+0000 7ff8ae59c640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff89006b230 0x7ff89006d6f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0036000 tx=0x7ff8a0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:19.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.271+0000 7ff8a6ffd640 1 -- 172.21.15.16:0/2857179230 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff898054a80 con 0x7ff8b0160920 2023-11-24T06:30:19.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.458+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8b00d3a80 con 0x7ff8b0160920 2023-11-24T06:30:19.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.461+0000 7ff8a6ffd640 1 -- 172.21.15.16:0/2857179230 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16673 (secure 0 0 0) 0x7ff898017090 con 0x7ff8b0160920 2023-11-24T06:30:19.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:19.465 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":60,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:12.048861+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:29:50.546182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:29:51.698824+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":60,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff89006b230 msgr2=0x7ff89006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff89006b230 0x7ff89006d6f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0036000 tx=0x7ff8a0008030 comp rx=0 tx=0).stop 2023-11-24T06:30:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8b0160920 msgr2=0x7ff8b01f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8b0160920 0x7ff8b01f2fb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e9b0 tx=0x7ff89800ee70 comp rx=0 tx=0).stop 2023-11-24T06:30:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 shutdown_connections 2023-11-24T06:30:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff89006b230 0x7ff89006d6f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8b0161010 0x7ff8b00b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8b0160920 0x7ff8b01f2fb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 --2- 172.21.15.16:0/2857179230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8b00cf9f0 0x7ff8b00ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 >> 172.21.15.16:0/2857179230 conn(0x7ff8b00d3090 msgr2=0x7ff8b006aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 shutdown_connections 2023-11-24T06:30:19.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.464+0000 7ff8af59e640 1 -- 172.21.15.16:0/2857179230 wait complete. 2023-11-24T06:30:19.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:30:19.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.703+0000 7febf6de5640 1 Processor -- start 2023-11-24T06:30:19.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.704+0000 7febf6de5640 1 -- start start 2023-11-24T06:30:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.705+0000 7febf6de5640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7febf00ae5a0 0x7febf015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.705+0000 7febf6de5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf0070c40 con 0x7febf00b3ae0 2023-11-24T06:30:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.705+0000 7febf6de5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf006c470 con 0x7febf00cf290 2023-11-24T06:30:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.705+0000 7febf6de5640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf0071450 con 0x7febf00ae5a0 2023-11-24T06:30:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.706+0000 7febeffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7febf00b3ae0 0x7febf006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51762/0 (socket says 172.21.15.16:51762) 2023-11-24T06:30:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.706+0000 7febeffff640 1 -- 172.21.15.16:0/1071501834 learned_addr learned my addr 172.21.15.16:0/1071501834 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:19.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.706+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237741371 0 0) 0x7febf0070c40 con 0x7febf00b3ae0 2023-11-24T06:30:19.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.706+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febe0003520 con 0x7febf00b3ae0 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.706+0000 7febf4b81640 1 --2- 172.21.15.16:0/1071501834 >> v2:172.21.15.173:3300/0 conn(0x7febf00ae5a0 0x7febf015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 132896855 0 0) 0x7febe0003520 con 0x7febf00b3ae0 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 >> v2:172.21.15.173:3300/0 conn(0x7febf00ae5a0 msgr2=0x7febf015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 --2- 172.21.15.16:0/1071501834 >> v2:172.21.15.173:3300/0 conn(0x7febf00ae5a0 0x7febf015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 >> v1:172.21.15.173:6789/0 conn(0x7febf00cf290 legacy=0x7febf00b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf006c750 con 0x7febf00b3ae0 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7febdc001670 con 0x7febf00b3ae0 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 >> v1:172.21.15.16:6789/0 conn(0x7febf00b3ae0 legacy=0x7febf006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00039c0 0x7febe0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe00047c0 0x7febe0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe0005180 0x7febe000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf0071450 con 0x7febe00047c0 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe000c060 con 0x7febe0005180 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe000c220 con 0x7febe00039c0 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.707+0000 7febeffff640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe00047c0 0x7febe0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.708+0000 7febf5382640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe0005180 0x7febe000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.708+0000 7febf4b81640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00039c0 0x7febe0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.708+0000 7febeffff640 1 -- 172.21.15.16:0/1071501834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00039c0 msgr2=0x7febe0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.708+0000 7febeffff640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00039c0 0x7febe0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.708+0000 7febeffff640 1 -- 172.21.15.16:0/1071501834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe0005180 msgr2=0x7febe000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.708+0000 7febeffff640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe0005180 0x7febe000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.708+0000 7febeffff640 1 -- 172.21.15.16:0/1071501834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe0003520 con 0x7febe00047c0 2023-11-24T06:30:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.708+0000 7febf5382640 1 -- 172.21.15.16:0/1071501834 reap_dead start 2023-11-24T06:30:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.708+0000 7febf4b81640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00039c0 0x7febe0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.709+0000 7febeffff640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe00047c0 0x7febe0004c40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7febdc007fc0 tx=0x7febe00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d537caecaefb5ab5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.709+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc005ad0 con 0x7febe00047c0 2023-11-24T06:30:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.709+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc005c30 con 0x7febe00047c0 2023-11-24T06:30:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.709+0000 7febef7fe640 1 -- 172.21.15.16:0/1071501834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc02ca00 con 0x7febe00047c0 2023-11-24T06:30:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.710+0000 7febf6de5640 1 -- 172.21.15.16:0/1071501834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe00047c0 msgr2=0x7febe0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.710+0000 7febf6de5640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe00047c0 0x7febe0004c40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7febdc007fc0 tx=0x7febe00057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.710+0000 7febf6de5640 1 -- 172.21.15.16:0/1071501834 shutdown_connections 2023-11-24T06:30:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.710+0000 7febf6de5640 1 --2- 172.21.15.16:0/1071501834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe00047c0 0x7febe0004c40 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.710+0000 7febf6de5640 1 -- 172.21.15.16:0/1071501834 >> 172.21.15.16:0/1071501834 conn(0x7febf006a800 msgr2=0x7febf006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.710+0000 7febf6de5640 1 -- 172.21.15.16:0/1071501834 shutdown_connections 2023-11-24T06:30:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.710+0000 7febf6de5640 1 -- 172.21.15.16:0/1071501834 wait complete. 2023-11-24T06:30:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.713+0000 7febf6de5640 1 Processor -- start 2023-11-24T06:30:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.713+0000 7febf6de5640 1 -- start start 2023-11-24T06:30:19.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.714+0000 7febf6de5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00047c0 0x7febf014c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.714+0000 7febf6de5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf01f6fb0 0x7febf0150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.714+0000 7febf6de5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0150510 0x7febf00b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.714+0000 7febf6de5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf01f6b90 con 0x7febf0150510 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.714+0000 7febf6de5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf0151a50 con 0x7febf01f6fb0 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.714+0000 7febf6de5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf01f62c0 con 0x7febe00047c0 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febf5382640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0150510 0x7febf00b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febf5382640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0150510 0x7febf00b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55364/0 (socket says 172.21.15.16:55364) 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febf5382640 1 -- 172.21.15.16:0/2149477308 learned_addr learned my addr 172.21.15.16:0/2149477308 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febf4b81640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00047c0 0x7febf014c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febeffff640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf01f6fb0 0x7febf0150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febf5382640 1 -- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00047c0 msgr2=0x7febf014c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febf5382640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00047c0 0x7febf014c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febf5382640 1 -- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf01f6fb0 msgr2=0x7febf0150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febf5382640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf01f6fb0 0x7febf0150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.715+0000 7febf5382640 1 -- 172.21.15.16:0/2149477308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf014c5a0 con 0x7febf0150510 2023-11-24T06:30:19.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.716+0000 7febf5382640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0150510 0x7febf00b26f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7febe8007fd0 tx=0x7febe800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.716+0000 7febed7fa640 1 -- 172.21.15.16:0/2149477308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8006b50 con 0x7febf0150510 2023-11-24T06:30:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.716+0000 7febed7fa640 1 -- 172.21.15.16:0/2149477308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe8006cd0 con 0x7febf0150510 2023-11-24T06:30:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.716+0000 7febed7fa640 1 -- 172.21.15.16:0/2149477308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8003bb0 con 0x7febf0150510 2023-11-24T06:30:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.716+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf0151770 con 0x7febf0150510 2023-11-24T06:30:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.716+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf01f67b0 con 0x7febf0150510 2023-11-24T06:30:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.717+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf01f6cd0 con 0x7febf0150510 2023-11-24T06:30:19.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.718+0000 7febed7fa640 1 -- 172.21.15.16:0/2149477308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7febe8003d10 con 0x7febf0150510 2023-11-24T06:30:19.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.719+0000 7febed7fa640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd406b2b0 0x7febd406d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:19.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.719+0000 7febed7fa640 1 -- 172.21.15.16:0/2149477308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5765+0+0 (secure 0 0 0) 0x7febe80895d0 con 0x7febf0150510 2023-11-24T06:30:19.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.720+0000 7febf4b81640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd406b2b0 0x7febd406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:19.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.721+0000 7febf4b81640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd406b2b0 0x7febd406d770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7febf00695b0 tx=0x7febe4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:19.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.723+0000 7febed7fa640 1 -- 172.21.15.16:0/2149477308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febf01f6cd0 con 0x7febf0150510 2023-11-24T06:30:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.960+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7febf01f6cd0 con 0x7febf0150510 2023-11-24T06:30:19.964 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-24T06:30:19.961+0000 7f082ae01640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.964+0000 7febed7fa640 1 -- 172.21.15.16:0/2149477308 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7febf01f6cd0 con 0x7febf0150510 2023-11-24T06:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:30:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.967+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd406b2b0 msgr2=0x7febd406d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.967+0000 7febf6de5640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd406b2b0 0x7febd406d770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7febf00695b0 tx=0x7febe4008040 comp rx=0 tx=0).stop 2023-11-24T06:30:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.967+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0150510 msgr2=0x7febf00b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.967+0000 7febf6de5640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0150510 0x7febf00b26f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7febe8007fd0 tx=0x7febe800ec10 comp rx=0 tx=0).stop 2023-11-24T06:30:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.968+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 shutdown_connections 2023-11-24T06:30:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.968+0000 7febf6de5640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd406b2b0 0x7febd406d770 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.968+0000 7febf6de5640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf0150510 0x7febf00b26f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.968+0000 7febf6de5640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febf01f6fb0 0x7febf0150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.968+0000 7febf6de5640 1 --2- 172.21.15.16:0/2149477308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febe00047c0 0x7febf014c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.968+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 >> 172.21.15.16:0/2149477308 conn(0x7febf006a800 msgr2=0x7febf00b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.968+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 shutdown_connections 2023-11-24T06:30:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:19.968+0000 7febf6de5640 1 -- 172.21.15.16:0/2149477308 wait complete. 2023-11-24T06:30:19.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:30:20.046 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:30:20.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.191+0000 7f9173901640 1 Processor -- start 2023-11-24T06:30:20.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.191+0000 7f9173901640 1 -- start start 2023-11-24T06:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f9173901640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f916c143ae0 0x7f916c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f9173901640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c0cf7d0 0x7f916c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f9173901640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0cfc30 con 0x7f916c15e1e0 2023-11-24T06:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f9173901640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0b1ee0 con 0x7f916c143ae0 2023-11-24T06:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f9173901640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b2780 con 0x7f916c0cf7d0 2023-11-24T06:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f917169d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f916c143ae0 0x7f916c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f9171e9e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f916c15e1e0 0x7f916c15a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51770/0 (socket says 172.21.15.16:51770) 2023-11-24T06:30:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f9171e9e640 1 -- 172.21.15.16:0/877402391 learned_addr learned my addr 172.21.15.16:0/877402391 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f9170e9c640 1 --2- 172.21.15.16:0/877402391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c0cf7d0 0x7f916c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f917169d640 1 -- 172.21.15.16:0/877402391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c0cf7d0 msgr2=0x7f916c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f917169d640 1 --2- 172.21.15.16:0/877402391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c0cf7d0 0x7f916c15dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f917169d640 1 -- 172.21.15.16:0/877402391 >> v1:172.21.15.16:6789/0 conn(0x7f916c15e1e0 legacy=0x7f916c15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f917169d640 1 -- 172.21.15.16:0/877402391 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c15e990 con 0x7f916c143ae0 2023-11-24T06:30:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.193+0000 7f9163fff640 1 -- 172.21.15.16:0/877402391 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55599072 0 0) 0x7f916c0cfc30 con 0x7f916c15e1e0 2023-11-24T06:30:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.195+0000 7f917169d640 1 --2- 172.21.15.16:0/877402391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c143ae0 0x7f916c0cf290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f91680097a0 tx=0x7f916802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ab46582a851814 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.195+0000 7f9163fff640 1 -- 172.21.15.16:0/877402391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916802d040 con 0x7f916c143ae0 2023-11-24T06:30:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.195+0000 7f9163fff640 1 -- 172.21.15.16:0/877402391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9168031810 con 0x7f916c143ae0 2023-11-24T06:30:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.195+0000 7f9163fff640 1 -- 172.21.15.16:0/877402391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168031ba0 con 0x7f916c143ae0 2023-11-24T06:30:20.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.196+0000 7f9173901640 1 -- 172.21.15.16:0/877402391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c143ae0 msgr2=0x7f916c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.196+0000 7f9173901640 1 --2- 172.21.15.16:0/877402391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c143ae0 0x7f916c0cf290 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f91680097a0 tx=0x7f916802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.196+0000 7f9173901640 1 -- 172.21.15.16:0/877402391 shutdown_connections 2023-11-24T06:30:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.196+0000 7f9173901640 1 --2- 172.21.15.16:0/877402391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c0cf7d0 0x7f916c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.196+0000 7f9173901640 1 --2- 172.21.15.16:0/877402391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c143ae0 0x7f916c0cf290 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.196+0000 7f9173901640 1 -- 172.21.15.16:0/877402391 >> 172.21.15.16:0/877402391 conn(0x7f916c06a800 msgr2=0x7f916c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.196+0000 7f9173901640 1 -- 172.21.15.16:0/877402391 shutdown_connections 2023-11-24T06:30:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.196+0000 7f9173901640 1 -- 172.21.15.16:0/877402391 wait complete. 2023-11-24T06:30:20.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.197+0000 7f9173901640 1 Processor -- start 2023-11-24T06:30:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.198+0000 7f9173901640 1 -- start start 2023-11-24T06:30:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9173901640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f916c0cf7d0 0x7f916c0ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9173901640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c143ae0 0x7f916c135440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9173901640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c15e1e0 0x7f916c159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9173901640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0ccf40 con 0x7f916c15e1e0 2023-11-24T06:30:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9173901640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c41b0 con 0x7f916c0cf7d0 2023-11-24T06:30:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9173901640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0cc610 con 0x7f916c143ae0 2023-11-24T06:30:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9171e9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c15e1e0 0x7f916c159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9171e9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c15e1e0 0x7f916c159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55392/0 (socket says 172.21.15.16:55392) 2023-11-24T06:30:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9171e9e640 1 -- 172.21.15.16:0/1268394478 learned_addr learned my addr 172.21.15.16:0/1268394478 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9170e9c640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c143ae0 0x7f916c135440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f917169d640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f916c0cf7d0 0x7f916c0ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9171e9e640 1 -- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c143ae0 msgr2=0x7f916c135440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9171e9e640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c143ae0 0x7f916c135440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9171e9e640 1 -- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f916c0cf7d0 msgr2=0x7f916c0ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9171e9e640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f916c0cf7d0 0x7f916c0ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.199+0000 7f9171e9e640 1 -- 172.21.15.16:0/1268394478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9168009400 con 0x7f916c15e1e0 2023-11-24T06:30:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.200+0000 7f9171e9e640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c15e1e0 0x7f916c159900 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f916400c780 tx=0x7f916400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.200+0000 7f9161ffb640 1 -- 172.21.15.16:0/1268394478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916400ce90 con 0x7f916c15e1e0 2023-11-24T06:30:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.200+0000 7f9161ffb640 1 -- 172.21.15.16:0/1268394478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916400a640 con 0x7f916c15e1e0 2023-11-24T06:30:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.200+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0cc8f0 con 0x7f916c15e1e0 2023-11-24T06:30:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.200+0000 7f9161ffb640 1 -- 172.21.15.16:0/1268394478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916400a9d0 con 0x7f916c15e1e0 2023-11-24T06:30:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.200+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c135980 con 0x7f916c15e1e0 2023-11-24T06:30:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.201+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c0b2bf0 con 0x7f916c15e1e0 2023-11-24T06:30:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.203+0000 7f9161ffb640 1 -- 172.21.15.16:0/1268394478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f916400ab30 con 0x7f916c15e1e0 2023-11-24T06:30:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.203+0000 7f9161ffb640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f915006b230 0x7f915006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.204+0000 7f9161ffb640 1 -- 172.21.15.16:0/1268394478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f9164089360 con 0x7f916c15e1e0 2023-11-24T06:30:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.204+0000 7f917169d640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f915006b230 0x7f915006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:20.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.205+0000 7f917169d640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f915006b230 0x7f915006d6f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9168036000 tx=0x7f916803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:20.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.208+0000 7f9161ffb640 1 -- 172.21.15.16:0/1268394478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f916c0b2bf0 con 0x7f916c15e1e0 2023-11-24T06:30:20.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.443+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f916c0b2bf0 con 0x7f916c15e1e0 2023-11-24T06:30:20.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.967+0000 7f9161ffb640 1 -- 172.21.15.16:0/1268394478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f916c0b2bf0 con 0x7f916c15e1e0 2023-11-24T06:30:20.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f915006b230 msgr2=0x7f915006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:20.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f915006b230 0x7f915006d6f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9168036000 tx=0x7f916803c040 comp rx=0 tx=0).stop 2023-11-24T06:30:20.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c15e1e0 msgr2=0x7f916c159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:20.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c15e1e0 0x7f916c159900 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f916400c780 tx=0x7f916400cc40 comp rx=0 tx=0).stop 2023-11-24T06:30:20.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 shutdown_connections 2023-11-24T06:30:20.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f915006b230 0x7f915006d6f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f916c15e1e0 0x7f916c159900 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f916c143ae0 0x7f916c135440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 --2- 172.21.15.16:0/1268394478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f916c0cf7d0 0x7f916c0ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:20.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 >> 172.21.15.16:0/1268394478 conn(0x7f916c06a800 msgr2=0x7f916c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:20.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 shutdown_connections 2023-11-24T06:30:20.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:20.971+0000 7f9173901640 1 -- 172.21.15.16:0/1268394478 wait complete. 2023-11-24T06:30:20.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:30:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.221+0000 7f2dc761e640 1 Processor -- start 2023-11-24T06:30:21.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.221+0000 7f2dc761e640 1 -- start start 2023-11-24T06:30:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.223+0000 7f2dc761e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2dc013df80 0x7f2dc015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.223+0000 7f2dc761e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2dc013e360 0x7f2dc00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.223+0000 7f2dc761e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc006c470 con 0x7f2dc015f9c0 2023-11-24T06:30:21.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.223+0000 7f2dc761e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc006c750 con 0x7f2dc013e360 2023-11-24T06:30:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.223+0000 7f2dc761e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc006ca30 con 0x7f2dc013df80 2023-11-24T06:30:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.223+0000 7f2dc4bb9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2dc013e360 0x7f2dc00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.223+0000 7f2dc4bb9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2dc013e360 0x7f2dc00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55408/0 (socket says 172.21.15.16:55408) 2023-11-24T06:30:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.223+0000 7f2dc4bb9640 1 -- 172.21.15.16:0/988373662 learned_addr learned my addr 172.21.15.16:0/988373662 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:21.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.224+0000 7f2dc53ba640 1 --2- 172.21.15.16:0/988373662 >> v2:172.21.15.173:3300/0 conn(0x7f2dc013df80 0x7f2dc015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.224+0000 7f2dc4bb9640 1 -- 172.21.15.16:0/988373662 >> v2:172.21.15.173:3300/0 conn(0x7f2dc013df80 msgr2=0x7f2dc015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.224+0000 7f2dc4bb9640 1 --2- 172.21.15.16:0/988373662 >> v2:172.21.15.173:3300/0 conn(0x7f2dc013df80 0x7f2dc015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.224+0000 7f2dc4bb9640 1 -- 172.21.15.16:0/988373662 >> v1:172.21.15.173:6789/0 conn(0x7f2dc015f9c0 legacy=0x7f2dc01408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.224+0000 7f2dc4bb9640 1 -- 172.21.15.16:0/988373662 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc006cd10 con 0x7f2dc013e360 2023-11-24T06:30:21.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.225+0000 7f2dc4bb9640 1 --2- 172.21.15.16:0/988373662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc013e360 0x7f2dc00cf290 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0072770 tx=0x7f2db002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21e1609e1fc40c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.226+0000 7f2db7fff640 1 -- 172.21.15.16:0/988373662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0031870 con 0x7f2dc013e360 2023-11-24T06:30:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.226+0000 7f2db7fff640 1 -- 172.21.15.16:0/988373662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db00319d0 con 0x7f2dc013e360 2023-11-24T06:30:21.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.226+0000 7f2db7fff640 1 -- 172.21.15.16:0/988373662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0031d60 con 0x7f2dc013e360 2023-11-24T06:30:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.226+0000 7f2dc761e640 1 -- 172.21.15.16:0/988373662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc013e360 msgr2=0x7f2dc00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.226+0000 7f2dc761e640 1 --2- 172.21.15.16:0/988373662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc013e360 0x7f2dc00cf290 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0072770 tx=0x7f2db002b910 comp rx=0 tx=0).stop 2023-11-24T06:30:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.226+0000 7f2dc761e640 1 -- 172.21.15.16:0/988373662 shutdown_connections 2023-11-24T06:30:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.227+0000 7f2dc761e640 1 --2- 172.21.15.16:0/988373662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc013e360 0x7f2dc00cf290 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.227+0000 7f2dc761e640 1 --2- 172.21.15.16:0/988373662 >> v2:172.21.15.173:3300/0 conn(0x7f2dc013df80 0x7f2dc015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.227+0000 7f2dc761e640 1 -- 172.21.15.16:0/988373662 >> 172.21.15.16:0/988373662 conn(0x7f2dc006a800 msgr2=0x7f2dc006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.227+0000 7f2dc761e640 1 -- 172.21.15.16:0/988373662 shutdown_connections 2023-11-24T06:30:21.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.227+0000 7f2dc761e640 1 -- 172.21.15.16:0/988373662 wait complete. 2023-11-24T06:30:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.228+0000 7f2dc761e640 1 Processor -- start 2023-11-24T06:30:21.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.228+0000 7f2dc761e640 1 -- start start 2023-11-24T06:30:21.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.229+0000 7f2dc761e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc013df80 0x7f2dc015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.229+0000 7f2dc761e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc013e360 0x7f2dc013d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.229+0000 7f2dc761e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc015f9c0 0x7f2dc0131580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.229+0000 7f2dc761e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc015e660 con 0x7f2dc015f9c0 2023-11-24T06:30:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.229+0000 7f2dc761e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc015d5c0 con 0x7f2dc013df80 2023-11-24T06:30:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.229+0000 7f2dc761e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc015d8a0 con 0x7f2dc013e360 2023-11-24T06:30:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc5bbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc015f9c0 0x7f2dc0131580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc5bbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc015f9c0 0x7f2dc0131580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55412/0 (socket says 172.21.15.16:55412) 2023-11-24T06:30:21.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc5bbb640 1 -- 172.21.15.16:0/1221895749 learned_addr learned my addr 172.21.15.16:0/1221895749 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc4bb9640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc013e360 0x7f2dc013d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc53ba640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc013df80 0x7f2dc015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc5bbb640 1 -- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc013e360 msgr2=0x7f2dc013d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc5bbb640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc013e360 0x7f2dc013d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc5bbb640 1 -- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc013df80 msgr2=0x7f2dc015cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc5bbb640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc013df80 0x7f2dc015cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.230+0000 7f2dc5bbb640 1 -- 172.21.15.16:0/1221895749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00094d0 con 0x7f2dc015f9c0 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.231+0000 7f2dc5bbb640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc015f9c0 0x7f2dc0131580 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2db800c700 tx=0x7f2db800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.231+0000 7f2db5ffb640 1 -- 172.21.15.16:0/1221895749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800aa50 con 0x7f2dc015f9c0 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.231+0000 7f2db5ffb640 1 -- 172.21.15.16:0/1221895749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db800abb0 con 0x7f2dc015f9c0 2023-11-24T06:30:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.231+0000 7f2db5ffb640 1 -- 172.21.15.16:0/1221895749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800ce10 con 0x7f2dc015f9c0 2023-11-24T06:30:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.231+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc015db80 con 0x7f2dc015f9c0 2023-11-24T06:30:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.231+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc015e140 con 0x7f2dc015f9c0 2023-11-24T06:30:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.233+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc015e330 con 0x7f2dc015f9c0 2023-11-24T06:30:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.234+0000 7f2db5ffb640 1 -- 172.21.15.16:0/1221895749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f2db8012420 con 0x7f2dc015f9c0 2023-11-24T06:30:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.235+0000 7f2db5ffb640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2da406b260 0x7f2da406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.235+0000 7f2db5ffb640 1 -- 172.21.15.16:0/1221895749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f2db808a310 con 0x7f2dc015f9c0 2023-11-24T06:30:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.235+0000 7f2dc53ba640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2da406b260 0x7f2da406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.237+0000 7f2dc53ba640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2da406b260 0x7f2da406d720 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc006fd0 tx=0x7f2dbc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.239+0000 7f2db5ffb640 1 -- 172.21.15.16:0/1221895749 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2db8054d30 con 0x7f2dc015f9c0 2023-11-24T06:30:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:21.426+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2dc00675d0 con 0x7f2dc015f9c0 2023-11-24T06:30:22.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.009+0000 7f2db5ffb640 1 -- 172.21.15.16:0/1221895749 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2db8003eb0 con 0x7f2dc015f9c0 2023-11-24T06:30:22.013 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:30:22.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2da406b260 msgr2=0x7f2da406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:22.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2da406b260 0x7f2da406d720 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc006fd0 tx=0x7f2dbc008040 comp rx=0 tx=0).stop 2023-11-24T06:30:22.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc015f9c0 msgr2=0x7f2dc0131580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc015f9c0 0x7f2dc0131580 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2db800c700 tx=0x7f2db800cbc0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 shutdown_connections 2023-11-24T06:30:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2da406b260 0x7f2da406d720 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2dc015f9c0 0x7f2dc0131580 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc013e360 0x7f2dc013d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 --2- 172.21.15.16:0/1221895749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc013df80 0x7f2dc015cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.014+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 >> 172.21.15.16:0/1221895749 conn(0x7f2dc006a800 msgr2=0x7f2dc00702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:22.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.015+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 shutdown_connections 2023-11-24T06:30:22.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.015+0000 7f2dc761e640 1 -- 172.21.15.16:0/1221895749 wait complete. 2023-11-24T06:30:22.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:30:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.246+0000 7fd69d305640 1 Processor -- start 2023-11-24T06:30:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.247+0000 7fd69d305640 1 -- start start 2023-11-24T06:30:22.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.248+0000 7fd69d305640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd69815d610 0x7fd6980ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.248+0000 7fd69d305640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980ced60 0x7fd6980d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.248+0000 7fd69d305640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980b3400 con 0x7fd6980d1d20 2023-11-24T06:30:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.248+0000 7fd69d305640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980d1390 con 0x7fd69815d610 2023-11-24T06:30:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.248+0000 7fd69d305640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980d21b0 con 0x7fd6980ced60 2023-11-24T06:30:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd6977fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd6980d1d20 0x7fd6980cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51782/0 (socket says 172.21.15.16:51782) 2023-11-24T06:30:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd6977fe640 1 -- 172.21.15.16:0/1114130175 learned_addr learned my addr 172.21.15.16:0/1114130175 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd696ffd640 1 --2- 172.21.15.16:0/1114130175 >> v2:172.21.15.173:3300/0 conn(0x7fd69815d610 0x7fd6980ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd6967fc640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980ced60 0x7fd6980d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026380171 0 0) 0x7fd6980b3400 con 0x7fd6980d1d20 2023-11-24T06:30:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd684003520 con 0x7fd6980d1d20 2023-11-24T06:30:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2390198417 0 0) 0x7fd684003520 con 0x7fd6980d1d20 2023-11-24T06:30:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980ced60 msgr2=0x7fd6980d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd695ffb640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980ced60 0x7fd6980d17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 >> v2:172.21.15.173:3300/0 conn(0x7fd69815d610 msgr2=0x7fd6980ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd695ffb640 1 --2- 172.21.15.16:0/1114130175 >> v2:172.21.15.173:3300/0 conn(0x7fd69815d610 0x7fd6980ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980b27f0 con 0x7fd6980d1d20 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.249+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fd690001670 con 0x7fd6980d1d20 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 >> v1:172.21.15.16:6789/0 conn(0x7fd6980d1d20 legacy=0x7fd6980cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd695ffb640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd684003970 0x7fd684003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd695ffb640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980ced60 0x7fd6840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd695ffb640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd684004d00 0x7fd684005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980d1390 con 0x7fd684003970 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980d21b0 con 0x7fd684004d00 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68400b980 con 0x7fd6980ced60 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd696ffd640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd684003970 0x7fd684003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:22.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd696ffd640 1 -- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980ced60 msgr2=0x7fd6840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd6967fc640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980ced60 0x7fd6840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd696ffd640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980ced60 0x7fd6840047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd696ffd640 1 -- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd684004d00 msgr2=0x7fd684005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd696ffd640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd684004d00 0x7fd684005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.250+0000 7fd696ffd640 1 -- 172.21.15.16:0/1114130175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd684003520 con 0x7fd684003970 2023-11-24T06:30:22.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.251+0000 7fd696ffd640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd684003970 0x7fd684003d90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd688009fc0 tx=0x7fd6840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c82d729b1ef1c3a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:22.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68802cc60 con 0x7fd684003970 2023-11-24T06:30:22.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68800f040 con 0x7fd684003970 2023-11-24T06:30:22.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd695ffb640 1 -- 172.21.15.16:0/1114130175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688034440 con 0x7fd684003970 2023-11-24T06:30:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd69d305640 1 -- 172.21.15.16:0/1114130175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd684003970 msgr2=0x7fd684003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd69d305640 1 --2- 172.21.15.16:0/1114130175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd684003970 0x7fd684003d90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd688009fc0 tx=0x7fd6840057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd6977fe640 1 -- 172.21.15.16:0/1114130175 reap_dead start 2023-11-24T06:30:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd69d305640 1 -- 172.21.15.16:0/1114130175 shutdown_connections 2023-11-24T06:30:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd69d305640 1 -- 172.21.15.16:0/1114130175 >> 172.21.15.16:0/1114130175 conn(0x7fd69809c540 msgr2=0x7fd6980d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd69d305640 1 -- 172.21.15.16:0/1114130175 shutdown_connections 2023-11-24T06:30:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.252+0000 7fd69d305640 1 -- 172.21.15.16:0/1114130175 wait complete. 2023-11-24T06:30:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.254+0000 7fd69d305640 1 Processor -- start 2023-11-24T06:30:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.254+0000 7fd69d305640 1 -- start start 2023-11-24T06:30:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.255+0000 7fd69d305640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd684003970 0x7fd6980b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.255+0000 7fd69d305640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980b6a60 0x7fd69815b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.255+0000 7fd69d305640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6980cc0e0 0x7fd6980cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.255+0000 7fd69d305640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980b7ae0 con 0x7fd6980cc0e0 2023-11-24T06:30:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.255+0000 7fd69d305640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980ccab0 con 0x7fd684003970 2023-11-24T06:30:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.255+0000 7fd69d305640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6981f3240 con 0x7fd6980b6a60 2023-11-24T06:30:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.255+0000 7fd6977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6980cc0e0 0x7fd6980cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.256+0000 7fd6977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6980cc0e0 0x7fd6980cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55438/0 (socket says 172.21.15.16:55438) 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.256+0000 7fd6977fe640 1 -- 172.21.15.16:0/1543292757 learned_addr learned my addr 172.21.15.16:0/1543292757 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.256+0000 7fd696ffd640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd684003970 0x7fd6980b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.256+0000 7fd6977fe640 1 -- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980b6a60 msgr2=0x7fd69815b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.256+0000 7fd6977fe640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980b6a60 0x7fd69815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.256+0000 7fd6977fe640 1 -- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd684003970 msgr2=0x7fd6980b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.256+0000 7fd6977fe640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd684003970 0x7fd6980b85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.256+0000 7fd6977fe640 1 -- 172.21.15.16:0/1543292757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980b6f60 con 0x7fd6980cc0e0 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.256+0000 7fd6977fe640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6980cc0e0 0x7fd6980cc570 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd690000f60 tx=0x7fd69000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.257+0000 7fd68f7fe640 1 -- 172.21.15.16:0/1543292757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690007230 con 0x7fd6980cc0e0 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.257+0000 7fd68f7fe640 1 -- 172.21.15.16:0/1543292757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69000ced0 con 0x7fd6980cc0e0 2023-11-24T06:30:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.257+0000 7fd68f7fe640 1 -- 172.21.15.16:0/1543292757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6900075d0 con 0x7fd6980cc0e0 2023-11-24T06:30:22.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.257+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6980b7240 con 0x7fd6980cc0e0 2023-11-24T06:30:22.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.257+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6980b7800 con 0x7fd6980cc0e0 2023-11-24T06:30:22.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.258+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6981f33b0 con 0x7fd6980cc0e0 2023-11-24T06:30:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.259+0000 7fd68f7fe640 1 -- 172.21.15.16:0/1543292757 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd6900047e0 con 0x7fd6980cc0e0 2023-11-24T06:30:22.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.260+0000 7fd68f7fe640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd67806b260 0x7fd67806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.260+0000 7fd68f7fe640 1 -- 172.21.15.16:0/1543292757 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5385+0+0 (secure 0 0 0) 0x7fd690089140 con 0x7fd6980cc0e0 2023-11-24T06:30:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.260+0000 7fd696ffd640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd67806b260 0x7fd67806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.261+0000 7fd696ffd640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd67806b260 0x7fd67806d720 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd688036000 tx=0x7fd68802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:22.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.264+0000 7fd68f7fe640 1 -- 172.21.15.16:0/1543292757 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6981f33b0 con 0x7fd6980cc0e0 2023-11-24T06:30:22.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:22.459+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd6981f33b0 con 0x7fd6980cc0e0 2023-11-24T06:30:23.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.141+0000 7fd68f7fe640 1 -- 172.21.15.16:0/1543292757 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd6981f33b0 con 0x7fd6980cc0e0 2023-11-24T06:30:23.145 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-24T06:30:23.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.145+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd67806b260 msgr2=0x7fd67806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.146+0000 7fd69d305640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd67806b260 0x7fd67806d720 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd688036000 tx=0x7fd68802e000 comp rx=0 tx=0).stop 2023-11-24T06:30:23.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.146+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6980cc0e0 msgr2=0x7fd6980cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.146+0000 7fd69d305640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6980cc0e0 0x7fd6980cc570 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd690000f60 tx=0x7fd69000cb90 comp rx=0 tx=0).stop 2023-11-24T06:30:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.147+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 shutdown_connections 2023-11-24T06:30:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.147+0000 7fd69d305640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd67806b260 0x7fd67806d720 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.147+0000 7fd69d305640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6980cc0e0 0x7fd6980cc570 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.147+0000 7fd69d305640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6980b6a60 0x7fd69815b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.147+0000 7fd69d305640 1 --2- 172.21.15.16:0/1543292757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd684003970 0x7fd6980b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.147+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 >> 172.21.15.16:0/1543292757 conn(0x7fd69809c540 msgr2=0x7fd6980b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:23.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.147+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 shutdown_connections 2023-11-24T06:30:23.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.147+0000 7fd69d305640 1 -- 172.21.15.16:0/1543292757 wait complete. 2023-11-24T06:30:23.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:30:23.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.391+0000 7f0668cbb640 1 Processor -- start 2023-11-24T06:30:23.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.391+0000 7f0668cbb640 1 -- start start 2023-11-24T06:30:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.392+0000 7f0668cbb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640ccb50 0x7f0664070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.392+0000 7f0668cbb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066415f4e0 con 0x7f066415bf10 2023-11-24T06:30:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.392+0000 7f0668cbb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066415c3c0 con 0x7f06640adf20 2023-11-24T06:30:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.392+0000 7f0668cbb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066415f970 con 0x7f06640ccb50 2023-11-24T06:30:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.393+0000 7f0663fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f066415bf10 0x7f0664143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50570/0 (socket says 172.21.15.16:50570) 2023-11-24T06:30:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.393+0000 7f0663fff640 1 -- 172.21.15.16:0/2167324411 learned_addr learned my addr 172.21.15.16:0/2167324411 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.393+0000 7f0662ffd640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640ccb50 0x7f0664070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.393+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986847620 0 0) 0x7f066415f4e0 con 0x7f066415bf10 2023-11-24T06:30:23.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.393+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0650003520 con 0x7f066415bf10 2023-11-24T06:30:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337862920 0 0) 0x7f0650003520 con 0x7f066415bf10 2023-11-24T06:30:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640ccb50 msgr2=0x7f0664070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640ccb50 0x7f0664070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 >> v1:172.21.15.173:6789/0 conn(0x7f06640adf20 legacy=0x7f06640cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0664071130 con 0x7f066415bf10 2023-11-24T06:30:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f065c001670 con 0x7f066415bf10 2023-11-24T06:30:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 >> v1:172.21.15.16:6789/0 conn(0x7f066415bf10 legacy=0x7f0664143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640ccb50 0x7f06500039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0650004d00 0x7f0650005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066415f970 con 0x7f0650003f70 2023-11-24T06:30:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065000bb70 con 0x7f0650004d00 2023-11-24T06:30:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065000bd30 con 0x7f06640ccb50 2023-11-24T06:30:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.394+0000 7f0662ffd640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.395+0000 7f06637fe640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640ccb50 0x7f06500039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.395+0000 7f0663fff640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0650004d00 0x7f0650005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.395+0000 7f0662ffd640 1 -- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640ccb50 msgr2=0x7f06500039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.395+0000 7f0662ffd640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640ccb50 0x7f06500039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.395+0000 7f0662ffd640 1 -- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0650004d00 msgr2=0x7f0650005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.395+0000 7f0662ffd640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0650004d00 0x7f0650005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.395+0000 7f0662ffd640 1 -- 172.21.15.16:0/2167324411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0650003520 con 0x7f0650003f70 2023-11-24T06:30:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.395+0000 7f0663fff640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0650004d00 0x7f0650005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.396+0000 7f0662ffd640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06500047c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f064c009f90 tx=0x7f06500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc1f1c9acce1adf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.396+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c02cbd0 con 0x7f0650003f70 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.396+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c00f040 con 0x7f0650003f70 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.396+0000 7f06627fc640 1 -- 172.21.15.16:0/2167324411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c034440 con 0x7f0650003f70 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.397+0000 7f0668cbb640 1 -- 172.21.15.16:0/2167324411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 msgr2=0x7f06500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.397+0000 7f0668cbb640 1 --2- 172.21.15.16:0/2167324411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06500047c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f064c009f90 tx=0x7f06500057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.397+0000 7f0663fff640 1 -- 172.21.15.16:0/2167324411 reap_dead start 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.397+0000 7f0668cbb640 1 -- 172.21.15.16:0/2167324411 shutdown_connections 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.397+0000 7f0668cbb640 1 -- 172.21.15.16:0/2167324411 >> 172.21.15.16:0/2167324411 conn(0x7f066406a7d0 msgr2=0x7f066406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.397+0000 7f0668cbb640 1 -- 172.21.15.16:0/2167324411 shutdown_connections 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.397+0000 7f0668cbb640 1 -- 172.21.15.16:0/2167324411 wait complete. 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.399+0000 7f0668cbb640 1 Processor -- start 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.399+0000 7f0668cbb640 1 -- start start 2023-11-24T06:30:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.400+0000 7f0668cbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06641f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.400+0000 7f0668cbb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640b3210 0x7f06640cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.400+0000 7f0668cbb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06640cb6d0 0x7f06641efc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.400+0000 7f0668cbb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06641f3ad0 con 0x7f0650003f70 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.400+0000 7f0668cbb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06640b1e90 con 0x7f06640cb6d0 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.400+0000 7f0668cbb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06640b2170 con 0x7f06640b3210 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.400+0000 7f06637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06641f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.400+0000 7f06637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06641f3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48486/0 (socket says 172.21.15.16:48486) 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.400+0000 7f06637fe640 1 -- 172.21.15.16:0/70691432 learned_addr learned my addr 172.21.15.16:0/70691432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.401+0000 7f0663fff640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06640cb6d0 0x7f06641efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.401+0000 7f0662ffd640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640b3210 0x7f06640cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.401+0000 7f06637fe640 1 -- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640b3210 msgr2=0x7f06640cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.401+0000 7f06637fe640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640b3210 0x7f06640cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.401+0000 7f06637fe640 1 -- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06640cb6d0 msgr2=0x7f06641efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.401+0000 7f06637fe640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06640cb6d0 0x7f06641efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.401+0000 7f06637fe640 1 -- 172.21.15.16:0/70691432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06640b2450 con 0x7f0650003f70 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.401+0000 7f06637fe640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06641f3590 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f065400e550 tx=0x7f065400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.402+0000 7f065bfff640 1 -- 172.21.15.16:0/70691432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06540052e0 con 0x7f0650003f70 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.402+0000 7f065bfff640 1 -- 172.21.15.16:0/70691432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0654005440 con 0x7f0650003f70 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.402+0000 7f065bfff640 1 -- 172.21.15.16:0/70691432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654003810 con 0x7f0650003f70 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.402+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06640b2730 con 0x7f0650003f70 2023-11-24T06:30:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.402+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06640b2cf0 con 0x7f0650003f70 2023-11-24T06:30:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.403+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06640b2eb0 con 0x7f0650003f70 2023-11-24T06:30:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.404+0000 7f065bfff640 1 -- 172.21.15.16:0/70691432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f0654003970 con 0x7f0650003f70 2023-11-24T06:30:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.405+0000 7f065bfff640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f064406b230 0x7f064406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.405+0000 7f065bfff640 1 -- 172.21.15.16:0/70691432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f06540897e0 con 0x7f0650003f70 2023-11-24T06:30:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.405+0000 7f0662ffd640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f064406b230 0x7f064406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.407+0000 7f0662ffd640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f064406b230 0x7f064406d6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f064c036000 tx=0x7f064c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:23.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.409+0000 7f065bfff640 1 -- 172.21.15.16:0/70691432 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06540545b0 con 0x7f0650003f70 2023-11-24T06:30:23.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.604+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0664067450 con 0x7f0650003f70 2023-11-24T06:30:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.611+0000 7f065bfff640 1 -- 172.21.15.16:0/70691432 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0654057bd0 con 0x7f0650003f70 2023-11-24T06:30:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f064406b230 msgr2=0x7f064406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f064406b230 0x7f064406d6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f064c036000 tx=0x7f064c02e000 comp rx=0 tx=0).stop 2023-11-24T06:30:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 msgr2=0x7f06641f3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06641f3590 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f065400e550 tx=0x7f065400ea10 comp rx=0 tx=0).stop 2023-11-24T06:30:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 shutdown_connections 2023-11-24T06:30:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f064406b230 0x7f064406d6f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06640cb6d0 0x7f06641efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06640b3210 0x7f06640cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 --2- 172.21.15.16:0/70691432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0650003f70 0x7f06641f3590 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 >> 172.21.15.16:0/70691432 conn(0x7f066406a7d0 msgr2=0x7f066406c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.614+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 shutdown_connections 2023-11-24T06:30:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.615+0000 7f0668cbb640 1 -- 172.21.15.16:0/70691432 wait complete. 2023-11-24T06:30:23.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T06:30:23.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:30:23.689 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:30:23.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.857+0000 7fc54c911640 1 Processor -- start 2023-11-24T06:30:23.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.858+0000 7fc54c911640 1 -- start start 2023-11-24T06:30:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.859+0000 7fc54c911640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc548143c40 0x7fc548071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.859+0000 7fc54c911640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc548070e50 con 0x7fc5480717d0 2023-11-24T06:30:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.859+0000 7fc54c911640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc548144020 con 0x7fc5480ccc10 2023-11-24T06:30:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.859+0000 7fc54c911640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5480723c0 con 0x7fc548143c40 2023-11-24T06:30:23.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.860+0000 7fc5477fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc548143c40 0x7fc548071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.860+0000 7fc53ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc5480717d0 0x7fc5480cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50580/0 (socket says 172.21.15.16:50580) 2023-11-24T06:30:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.860+0000 7fc53ffff640 1 -- 172.21.15.16:0/1290339147 learned_addr learned my addr 172.21.15.16:0/1290339147 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.860+0000 7fc5477fe640 1 -- 172.21.15.16:0/1290339147 >> v1:172.21.15.173:6789/0 conn(0x7fc5480ccc10 legacy=0x7fc54806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.860+0000 7fc5477fe640 1 -- 172.21.15.16:0/1290339147 >> v1:172.21.15.16:6789/0 conn(0x7fc5480717d0 legacy=0x7fc5480cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.860+0000 7fc5477fe640 1 -- 172.21.15.16:0/1290339147 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc548071e00 con 0x7fc548143c40 2023-11-24T06:30:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.860+0000 7fc546ffd640 1 -- 172.21.15.16:0/1290339147 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80445786 0 0) 0x7fc548070e50 con 0x7fc5480717d0 2023-11-24T06:30:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.862+0000 7fc5477fe640 1 --2- 172.21.15.16:0/1290339147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548143c40 0x7fc548071290 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc5380097a0 tx=0x7fc53802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29913807d3a6b69a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.862+0000 7fc546ffd640 1 -- 172.21.15.16:0/1290339147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53802e070 con 0x7fc548143c40 2023-11-24T06:30:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.862+0000 7fc546ffd640 1 -- 172.21.15.16:0/1290339147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5380315a0 con 0x7fc548143c40 2023-11-24T06:30:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.862+0000 7fc546ffd640 1 -- 172.21.15.16:0/1290339147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc538031930 con 0x7fc548143c40 2023-11-24T06:30:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.863+0000 7fc54c911640 1 -- 172.21.15.16:0/1290339147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548143c40 msgr2=0x7fc548071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.863+0000 7fc54c911640 1 --2- 172.21.15.16:0/1290339147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548143c40 0x7fc548071290 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc5380097a0 tx=0x7fc53802b840 comp rx=0 tx=0).stop 2023-11-24T06:30:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.863+0000 7fc54c911640 1 -- 172.21.15.16:0/1290339147 shutdown_connections 2023-11-24T06:30:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.863+0000 7fc54c911640 1 --2- 172.21.15.16:0/1290339147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc548143c40 0x7fc548071290 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.863+0000 7fc54c911640 1 -- 172.21.15.16:0/1290339147 >> 172.21.15.16:0/1290339147 conn(0x7fc54806a800 msgr2=0x7fc54806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.863+0000 7fc54c911640 1 -- 172.21.15.16:0/1290339147 shutdown_connections 2023-11-24T06:30:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.863+0000 7fc54c911640 1 -- 172.21.15.16:0/1290339147 wait complete. 2023-11-24T06:30:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.865+0000 7fc54c911640 1 Processor -- start 2023-11-24T06:30:23.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.865+0000 7fc54c911640 1 -- start start 2023-11-24T06:30:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.866+0000 7fc54c911640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5480717d0 0x7fc54815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.866+0000 7fc54c911640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480ccc10 0x7fc5480ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.866+0000 7fc54c911640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc548143c40 0x7fc5480ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.866+0000 7fc54c911640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc548132280 con 0x7fc5480ccc10 2023-11-24T06:30:23.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.866+0000 7fc54c911640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc548132560 con 0x7fc548143c40 2023-11-24T06:30:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.866+0000 7fc54c911640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc548132840 con 0x7fc5480717d0 2023-11-24T06:30:23.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.866+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480ccc10 0x7fc5480ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.866+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480ccc10 0x7fc5480ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48514/0 (socket says 172.21.15.16:48514) 2023-11-24T06:30:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.866+0000 7fc53ffff640 1 -- 172.21.15.16:0/2478612793 learned_addr learned my addr 172.21.15.16:0/2478612793 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc5477fe640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5480717d0 0x7fc54815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc53ffff640 1 -- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5480717d0 msgr2=0x7fc54815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc547fff640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc548143c40 0x7fc5480ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc53ffff640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5480717d0 0x7fc54815a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc53ffff640 1 -- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc548143c40 msgr2=0x7fc5480ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc53ffff640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc548143c40 0x7fc5480ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc53ffff640 1 -- 172.21.15.16:0/2478612793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc538009400 con 0x7fc5480ccc10 2023-11-24T06:30:23.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc547fff640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc548143c40 0x7fc5480ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:23.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc53ffff640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480ccc10 0x7fc5480ac8d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c430 tx=0x7fc53000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:23.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc544ff9640 1 -- 172.21.15.16:0/2478612793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5300074e0 con 0x7fc5480ccc10 2023-11-24T06:30:23.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc544ff9640 1 -- 172.21.15.16:0/2478612793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc530007660 con 0x7fc5480ccc10 2023-11-24T06:30:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.867+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc548132b20 con 0x7fc5480ccc10 2023-11-24T06:30:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.868+0000 7fc544ff9640 1 -- 172.21.15.16:0/2478612793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530004a20 con 0x7fc5480ccc10 2023-11-24T06:30:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.868+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc548131510 con 0x7fc5480ccc10 2023-11-24T06:30:23.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.869+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc548133040 con 0x7fc5480ccc10 2023-11-24T06:30:23.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.870+0000 7fc544ff9640 1 -- 172.21.15.16:0/2478612793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fc530004b80 con 0x7fc5480ccc10 2023-11-24T06:30:23.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.872+0000 7fc544ff9640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc52806b230 0x7fc52806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:23.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.872+0000 7fc544ff9640 1 -- 172.21.15.16:0/2478612793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc530089890 con 0x7fc5480ccc10 2023-11-24T06:30:23.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.872+0000 7fc5477fe640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc52806b230 0x7fc52806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.874+0000 7fc5477fe640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc52806b230 0x7fc52806d6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc538036000 tx=0x7fc53803d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:23.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:23.877+0000 7fc544ff9640 1 -- 172.21.15.16:0/2478612793 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc548133040 con 0x7fc5480ccc10 2023-11-24T06:30:24.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:24.144+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7fc548133040 con 0x7fc5480ccc10 2023-11-24T06:30:25.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.154+0000 7fc544ff9640 1 -- 172.21.15.16:0/2478612793 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v215) v1 ==== 166+0+0 (secure 0 0 0) 0x7fc548133040 con 0x7fc5480ccc10 2023-11-24T06:30:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.157+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc52806b230 msgr2=0x7fc52806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.157+0000 7fc54c911640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc52806b230 0x7fc52806d6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc538036000 tx=0x7fc53803d040 comp rx=0 tx=0).stop 2023-11-24T06:30:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.157+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480ccc10 msgr2=0x7fc5480ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.157+0000 7fc54c911640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480ccc10 0x7fc5480ac8d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c430 tx=0x7fc53000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.158+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 shutdown_connections 2023-11-24T06:30:25.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.158+0000 7fc54c911640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc52806b230 0x7fc52806d6f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.158+0000 7fc54c911640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc548143c40 0x7fc5480ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.158+0000 7fc54c911640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480ccc10 0x7fc5480ac8d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.158+0000 7fc54c911640 1 --2- 172.21.15.16:0/2478612793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5480717d0 0x7fc54815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.158+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 >> 172.21.15.16:0/2478612793 conn(0x7fc54806a800 msgr2=0x7fc5480adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.158+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 shutdown_connections 2023-11-24T06:30:25.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.158+0000 7fc54c911640 1 -- 172.21.15.16:0/2478612793 wait complete. 2023-11-24T06:30:25.184 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ok 2023-11-24T06:30:25.184 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) tests the scenario where 'fs subvolume snapshot info' command 2023-11-24T06:30:25.184 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:30:25.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.405+0000 7f5d0a901640 1 Processor -- start 2023-11-24T06:30:25.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.405+0000 7f5d0a901640 1 -- start start 2023-11-24T06:30:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.407+0000 7f5d0a901640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5d0413e360 0x7f5d040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.407+0000 7f5d0a901640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0415c5f0 con 0x7f5d0415f9c0 2023-11-24T06:30:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.407+0000 7f5d0a901640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0406c470 con 0x7f5d0413df80 2023-11-24T06:30:25.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.407+0000 7f5d0a901640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0406c750 con 0x7f5d0413e360 2023-11-24T06:30:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.407+0000 7f5d08e9e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d0415f9c0 0x7f5d040cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50594/0 (socket says 172.21.15.16:50594) 2023-11-24T06:30:25.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.407+0000 7f5d08e9e640 1 -- 172.21.15.16:0/3886696605 learned_addr learned my addr 172.21.15.16:0/3886696605 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5cfb7fe640 1 --2- 172.21.15.16:0/3886696605 >> v2:172.21.15.173:3300/0 conn(0x7f5d0413e360 0x7f5d040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908376151 0 0) 0x7f5d0415c5f0 con 0x7f5d0415f9c0 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf0003520 con 0x7f5d0415f9c0 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1724029802 0 0) 0x7f5cf0003520 con 0x7f5d0415f9c0 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 >> v2:172.21.15.173:3300/0 conn(0x7f5d0413e360 msgr2=0x7f5d040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5d037fe640 1 --2- 172.21.15.16:0/3886696605 >> v2:172.21.15.173:3300/0 conn(0x7f5d0413e360 0x7f5d040cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 >> v1:172.21.15.173:6789/0 conn(0x7f5d0413df80 legacy=0x7f5d0415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0406ca30 con 0x7f5d0415f9c0 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f5cfc001690 con 0x7f5d0415f9c0 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.408+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 >> v1:172.21.15.16:6789/0 conn(0x7f5d0415f9c0 legacy=0x7f5d040cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d037fe640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5cf0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d037fe640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cf00047c0 0x7f5cf0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d037fe640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cf0005180 0x7f5cf000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0406c750 con 0x7f5cf00039c0 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf000c080 con 0x7f5cf00047c0 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf000c240 con 0x7f5cf0005180 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d03fff640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5cf0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5cfb7fe640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cf00047c0 0x7f5cf0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d08e9e640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cf0005180 0x7f5cf000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d03fff640 1 -- 172.21.15.16:0/3886696605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cf0005180 msgr2=0x7f5cf000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d03fff640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cf0005180 0x7f5cf000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d03fff640 1 -- 172.21.15.16:0/3886696605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cf00047c0 msgr2=0x7f5cf0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d03fff640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cf00047c0 0x7f5cf0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d03fff640 1 -- 172.21.15.16:0/3886696605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0003520 con 0x7f5cf00039c0 2023-11-24T06:30:25.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.409+0000 7f5d08e9e640 1 -- 172.21.15.16:0/3886696605 reap_dead start 2023-11-24T06:30:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.411+0000 7f5d03fff640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5cf0003de0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4001e50 tx=0x7f5cf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d078f9f0aae1b759 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.411+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4003b70 con 0x7f5cf00039c0 2023-11-24T06:30:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.411+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf4003cd0 con 0x7f5cf00039c0 2023-11-24T06:30:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.411+0000 7f5d037fe640 1 -- 172.21.15.16:0/3886696605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4005310 con 0x7f5cf00039c0 2023-11-24T06:30:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.411+0000 7f5d0a901640 1 -- 172.21.15.16:0/3886696605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 msgr2=0x7f5cf0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.411+0000 7f5d0a901640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5cf0003de0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4001e50 tx=0x7f5cf00057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.412+0000 7f5d0a901640 1 -- 172.21.15.16:0/3886696605 shutdown_connections 2023-11-24T06:30:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.412+0000 7f5d0a901640 1 --2- 172.21.15.16:0/3886696605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5cf0003de0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.412+0000 7f5d0a901640 1 -- 172.21.15.16:0/3886696605 >> 172.21.15.16:0/3886696605 conn(0x7f5d0406a800 msgr2=0x7f5d0406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.412+0000 7f5d0a901640 1 -- 172.21.15.16:0/3886696605 shutdown_connections 2023-11-24T06:30:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.412+0000 7f5d0a901640 1 -- 172.21.15.16:0/3886696605 wait complete. 2023-11-24T06:30:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.414+0000 7f5d0a901640 1 Processor -- start 2023-11-24T06:30:25.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.414+0000 7f5d0a901640 1 -- start start 2023-11-24T06:30:25.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d0a901640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5d0413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d0a901640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d04070fe0 0x7f5d0414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d0a901640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0414f970 0x7f5d0414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d0a901640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04150330 con 0x7f5cf00039c0 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d0a901640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040a8530 con 0x7f5d0414f970 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d0a901640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0414eb30 con 0x7f5d04070fe0 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5d0413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5d0413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48536/0 (socket says 172.21.15.16:48536) 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d03fff640 1 -- 172.21.15.16:0/871194688 learned_addr learned my addr 172.21.15.16:0/871194688 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5cfb7fe640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d04070fe0 0x7f5d0414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.415+0000 7f5d08e9e640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0414f970 0x7f5d0414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d03fff640 1 -- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d04070fe0 msgr2=0x7f5d0414e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d03fff640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d04070fe0 0x7f5d0414e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d03fff640 1 -- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0414f970 msgr2=0x7f5d0414fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d03fff640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0414f970 0x7f5d0414fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d03fff640 1 -- 172.21.15.16:0/871194688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0414ee10 con 0x7f5cf00039c0 2023-11-24T06:30:25.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d08e9e640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0414f970 0x7f5d0414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:25.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d03fff640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5d0413d8f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5cf402bf70 tx=0x7f5cf40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:25.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d017fa640 1 -- 172.21.15.16:0/871194688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf403e070 con 0x7f5cf00039c0 2023-11-24T06:30:25.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d017fa640 1 -- 172.21.15.16:0/871194688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf402f440 con 0x7f5cf00039c0 2023-11-24T06:30:25.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d017fa640 1 -- 172.21.15.16:0/871194688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf402f7b0 con 0x7f5cf00039c0 2023-11-24T06:30:25.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.416+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0414f0f0 con 0x7f5cf00039c0 2023-11-24T06:30:25.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.417+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0414e840 con 0x7f5cf00039c0 2023-11-24T06:30:25.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.418+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d040675d0 con 0x7f5cf00039c0 2023-11-24T06:30:25.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.419+0000 7f5d017fa640 1 -- 172.21.15.16:0/871194688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5cf402f910 con 0x7f5cf00039c0 2023-11-24T06:30:25.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.419+0000 7f5d017fa640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5ce406b280 0x7f5ce406d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.420+0000 7f5d017fa640 1 -- 172.21.15.16:0/871194688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5cf40ab260 con 0x7f5cf00039c0 2023-11-24T06:30:25.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.420+0000 7f5cfb7fe640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5ce406b280 0x7f5ce406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.421+0000 7f5cfb7fe640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5ce406b280 0x7f5ce406d740 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5d040cd190 tx=0x7f5cec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.424+0000 7f5d017fa640 1 -- 172.21.15.16:0/871194688 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5cf40761c0 con 0x7f5cf00039c0 2023-11-24T06:30:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.635+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d0414f740 con 0x7f5cf00039c0 2023-11-24T06:30:25.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.637+0000 7f5d017fa640 1 -- 172.21.15.16:0/871194688 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+12984 (secure 0 0 0) 0x7f5cf40797e0 con 0x7f5cf00039c0 2023-11-24T06:30:25.640 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:25.641 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:23.132062+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6833/515502120":"2023-11-25T06:30:19.962208+0000","172.21.15.16:6832/515502120":"2023-11-25T06:30:19.962208+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5ce406b280 msgr2=0x7f5ce406d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5ce406b280 0x7f5ce406d740 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5d040cd190 tx=0x7f5cec008040 comp rx=0 tx=0).stop 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 msgr2=0x7f5d0413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5d0413d8f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5cf402bf70 tx=0x7f5cf40014a0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 shutdown_connections 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5ce406b280 0x7f5ce406d740 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0414f970 0x7f5d0414fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d04070fe0 0x7f5d0414e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 --2- 172.21.15.16:0/871194688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf00039c0 0x7f5d0413d8f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 >> 172.21.15.16:0/871194688 conn(0x7f5d0406a800 msgr2=0x7f5d0415d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 shutdown_connections 2023-11-24T06:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.640+0000 7f5d0a901640 1 -- 172.21.15.16:0/871194688 wait complete. 2023-11-24T06:30:25.662 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:30:25.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:30:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.889+0000 7f16594c2640 1 Processor -- start 2023-11-24T06:30:25.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.889+0000 7f16594c2640 1 -- start start 2023-11-24T06:30:25.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.890+0000 7f16594c2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f165409a170 0x7f16540b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.890+0000 7f16594c2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f165406c6b0 0x7f16540cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.890+0000 7f16594c2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f165406cea0 con 0x7f16540cd240 2023-11-24T06:30:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.890+0000 7f16594c2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f165406cbc0 con 0x7f165409a170 2023-11-24T06:30:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.890+0000 7f16594c2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165406f350 con 0x7f165406c6b0 2023-11-24T06:30:25.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.891+0000 7f1652ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f16540cd240 0x7f1654099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50596/0 (socket says 172.21.15.16:50596) 2023-11-24T06:30:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.891+0000 7f1652ffd640 1 -- 172.21.15.16:0/4096685813 learned_addr learned my addr 172.21.15.16:0/4096685813 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:25.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.891+0000 7f16527fc640 1 --2- 172.21.15.16:0/4096685813 >> v2:172.21.15.16:3300/0 conn(0x7f165409a170 0x7f16540b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.891+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705731929 0 0) 0x7f165406cea0 con 0x7f16540cd240 2023-11-24T06:30:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.891+0000 7f16537fe640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f165406c6b0 0x7f16540cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.891+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1640003520 con 0x7f16540cd240 2023-11-24T06:30:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1758876809 0 0) 0x7f1640003520 con 0x7f16540cd240 2023-11-24T06:30:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f165406c6b0 msgr2=0x7f16540cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f165406c6b0 0x7f16540cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 >> v2:172.21.15.16:3300/0 conn(0x7f165409a170 msgr2=0x7f16540b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 --2- 172.21.15.16:0/4096685813 >> v2:172.21.15.16:3300/0 conn(0x7f165409a170 0x7f16540b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165406f630 con 0x7f16540cd240 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f16440019f0 con 0x7f16540cd240 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 >> v1:172.21.15.16:6789/0 conn(0x7f16540cd240 legacy=0x7f1654099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1640003970 0x7f1640003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f165406c6b0 0x7f16400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640004d00 0x7f1640005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165406cbc0 con 0x7f1640004d00 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165406f350 con 0x7f1640003970 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.892+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164000b980 con 0x7f165406c6b0 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.893+0000 7f16537fe640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640004d00 0x7f1640005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.893+0000 7f1652ffd640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1640003970 0x7f1640003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.893+0000 7f16527fc640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f165406c6b0 0x7f16400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.893+0000 7f16537fe640 1 -- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f165406c6b0 msgr2=0x7f16400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.893+0000 7f16537fe640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f165406c6b0 0x7f16400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.893+0000 7f16537fe640 1 -- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1640003970 msgr2=0x7f1640003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.893+0000 7f16537fe640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1640003970 0x7f1640003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.893+0000 7f16537fe640 1 -- 172.21.15.16:0/4096685813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640003520 con 0x7f1640004d00 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.893+0000 7f16527fc640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f165406c6b0 0x7f16400047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.894+0000 7f16537fe640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640004d00 0x7f1640005180 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f164c009a90 tx=0x7f16400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6a59546276de62 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c02c6b0 con 0x7f1640004d00 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c02c810 con 0x7f1640004d00 2023-11-24T06:30:25.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f1651ffb640 1 -- 172.21.15.16:0/4096685813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c02cbc0 con 0x7f1640004d00 2023-11-24T06:30:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f16594c2640 1 -- 172.21.15.16:0/4096685813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640004d00 msgr2=0x7f1640005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f16594c2640 1 --2- 172.21.15.16:0/4096685813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1640004d00 0x7f1640005180 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f164c009a90 tx=0x7f16400057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f16537fe640 1 -- 172.21.15.16:0/4096685813 reap_dead start 2023-11-24T06:30:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f16594c2640 1 -- 172.21.15.16:0/4096685813 shutdown_connections 2023-11-24T06:30:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f16594c2640 1 -- 172.21.15.16:0/4096685813 >> 172.21.15.16:0/4096685813 conn(0x7f16540b2d00 msgr2=0x7f16540b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f16594c2640 1 -- 172.21.15.16:0/4096685813 shutdown_connections 2023-11-24T06:30:25.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.895+0000 7f16594c2640 1 -- 172.21.15.16:0/4096685813 wait complete. 2023-11-24T06:30:25.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.898+0000 7f16594c2640 1 Processor -- start 2023-11-24T06:30:25.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.898+0000 7f16594c2640 1 -- start start 2023-11-24T06:30:25.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.899+0000 7f16594c2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1640004d00 0x7f165415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.899+0000 7f16594c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f165415ccf0 0x7f16540b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.899+0000 7f16594c2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16540b45a0 0x7f16541486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.899+0000 7f16594c2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165414c6e0 con 0x7f165415ccf0 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.899+0000 7f16594c2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540b4a20 con 0x7f16540b45a0 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.899+0000 7f16594c2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165415de30 con 0x7f1640004d00 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.899+0000 7f16527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f165415ccf0 0x7f16540b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.899+0000 7f16527fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f165415ccf0 0x7f16540b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48568/0 (socket says 172.21.15.16:48568) 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.899+0000 7f16527fc640 1 -- 172.21.15.16:0/1232658811 learned_addr learned my addr 172.21.15.16:0/1232658811 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.900+0000 7f1652ffd640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1640004d00 0x7f165415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.900+0000 7f16537fe640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16540b45a0 0x7f16541486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.900+0000 7f16527fc640 1 -- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1640004d00 msgr2=0x7f165415e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.900+0000 7f16527fc640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1640004d00 0x7f165415e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.900+0000 7f16527fc640 1 -- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16540b45a0 msgr2=0x7f16541486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:25.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.900+0000 7f16527fc640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16540b45a0 0x7f16541486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:25.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.900+0000 7f16527fc640 1 -- 172.21.15.16:0/1232658811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165415d220 con 0x7f165415ccf0 2023-11-24T06:30:25.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.900+0000 7f16527fc640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f165415ccf0 0x7f16540b4060 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f163c0109d0 tx=0x7f163c010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:25.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.901+0000 7f164b7fe640 1 -- 172.21.15.16:0/1232658811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c00ec20 con 0x7f165415ccf0 2023-11-24T06:30:25.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.901+0000 7f164b7fe640 1 -- 172.21.15.16:0/1232658811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c00ed80 con 0x7f165415ccf0 2023-11-24T06:30:25.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.901+0000 7f164b7fe640 1 -- 172.21.15.16:0/1232658811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c01c620 con 0x7f165415ccf0 2023-11-24T06:30:25.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.901+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165415d500 con 0x7f165415ccf0 2023-11-24T06:30:25.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.901+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165415dac0 con 0x7f165415ccf0 2023-11-24T06:30:25.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.902+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165415dc50 con 0x7f165415ccf0 2023-11-24T06:30:25.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.903+0000 7f164b7fe640 1 -- 172.21.15.16:0/1232658811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f163c01c780 con 0x7f165415ccf0 2023-11-24T06:30:25.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.904+0000 7f164b7fe640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f163406b230 0x7f163406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:25.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.904+0000 7f164b7fe640 1 -- 172.21.15.16:0/1232658811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f163c08a100 con 0x7f165415ccf0 2023-11-24T06:30:25.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.905+0000 7f1652ffd640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f163406b230 0x7f163406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.906+0000 7f1652ffd640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f163406b230 0x7f163406d6f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f16440013d0 tx=0x7f1644009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:25.909+0000 7f164b7fe640 1 -- 172.21.15.16:0/1232658811 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f163c054f80 con 0x7f165415ccf0 2023-11-24T06:30:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.105+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16540675d0 con 0x7f165415ccf0 2023-11-24T06:30:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.106+0000 7f164b7fe640 1 -- 172.21.15.16:0/1232658811 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f163c0585a0 con 0x7f165415ccf0 2023-11-24T06:30:26.109 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:30:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f163406b230 msgr2=0x7f163406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f163406b230 0x7f163406d6f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f16440013d0 tx=0x7f1644009040 comp rx=0 tx=0).stop 2023-11-24T06:30:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f165415ccf0 msgr2=0x7f16540b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f165415ccf0 0x7f16540b4060 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f163c0109d0 tx=0x7f163c010e90 comp rx=0 tx=0).stop 2023-11-24T06:30:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 shutdown_connections 2023-11-24T06:30:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f163406b230 0x7f163406d6f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16540b45a0 0x7f16541486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f165415ccf0 0x7f16540b4060 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 --2- 172.21.15.16:0/1232658811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1640004d00 0x7f165415e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 >> 172.21.15.16:0/1232658811 conn(0x7f16540b2d00 msgr2=0x7f16541607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 shutdown_connections 2023-11-24T06:30:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.109+0000 7f16594c2640 1 -- 172.21.15.16:0/1232658811 wait complete. 2023-11-24T06:30:26.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T06:30:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.354+0000 7f7052e9f640 1 Processor -- start 2023-11-24T06:30:26.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.354+0000 7f7052e9f640 1 -- start start 2023-11-24T06:30:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.356+0000 7f7052e9f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f704c143c40 0x7f704c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.356+0000 7f7052e9f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f704c0ccc10 0x7f704c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.356+0000 7f7052e9f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c070e50 con 0x7f704c0717d0 2023-11-24T06:30:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.356+0000 7f7052e9f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704c0723c0 con 0x7f704c0ccc10 2023-11-24T06:30:26.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.356+0000 7f7052e9f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704c144020 con 0x7f704c143c40 2023-11-24T06:30:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.356+0000 7f705169c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f704c0717d0 0x7f704c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50612/0 (socket says 172.21.15.16:50612) 2023-11-24T06:30:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.356+0000 7f705269e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f704c0ccc10 0x7f704c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.356+0000 7f705169c640 1 -- 172.21.15.16:0/289891695 learned_addr learned my addr 172.21.15.16:0/289891695 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:26.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.357+0000 7f7051e9d640 1 --2- 172.21.15.16:0/289891695 >> v2:172.21.15.173:3300/0 conn(0x7f704c143c40 0x7f704c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.357+0000 7f705269e640 1 -- 172.21.15.16:0/289891695 >> v2:172.21.15.173:3300/0 conn(0x7f704c143c40 msgr2=0x7f704c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.357+0000 7f705269e640 1 --2- 172.21.15.16:0/289891695 >> v2:172.21.15.173:3300/0 conn(0x7f704c143c40 0x7f704c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.357+0000 7f705269e640 1 -- 172.21.15.16:0/289891695 >> v1:172.21.15.16:6789/0 conn(0x7f704c0717d0 legacy=0x7f704c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.357+0000 7f705269e640 1 -- 172.21.15.16:0/289891695 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c071e00 con 0x7f704c0ccc10 2023-11-24T06:30:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.357+0000 7f7050e9b640 1 -- 172.21.15.16:0/289891695 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440872796 0 0) 0x7f704c070e50 con 0x7f704c0717d0 2023-11-24T06:30:26.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.358+0000 7f705269e640 1 --2- 172.21.15.16:0/289891695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 0x7f704c06d160 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f704c06e410 tx=0x7f704402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3cbf43136f9727 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.358+0000 7f7050e9b640 1 -- 172.21.15.16:0/289891695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704402cbf0 con 0x7f704c0ccc10 2023-11-24T06:30:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7050e9b640 1 -- 172.21.15.16:0/289891695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704402cd50 con 0x7f704c0ccc10 2023-11-24T06:30:26.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7050e9b640 1 -- 172.21.15.16:0/289891695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044034660 con 0x7f704c0ccc10 2023-11-24T06:30:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7052e9f640 1 -- 172.21.15.16:0/289891695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 msgr2=0x7f704c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7052e9f640 1 --2- 172.21.15.16:0/289891695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 0x7f704c06d160 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f704c06e410 tx=0x7f704402be40 comp rx=0 tx=0).stop 2023-11-24T06:30:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7052e9f640 1 -- 172.21.15.16:0/289891695 shutdown_connections 2023-11-24T06:30:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7052e9f640 1 --2- 172.21.15.16:0/289891695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 0x7f704c06d160 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7052e9f640 1 --2- 172.21.15.16:0/289891695 >> v2:172.21.15.173:3300/0 conn(0x7f704c143c40 0x7f704c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7052e9f640 1 -- 172.21.15.16:0/289891695 >> 172.21.15.16:0/289891695 conn(0x7f704c06a800 msgr2=0x7f704c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7052e9f640 1 -- 172.21.15.16:0/289891695 shutdown_connections 2023-11-24T06:30:26.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.359+0000 7f7052e9f640 1 -- 172.21.15.16:0/289891695 wait complete. 2023-11-24T06:30:26.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.362+0000 7f7052e9f640 1 Processor -- start 2023-11-24T06:30:26.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.362+0000 7f7052e9f640 1 -- start start 2023-11-24T06:30:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.363+0000 7f7052e9f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704c0717d0 0x7f704c12b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.363+0000 7f7052e9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 0x7f704c0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.363+0000 7f7052e9f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704c143c40 0x7f704c12d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.363+0000 7f7052e9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0c96e0 con 0x7f704c0ccc10 2023-11-24T06:30:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.363+0000 7f7052e9f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c12e430 con 0x7f704c143c40 2023-11-24T06:30:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.363+0000 7f7052e9f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0c8260 con 0x7f704c0717d0 2023-11-24T06:30:26.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.363+0000 7f705169c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 0x7f704c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705169c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 0x7f704c0c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48584/0 (socket says 172.21.15.16:48584) 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705169c640 1 -- 172.21.15.16:0/98969115 learned_addr learned my addr 172.21.15.16:0/98969115 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f7051e9d640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704c0717d0 0x7f704c12b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705269e640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704c143c40 0x7f704c12d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705169c640 1 -- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704c0717d0 msgr2=0x7f704c12b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705169c640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704c0717d0 0x7f704c12b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705169c640 1 -- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704c143c40 msgr2=0x7f704c12d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705169c640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704c143c40 0x7f704c12d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705169c640 1 -- 172.21.15.16:0/98969115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044009a70 con 0x7f704c0ccc10 2023-11-24T06:30:26.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f7051e9d640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704c0717d0 0x7f704c12b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705269e640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704c143c40 0x7f704c12d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.364+0000 7f705169c640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 0x7f704c0c7d20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f703c00c430 tx=0x7f703c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.365+0000 7f70427fc640 1 -- 172.21.15.16:0/98969115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c0074e0 con 0x7f704c0ccc10 2023-11-24T06:30:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.365+0000 7f70427fc640 1 -- 172.21.15.16:0/98969115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c007660 con 0x7f704c0ccc10 2023-11-24T06:30:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.365+0000 7f70427fc640 1 -- 172.21.15.16:0/98969115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c004a20 con 0x7f704c0ccc10 2023-11-24T06:30:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.365+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c12de70 con 0x7f704c0ccc10 2023-11-24T06:30:26.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.365+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c0c98f0 con 0x7f704c0ccc10 2023-11-24T06:30:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.366+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c0cd6f0 con 0x7f704c0ccc10 2023-11-24T06:30:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.367+0000 7f70427fc640 1 -- 172.21.15.16:0/98969115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f703c004b80 con 0x7f704c0ccc10 2023-11-24T06:30:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.368+0000 7f70427fc640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006b230 0x7f703006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:26.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.368+0000 7f70427fc640 1 -- 172.21.15.16:0/98969115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f703c0897c0 con 0x7f704c0ccc10 2023-11-24T06:30:26.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.369+0000 7f7051e9d640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006b230 0x7f703006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:26.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.370+0000 7f7051e9d640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006b230 0x7f703006d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7048006fd0 tx=0x7f7048008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:26.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.372+0000 7f70427fc640 1 -- 172.21.15.16:0/98969115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f703c054660 con 0x7f704c0ccc10 2023-11-24T06:30:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:26.639+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7f704c067450 con 0x7f704c0ccc10 2023-11-24T06:30:27.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.168+0000 7f70427fc640 1 -- 172.21.15.16:0/98969115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v217) v1 ==== 169+0+0 (secure 0 0 0) 0x7f703c057c80 con 0x7f704c0ccc10 2023-11-24T06:30:27.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006b230 msgr2=0x7f703006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:27.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006b230 0x7f703006d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7048006fd0 tx=0x7f7048008040 comp rx=0 tx=0).stop 2023-11-24T06:30:27.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 msgr2=0x7f704c0c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:27.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 0x7f704c0c7d20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f703c00c430 tx=0x7f703c00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 shutdown_connections 2023-11-24T06:30:27.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f703006b230 0x7f703006d6f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f704c143c40 0x7f704c12d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f704c0ccc10 0x7f704c0c7d20 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 --2- 172.21.15.16:0/98969115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f704c0717d0 0x7f704c12b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.174+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 >> 172.21.15.16:0/98969115 conn(0x7f704c06a800 msgr2=0x7f704c0aeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:27.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.175+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 shutdown_connections 2023-11-24T06:30:27.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.175+0000 7f7052e9f640 1 -- 172.21.15.16:0/98969115 wait complete. 2023-11-24T06:30:27.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:30:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.427+0000 7f5fdb83d640 1 Processor -- start 2023-11-24T06:30:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.427+0000 7f5fdb83d640 1 -- start start 2023-11-24T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.429+0000 7f5fdb83d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5fd413df80 0x7f5fd415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.429+0000 7f5fdb83d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5fd40cf9a0 0x7f5fd40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.429+0000 7f5fdb83d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd406c470 con 0x7f5fd413e360 2023-11-24T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.429+0000 7f5fdb83d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd406c750 con 0x7f5fd413df80 2023-11-24T06:30:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.429+0000 7f5fdb83d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd406ca30 con 0x7f5fd40cf9a0 2023-11-24T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.429+0000 7f5fd95d9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5fd413df80 0x7f5fd415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.429+0000 7f5fd95d9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5fd413df80 0x7f5fd415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48592/0 (socket says 172.21.15.16:48592) 2023-11-24T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.429+0000 7f5fd95d9640 1 -- 172.21.15.16:0/3979237417 learned_addr learned my addr 172.21.15.16:0/3979237417 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.430+0000 7f5fd95d9640 1 -- 172.21.15.16:0/3979237417 >> v2:172.21.15.173:3300/0 conn(0x7f5fd40cf9a0 msgr2=0x7f5fd40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.430+0000 7f5fd9dda640 1 --2- 172.21.15.16:0/3979237417 >> v2:172.21.15.173:3300/0 conn(0x7f5fd40cf9a0 0x7f5fd40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.430+0000 7f5fd95d9640 1 --2- 172.21.15.16:0/3979237417 >> v2:172.21.15.173:3300/0 conn(0x7f5fd40cf9a0 0x7f5fd40cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.430+0000 7f5fd95d9640 1 -- 172.21.15.16:0/3979237417 >> v1:172.21.15.173:6789/0 conn(0x7f5fd413e360 legacy=0x7f5fd40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.430+0000 7f5fd95d9640 1 -- 172.21.15.16:0/3979237417 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd406cd10 con 0x7f5fd413df80 2023-11-24T06:30:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.430+0000 7f5fd9dda640 1 --2- 172.21.15.16:0/3979237417 >> v2:172.21.15.173:3300/0 conn(0x7f5fd40cf9a0 0x7f5fd40cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.431+0000 7f5fd95d9640 1 --2- 172.21.15.16:0/3979237417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 0x7f5fd415bee0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40ccb10 tx=0x7f5fc402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4425cb5b09fda5ce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.431+0000 7f5fd3fff640 1 -- 172.21.15.16:0/3979237417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc402ca80 con 0x7f5fd413df80 2023-11-24T06:30:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.431+0000 7f5fd3fff640 1 -- 172.21.15.16:0/3979237417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc402cbe0 con 0x7f5fd413df80 2023-11-24T06:30:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.431+0000 7f5fd3fff640 1 -- 172.21.15.16:0/3979237417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40343f0 con 0x7f5fd413df80 2023-11-24T06:30:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.432+0000 7f5fdb83d640 1 -- 172.21.15.16:0/3979237417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 msgr2=0x7f5fd415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.432+0000 7f5fdb83d640 1 --2- 172.21.15.16:0/3979237417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 0x7f5fd415bee0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40ccb10 tx=0x7f5fc402bee0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.432+0000 7f5fdb83d640 1 -- 172.21.15.16:0/3979237417 shutdown_connections 2023-11-24T06:30:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.432+0000 7f5fdb83d640 1 --2- 172.21.15.16:0/3979237417 >> v2:172.21.15.173:3300/0 conn(0x7f5fd40cf9a0 0x7f5fd40cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.432+0000 7f5fdb83d640 1 --2- 172.21.15.16:0/3979237417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 0x7f5fd415bee0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.432+0000 7f5fdb83d640 1 -- 172.21.15.16:0/3979237417 >> 172.21.15.16:0/3979237417 conn(0x7f5fd406a800 msgr2=0x7f5fd406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.432+0000 7f5fdb83d640 1 -- 172.21.15.16:0/3979237417 shutdown_connections 2023-11-24T06:30:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.432+0000 7f5fdb83d640 1 -- 172.21.15.16:0/3979237417 wait complete. 2023-11-24T06:30:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.434+0000 7f5fdb83d640 1 Processor -- start 2023-11-24T06:30:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.434+0000 7f5fdb83d640 1 -- start start 2023-11-24T06:30:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.435+0000 7f5fdb83d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fd40cf9a0 0x7f5fd40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.435+0000 7f5fdb83d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 0x7f5fd4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.435+0000 7f5fdb83d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fd413e360 0x7f5fd40ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.435+0000 7f5fdb83d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd415da90 con 0x7f5fd413df80 2023-11-24T06:30:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.435+0000 7f5fdb83d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd413dc30 con 0x7f5fd40cf9a0 2023-11-24T06:30:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.435+0000 7f5fdb83d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd4153a20 con 0x7f5fd413e360 2023-11-24T06:30:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.436+0000 7f5fd8dd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 0x7f5fd4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.436+0000 7f5fd8dd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 0x7f5fd4070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48598/0 (socket says 172.21.15.16:48598) 2023-11-24T06:30:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.436+0000 7f5fd8dd8640 1 -- 172.21.15.16:0/718448480 learned_addr learned my addr 172.21.15.16:0/718448480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.436+0000 7f5fd95d9640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fd40cf9a0 0x7f5fd40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.436+0000 7f5fd8dd8640 1 -- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fd413e360 msgr2=0x7f5fd40ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.436+0000 7f5fd8dd8640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fd413e360 0x7f5fd40ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.436+0000 7f5fd8dd8640 1 -- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fd40cf9a0 msgr2=0x7f5fd40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.436+0000 7f5fd8dd8640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fd40cf9a0 0x7f5fd40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.436+0000 7f5fd8dd8640 1 -- 172.21.15.16:0/718448480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc4009b30 con 0x7f5fd413df80 2023-11-24T06:30:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.437+0000 7f5fd8dd8640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 0x7f5fd4070800 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c430 tx=0x7f5fc800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.437+0000 7f5fd1ffb640 1 -- 172.21.15.16:0/718448480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8011790 con 0x7f5fd413df80 2023-11-24T06:30:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.437+0000 7f5fd1ffb640 1 -- 172.21.15.16:0/718448480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc80118f0 con 0x7f5fd413df80 2023-11-24T06:30:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.437+0000 7f5fd1ffb640 1 -- 172.21.15.16:0/718448480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8011ca0 con 0x7f5fd413df80 2023-11-24T06:30:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.437+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd415ca80 con 0x7f5fd413df80 2023-11-24T06:30:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.437+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd415cfb0 con 0x7f5fd413df80 2023-11-24T06:30:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.438+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd40b4330 con 0x7f5fd413df80 2023-11-24T06:30:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.440+0000 7f5fd1ffb640 1 -- 172.21.15.16:0/718448480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5fc801c460 con 0x7f5fd413df80 2023-11-24T06:30:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.441+0000 7f5fd1ffb640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fc006b230 0x7f5fc006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.441+0000 7f5fd1ffb640 1 -- 172.21.15.16:0/718448480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5fc8089950 con 0x7f5fd413df80 2023-11-24T06:30:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.441+0000 7f5fd95d9640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fc006b230 0x7f5fc006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:27.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.443+0000 7f5fd95d9640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fc006b230 0x7f5fc006d6f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4035000 tx=0x7f5fc4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.446+0000 7f5fd1ffb640 1 -- 172.21.15.16:0/718448480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5fc80546e0 con 0x7f5fd413df80 2023-11-24T06:30:27.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.647+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5fd40675d0 con 0x7f5fd413df80 2023-11-24T06:30:27.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.654+0000 7f5fd1ffb640 1 -- 172.21.15.16:0/718448480 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5fc8057d00 con 0x7f5fd413df80 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.657+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fc006b230 msgr2=0x7f5fc006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.657+0000 7f5fdb83d640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fc006b230 0x7f5fc006d6f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4035000 tx=0x7f5fc4030000 comp rx=0 tx=0).stop 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.657+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 msgr2=0x7f5fd4070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.657+0000 7f5fdb83d640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 0x7f5fd4070800 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c430 tx=0x7f5fc800c8f0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.658+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 shutdown_connections 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.658+0000 7f5fdb83d640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fc006b230 0x7f5fc006d6f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.658+0000 7f5fdb83d640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5fd413e360 0x7f5fd40ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.658+0000 7f5fdb83d640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd413df80 0x7f5fd4070800 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.658+0000 7f5fdb83d640 1 --2- 172.21.15.16:0/718448480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fd40cf9a0 0x7f5fd40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:27.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.658+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 >> 172.21.15.16:0/718448480 conn(0x7f5fd406a800 msgr2=0x7f5fd406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.658+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 shutdown_connections 2023-11-24T06:30:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:27.658+0000 7f5fdb83d640 1 -- 172.21.15.16:0/718448480 wait complete. 2023-11-24T06:30:27.679 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:30:27.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:30:27.795 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:30:27.918 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:30:27.918 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:30:27.918 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:30:27.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:30:27.982 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:30:27.982 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:30:28.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:30:28.043 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:30:28.043 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:30:28.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:30:28.105 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:30:28.106 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:30:28.164 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:30:28.165 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:30:28.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:30:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.392+0000 7ff5213c4640 1 Processor -- start 2023-11-24T06:30:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.392+0000 7ff5213c4640 1 -- start start 2023-11-24T06:30:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.394+0000 7ff5213c4640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff51c13e360 0x7ff51c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.394+0000 7ff5213c4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51c15c5f0 con 0x7ff51c15f9c0 2023-11-24T06:30:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.394+0000 7ff5213c4640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51c06c470 con 0x7ff51c13df80 2023-11-24T06:30:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.394+0000 7ff5213c4640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51c06c750 con 0x7ff51c13e360 2023-11-24T06:30:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.394+0000 7ff51b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff51c15f9c0 0x7ff51c0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50628/0 (socket says 172.21.15.16:50628) 2023-11-24T06:30:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.394+0000 7ff51b7fe640 1 -- 172.21.15.16:0/588955428 learned_addr learned my addr 172.21.15.16:0/588955428 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff51a7fc640 1 --2- 172.21.15.16:0/588955428 >> v2:172.21.15.173:3300/0 conn(0x7ff51c13e360 0x7ff51c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984833449 0 0) 0x7ff51c15c5f0 con 0x7ff51c15f9c0 2023-11-24T06:30:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff504003520 con 0x7ff51c15f9c0 2023-11-24T06:30:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 389038533 0 0) 0x7ff504003520 con 0x7ff51c15f9c0 2023-11-24T06:30:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 >> v2:172.21.15.173:3300/0 conn(0x7ff51c13e360 msgr2=0x7ff51c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 --2- 172.21.15.16:0/588955428 >> v2:172.21.15.173:3300/0 conn(0x7ff51c13e360 0x7ff51c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 >> v1:172.21.15.173:6789/0 conn(0x7ff51c13df80 legacy=0x7ff51c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c06ca30 con 0x7ff51c15f9c0 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7ff514001a10 con 0x7ff51c15f9c0 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 >> v1:172.21.15.16:6789/0 conn(0x7ff51c15f9c0 legacy=0x7ff51c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.395+0000 7ff519ffb640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff504003970 0x7ff504003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff519ffb640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff504004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff519ffb640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff504005180 0x7ff50400fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c06c750 con 0x7ff5040047c0 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040104e0 con 0x7ff504005180 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040106a0 con 0x7ff504003970 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51a7fc640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff504004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51affd640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff504003970 0x7ff504003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51a7fc640 1 -- 172.21.15.16:0/588955428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff504003970 msgr2=0x7ff504003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51b7fe640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff504005180 0x7ff50400fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51a7fc640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff504003970 0x7ff504003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51a7fc640 1 -- 172.21.15.16:0/588955428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff504005180 msgr2=0x7ff50400fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51a7fc640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff504005180 0x7ff50400fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51a7fc640 1 -- 172.21.15.16:0/588955428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504003520 con 0x7ff5040047c0 2023-11-24T06:30:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51b7fe640 1 -- 172.21.15.16:0/588955428 reap_dead start 2023-11-24T06:30:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.396+0000 7ff51affd640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff504003970 0x7ff504003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.397+0000 7ff51a7fc640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff504004c40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff51000ba10 tx=0x7ff5040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4f683a454b25b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.398+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510032630 con 0x7ff5040047c0 2023-11-24T06:30:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.398+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510032790 con 0x7ff5040047c0 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.398+0000 7ff519ffb640 1 -- 172.21.15.16:0/588955428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510032b20 con 0x7ff5040047c0 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.398+0000 7ff5213c4640 1 -- 172.21.15.16:0/588955428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 msgr2=0x7ff504004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.398+0000 7ff5213c4640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff504004c40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff51000ba10 tx=0x7ff5040057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.398+0000 7ff5213c4640 1 -- 172.21.15.16:0/588955428 shutdown_connections 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.398+0000 7ff5213c4640 1 --2- 172.21.15.16:0/588955428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff504004c40 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.398+0000 7ff5213c4640 1 -- 172.21.15.16:0/588955428 >> 172.21.15.16:0/588955428 conn(0x7ff51c06a800 msgr2=0x7ff51c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.399+0000 7ff5213c4640 1 -- 172.21.15.16:0/588955428 shutdown_connections 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.399+0000 7ff5213c4640 1 -- 172.21.15.16:0/588955428 wait complete. 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.400+0000 7ff5213c4640 1 Processor -- start 2023-11-24T06:30:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.401+0000 7ff5213c4640 1 -- start start 2023-11-24T06:30:28.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.401+0000 7ff5213c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff51c13d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.401+0000 7ff5213c4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c071ef0 0x7ff51c0afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff5213c4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0affc0 0x7ff51c159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff5213c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0b0440 con 0x7ff5040047c0 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff5213c4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c13d7a0 con 0x7ff51c0affc0 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff5213c4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c0712e0 con 0x7ff51c071ef0 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff51c13d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff51c13d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48626/0 (socket says 172.21.15.16:48626) 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51affd640 1 -- 172.21.15.16:0/2955797515 learned_addr learned my addr 172.21.15.16:0/2955797515 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51a7fc640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c071ef0 0x7ff51c0afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51b7fe640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0affc0 0x7ff51c159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51affd640 1 -- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c071ef0 msgr2=0x7ff51c0afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51affd640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c071ef0 0x7ff51c0afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51affd640 1 -- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0affc0 msgr2=0x7ff51c159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51affd640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0affc0 0x7ff51c159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51affd640 1 -- 172.21.15.16:0/2955797515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c0b38a0 con 0x7ff5040047c0 2023-11-24T06:30:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.402+0000 7ff51b7fe640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0affc0 0x7ff51c159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.403+0000 7ff51affd640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff51c13d260 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00d4a0 tx=0x7ff50c00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.403+0000 7ff5037fe640 1 -- 172.21.15.16:0/2955797515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c005480 con 0x7ff5040047c0 2023-11-24T06:30:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.403+0000 7ff5037fe640 1 -- 172.21.15.16:0/2955797515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c007030 con 0x7ff5040047c0 2023-11-24T06:30:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.403+0000 7ff5037fe640 1 -- 172.21.15.16:0/2955797515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c007400 con 0x7ff5040047c0 2023-11-24T06:30:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.403+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51c071000 con 0x7ff5040047c0 2023-11-24T06:30:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.403+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51c0b35c0 con 0x7ff5040047c0 2023-11-24T06:30:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.405+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51c0675d0 con 0x7ff5040047c0 2023-11-24T06:30:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.406+0000 7ff5037fe640 1 -- 172.21.15.16:0/2955797515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff50c007560 con 0x7ff5040047c0 2023-11-24T06:30:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.406+0000 7ff5037fe640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff50806b230 0x7ff50806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.407+0000 7ff5037fe640 1 -- 172.21.15.16:0/2955797515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff50c088fb0 con 0x7ff5040047c0 2023-11-24T06:30:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.407+0000 7ff51a7fc640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff50806b230 0x7ff50806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.408+0000 7ff51a7fc640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff50806b230 0x7ff50806d6f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0cc790 tx=0x7ff510030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.411+0000 7ff5037fe640 1 -- 172.21.15.16:0/2955797515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff50c053d20 con 0x7ff5040047c0 2023-11-24T06:30:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.644+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff51c14ff00 con 0x7ff5040047c0 2023-11-24T06:30:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.645+0000 7ff5037fe640 1 -- 172.21.15.16:0/2955797515 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3699 (secure 0 0 0) 0x7ff50c019020 con 0x7ff5040047c0 2023-11-24T06:30:28.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:28.648 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3782726360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3782726360},{"type":"v1","addr":"172.21.15.173:6835","nonce":3782726360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1586323927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1586323927},{"type":"v1","addr":"172.21.15.16:6835","nonce":1586323927}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1730788036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1730788036},{"type":"v1","addr":"172.21.15.173:6837","nonce":1730788036}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/27468868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":27468868},{"type":"v1","addr":"172.21.15.16:6833","nonce":27468868}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[]} 2023-11-24T06:30:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 37 2023-11-24T06:30:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff50806b230 msgr2=0x7ff50806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff50806b230 0x7ff50806d6f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0cc790 tx=0x7ff510030000 comp rx=0 tx=0).stop 2023-11-24T06:30:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 msgr2=0x7ff51c13d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff51c13d260 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00d4a0 tx=0x7ff50c00d960 comp rx=0 tx=0).stop 2023-11-24T06:30:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 shutdown_connections 2023-11-24T06:30:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff50806b230 0x7ff50806d6f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0affc0 0x7ff51c159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c071ef0 0x7ff51c0afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 --2- 172.21.15.16:0/2955797515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5040047c0 0x7ff51c13d260 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.647+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 >> 172.21.15.16:0/2955797515 conn(0x7ff51c06a800 msgr2=0x7ff51c0cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.648+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 shutdown_connections 2023-11-24T06:30:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.648+0000 7ff5213c4640 1 -- 172.21.15.16:0/2955797515 wait complete. 2023-11-24T06:30:28.669 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:30:28.669 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:30:28.670 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:30:28.670 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:28.670 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:30:28.670 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:30:28.670 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:30:28.671 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:28.671 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:30:28.671 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:30:28.671 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:30:28.671 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:28.672 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:30:28.672 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:30:28.672 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:30:28.672 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:30:28.687 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:30:28.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:30:28.689 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:30:28.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:30:28.693 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:30:28.693 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:30:28.695 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:30:28.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:30:28.737 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:30:28.737 DEBUG:teuthology.parallel:result is None 2023-11-24T06:30:28.742 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:30:28.743 DEBUG:teuthology.parallel:result is None 2023-11-24T06:30:28.745 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:30:28.745 DEBUG:teuthology.parallel:result is None 2023-11-24T06:30:28.750 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:30:28.750 DEBUG:teuthology.parallel:result is None 2023-11-24T06:30:28.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:30:28.902 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2023-11-24T06:30:28.902 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-24T06:30:28.903 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2023-11-24T06:30:28.903 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:30:28.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.939+0000 7fa01a301640 1 Processor -- start 2023-11-24T06:30:28.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.940+0000 7fa01a301640 1 -- start start 2023-11-24T06:30:28.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.941+0000 7fa01a301640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0140ae5a0 0x7fa01415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.941+0000 7fa01a301640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa014070c40 con 0x7fa0140cf290 2023-11-24T06:30:28.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.941+0000 7fa01a301640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01406c470 con 0x7fa0140b3ae0 2023-11-24T06:30:28.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.941+0000 7fa01a301640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa014071450 con 0x7fa0140ae5a0 2023-11-24T06:30:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.941+0000 7fa013fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0140ae5a0 0x7fa01415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.941+0000 7fa01889e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa0140cf290 0x7fa0140b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50648/0 (socket says 172.21.15.16:50648) 2023-11-24T06:30:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.941+0000 7fa01889e640 1 -- 172.21.15.16:0/1197606445 learned_addr learned my addr 172.21.15.16:0/1197606445 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.942+0000 7fa013fff640 1 -- 172.21.15.16:0/1197606445 >> v1:172.21.15.173:6789/0 conn(0x7fa0140b3ae0 legacy=0x7fa01406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.942+0000 7fa013fff640 1 -- 172.21.15.16:0/1197606445 >> v1:172.21.15.16:6789/0 conn(0x7fa0140cf290 legacy=0x7fa0140b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.942+0000 7fa013fff640 1 -- 172.21.15.16:0/1197606445 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01406c750 con 0x7fa0140ae5a0 2023-11-24T06:30:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.943+0000 7fa013fff640 1 --2- 172.21.15.16:0/1197606445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140ae5a0 0x7fa01415e430 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa0040097a0 tx=0x7fa00402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298fa032827ed407 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.943+0000 7fa012ffd640 1 -- 172.21.15.16:0/1197606445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004030610 con 0x7fa0140ae5a0 2023-11-24T06:30:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.943+0000 7fa012ffd640 1 -- 172.21.15.16:0/1197606445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004030770 con 0x7fa0140ae5a0 2023-11-24T06:30:28.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.943+0000 7fa012ffd640 1 -- 172.21.15.16:0/1197606445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004030b00 con 0x7fa0140ae5a0 2023-11-24T06:30:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.944+0000 7fa01a301640 1 -- 172.21.15.16:0/1197606445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140ae5a0 msgr2=0x7fa01415e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.944+0000 7fa01a301640 1 --2- 172.21.15.16:0/1197606445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140ae5a0 0x7fa01415e430 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa0040097a0 tx=0x7fa00402b960 comp rx=0 tx=0).stop 2023-11-24T06:30:28.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.944+0000 7fa01a301640 1 -- 172.21.15.16:0/1197606445 shutdown_connections 2023-11-24T06:30:28.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.944+0000 7fa01a301640 1 --2- 172.21.15.16:0/1197606445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140ae5a0 0x7fa01415e430 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.944+0000 7fa01a301640 1 -- 172.21.15.16:0/1197606445 >> 172.21.15.16:0/1197606445 conn(0x7fa01406a800 msgr2=0x7fa01406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:28.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.944+0000 7fa01a301640 1 -- 172.21.15.16:0/1197606445 shutdown_connections 2023-11-24T06:30:28.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.944+0000 7fa01a301640 1 -- 172.21.15.16:0/1197606445 wait complete. 2023-11-24T06:30:28.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.946+0000 7fa01a301640 1 Processor -- start 2023-11-24T06:30:28.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.946+0000 7fa01a301640 1 -- start start 2023-11-24T06:30:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa01a301640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0140ae5a0 0x7fa01414bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa01a301640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0140b3ae0 0x7fa014147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa01a301640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140cf290 0x7fa014157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa01a301640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140b07c0 con 0x7fa0140cf290 2023-11-24T06:30:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa01a301640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa014158040 con 0x7fa0140ae5a0 2023-11-24T06:30:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa01a301640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140afb40 con 0x7fa0140b3ae0 2023-11-24T06:30:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa01889e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140cf290 0x7fa014157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa01889e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140cf290 0x7fa014157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48654/0 (socket says 172.21.15.16:48654) 2023-11-24T06:30:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa01889e640 1 -- 172.21.15.16:0/2488072041 learned_addr learned my addr 172.21.15.16:0/2488072041 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:28.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa013fff640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0140ae5a0 0x7fa01414bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.947+0000 7fa0137fe640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0140b3ae0 0x7fa014147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa01889e640 1 -- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0140b3ae0 msgr2=0x7fa014147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa01889e640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0140b3ae0 0x7fa014147bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa01889e640 1 -- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0140ae5a0 msgr2=0x7fa01414bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:28.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa01889e640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0140ae5a0 0x7fa01414bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:28.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa01889e640 1 -- 172.21.15.16:0/2488072041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa004009400 con 0x7fa0140cf290 2023-11-24T06:30:28.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa01889e640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140cf290 0x7fa014157b00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00ca80 tx=0x7fa00c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:28.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa010ff9640 1 -- 172.21.15.16:0/2488072041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00ac30 con 0x7fa0140cf290 2023-11-24T06:30:28.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa010ff9640 1 -- 172.21.15.16:0/2488072041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c00ad90 con 0x7fa0140cf290 2023-11-24T06:30:28.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa010ff9640 1 -- 172.21.15.16:0/2488072041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c012600 con 0x7fa0140cf290 2023-11-24T06:30:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0140afe20 con 0x7fa0140cf290 2023-11-24T06:30:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.948+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0140b03e0 con 0x7fa0140cf290 2023-11-24T06:30:28.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.950+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0140675d0 con 0x7fa0140cf290 2023-11-24T06:30:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.951+0000 7fa010ff9640 1 -- 172.21.15.16:0/2488072041 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa00c007070 con 0x7fa0140cf290 2023-11-24T06:30:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.952+0000 7fa010ff9640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa00006b230 0x7fa00006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.952+0000 7fa010ff9640 1 -- 172.21.15.16:0/2488072041 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa00c089790 con 0x7fa0140cf290 2023-11-24T06:30:28.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.952+0000 7fa013fff640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa00006b230 0x7fa00006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:28.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.953+0000 7fa013fff640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa00006b230 0x7fa00006d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa004036000 tx=0x7fa004008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:28.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:28.956+0000 7fa010ff9640 1 -- 172.21.15.16:0/2488072041 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa00c054660 con 0x7fa0140cf290 2023-11-24T06:30:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.162+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa0140b0900 con 0x7fa0140cf290 2023-11-24T06:30:29.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.193+0000 7fa010ff9640 1 -- 172.21.15.16:0/2488072041 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa00c017080 con 0x7fa0140cf290 2023-11-24T06:30:29.196 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-11-24T06:30:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa00006b230 msgr2=0x7fa00006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa00006b230 0x7fa00006d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa004036000 tx=0x7fa004008030 comp rx=0 tx=0).stop 2023-11-24T06:30:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140cf290 msgr2=0x7fa014157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140cf290 0x7fa014157b00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00ca80 tx=0x7fa00c00cf40 comp rx=0 tx=0).stop 2023-11-24T06:30:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 shutdown_connections 2023-11-24T06:30:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa00006b230 0x7fa00006d6f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0140cf290 0x7fa014157b00 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0140b3ae0 0x7fa014147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 --2- 172.21.15.16:0/2488072041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0140ae5a0 0x7fa01414bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 >> 172.21.15.16:0/2488072041 conn(0x7fa01406a800 msgr2=0x7fa01406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.196+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 shutdown_connections 2023-11-24T06:30:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.197+0000 7fa01a301640 1 -- 172.21.15.16:0/2488072041 wait complete. 2023-11-24T06:30:29.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:30:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.429+0000 7effbefcf640 1 Processor -- start 2023-11-24T06:30:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.429+0000 7effbefcf640 1 -- start start 2023-11-24T06:30:29.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.431+0000 7effbefcf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 0x7effb815c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.431+0000 7effbefcf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effb806c470 0x7effb80cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.431+0000 7effbefcf640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7effb806dab0 0x7effb80ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.431+0000 7effbefcf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb806d690 con 0x7effb806c470 2023-11-24T06:30:29.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.431+0000 7effbefcf640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb80cebb0 con 0x7effb806dab0 2023-11-24T06:30:29.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.431+0000 7effbefcf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb815c5c0 con 0x7effb81447f0 2023-11-24T06:30:29.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbd7cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effb806c470 0x7effb80cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbd7cc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effb806c470 0x7effb80cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48658/0 (socket says 172.21.15.16:48658) 2023-11-24T06:30:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbd7cc640 1 -- 172.21.15.16:0/972321947 learned_addr learned my addr 172.21.15.16:0/972321947 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbe7ce640 1 --2- 172.21.15.16:0/972321947 >> v2:172.21.15.173:3300/0 conn(0x7effb806dab0 0x7effb80ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbdfcd640 1 --2- 172.21.15.16:0/972321947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 0x7effb815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbd7cc640 1 -- 172.21.15.16:0/972321947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 msgr2=0x7effb815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbd7cc640 1 --2- 172.21.15.16:0/972321947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 0x7effb815c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbd7cc640 1 -- 172.21.15.16:0/972321947 >> v2:172.21.15.173:3300/0 conn(0x7effb806dab0 msgr2=0x7effb80ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbd7cc640 1 --2- 172.21.15.16:0/972321947 >> v2:172.21.15.173:3300/0 conn(0x7effb806dab0 0x7effb80ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.432+0000 7effbd7cc640 1 -- 172.21.15.16:0/972321947 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb806ca10 con 0x7effb806c470 2023-11-24T06:30:29.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.434+0000 7effbd7cc640 1 --2- 172.21.15.16:0/972321947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806c470 0x7effb80cc5e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7effb815eb50 tx=0x7effa802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a17db40801927a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.434+0000 7effbcfcb640 1 -- 172.21.15.16:0/972321947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa80317d0 con 0x7effb806c470 2023-11-24T06:30:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.434+0000 7effbcfcb640 1 -- 172.21.15.16:0/972321947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa8031930 con 0x7effb806c470 2023-11-24T06:30:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.434+0000 7effbcfcb640 1 -- 172.21.15.16:0/972321947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8031cc0 con 0x7effb806c470 2023-11-24T06:30:29.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.435+0000 7effbefcf640 1 -- 172.21.15.16:0/972321947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806c470 msgr2=0x7effb80cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.435+0000 7effbefcf640 1 --2- 172.21.15.16:0/972321947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806c470 0x7effb80cc5e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7effb815eb50 tx=0x7effa802b870 comp rx=0 tx=0).stop 2023-11-24T06:30:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.435+0000 7effbefcf640 1 -- 172.21.15.16:0/972321947 shutdown_connections 2023-11-24T06:30:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.435+0000 7effbefcf640 1 --2- 172.21.15.16:0/972321947 >> v2:172.21.15.173:3300/0 conn(0x7effb806dab0 0x7effb80ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.435+0000 7effbefcf640 1 --2- 172.21.15.16:0/972321947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806c470 0x7effb80cc5e0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.435+0000 7effbefcf640 1 --2- 172.21.15.16:0/972321947 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 0x7effb815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.435+0000 7effbefcf640 1 -- 172.21.15.16:0/972321947 >> 172.21.15.16:0/972321947 conn(0x7effb806a3e0 msgr2=0x7effb806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:29.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.435+0000 7effbefcf640 1 -- 172.21.15.16:0/972321947 shutdown_connections 2023-11-24T06:30:29.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.435+0000 7effbefcf640 1 -- 172.21.15.16:0/972321947 wait complete. 2023-11-24T06:30:29.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.437+0000 7effbefcf640 1 Processor -- start 2023-11-24T06:30:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.437+0000 7effbefcf640 1 -- start start 2023-11-24T06:30:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.438+0000 7effbefcf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb806c470 0x7effb813f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.438+0000 7effbefcf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806dab0 0x7effb8158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.438+0000 7effbefcf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 0x7effb81587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.438+0000 7effbefcf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb813eba0 con 0x7effb806dab0 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.438+0000 7effbefcf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb80aec20 con 0x7effb806c470 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.438+0000 7effbefcf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb80aef00 con 0x7effb81447f0 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.438+0000 7effbd7cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806dab0 0x7effb8158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.438+0000 7effbd7cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806dab0 0x7effb8158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48674/0 (socket says 172.21.15.16:48674) 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.438+0000 7effbd7cc640 1 -- 172.21.15.16:0/1660909070 learned_addr learned my addr 172.21.15.16:0/1660909070 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effbe7ce640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 0x7effb81587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effbdfcd640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb806c470 0x7effb813f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effbd7cc640 1 -- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 msgr2=0x7effb81587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effbd7cc640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 0x7effb81587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effbd7cc640 1 -- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb806c470 msgr2=0x7effb813f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effbd7cc640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb806c470 0x7effb813f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effbd7cc640 1 -- 172.21.15.16:0/1660909070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80094d0 con 0x7effb806dab0 2023-11-24T06:30:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effbd7cc640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806dab0 0x7effb8158260 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7effa8000c00 tx=0x7effa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effae7fc640 1 -- 172.21.15.16:0/1660909070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa80328f0 con 0x7effb806dab0 2023-11-24T06:30:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effae7fc640 1 -- 172.21.15.16:0/1660909070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa8032a70 con 0x7effb806dab0 2023-11-24T06:30:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effae7fc640 1 -- 172.21.15.16:0/1660909070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8032e20 con 0x7effb806dab0 2023-11-24T06:30:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.439+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb80af1e0 con 0x7effb806dab0 2023-11-24T06:30:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.440+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb813fdd0 con 0x7effb806dab0 2023-11-24T06:30:29.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.441+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb813ffc0 con 0x7effb806dab0 2023-11-24T06:30:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.442+0000 7effae7fc640 1 -- 172.21.15.16:0/1660909070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7effa8030070 con 0x7effb806dab0 2023-11-24T06:30:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.443+0000 7effae7fc640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff9c06b230 0x7eff9c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.443+0000 7effae7fc640 1 -- 172.21.15.16:0/1660909070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7effa80abb20 con 0x7effb806dab0 2023-11-24T06:30:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.443+0000 7effbdfcd640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff9c06b230 0x7eff9c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.444+0000 7effbdfcd640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff9c06b230 0x7eff9c06d6f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7effb40070b0 tx=0x7effb4007040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:29.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.447+0000 7effae7fc640 1 -- 172.21.15.16:0/1660909070 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7effa80768c0 con 0x7effb806dab0 2023-11-24T06:30:29.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.711+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7effb80af700 con 0x7effb806dab0 2023-11-24T06:30:29.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.712+0000 7effae7fc640 1 -- 172.21.15.16:0/1660909070 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7effb80af700 con 0x7effb806dab0 2023-11-24T06:30:29.715 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBl2seAMxAAL8jbBlU0yStjuIivg2ehGQ==", 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.716 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBlbyW+GxAAy15inYIiNwnGVHj7Ghs5hg==", 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.717 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl79LhCRAALZnwoYyKju4aeWNS+jF31A==", 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-11-24T06:30:29.718 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl/NBDLBAAR4qYgD5THE/yNeZT9XgK5g==", 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD4QWBlarihIxAAczpFVQAODZogIKH0TTdMhw==", 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:30:29.719 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD8QWBlITYFAhAAMlJyCTojY7WZihUegTcBcQ==", 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD/QWBlA/N0DRAAThU0hfAL0+MggufKZC4K8g==", 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-11-24T06:30:29.721 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQACQmBlM7E3JxAAdmfhQHbjK/JGHgeD0urDZw==", 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAKQmBlrM97ARAApYaqyASXJqB5ITNLeBec/w==", 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:30:29.722 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQANQmBl3s5dHBAA3spWOZQiqazGS6isl8ft3Q==", 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.723 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.724 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-11-24T06:30:29.724 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAQQmBlKVmeNhAAuV5j9qZ2NNvEePQx77EM6Q==", 2023-11-24T06:30:29.724 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.724 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:30:29.724 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:30:29.724 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.724 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.725 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.725 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-11-24T06:30:29.725 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAUQmBlwADrBRAAvS3M6+H3X+OQD1h6xwhnBQ==", 2023-11-24T06:30:29.725 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.725 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:30:29.726 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:30:29.726 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.726 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.726 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.726 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-11-24T06:30:29.727 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA==", 2023-11-24T06:30:29.727 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.727 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:30:29.727 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-11-24T06:30:29.727 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:30:29.728 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.728 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.728 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-11-24T06:30:29.728 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBlKhfoFxAA8A+MObA/cjawPN7aclqS9Q==", 2023-11-24T06:30:29.729 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.729 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:30:29.729 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:30:29.729 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:30:29.729 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:30:29.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.730 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.730 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-11-24T06:30:29.730 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBliwu9CxAAPcBaB9CWK2QqRD1k2x+4ig==", 2023-11-24T06:30:29.730 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.730 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:30:29.731 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:30:29.731 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:30:29.731 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:30:29.731 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.732 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.732 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-11-24T06:30:29.732 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBl8HWRIhAAqzUXni0Uqgbmi/imvnNIZg==", 2023-11-24T06:30:29.732 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.732 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:30:29.732 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:30:29.733 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:30:29.733 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:30:29.733 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.733 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.734 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-11-24T06:30:29.734 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDzQWBlo6bBIRAADWG8Peciy2ws4wHagrfFZw==", 2023-11-24T06:30:29.734 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.734 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:30:29.734 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:30:29.734 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlz+KlJhAAXtiyVVA72YrbDKpNSBUnwA==", 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:30:29.735 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlN4GDGhAAF9YIRqu011Xj6oWUv9LYlw==", 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:29.736 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-11-24T06:30:29.737 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:30:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.714+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff9c06b230 msgr2=0x7eff9c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.714+0000 7effbefcf640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff9c06b230 0x7eff9c06d6f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7effb40070b0 tx=0x7effb4007040 comp rx=0 tx=0).stop 2023-11-24T06:30:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.714+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806dab0 msgr2=0x7effb8158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.714+0000 7effbefcf640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806dab0 0x7effb8158260 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7effa8000c00 tx=0x7effa80046c0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.715+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 shutdown_connections 2023-11-24T06:30:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.715+0000 7effbefcf640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7eff9c06b230 0x7eff9c06d6f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.715+0000 7effbefcf640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effb81447f0 0x7effb81587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.715+0000 7effbefcf640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effb806dab0 0x7effb8158260 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.715+0000 7effbefcf640 1 --2- 172.21.15.16:0/1660909070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effb806c470 0x7effb813f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.715+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 >> 172.21.15.16:0/1660909070 conn(0x7effb806a3e0 msgr2=0x7effb815aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:29.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.715+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 shutdown_connections 2023-11-24T06:30:29.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.715+0000 7effbefcf640 1 -- 172.21.15.16:0/1660909070 wait complete. 2023-11-24T06:30:29.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.958+0000 7f113bfff640 1 Processor -- start 2023-11-24T06:30:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.958+0000 7f113bfff640 1 -- start start 2023-11-24T06:30:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f113c143c40 0x7f113c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 0x7f113c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113bfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f113c0cca40 0x7f113c15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113bfff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113c0723c0 con 0x7f113c143c40 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113bfff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113c070e50 con 0x7f113c0cca40 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0cce70 con 0x7f113c0717d0 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f113c143c40 0x7f113c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f113c143c40 0x7f113c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48678/0 (socket says 172.21.15.16:48678) 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113affd640 1 -- 172.21.15.16:0/4058759967 learned_addr learned my addr 172.21.15.16:0/4058759967 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.960+0000 7f113b7fe640 1 --2- 172.21.15.16:0/4058759967 >> v2:172.21.15.173:3300/0 conn(0x7f113c0cca40 0x7f113c15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.961+0000 7f11327fc640 1 --2- 172.21.15.16:0/4058759967 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 0x7f113c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.961+0000 7f113affd640 1 -- 172.21.15.16:0/4058759967 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 msgr2=0x7f113c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.961+0000 7f113affd640 1 --2- 172.21.15.16:0/4058759967 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 0x7f113c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.961+0000 7f113affd640 1 -- 172.21.15.16:0/4058759967 >> v2:172.21.15.173:3300/0 conn(0x7f113c0cca40 msgr2=0x7f113c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.961+0000 7f113affd640 1 --2- 172.21.15.16:0/4058759967 >> v2:172.21.15.173:3300/0 conn(0x7f113c0cca40 0x7f113c15eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.961+0000 7f113affd640 1 -- 172.21.15.16:0/4058759967 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c144020 con 0x7f113c143c40 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.962+0000 7f113affd640 1 --2- 172.21.15.16:0/4058759967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 0x7f113c071290 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f113c0cc3f0 tx=0x7f112c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4486020361c56ac0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113a7fc640 1 -- 172.21.15.16:0/4058759967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c0317e0 con 0x7f113c143c40 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113a7fc640 1 -- 172.21.15.16:0/4058759967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c031940 con 0x7f113c143c40 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113a7fc640 1 -- 172.21.15.16:0/4058759967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c031cd0 con 0x7f113c143c40 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113bfff640 1 -- 172.21.15.16:0/4058759967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 msgr2=0x7f113c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113bfff640 1 --2- 172.21.15.16:0/4058759967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 0x7f113c071290 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f113c0cc3f0 tx=0x7f112c02b880 comp rx=0 tx=0).stop 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113bfff640 1 -- 172.21.15.16:0/4058759967 shutdown_connections 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113bfff640 1 --2- 172.21.15.16:0/4058759967 >> v2:172.21.15.173:3300/0 conn(0x7f113c0cca40 0x7f113c15eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113bfff640 1 --2- 172.21.15.16:0/4058759967 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 0x7f113c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113bfff640 1 --2- 172.21.15.16:0/4058759967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 0x7f113c071290 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113bfff640 1 -- 172.21.15.16:0/4058759967 >> 172.21.15.16:0/4058759967 conn(0x7f113c06a800 msgr2=0x7f113c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113bfff640 1 -- 172.21.15.16:0/4058759967 shutdown_connections 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.963+0000 7f113bfff640 1 -- 172.21.15.16:0/4058759967 wait complete. 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.965+0000 7f113bfff640 1 Processor -- start 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.965+0000 7f113bfff640 1 -- start start 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.966+0000 7f113bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 0x7f113c1381c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.966+0000 7f113bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f113c0cca40 0x7f113c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.966+0000 7f113bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 0x7f113c13add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.966+0000 7f113bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c13eaf0 con 0x7f113c143c40 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.966+0000 7f113bfff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c1388a0 con 0x7f113c0cca40 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.966+0000 7f113bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0b0870 con 0x7f113c0717d0 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 0x7f113c13add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113affd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 0x7f113c1381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113affd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 0x7f113c1381c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:43322/0 (socket says 172.21.15.16:43322) 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113affd640 1 -- 172.21.15.16:0/3291007639 learned_addr learned my addr 172.21.15.16:0/3291007639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 0x7f113c13add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48682/0 (socket says 172.21.15.16:48682) 2023-11-24T06:30:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f11327fc640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f113c0cca40 0x7f113c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113b7fe640 1 -- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 msgr2=0x7f113c1381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113b7fe640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 0x7f113c1381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113b7fe640 1 -- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f113c0cca40 msgr2=0x7f113c0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113b7fe640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f113c0cca40 0x7f113c0adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.967+0000 7f113b7fe640 1 -- 172.21.15.16:0/3291007639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c0094d0 con 0x7f113c143c40 2023-11-24T06:30:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.968+0000 7f113b7fe640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 0x7f113c13add0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1134007dc0 tx=0x7f113400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.968+0000 7f1138ff9640 1 -- 172.21.15.16:0/3291007639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134018070 con 0x7f113c143c40 2023-11-24T06:30:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.968+0000 7f1138ff9640 1 -- 172.21.15.16:0/3291007639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113400edd0 con 0x7f113c143c40 2023-11-24T06:30:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.968+0000 7f1138ff9640 1 -- 172.21.15.16:0/3291007639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134003bb0 con 0x7f113c143c40 2023-11-24T06:30:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.968+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113c0b02b0 con 0x7f113c143c40 2023-11-24T06:30:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.968+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113c13a8b0 con 0x7f113c143c40 2023-11-24T06:30:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.969+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113c15b4c0 con 0x7f113c143c40 2023-11-24T06:30:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.970+0000 7f1138ff9640 1 -- 172.21.15.16:0/3291007639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1134003d10 con 0x7f113c143c40 2023-11-24T06:30:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.971+0000 7f1138ff9640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f112006b230 0x7f112006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.971+0000 7f1138ff9640 1 -- 172.21.15.16:0/3291007639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1134089ba0 con 0x7f113c143c40 2023-11-24T06:30:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.972+0000 7f113affd640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f112006b230 0x7f112006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.973+0000 7f113affd640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f112006b230 0x7f112006d6f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f112c034040 tx=0x7f112c03d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:29.975+0000 7f1138ff9640 1 -- 172.21.15.16:0/3291007639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1134054980 con 0x7f113c143c40 2023-11-24T06:30:30.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.173+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f113c067450 con 0x7f113c143c40 2023-11-24T06:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.174+0000 7f1138ff9640 1 -- 172.21.15.16:0/3291007639 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+12857 (secure 0 0 0) 0x7f1134057fa0 con 0x7f113c143c40 2023-11-24T06:30:30.178 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:30.178 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":67,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:29.208992+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f112006b230 msgr2=0x7f112006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f112006b230 0x7f112006d6f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f112c034040 tx=0x7f112c03d040 comp rx=0 tx=0).stop 2023-11-24T06:30:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 msgr2=0x7f113c13add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 0x7f113c13add0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1134007dc0 tx=0x7f113400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 shutdown_connections 2023-11-24T06:30:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f112006b230 0x7f112006d6f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113c143c40 0x7f113c13add0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f113c0cca40 0x7f113c0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 --2- 172.21.15.16:0/3291007639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f113c0717d0 0x7f113c1381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 >> 172.21.15.16:0/3291007639 conn(0x7f113c06a800 msgr2=0x7f113c142790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.177+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 shutdown_connections 2023-11-24T06:30:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.178+0000 7f113bfff640 1 -- 172.21.15.16:0/3291007639 wait complete. 2023-11-24T06:30:30.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:30:30.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:30:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.418+0000 7f88d6fcb640 1 Processor -- start 2023-11-24T06:30:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.418+0000 7f88d6fcb640 1 -- start start 2023-11-24T06:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.420+0000 7f88d6fcb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88d00714a0 0x7f88d00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.420+0000 7f88d6fcb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d0066430 con 0x7f88d006e5a0 2023-11-24T06:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.420+0000 7f88d6fcb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d006a560 con 0x7f88d0160770 2023-11-24T06:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.420+0000 7f88d6fcb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d006a820 con 0x7f88d00714a0 2023-11-24T06:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.421+0000 7f88d67ca640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f88d006e5a0 0x7f88d0160e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50652/0 (socket says 172.21.15.16:50652) 2023-11-24T06:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.421+0000 7f88d67ca640 1 -- 172.21.15.16:0/1129408097 learned_addr learned my addr 172.21.15.16:0/1129408097 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.421+0000 7f88d57c8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88d00714a0 0x7f88d00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.421+0000 7f88d57c8640 1 -- 172.21.15.16:0/1129408097 >> v1:172.21.15.173:6789/0 conn(0x7f88d0160770 legacy=0x7f88d0070d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.421+0000 7f88d57c8640 1 -- 172.21.15.16:0/1129408097 >> v1:172.21.15.16:6789/0 conn(0x7f88d006e5a0 legacy=0x7f88d0160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.421+0000 7f88d57c8640 1 -- 172.21.15.16:0/1129408097 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d006ab00 con 0x7f88d00714a0 2023-11-24T06:30:30.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.422+0000 7f88d57c8640 1 --2- 172.21.15.16:0/1129408097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 0x7f88d00ac8d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f88c00097a0 tx=0x7f88c002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f233ea6a94109df server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.423+0000 7f88d4fc7640 1 -- 172.21.15.16:0/1129408097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0030610 con 0x7f88d00714a0 2023-11-24T06:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.423+0000 7f88d4fc7640 1 -- 172.21.15.16:0/1129408097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c0030770 con 0x7f88d00714a0 2023-11-24T06:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.423+0000 7f88d4fc7640 1 -- 172.21.15.16:0/1129408097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0030b00 con 0x7f88d00714a0 2023-11-24T06:30:30.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.423+0000 7f88d6fcb640 1 -- 172.21.15.16:0/1129408097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 msgr2=0x7f88d00ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.423+0000 7f88d6fcb640 1 --2- 172.21.15.16:0/1129408097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 0x7f88d00ac8d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f88c00097a0 tx=0x7f88c002b960 comp rx=0 tx=0).stop 2023-11-24T06:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.424+0000 7f88d6fcb640 1 -- 172.21.15.16:0/1129408097 shutdown_connections 2023-11-24T06:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.424+0000 7f88d6fcb640 1 --2- 172.21.15.16:0/1129408097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 0x7f88d00ac8d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.424+0000 7f88d6fcb640 1 -- 172.21.15.16:0/1129408097 >> 172.21.15.16:0/1129408097 conn(0x7f88d00b15b0 msgr2=0x7f88d00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.424+0000 7f88d6fcb640 1 -- 172.21.15.16:0/1129408097 shutdown_connections 2023-11-24T06:30:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.424+0000 7f88d6fcb640 1 -- 172.21.15.16:0/1129408097 wait complete. 2023-11-24T06:30:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.425+0000 7f88d6fcb640 1 Processor -- start 2023-11-24T06:30:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.426+0000 7f88d6fcb640 1 -- start start 2023-11-24T06:30:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d6fcb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88d006e5a0 0x7f88d01e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d6fcb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 0x7f88d0159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d6fcb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88d0160770 0x7f88d01417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d6fcb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0159560 con 0x7f88d00714a0 2023-11-24T06:30:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d6fcb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01e9730 con 0x7f88d0160770 2023-11-24T06:30:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d6fcb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0152a20 con 0x7f88d006e5a0 2023-11-24T06:30:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d57c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 0x7f88d0159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d57c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 0x7f88d0159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48694/0 (socket says 172.21.15.16:48694) 2023-11-24T06:30:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d57c8640 1 -- 172.21.15.16:0/430560069 learned_addr learned my addr 172.21.15.16:0/430560069 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d5fc9640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88d006e5a0 0x7f88d01e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d67ca640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88d0160770 0x7f88d01417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d57c8640 1 -- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88d006e5a0 msgr2=0x7f88d01e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d57c8640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88d006e5a0 0x7f88d01e91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d57c8640 1 -- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88d0160770 msgr2=0x7f88d01417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d57c8640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88d0160770 0x7f88d01417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d57c8640 1 -- 172.21.15.16:0/430560069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c0009400 con 0x7f88d00714a0 2023-11-24T06:30:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.427+0000 7f88d5fc9640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88d006e5a0 0x7f88d01e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.428+0000 7f88d57c8640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 0x7f88d0159980 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009890 tx=0x7f88c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.428+0000 7f88c67fc640 1 -- 172.21.15.16:0/430560069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c003e040 con 0x7f88d00714a0 2023-11-24T06:30:30.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.428+0000 7f88c67fc640 1 -- 172.21.15.16:0/430560069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c0039ce0 con 0x7f88d00714a0 2023-11-24T06:30:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.428+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d0158ea0 con 0x7f88d00714a0 2023-11-24T06:30:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.428+0000 7f88c67fc640 1 -- 172.21.15.16:0/430560069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0043680 con 0x7f88d00714a0 2023-11-24T06:30:30.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.428+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d0159ec0 con 0x7f88d00714a0 2023-11-24T06:30:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.429+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d0158d00 con 0x7f88d00714a0 2023-11-24T06:30:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.431+0000 7f88c67fc640 1 -- 172.21.15.16:0/430560069 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f88c00423e0 con 0x7f88d00714a0 2023-11-24T06:30:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.432+0000 7f88c67fc640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f88b406b260 0x7f88b406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.432+0000 7f88c67fc640 1 -- 172.21.15.16:0/430560069 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f88c00abee0 con 0x7f88d00714a0 2023-11-24T06:30:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.433+0000 7f88d5fc9640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f88b406b260 0x7f88b406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.434+0000 7f88d5fc9640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f88b406b260 0x7f88b406d720 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f88cc000f60 tx=0x7f88cc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:30.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.436+0000 7f88c67fc640 1 -- 172.21.15.16:0/430560069 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88d0158d00 con 0x7f88d00714a0 2023-11-24T06:30:30.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.626+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88d0158d00 con 0x7f88d00714a0 2023-11-24T06:30:30.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.627+0000 7f88c67fc640 1 -- 172.21.15.16:0/430560069 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88d0158d00 con 0x7f88d00714a0 2023-11-24T06:30:30.631 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:30:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f88b406b260 msgr2=0x7f88b406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f88b406b260 0x7f88b406d720 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f88cc000f60 tx=0x7f88cc009040 comp rx=0 tx=0).stop 2023-11-24T06:30:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 msgr2=0x7f88d0159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 0x7f88d0159980 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009890 tx=0x7f88c0004730 comp rx=0 tx=0).stop 2023-11-24T06:30:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 shutdown_connections 2023-11-24T06:30:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f88b406b260 0x7f88b406d720 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88d0160770 0x7f88d01417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d00714a0 0x7f88d0159980 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 --2- 172.21.15.16:0/430560069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88d006e5a0 0x7f88d01e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 >> 172.21.15.16:0/430560069 conn(0x7f88d00b15b0 msgr2=0x7f88d00b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 shutdown_connections 2023-11-24T06:30:30.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.630+0000 7f88d6fcb640 1 -- 172.21.15.16:0/430560069 wait complete. 2023-11-24T06:30:30.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:30:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.876+0000 7f069b59e640 1 Processor -- start 2023-11-24T06:30:30.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.876+0000 7f069b59e640 1 -- start start 2023-11-24T06:30:30.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.877+0000 7f069b59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f069c15d590 0x7f069c0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.877+0000 7f069b59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c0c7110 con 0x7f069c0c6750 2023-11-24T06:30:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.877+0000 7f069b59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c0c6bb0 con 0x7f069c09a170 2023-11-24T06:30:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.877+0000 7f069b59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0c6e30 con 0x7f069c15d590 2023-11-24T06:30:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.878+0000 7f0699d9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f069c0c6750 0x7f069c099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50658/0 (socket says 172.21.15.16:50658) 2023-11-24T06:30:30.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.878+0000 7f0699d9b640 1 -- 172.21.15.16:0/4021037592 learned_addr learned my addr 172.21.15.16:0/4021037592 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.878+0000 7f069a59c640 1 --2- 172.21.15.16:0/4021037592 >> v2:172.21.15.173:3300/0 conn(0x7f069c15d590 0x7f069c0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.878+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486704232 0 0) 0x7f069c0c7110 con 0x7f069c0c6750 2023-11-24T06:30:30.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0688003520 con 0x7f069c0c6750 2023-11-24T06:30:30.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1067029895 0 0) 0x7f0688003520 con 0x7f069c0c6750 2023-11-24T06:30:30.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 >> v2:172.21.15.173:3300/0 conn(0x7f069c15d590 msgr2=0x7f069c0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 --2- 172.21.15.16:0/4021037592 >> v2:172.21.15.173:3300/0 conn(0x7f069c15d590 0x7f069c0d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 >> v1:172.21.15.173:6789/0 conn(0x7f069c09a170 legacy=0x7f069c0aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0adf30 con 0x7f069c0c6750 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f0684001670 con 0x7f069c0c6750 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 >> v1:172.21.15.16:6789/0 conn(0x7f069c0c6750 legacy=0x7f069c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06880039c0 0x7f0688003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06880047c0 0x7f0688004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f068800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0c6e30 con 0x7f0688005180 2023-11-24T06:30:30.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068800c080 con 0x7f06880047c0 2023-11-24T06:30:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068800c240 con 0x7f06880039c0 2023-11-24T06:30:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.879+0000 7f069ad9d640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f068800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.880+0000 7f069a59c640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06880039c0 0x7f0688003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.880+0000 7f0699d9b640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06880047c0 0x7f0688004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.880+0000 7f069ad9d640 1 -- 172.21.15.16:0/4021037592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06880039c0 msgr2=0x7f0688003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.880+0000 7f069ad9d640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f06880039c0 0x7f0688003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.880+0000 7f069ad9d640 1 -- 172.21.15.16:0/4021037592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06880047c0 msgr2=0x7f0688004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.880+0000 7f069ad9d640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06880047c0 0x7f0688004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.880+0000 7f069ad9d640 1 -- 172.21.15.16:0/4021037592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0688003520 con 0x7f0688005180 2023-11-24T06:30:30.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.880+0000 7f0699d9b640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06880047c0 0x7f0688004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.881+0000 7f069ad9d640 1 -- 172.21.15.16:0/4021037592 reap_dead start 2023-11-24T06:30:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.881+0000 7f069ad9d640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f068800b980 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f069c0718a0 tx=0x7f06880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b22b1a4790730c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.881+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694003b70 con 0x7f0688005180 2023-11-24T06:30:30.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.881+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694003cd0 con 0x7f0688005180 2023-11-24T06:30:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.881+0000 7f069959a640 1 -- 172.21.15.16:0/4021037592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694005310 con 0x7f0688005180 2023-11-24T06:30:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.882+0000 7f069b59e640 1 -- 172.21.15.16:0/4021037592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 msgr2=0x7f068800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.882+0000 7f069b59e640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f068800b980 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f069c0718a0 tx=0x7f06880057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.882+0000 7f069b59e640 1 -- 172.21.15.16:0/4021037592 shutdown_connections 2023-11-24T06:30:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.882+0000 7f069b59e640 1 --2- 172.21.15.16:0/4021037592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f068800b980 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.882+0000 7f069b59e640 1 -- 172.21.15.16:0/4021037592 >> 172.21.15.16:0/4021037592 conn(0x7f069c0d0e80 msgr2=0x7f069c0d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.882+0000 7f069b59e640 1 -- 172.21.15.16:0/4021037592 shutdown_connections 2023-11-24T06:30:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.882+0000 7f069b59e640 1 -- 172.21.15.16:0/4021037592 wait complete. 2023-11-24T06:30:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.884+0000 7f069b59e640 1 Processor -- start 2023-11-24T06:30:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.884+0000 7f069b59e640 1 -- start start 2023-11-24T06:30:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.885+0000 7f069b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f069c143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.885+0000 7f069b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f069c160070 0x7f069c1604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.885+0000 7f069b59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f069c151dd0 0x7f069c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.885+0000 7f069b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c153b50 con 0x7f0688005180 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.885+0000 7f069b59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c160a30 con 0x7f069c151dd0 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.885+0000 7f069b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c1536c0 con 0x7f069c160070 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.885+0000 7f069a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f069c143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.885+0000 7f069a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f069c143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48704/0 (socket says 172.21.15.16:48704) 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f069a59c640 1 -- 172.21.15.16:0/2136068906 learned_addr learned my addr 172.21.15.16:0/2136068906 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f069a59c640 1 -- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f069c160070 msgr2=0x7f069c1604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f069a59c640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f069c160070 0x7f069c1604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f069a59c640 1 -- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f069c151dd0 msgr2=0x7f069c152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f069a59c640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f069c151dd0 0x7f069c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f069a59c640 1 -- 172.21.15.16:0/2136068906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c1504e0 con 0x7f0688005180 2023-11-24T06:30:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f069a59c640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f069c143c90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f068c011540 tx=0x7f068c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f0692ffd640 1 -- 172.21.15.16:0/2136068906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c015930 con 0x7f0688005180 2023-11-24T06:30:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f0692ffd640 1 -- 172.21.15.16:0/2136068906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c015a90 con 0x7f0688005180 2023-11-24T06:30:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.886+0000 7f0692ffd640 1 -- 172.21.15.16:0/2136068906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c015e60 con 0x7f0688005180 2023-11-24T06:30:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.887+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c1441d0 con 0x7f0688005180 2023-11-24T06:30:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.887+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c152d90 con 0x7f0688005180 2023-11-24T06:30:30.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.888+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c067b60 con 0x7f0688005180 2023-11-24T06:30:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.889+0000 7f0692ffd640 1 -- 172.21.15.16:0/2136068906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f068c00f460 con 0x7f0688005180 2023-11-24T06:30:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.890+0000 7f0692ffd640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f067c06b230 0x7f067c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.890+0000 7f0692ffd640 1 -- 172.21.15.16:0/2136068906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f068c08a260 con 0x7f0688005180 2023-11-24T06:30:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.890+0000 7f0699d9b640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f067c06b230 0x7f067c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.892+0000 7f0699d9b640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f067c06b230 0x7f067c06d6f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0684000c00 tx=0x7f0684009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:30.894+0000 7f0692ffd640 1 -- 172.21.15.16:0/2136068906 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f068c055040 con 0x7f0688005180 2023-11-24T06:30:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.090+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f069c0665e0 con 0x7f0688005180 2023-11-24T06:30:31.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.091+0000 7f0692ffd640 1 -- 172.21.15.16:0/2136068906 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) v1 ==== 84+0+65 (secure 0 0 0) 0x7f068c058660 con 0x7f0688005180 2023-11-24T06:30:31.095 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:31.095 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:30:31.095 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:31.095 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-24T06:30:31.095 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-24T06:30:31.095 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:31.095 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:30:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f067c06b230 msgr2=0x7f067c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f067c06b230 0x7f067c06d6f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0684000c00 tx=0x7f0684009040 comp rx=0 tx=0).stop 2023-11-24T06:30:31.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 msgr2=0x7f069c143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f069c143c90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f068c011540 tx=0x7f068c011a00 comp rx=0 tx=0).stop 2023-11-24T06:30:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 shutdown_connections 2023-11-24T06:30:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f067c06b230 0x7f067c06d6f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f069c151dd0 0x7f069c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f069c160070 0x7f069c1604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 --2- 172.21.15.16:0/2136068906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0688005180 0x7f069c143c90 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 >> 172.21.15.16:0/2136068906 conn(0x7f069c0d0e80 msgr2=0x7f069c157650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 shutdown_connections 2023-11-24T06:30:31.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.094+0000 7f069b59e640 1 -- 172.21.15.16:0/2136068906 wait complete. 2023-11-24T06:30:31.116 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:30:31.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:30:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.352+0000 7f48fa87e640 1 Processor -- start 2023-11-24T06:30:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.352+0000 7f48fa87e640 1 -- start start 2023-11-24T06:30:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.353+0000 7f48fa87e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f48f4071410 0x7f48f4146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.354+0000 7f48fa87e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f40b2e10 con 0x7f48f4071f00 2023-11-24T06:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.354+0000 7f48fa87e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f406cc50 con 0x7f48f406c7c0 2023-11-24T06:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.354+0000 7f48fa87e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f406cec0 con 0x7f48f4071410 2023-11-24T06:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.354+0000 7f48fa07d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f48f4071f00 0x7f48f4166cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50672/0 (socket says 172.21.15.16:50672) 2023-11-24T06:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.354+0000 7f48fa07d640 1 -- 172.21.15.16:0/337650253 learned_addr learned my addr 172.21.15.16:0/337650253 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.354+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805091766 0 0) 0x7f48f40b2e10 con 0x7f48f4071f00 2023-11-24T06:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.354+0000 7f48f987c640 1 --2- 172.21.15.16:0/337650253 >> v2:172.21.15.173:3300/0 conn(0x7f48f4071410 0x7f48f4146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.354+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e0003520 con 0x7f48f4071f00 2023-11-24T06:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3687942198 0 0) 0x7f48e0003520 con 0x7f48f4071f00 2023-11-24T06:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 >> v2:172.21.15.173:3300/0 conn(0x7f48f4071410 msgr2=0x7f48f4146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 --2- 172.21.15.16:0/337650253 >> v2:172.21.15.173:3300/0 conn(0x7f48f4071410 0x7f48f4146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 >> v1:172.21.15.173:6789/0 conn(0x7f48f406c7c0 legacy=0x7f48f40717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f406d1a0 con 0x7f48f4071f00 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f48ec0019f0 con 0x7f48f4071f00 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 >> v1:172.21.15.16:6789/0 conn(0x7f48f4071f00 legacy=0x7f48f4166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 --2- 172.21.15.16:0/337650253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48e0003970 0x7f48e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 --2- 172.21.15.16:0/337650253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48e00047c0 0x7f48e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 --2- 172.21.15.16:0/337650253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0005180 0x7f48e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f406cec0 con 0x7f48e0003970 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e000c020 con 0x7f48e00047c0 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e000c1e0 con 0x7f48e0005180 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.355+0000 7f48f987c640 1 --2- 172.21.15.16:0/337650253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48e0003970 0x7f48e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.356+0000 7f48f987c640 1 -- 172.21.15.16:0/337650253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0005180 msgr2=0x7f48e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.356+0000 7f48f987c640 1 --2- 172.21.15.16:0/337650253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0005180 0x7f48e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.356+0000 7f48f987c640 1 -- 172.21.15.16:0/337650253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48e00047c0 msgr2=0x7f48e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:31.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.356+0000 7f48f987c640 1 --2- 172.21.15.16:0/337650253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48e00047c0 0x7f48e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.356+0000 7f48f987c640 1 -- 172.21.15.16:0/337650253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e0003520 con 0x7f48e0003970 2023-11-24T06:30:31.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.356+0000 7f48fa07d640 1 -- 172.21.15.16:0/337650253 reap_dead start 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.357+0000 7f48f987c640 1 --2- 172.21.15.16:0/337650253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48e0003970 0x7f48e0003d90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009ac0 tx=0x7f48e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70e80dbf6253044 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.357+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f002c6b0 con 0x7f48e0003970 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.357+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f002c810 con 0x7f48e0003970 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.357+0000 7f48f887a640 1 -- 172.21.15.16:0/337650253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f002cba0 con 0x7f48e0003970 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.358+0000 7f48fa87e640 1 -- 172.21.15.16:0/337650253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48e0003970 msgr2=0x7f48e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.358+0000 7f48fa87e640 1 --2- 172.21.15.16:0/337650253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48e0003970 0x7f48e0003d90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009ac0 tx=0x7f48e00057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.358+0000 7f48fa87e640 1 -- 172.21.15.16:0/337650253 shutdown_connections 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.358+0000 7f48fa87e640 1 --2- 172.21.15.16:0/337650253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48e0003970 0x7f48e0003d90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.358+0000 7f48fa87e640 1 -- 172.21.15.16:0/337650253 >> 172.21.15.16:0/337650253 conn(0x7f48f406a800 msgr2=0x7f48f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.358+0000 7f48fa87e640 1 -- 172.21.15.16:0/337650253 shutdown_connections 2023-11-24T06:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.358+0000 7f48fa87e640 1 -- 172.21.15.16:0/337650253 wait complete. 2023-11-24T06:30:31.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.360+0000 7f48fa87e640 1 Processor -- start 2023-11-24T06:30:31.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.360+0000 7f48fa87e640 1 -- start start 2023-11-24T06:30:31.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.361+0000 7f48fa87e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0003970 0x7f48f415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:31.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.361+0000 7f48fa87e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40c6750 0x7f48f40c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:31.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.361+0000 7f48fa87e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48f4144250 0x7f48f4144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:31.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.361+0000 7f48fa87e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c8740 con 0x7f48f40c6750 2023-11-24T06:30:31.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.361+0000 7f48fa87e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f40c7110 con 0x7f48f4144250 2023-11-24T06:30:31.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.361+0000 7f48fa87e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f4144fb0 con 0x7f48e0003970 2023-11-24T06:30:31.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.362+0000 7f48f907b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40c6750 0x7f48f40c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.362+0000 7f48f907b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40c6750 0x7f48f40c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48720/0 (socket says 172.21.15.16:48720) 2023-11-24T06:30:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.362+0000 7f48f907b640 1 -- 172.21.15.16:0/873707127 learned_addr learned my addr 172.21.15.16:0/873707127 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.362+0000 7f48f987c640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0003970 0x7f48f415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.362+0000 7f48f907b640 1 -- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0003970 msgr2=0x7f48f415c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.362+0000 7f48f907b640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0003970 0x7f48f415c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.362+0000 7f48f907b640 1 -- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48f4144250 msgr2=0x7f48f4144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.362+0000 7f48f907b640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48f4144250 0x7f48f4144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.362+0000 7f48f907b640 1 -- 172.21.15.16:0/873707127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40c8950 con 0x7f48f40c6750 2023-11-24T06:30:31.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.363+0000 7f48f907b640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40c6750 0x7f48f40c6bd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c530 tx=0x7f48e400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.363+0000 7f48ea7fc640 1 -- 172.21.15.16:0/873707127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e40052d0 con 0x7f48f40c6750 2023-11-24T06:30:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.363+0000 7f48ea7fc640 1 -- 172.21.15.16:0/873707127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e4005430 con 0x7f48f40c6750 2023-11-24T06:30:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.363+0000 7f48ea7fc640 1 -- 172.21.15.16:0/873707127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4012870 con 0x7f48f40c6750 2023-11-24T06:30:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.363+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f40c8c30 con 0x7f48f40c6750 2023-11-24T06:30:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.363+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f40c91f0 con 0x7f48f40c6750 2023-11-24T06:30:31.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.364+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f40c93b0 con 0x7f48f40c6750 2023-11-24T06:30:31.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.365+0000 7f48ea7fc640 1 -- 172.21.15.16:0/873707127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f48e40129d0 con 0x7f48f40c6750 2023-11-24T06:30:31.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.366+0000 7f48ea7fc640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d806b260 0x7f48d806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:31.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.366+0000 7f48ea7fc640 1 -- 172.21.15.16:0/873707127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f48e40894f0 con 0x7f48f40c6750 2023-11-24T06:30:31.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.367+0000 7f48f987c640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d806b260 0x7f48d806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:31.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.368+0000 7f48f987c640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d806b260 0x7f48d806d720 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f48f0036000 tx=0x7f48f0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:31.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.370+0000 7f48ea7fc640 1 -- 172.21.15.16:0/873707127 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48e4057df0 con 0x7f48f40c6750 2023-11-24T06:30:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:31.576+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f48f4067450 con 0x7f48f40c6750 2023-11-24T06:30:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.209+0000 7f48ea7fc640 1 -- 172.21.15.16:0/873707127 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) v1 ==== 138+0+0 (secure 0 0 0) 0x7f48e4057980 con 0x7f48f40c6750 2023-11-24T06:30:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-24T06:30:32.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d806b260 msgr2=0x7f48d806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:32.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d806b260 0x7f48d806d720 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f48f0036000 tx=0x7f48f0030000 comp rx=0 tx=0).stop 2023-11-24T06:30:32.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40c6750 msgr2=0x7f48f40c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40c6750 0x7f48f40c6bd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c530 tx=0x7f48e400c9f0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 shutdown_connections 2023-11-24T06:30:32.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d806b260 0x7f48d806d720 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48f4144250 0x7f48f4144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f40c6750 0x7f48f40c6bd0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 --2- 172.21.15.16:0/873707127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0003970 0x7f48f415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 >> 172.21.15.16:0/873707127 conn(0x7f48f406a800 msgr2=0x7f48f4142ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 shutdown_connections 2023-11-24T06:30:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.213+0000 7f48fa87e640 1 -- 172.21.15.16:0/873707127 wait complete. 2023-11-24T06:30:32.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:30:32.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.460+0000 7fea58968640 1 Processor -- start 2023-11-24T06:30:32.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.461+0000 7fea58968640 1 -- start start 2023-11-24T06:30:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.462+0000 7fea58968640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea54141360 0x7fea540cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.462+0000 7fea58968640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea540cdcc0 0x7fea5406c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.462+0000 7fea58968640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5413e440 con 0x7fea5415ed20 2023-11-24T06:30:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.462+0000 7fea58968640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea541417a0 con 0x7fea540cdcc0 2023-11-24T06:30:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.463+0000 7fea58968640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea540ce1d0 con 0x7fea54141360 2023-11-24T06:30:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.463+0000 7fea52d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea540cdcc0 0x7fea5406c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.463+0000 7fea52d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea540cdcc0 0x7fea5406c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48722/0 (socket says 172.21.15.16:48722) 2023-11-24T06:30:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.463+0000 7fea52d9d640 1 -- 172.21.15.16:0/3909614689 learned_addr learned my addr 172.21.15.16:0/3909614689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.463+0000 7fea51d9b640 1 --2- 172.21.15.16:0/3909614689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea54141360 0x7fea540cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.463+0000 7fea52d9d640 1 -- 172.21.15.16:0/3909614689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea54141360 msgr2=0x7fea540cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.463+0000 7fea52d9d640 1 --2- 172.21.15.16:0/3909614689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea54141360 0x7fea540cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.464+0000 7fea52d9d640 1 -- 172.21.15.16:0/3909614689 >> v1:172.21.15.173:6789/0 conn(0x7fea5415ed20 legacy=0x7fea5415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.464+0000 7fea52d9d640 1 -- 172.21.15.16:0/3909614689 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea540ce4b0 con 0x7fea540cdcc0 2023-11-24T06:30:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.464+0000 7fea51d9b640 1 --2- 172.21.15.16:0/3909614689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea54141360 0x7fea540cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:30:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.465+0000 7fea52d9d640 1 --2- 172.21.15.16:0/3909614689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea540cdcc0 0x7fea5406c850 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009870 tx=0x7fea4c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a986d5b103f2cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.465+0000 7fea5159a640 1 -- 172.21.15.16:0/3909614689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c031640 con 0x7fea540cdcc0 2023-11-24T06:30:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.465+0000 7fea5159a640 1 -- 172.21.15.16:0/3909614689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c0317a0 con 0x7fea540cdcc0 2023-11-24T06:30:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.465+0000 7fea5159a640 1 -- 172.21.15.16:0/3909614689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c031b30 con 0x7fea540cdcc0 2023-11-24T06:30:32.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.466+0000 7fea58968640 1 -- 172.21.15.16:0/3909614689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea540cdcc0 msgr2=0x7fea5406c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:32.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.466+0000 7fea58968640 1 --2- 172.21.15.16:0/3909614689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea540cdcc0 0x7fea5406c850 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009870 tx=0x7fea4c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.466+0000 7fea58968640 1 -- 172.21.15.16:0/3909614689 shutdown_connections 2023-11-24T06:30:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.466+0000 7fea58968640 1 --2- 172.21.15.16:0/3909614689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea540cdcc0 0x7fea5406c850 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.466+0000 7fea58968640 1 --2- 172.21.15.16:0/3909614689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea54141360 0x7fea540cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.466+0000 7fea58968640 1 -- 172.21.15.16:0/3909614689 >> 172.21.15.16:0/3909614689 conn(0x7fea540b0be0 msgr2=0x7fea540b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.466+0000 7fea58968640 1 -- 172.21.15.16:0/3909614689 shutdown_connections 2023-11-24T06:30:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.466+0000 7fea58968640 1 -- 172.21.15.16:0/3909614689 wait complete. 2023-11-24T06:30:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.468+0000 7fea58968640 1 Processor -- start 2023-11-24T06:30:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.468+0000 7fea58968640 1 -- start start 2023-11-24T06:30:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea58968640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea540cdcc0 0x7fea5413bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea58968640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea54141360 0x7fea541511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea58968640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea5415ed20 0x7fea540aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea58968640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea540725a0 con 0x7fea54141360 2023-11-24T06:30:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea58968640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54071a20 con 0x7fea5415ed20 2023-11-24T06:30:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea58968640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea54071d00 con 0x7fea540cdcc0 2023-11-24T06:30:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea51d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea54141360 0x7fea541511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:32.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea51d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea54141360 0x7fea541511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48730/0 (socket says 172.21.15.16:48730) 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea51d9b640 1 -- 172.21.15.16:0/3067940418 learned_addr learned my addr 172.21.15.16:0/3067940418 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea5259c640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea540cdcc0 0x7fea5413bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea51d9b640 1 -- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea540cdcc0 msgr2=0x7fea5413bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea51d9b640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea540cdcc0 0x7fea5413bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea51d9b640 1 -- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea5415ed20 msgr2=0x7fea540aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.470+0000 7fea52d9d640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea5415ed20 0x7fea540aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea51d9b640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea5415ed20 0x7fea540aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea51d9b640 1 -- 172.21.15.16:0/3067940418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c0094d0 con 0x7fea54141360 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea5259c640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea540cdcc0 0x7fea5413bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:32.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea52d9d640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea5415ed20 0x7fea540aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea51d9b640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea54141360 0x7fea541511a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e7e0 tx=0x7fea3c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea4affd640 1 -- 172.21.15.16:0/3067940418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c004030 con 0x7fea54141360 2023-11-24T06:30:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea4affd640 1 -- 172.21.15.16:0/3067940418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c004190 con 0x7fea54141360 2023-11-24T06:30:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea4affd640 1 -- 172.21.15.16:0/3067940418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c004520 con 0x7fea54141360 2023-11-24T06:30:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea54071fe0 con 0x7fea54141360 2023-11-24T06:30:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.471+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea540ae170 con 0x7fea54141360 2023-11-24T06:30:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.472+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea54067670 con 0x7fea54141360 2023-11-24T06:30:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.475+0000 7fea4affd640 1 -- 172.21.15.16:0/3067940418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fea3c00c6d0 con 0x7fea54141360 2023-11-24T06:30:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.476+0000 7fea4affd640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3406b230 0x7fea3406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.476+0000 7fea4affd640 1 -- 172.21.15.16:0/3067940418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fea3c089cb0 con 0x7fea54141360 2023-11-24T06:30:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.477+0000 7fea5259c640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3406b230 0x7fea3406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:32.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.478+0000 7fea5259c640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3406b230 0x7fea3406d6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea440012e0 tx=0x7fea44009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:32.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.480+0000 7fea4affd640 1 -- 172.21.15.16:0/3067940418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea3c054950 con 0x7fea54141360 2023-11-24T06:30:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:32.677+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fea541405e0 con 0x7fea54141360 2023-11-24T06:30:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.222+0000 7fea4affd640 1 -- 172.21.15.16:0/3067940418 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) v1 ==== 170+0+0 (secure 0 0 0) 0x7fea3c057f70 con 0x7fea54141360 2023-11-24T06:30:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-24T06:30:33.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.225+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3406b230 msgr2=0x7fea3406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:33.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.225+0000 7fea58968640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3406b230 0x7fea3406d6f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea440012e0 tx=0x7fea44009040 comp rx=0 tx=0).stop 2023-11-24T06:30:33.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.225+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea54141360 msgr2=0x7fea541511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.225+0000 7fea58968640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea54141360 0x7fea541511a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00e7e0 tx=0x7fea3c00eca0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.226+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 shutdown_connections 2023-11-24T06:30:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.226+0000 7fea58968640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea3406b230 0x7fea3406d6f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.226+0000 7fea58968640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea5415ed20 0x7fea540aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.226+0000 7fea58968640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea54141360 0x7fea541511a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.226+0000 7fea58968640 1 --2- 172.21.15.16:0/3067940418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea540cdcc0 0x7fea5413bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.226+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 >> 172.21.15.16:0/3067940418 conn(0x7fea540b0be0 msgr2=0x7fea5406d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.226+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 shutdown_connections 2023-11-24T06:30:33.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.226+0000 7fea58968640 1 -- 172.21.15.16:0/3067940418 wait complete. 2023-11-24T06:30:33.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:30:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.455+0000 7f0f3b7ce640 1 Processor -- start 2023-11-24T06:30:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.456+0000 7f0f3b7ce640 1 -- start start 2023-11-24T06:30:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.457+0000 7f0f3b7ce640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3413df80 0x7f0f3415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.457+0000 7f0f3b7ce640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0f3413e360 0x7f0f340cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.457+0000 7f0f3b7ce640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f3406c470 con 0x7f0f3415f9c0 2023-11-24T06:30:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.457+0000 7f0f3b7ce640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3406c750 con 0x7f0f3413e360 2023-11-24T06:30:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.457+0000 7f0f3b7ce640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3406ca30 con 0x7f0f3413df80 2023-11-24T06:30:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.458+0000 7f0f3956a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3413df80 0x7f0f3415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.458+0000 7f0f39d6b640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f0f3415f9c0 0x7f0f341408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:33760/0 (socket says 172.21.15.16:33760) 2023-11-24T06:30:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.458+0000 7f0f39d6b640 1 -- 172.21.15.16:0/3402192903 learned_addr learned my addr 172.21.15.16:0/3402192903 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.458+0000 7f0f38d69640 1 --2- 172.21.15.16:0/3402192903 >> v2:172.21.15.173:3300/0 conn(0x7f0f3413e360 0x7f0f340cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.459+0000 7f0f38d69640 1 -- 172.21.15.16:0/3402192903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3413df80 msgr2=0x7f0f3415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.459+0000 7f0f38d69640 1 --2- 172.21.15.16:0/3402192903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3413df80 0x7f0f3415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.459+0000 7f0f38d69640 1 -- 172.21.15.16:0/3402192903 >> v1:172.21.15.173:6789/0 conn(0x7f0f3415f9c0 legacy=0x7f0f341408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.459+0000 7f0f38d69640 1 -- 172.21.15.16:0/3402192903 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3406cd10 con 0x7f0f3413e360 2023-11-24T06:30:33.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.460+0000 7f0f38d69640 1 --2- 172.21.15.16:0/3402192903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 0x7f0f340cf290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0f240097a0 tx=0x7f0f2402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6a4efd51294de0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.461+0000 7f0f2bfff640 1 -- 172.21.15.16:0/3402192903 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2402d040 con 0x7f0f3413e360 2023-11-24T06:30:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.461+0000 7f0f2bfff640 1 -- 172.21.15.16:0/3402192903 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f24031810 con 0x7f0f3413e360 2023-11-24T06:30:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.461+0000 7f0f2bfff640 1 -- 172.21.15.16:0/3402192903 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24031ba0 con 0x7f0f3413e360 2023-11-24T06:30:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.461+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/3402192903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 msgr2=0x7f0f340cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.461+0000 7f0f3b7ce640 1 --2- 172.21.15.16:0/3402192903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 0x7f0f340cf290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0f240097a0 tx=0x7f0f2402b8b0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.461+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/3402192903 shutdown_connections 2023-11-24T06:30:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.461+0000 7f0f3b7ce640 1 --2- 172.21.15.16:0/3402192903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 0x7f0f340cf290 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.461+0000 7f0f3b7ce640 1 --2- 172.21.15.16:0/3402192903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3413df80 0x7f0f3415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.461+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/3402192903 >> 172.21.15.16:0/3402192903 conn(0x7f0f3406a800 msgr2=0x7f0f3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.462+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/3402192903 shutdown_connections 2023-11-24T06:30:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.462+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/3402192903 wait complete. 2023-11-24T06:30:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.463+0000 7f0f3b7ce640 1 Processor -- start 2023-11-24T06:30:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.464+0000 7f0f3b7ce640 1 -- start start 2023-11-24T06:30:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.464+0000 7f0f3b7ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f3413df80 0x7f0f34132180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.464+0000 7f0f3b7ce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 0x7f0f3414d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3b7ce640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3415f9c0 0x7f0f3414d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3b7ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f34150050 con 0x7f0f3413df80 2023-11-24T06:30:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3b7ce640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f34150330 con 0x7f0f3413e360 2023-11-24T06:30:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3b7ce640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3415c9f0 con 0x7f0f3415f9c0 2023-11-24T06:30:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3956a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f3413df80 0x7f0f34132180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3956a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f3413df80 0x7f0f34132180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36322/0 (socket says 172.21.15.16:36322) 2023-11-24T06:30:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3956a640 1 -- 172.21.15.16:0/707191897 learned_addr learned my addr 172.21.15.16:0/707191897 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f39d6b640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3415f9c0 0x7f0f3414d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f38d69640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 0x7f0f3414d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3956a640 1 -- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3415f9c0 msgr2=0x7f0f3414d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3956a640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3415f9c0 0x7f0f3414d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3956a640 1 -- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 msgr2=0x7f0f3414d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3956a640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 0x7f0f3414d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f3956a640 1 -- 172.21.15.16:0/707191897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f24009400 con 0x7f0f3413df80 2023-11-24T06:30:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.465+0000 7f0f38d69640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 0x7f0f3414d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.466+0000 7f0f3956a640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f3413df80 0x7f0f34132180 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000e870 tx=0x7f0f3000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.466+0000 7f0f29ffb640 1 -- 172.21.15.16:0/707191897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f300094f0 con 0x7f0f3413df80 2023-11-24T06:30:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.466+0000 7f0f29ffb640 1 -- 172.21.15.16:0/707191897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3000c940 con 0x7f0f3413df80 2023-11-24T06:30:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.466+0000 7f0f29ffb640 1 -- 172.21.15.16:0/707191897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000ccf0 con 0x7f0f3413df80 2023-11-24T06:30:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.466+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3415ccd0 con 0x7f0f3413df80 2023-11-24T06:30:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.466+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3414f290 con 0x7f0f3413df80 2023-11-24T06:30:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.468+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3415c850 con 0x7f0f3413df80 2023-11-24T06:30:33.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.469+0000 7f0f29ffb640 1 -- 172.21.15.16:0/707191897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f0f30013470 con 0x7f0f3413df80 2023-11-24T06:30:33.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.470+0000 7f0f29ffb640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f1806b230 0x7f0f1806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:33.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.470+0000 7f0f29ffb640 1 -- 172.21.15.16:0/707191897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0f3008a3e0 con 0x7f0f3413df80 2023-11-24T06:30:33.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.471+0000 7f0f38d69640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f1806b230 0x7f0f1806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:33.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.472+0000 7f0f38d69640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f1806b230 0x7f0f1806d6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0f24036000 tx=0x7f0f2403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.474+0000 7f0f29ffb640 1 -- 172.21.15.16:0/707191897 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f30055010 con 0x7f0f3413df80 2023-11-24T06:30:33.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:33.705+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0f3415c850 con 0x7f0f3413df80 2023-11-24T06:30:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.226+0000 7f0f29ffb640 1 -- 172.21.15.16:0/707191897 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0f3415c850 con 0x7f0f3413df80 2023-11-24T06:30:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 10 and data pool 11 2023-11-24T06:30:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.231+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f1806b230 msgr2=0x7f0f1806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.231+0000 7f0f3b7ce640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f1806b230 0x7f0f1806d6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0f24036000 tx=0x7f0f2403c040 comp rx=0 tx=0).stop 2023-11-24T06:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.231+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f3413df80 msgr2=0x7f0f34132180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.231+0000 7f0f3b7ce640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f3413df80 0x7f0f34132180 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000e870 tx=0x7f0f3000ed30 comp rx=0 tx=0).stop 2023-11-24T06:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.231+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 shutdown_connections 2023-11-24T06:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.231+0000 7f0f3b7ce640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0f1806b230 0x7f0f1806d6f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.231+0000 7f0f3b7ce640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f3415f9c0 0x7f0f3414d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.231+0000 7f0f3b7ce640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f3413e360 0x7f0f3414d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.231+0000 7f0f3b7ce640 1 --2- 172.21.15.16:0/707191897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f3413df80 0x7f0f34132180 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.232+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 >> 172.21.15.16:0/707191897 conn(0x7f0f3406a800 msgr2=0x7f0f340b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:34.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.232+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 shutdown_connections 2023-11-24T06:30:34.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.233+0000 7f0f3b7ce640 1 -- 172.21.15.16:0/707191897 wait complete. 2023-11-24T06:30:34.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.453+0000 7fd7cd2bb640 1 Processor -- start 2023-11-24T06:30:34.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.453+0000 7fd7cd2bb640 1 -- start start 2023-11-24T06:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.455+0000 7fd7cd2bb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd7c80cf9a0 0x7fd7c815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.455+0000 7fd7cd2bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c815e1e0 0x7fd7c80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.455+0000 7fd7cd2bb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c80b1ee0 con 0x7fd7c8143ae0 2023-11-24T06:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.455+0000 7fd7cd2bb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c80b2780 con 0x7fd7c80cf9a0 2023-11-24T06:30:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.455+0000 7fd7cd2bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c815e990 con 0x7fd7c815e1e0 2023-11-24T06:30:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.456+0000 7fd7c77fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c815e1e0 0x7fd7c80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.456+0000 7fd7c77fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c815e1e0 0x7fd7c80702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:49972/0 (socket says 172.21.15.16:49972) 2023-11-24T06:30:34.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.456+0000 7fd7c77fe640 1 -- 172.21.15.16:0/2342196524 learned_addr learned my addr 172.21.15.16:0/2342196524 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:34.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.456+0000 7fd7c67fc640 1 --2- 172.21.15.16:0/2342196524 >> v2:172.21.15.173:3300/0 conn(0x7fd7c80cf9a0 0x7fd7c815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.456+0000 7fd7c77fe640 1 -- 172.21.15.16:0/2342196524 >> v2:172.21.15.173:3300/0 conn(0x7fd7c80cf9a0 msgr2=0x7fd7c815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.456+0000 7fd7c77fe640 1 --2- 172.21.15.16:0/2342196524 >> v2:172.21.15.173:3300/0 conn(0x7fd7c80cf9a0 0x7fd7c815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.457+0000 7fd7c77fe640 1 -- 172.21.15.16:0/2342196524 >> v1:172.21.15.173:6789/0 conn(0x7fd7c8143ae0 legacy=0x7fd7c80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.457+0000 7fd7c77fe640 1 -- 172.21.15.16:0/2342196524 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c80b2130 con 0x7fd7c815e1e0 2023-11-24T06:30:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.458+0000 7fd7c77fe640 1 --2- 172.21.15.16:0/2342196524 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c815e1e0 0x7fd7c80702c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000d150 tx=0x7fd7c002e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dac3961b528299b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.458+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/2342196524 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000b0d0 con 0x7fd7c815e1e0 2023-11-24T06:30:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.459+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/2342196524 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c000b230 con 0x7fd7c815e1e0 2023-11-24T06:30:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.459+0000 7fd7c5ffb640 1 -- 172.21.15.16:0/2342196524 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0009600 con 0x7fd7c815e1e0 2023-11-24T06:30:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.459+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/2342196524 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c815e1e0 msgr2=0x7fd7c80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.459+0000 7fd7cd2bb640 1 --2- 172.21.15.16:0/2342196524 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c815e1e0 0x7fd7c80702c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000d150 tx=0x7fd7c002e160 comp rx=0 tx=0).stop 2023-11-24T06:30:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.459+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/2342196524 shutdown_connections 2023-11-24T06:30:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.459+0000 7fd7cd2bb640 1 --2- 172.21.15.16:0/2342196524 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c815e1e0 0x7fd7c80702c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.459+0000 7fd7cd2bb640 1 --2- 172.21.15.16:0/2342196524 >> v2:172.21.15.173:3300/0 conn(0x7fd7c80cf9a0 0x7fd7c815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.459+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/2342196524 >> 172.21.15.16:0/2342196524 conn(0x7fd7c806a800 msgr2=0x7fd7c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.459+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/2342196524 shutdown_connections 2023-11-24T06:30:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.460+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/2342196524 wait complete. 2023-11-24T06:30:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.461+0000 7fd7cd2bb640 1 Processor -- start 2023-11-24T06:30:34.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.461+0000 7fd7cd2bb640 1 -- start start 2023-11-24T06:30:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.462+0000 7fd7cd2bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7c80cf9a0 0x7fd7c81e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.462+0000 7fd7cd2bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c8143ae0 0x7fd7c8141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.462+0000 7fd7cd2bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c815e1e0 0x7fd7c81e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.462+0000 7fd7cd2bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81e40a0 con 0x7fd7c815e1e0 2023-11-24T06:30:34.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.462+0000 7fd7cd2bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81e4380 con 0x7fd7c80cf9a0 2023-11-24T06:30:34.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.462+0000 7fd7cd2bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c81e4660 con 0x7fd7c8143ae0 2023-11-24T06:30:34.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c815e1e0 0x7fd7c81e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c815e1e0 0x7fd7c81e61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36334/0 (socket says 172.21.15.16:36334) 2023-11-24T06:30:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c77fe640 1 -- 172.21.15.16:0/1251193476 learned_addr learned my addr 172.21.15.16:0/1251193476 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c6ffd640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7c80cf9a0 0x7fd7c81e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c67fc640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c8143ae0 0x7fd7c8141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c77fe640 1 -- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c8143ae0 msgr2=0x7fd7c8141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c77fe640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c8143ae0 0x7fd7c8141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c77fe640 1 -- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7c80cf9a0 msgr2=0x7fd7c81e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c77fe640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7c80cf9a0 0x7fd7c81e3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c77fe640 1 -- 172.21.15.16:0/1251193476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c000bd40 con 0x7fd7c815e1e0 2023-11-24T06:30:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.463+0000 7fd7c67fc640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c8143ae0 0x7fd7c8141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.464+0000 7fd7c77fe640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c815e1e0 0x7fd7c81e61e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0038040 tx=0x7fd7c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:34.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.464+0000 7fd7bf7fe640 1 -- 172.21.15.16:0/1251193476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c000b0d0 con 0x7fd7c815e1e0 2023-11-24T06:30:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.464+0000 7fd7bf7fe640 1 -- 172.21.15.16:0/1251193476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c003f850 con 0x7fd7c815e1e0 2023-11-24T06:30:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.464+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c81e4940 con 0x7fd7c815e1e0 2023-11-24T06:30:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.464+0000 7fd7bf7fe640 1 -- 172.21.15.16:0/1251193476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c003fc00 con 0x7fd7c815e1e0 2023-11-24T06:30:34.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.464+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c8141d70 con 0x7fd7c815e1e0 2023-11-24T06:30:34.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.465+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c80675d0 con 0x7fd7c815e1e0 2023-11-24T06:30:34.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.466+0000 7fd7bf7fe640 1 -- 172.21.15.16:0/1251193476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd7c00403f0 con 0x7fd7c815e1e0 2023-11-24T06:30:34.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.467+0000 7fd7bf7fe640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd7a806b230 0x7fd7a806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.467+0000 7fd7bf7fe640 1 -- 172.21.15.16:0/1251193476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7c00ad150 con 0x7fd7c815e1e0 2023-11-24T06:30:34.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.468+0000 7fd7c6ffd640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd7a806b230 0x7fd7a806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.469+0000 7fd7c6ffd640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd7a806b230 0x7fd7a806d6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8000f60 tx=0x7fd7b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:34.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.471+0000 7fd7bf7fe640 1 -- 172.21.15.16:0/1251193476 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7c0077be0 con 0x7fd7c815e1e0 2023-11-24T06:30:34.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.664+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7c806b3f0 con 0x7fd7c815e1e0 2023-11-24T06:30:34.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.667+0000 7fd7bf7fe640 1 -- 172.21.15.16:0/1251193476 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16581 (secure 0 0 0) 0x7fd7c007b200 con 0x7fd7c815e1e0 2023-11-24T06:30:34.671 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:34.671 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:33.707172+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:30:31.631710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:30:32.732622+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:34.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd7a806b230 msgr2=0x7fd7a806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd7a806b230 0x7fd7a806d6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8000f60 tx=0x7fd7b8009040 comp rx=0 tx=0).stop 2023-11-24T06:30:34.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c815e1e0 msgr2=0x7fd7c81e61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c815e1e0 0x7fd7c81e61e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0038040 tx=0x7fd7c0004a40 comp rx=0 tx=0).stop 2023-11-24T06:30:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 shutdown_connections 2023-11-24T06:30:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd7a806b230 0x7fd7a806d6f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7c815e1e0 0x7fd7c81e61e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7c8143ae0 0x7fd7c8141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 --2- 172.21.15.16:0/1251193476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7c80cf9a0 0x7fd7c81e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 >> 172.21.15.16:0/1251193476 conn(0x7fd7c806a800 msgr2=0x7fd7c806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 shutdown_connections 2023-11-24T06:30:34.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.670+0000 7fd7cd2bb640 1 -- 172.21.15.16:0/1251193476 wait complete. 2023-11-24T06:30:34.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.904+0000 7fec208ad640 1 Processor -- start 2023-11-24T06:30:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.905+0000 7fec208ad640 1 -- start start 2023-11-24T06:30:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.906+0000 7fec208ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fec1c0b34e0 0x7fec1c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.906+0000 7fec208ad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec1c0b4140 0x7fec1c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.906+0000 7fec208ad640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1c0c7bf0 con 0x7fec1c0d1b90 2023-11-24T06:30:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.906+0000 7fec208ad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c0d2020 con 0x7fec1c0b34e0 2023-11-24T06:30:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.906+0000 7fec208ad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0d0420 con 0x7fec1c0b4140 2023-11-24T06:30:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.907+0000 7fec1bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fec1c0d1b90 0x7fec1c0cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44706/0 (socket says 172.21.15.16:44706) 2023-11-24T06:30:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.907+0000 7fec1bfff640 1 -- 172.21.15.16:0/3784254194 learned_addr learned my addr 172.21.15.16:0/3784254194 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.907+0000 7fec1b7fe640 1 --2- 172.21.15.16:0/3784254194 >> v2:172.21.15.16:3300/0 conn(0x7fec1c0b34e0 0x7fec1c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.907+0000 7fec1affd640 1 --2- 172.21.15.16:0/3784254194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec1c0b4140 0x7fec1c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.907+0000 7fec1b7fe640 1 -- 172.21.15.16:0/3784254194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec1c0b4140 msgr2=0x7fec1c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:34.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.907+0000 7fec1b7fe640 1 --2- 172.21.15.16:0/3784254194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec1c0b4140 0x7fec1c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.907+0000 7fec1b7fe640 1 -- 172.21.15.16:0/3784254194 >> v1:172.21.15.16:6789/0 conn(0x7fec1c0d1b90 legacy=0x7fec1c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.907+0000 7fec1b7fe640 1 -- 172.21.15.16:0/3784254194 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c0d0e70 con 0x7fec1c0b34e0 2023-11-24T06:30:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.907+0000 7fec1a7fc640 1 -- 172.21.15.16:0/3784254194 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407870150 0 0) 0x7fec1c0c7bf0 con 0x7fec1c0d1b90 2023-11-24T06:30:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.908+0000 7fec1b7fe640 1 --2- 172.21.15.16:0/3784254194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0b34e0 0x7fec1c0b3c00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0097a0 tx=0x7fec0c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89f882100ca0db3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec1a7fc640 1 -- 172.21.15.16:0/3784254194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c031830 con 0x7fec1c0b34e0 2023-11-24T06:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec1a7fc640 1 -- 172.21.15.16:0/3784254194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c031990 con 0x7fec1c0b34e0 2023-11-24T06:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec1a7fc640 1 -- 172.21.15.16:0/3784254194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c031d20 con 0x7fec1c0b34e0 2023-11-24T06:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec208ad640 1 -- 172.21.15.16:0/3784254194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0b34e0 msgr2=0x7fec1c0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec208ad640 1 --2- 172.21.15.16:0/3784254194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0b34e0 0x7fec1c0b3c00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0097a0 tx=0x7fec0c02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec208ad640 1 -- 172.21.15.16:0/3784254194 shutdown_connections 2023-11-24T06:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec208ad640 1 --2- 172.21.15.16:0/3784254194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec1c0b4140 0x7fec1c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec208ad640 1 --2- 172.21.15.16:0/3784254194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0b34e0 0x7fec1c0b3c00 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec208ad640 1 -- 172.21.15.16:0/3784254194 >> 172.21.15.16:0/3784254194 conn(0x7fec1c0d3090 msgr2=0x7fec1c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec208ad640 1 -- 172.21.15.16:0/3784254194 shutdown_connections 2023-11-24T06:30:34.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.909+0000 7fec208ad640 1 -- 172.21.15.16:0/3784254194 wait complete. 2023-11-24T06:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.911+0000 7fec208ad640 1 Processor -- start 2023-11-24T06:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.911+0000 7fec208ad640 1 -- start start 2023-11-24T06:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.912+0000 7fec208ad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec1c0b34e0 0x7fec1c156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.912+0000 7fec208ad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec1c0b4140 0x7fec1c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.912+0000 7fec208ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0d1b90 0x7fec1c0b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.912+0000 7fec208ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0b1f00 con 0x7fec1c0d1b90 2023-11-24T06:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.912+0000 7fec208ad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c156ba0 con 0x7fec1c0b34e0 2023-11-24T06:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.912+0000 7fec208ad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0b6d90 con 0x7fec1c0b4140 2023-11-24T06:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.912+0000 7fec1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0d1b90 0x7fec1c0b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.912+0000 7fec1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0d1b90 0x7fec1c0b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36356/0 (socket says 172.21.15.16:36356) 2023-11-24T06:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.912+0000 7fec1bfff640 1 -- 172.21.15.16:0/1079871794 learned_addr learned my addr 172.21.15.16:0/1079871794 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec1bfff640 1 -- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec1c0b4140 msgr2=0x7fec1c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec1bfff640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec1c0b4140 0x7fec1c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec1bfff640 1 -- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec1c0b34e0 msgr2=0x7fec1c156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec1b7fe640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec1c0b34e0 0x7fec1c156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec1bfff640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec1c0b34e0 0x7fec1c156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec1bfff640 1 -- 172.21.15.16:0/1079871794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c009400 con 0x7fec1c0d1b90 2023-11-24T06:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec1b7fe640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec1c0b34e0 0x7fec1c156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec1bfff640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0d1b90 0x7fec1c0b7980 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fec1400c780 tx=0x7fec1400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:34.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec13fff640 1 -- 172.21.15.16:0/1079871794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1400ce90 con 0x7fec1c0d1b90 2023-11-24T06:30:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec13fff640 1 -- 172.21.15.16:0/1079871794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1400a640 con 0x7fec1c0d1b90 2023-11-24T06:30:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec13fff640 1 -- 172.21.15.16:0/1079871794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1400a9d0 con 0x7fec1c0d1b90 2023-11-24T06:30:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.913+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c0b1280 con 0x7fec1c0d1b90 2023-11-24T06:30:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.914+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c0b1840 con 0x7fec1c0d1b90 2023-11-24T06:30:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.915+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c0d1350 con 0x7fec1c0d1b90 2023-11-24T06:30:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.916+0000 7fec13fff640 1 -- 172.21.15.16:0/1079871794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fec1400ab30 con 0x7fec1c0d1b90 2023-11-24T06:30:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.916+0000 7fec13fff640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febfc06b230 0x7febfc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.916+0000 7fec13fff640 1 -- 172.21.15.16:0/1079871794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec14089190 con 0x7fec1c0d1b90 2023-11-24T06:30:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.917+0000 7fec1b7fe640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febfc06b230 0x7febfc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.918+0000 7fec1b7fe640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febfc06b230 0x7febfc06d6f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fec0c036000 tx=0x7fec0c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:34.921+0000 7fec13fff640 1 -- 172.21.15.16:0/1079871794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec1c0d1350 con 0x7fec1c0d1b90 2023-11-24T06:30:35.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.108+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec1c0d1350 con 0x7fec1c0d1b90 2023-11-24T06:30:35.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.111+0000 7fec13fff640 1 -- 172.21.15.16:0/1079871794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16581 (secure 0 0 0) 0x7fec1c0d1350 con 0x7fec1c0d1b90 2023-11-24T06:30:35.115 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:35.115 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:33.707172+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:30:31.631710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:30:32.732622+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febfc06b230 msgr2=0x7febfc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febfc06b230 0x7febfc06d6f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fec0c036000 tx=0x7fec0c008030 comp rx=0 tx=0).stop 2023-11-24T06:30:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0d1b90 msgr2=0x7fec1c0b7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0d1b90 0x7fec1c0b7980 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fec1400c780 tx=0x7fec1400cc40 comp rx=0 tx=0).stop 2023-11-24T06:30:35.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 shutdown_connections 2023-11-24T06:30:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febfc06b230 0x7febfc06d6f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec1c0d1b90 0x7fec1c0b7980 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec1c0b4140 0x7fec1c0b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 --2- 172.21.15.16:0/1079871794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec1c0b34e0 0x7fec1c156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 >> 172.21.15.16:0/1079871794 conn(0x7fec1c0d3090 msgr2=0x7fec1c06c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 shutdown_connections 2023-11-24T06:30:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.114+0000 7fec208ad640 1 -- 172.21.15.16:0/1079871794 wait complete. 2023-11-24T06:30:35.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:30:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.361+0000 7febf2183640 1 Processor -- start 2023-11-24T06:30:35.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.362+0000 7febf2183640 1 -- start start 2023-11-24T06:30:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.363+0000 7febf2183640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec160a20 0x7febec0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.363+0000 7febf2183640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febec0d1e00 con 0x7febec160e00 2023-11-24T06:30:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.363+0000 7febf2183640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febec0b38a0 con 0x7febec0b3b20 2023-11-24T06:30:35.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.363+0000 7febf2183640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febec0d3c40 con 0x7febec160a20 2023-11-24T06:30:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.363+0000 7febf0980640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7febec160e00 0x7febec0b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44714/0 (socket says 172.21.15.16:44714) 2023-11-24T06:30:35.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.363+0000 7febf0980640 1 -- 172.21.15.16:0/160686889 learned_addr learned my addr 172.21.15.16:0/160686889 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666586 0 0) 0x7febec0d1e00 con 0x7febec160e00 2023-11-24T06:30:35.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd8003520 con 0x7febec160e00 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febf1181640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec160a20 0x7febec0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665548913 0 0) 0x7febd8003520 con 0x7febec160e00 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec160a20 msgr2=0x7febec0cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec160a20 0x7febec0cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 >> v1:172.21.15.173:6789/0 conn(0x7febec0b3b20 legacy=0x7febec0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec0d3e50 con 0x7febec160e00 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7febdc001670 con 0x7febec160e00 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 >> v1:172.21.15.16:6789/0 conn(0x7febec160e00 legacy=0x7febec0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80039a0 0x7febd8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd80047c0 0x7febd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec160a20 0x7febd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:35.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec0d3c40 con 0x7febd80039a0 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd800bab0 con 0x7febd80047c0 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.364+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd800bc70 con 0x7febec160a20 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.365+0000 7febf1181640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80039a0 0x7febd8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.365+0000 7febf1982640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec160a20 0x7febd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.365+0000 7febf0980640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd80047c0 0x7febd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.365+0000 7febf1181640 1 -- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec160a20 msgr2=0x7febd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.365+0000 7febf1181640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec160a20 0x7febd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.365+0000 7febf1181640 1 -- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd80047c0 msgr2=0x7febd8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.365+0000 7febf1181640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd80047c0 0x7febd8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.365+0000 7febf1181640 1 -- 172.21.15.16:0/160686889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd8003520 con 0x7febd80039a0 2023-11-24T06:30:35.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.366+0000 7febf1181640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80039a0 0x7febd8003dc0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7febe800ba30 tx=0x7febd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d12831e40739653 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.366+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8032640 con 0x7febd80039a0 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.367+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe80327a0 con 0x7febd80039a0 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.367+0000 7febe3fff640 1 -- 172.21.15.16:0/160686889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8032b30 con 0x7febd80039a0 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.367+0000 7febf2183640 1 -- 172.21.15.16:0/160686889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80039a0 msgr2=0x7febd8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.367+0000 7febf2183640 1 --2- 172.21.15.16:0/160686889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80039a0 0x7febd8003dc0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7febe800ba30 tx=0x7febd80057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.367+0000 7febf1982640 1 -- 172.21.15.16:0/160686889 reap_dead start 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.367+0000 7febf2183640 1 -- 172.21.15.16:0/160686889 shutdown_connections 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.367+0000 7febf2183640 1 -- 172.21.15.16:0/160686889 >> 172.21.15.16:0/160686889 conn(0x7febec0d3120 msgr2=0x7febec0d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.367+0000 7febf2183640 1 -- 172.21.15.16:0/160686889 shutdown_connections 2023-11-24T06:30:35.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.367+0000 7febf2183640 1 -- 172.21.15.16:0/160686889 wait complete. 2023-11-24T06:30:35.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.369+0000 7febf2183640 1 Processor -- start 2023-11-24T06:30:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.369+0000 7febf2183640 1 -- start start 2023-11-24T06:30:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.370+0000 7febf2183640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd80039a0 0x7febec1f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.370+0000 7febf2183640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec0c7e70 0x7febec0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.370+0000 7febf2183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febec0c8290 0x7febec0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.370+0000 7febf2183640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec0c6b80 con 0x7febec0c8290 2023-11-24T06:30:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.370+0000 7febf2183640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec0c6dd0 con 0x7febd80039a0 2023-11-24T06:30:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.370+0000 7febf2183640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febec0c70b0 con 0x7febec0c7e70 2023-11-24T06:30:35.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.370+0000 7febf1982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febec0c8290 0x7febec0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.370+0000 7febf1982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febec0c8290 0x7febec0c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36372/0 (socket says 172.21.15.16:36372) 2023-11-24T06:30:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.370+0000 7febf1982640 1 -- 172.21.15.16:0/3773850187 learned_addr learned my addr 172.21.15.16:0/3773850187 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.371+0000 7febf1181640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd80039a0 0x7febec1f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:35.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.371+0000 7febf0980640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec0c7e70 0x7febec0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.371+0000 7febf1982640 1 -- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec0c7e70 msgr2=0x7febec0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.371+0000 7febf1982640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec0c7e70 0x7febec0c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.371+0000 7febf1982640 1 -- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd80039a0 msgr2=0x7febec1f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:35.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.371+0000 7febf1982640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd80039a0 0x7febec1f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.371+0000 7febf1982640 1 -- 172.21.15.16:0/3773850187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec0c7390 con 0x7febec0c8290 2023-11-24T06:30:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.371+0000 7febf1982640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febec0c8290 0x7febec0c96e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7febec0d30f0 tx=0x7febe400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.371+0000 7febe1ffb640 1 -- 172.21.15.16:0/3773850187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4003970 con 0x7febec0c8290 2023-11-24T06:30:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.372+0000 7febe1ffb640 1 -- 172.21.15.16:0/3773850187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe4003ad0 con 0x7febec0c8290 2023-11-24T06:30:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.372+0000 7febe1ffb640 1 -- 172.21.15.16:0/3773850187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4003e80 con 0x7febec0c8290 2023-11-24T06:30:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.372+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec0c7670 con 0x7febec0c8290 2023-11-24T06:30:35.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.372+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febec1f6c50 con 0x7febec0c8290 2023-11-24T06:30:35.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.373+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febec0c7b90 con 0x7febec0c8290 2023-11-24T06:30:35.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.374+0000 7febe1ffb640 1 -- 172.21.15.16:0/3773850187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7febe40050d0 con 0x7febec0c8290 2023-11-24T06:30:35.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.375+0000 7febe1ffb640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd006b260 0x7febd006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:35.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.375+0000 7febe1ffb640 1 -- 172.21.15.16:0/3773850187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febe4089b40 con 0x7febec0c8290 2023-11-24T06:30:35.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.376+0000 7febf1181640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd006b260 0x7febd006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:35.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.377+0000 7febf1181640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd006b260 0x7febd006d720 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7febe800ba00 tx=0x7febe803d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:35.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.379+0000 7febe1ffb640 1 -- 172.21.15.16:0/3773850187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febec0c7b90 con 0x7febec0c8290 2023-11-24T06:30:35.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:35.616+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7febec0c7b90 con 0x7febec0c8290 2023-11-24T06:30:36.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.263+0000 7febe1ffb640 1 -- 172.21.15.16:0/3773850187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7febec0c7b90 con 0x7febec0c8290 2023-11-24T06:30:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.266+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd006b260 msgr2=0x7febd006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.266+0000 7febf2183640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd006b260 0x7febd006d720 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7febe800ba00 tx=0x7febe803d040 comp rx=0 tx=0).stop 2023-11-24T06:30:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.266+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febec0c8290 msgr2=0x7febec0c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.266+0000 7febf2183640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febec0c8290 0x7febec0c96e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7febec0d30f0 tx=0x7febe400ec10 comp rx=0 tx=0).stop 2023-11-24T06:30:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.267+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 shutdown_connections 2023-11-24T06:30:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.267+0000 7febf2183640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febd006b260 0x7febd006d720 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.267+0000 7febf2183640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febec0c8290 0x7febec0c96e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.267+0000 7febf2183640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febec0c7e70 0x7febec0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.267+0000 7febf2183640 1 --2- 172.21.15.16:0/3773850187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd80039a0 0x7febec1f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.267+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 >> 172.21.15.16:0/3773850187 conn(0x7febec0d3120 msgr2=0x7febec06b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.267+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 shutdown_connections 2023-11-24T06:30:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.267+0000 7febf2183640 1 -- 172.21.15.16:0/3773850187 wait complete. 2023-11-24T06:30:36.291 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:30:36.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:30:36.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.514+0000 7f5866528640 1 Processor -- start 2023-11-24T06:30:36.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.515+0000 7f5866528640 1 -- start start 2023-11-24T06:30:36.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.516+0000 7f5866528640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 0x7f5860070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.517+0000 7f5866528640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f586015bf10 0x7f58600cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.517+0000 7f5866528640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586015f4e0 con 0x7f58600cc980 2023-11-24T06:30:36.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.517+0000 7f5866528640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586015f970 con 0x7f586015bf10 2023-11-24T06:30:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.517+0000 7f5866528640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58600cce10 con 0x7f58600adf20 2023-11-24T06:30:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.517+0000 7f5865d27640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58600cc980 0x7f5860143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44722/0 (socket says 172.21.15.16:44722) 2023-11-24T06:30:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.517+0000 7f5865d27640 1 -- 172.21.15.16:0/441160696 learned_addr learned my addr 172.21.15.16:0/441160696 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:36.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.517+0000 7f5864d25640 1 --2- 172.21.15.16:0/441160696 >> v2:172.21.15.16:3300/0 conn(0x7f586015bf10 0x7f58600cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:36.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.517+0000 7f5865526640 1 --2- 172.21.15.16:0/441160696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 0x7f5860070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.518+0000 7f5857fff640 1 -- 172.21.15.16:0/441160696 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594610845 0 0) 0x7f586015f4e0 con 0x7f58600cc980 2023-11-24T06:30:36.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.518+0000 7f5857fff640 1 -- 172.21.15.16:0/441160696 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f584c003520 con 0x7f58600cc980 2023-11-24T06:30:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.518+0000 7f5864d25640 1 -- 172.21.15.16:0/441160696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 msgr2=0x7f5860070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.518+0000 7f5864d25640 1 --2- 172.21.15.16:0/441160696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 0x7f5860070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.518+0000 7f5864d25640 1 -- 172.21.15.16:0/441160696 >> v1:172.21.15.16:6789/0 conn(0x7f58600cc980 legacy=0x7f5860143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.518+0000 7f5864d25640 1 -- 172.21.15.16:0/441160696 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860071130 con 0x7f586015bf10 2023-11-24T06:30:36.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.518+0000 7f5865526640 1 --2- 172.21.15.16:0/441160696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 0x7f5860070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:30:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.519+0000 7f5864d25640 1 --2- 172.21.15.16:0/441160696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 0x7f58600cc440 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f58500097a0 tx=0x7f585002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4c28294492a351 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.519+0000 7f5857fff640 1 -- 172.21.15.16:0/441160696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850031780 con 0x7f586015bf10 2023-11-24T06:30:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.519+0000 7f5857fff640 1 -- 172.21.15.16:0/441160696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58500318e0 con 0x7f586015bf10 2023-11-24T06:30:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.519+0000 7f5857fff640 1 -- 172.21.15.16:0/441160696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850031c50 con 0x7f586015bf10 2023-11-24T06:30:36.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.520+0000 7f5866528640 1 -- 172.21.15.16:0/441160696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 msgr2=0x7f58600cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.520+0000 7f5866528640 1 --2- 172.21.15.16:0/441160696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 0x7f58600cc440 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f58500097a0 tx=0x7f585002b820 comp rx=0 tx=0).stop 2023-11-24T06:30:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.520+0000 7f5866528640 1 -- 172.21.15.16:0/441160696 shutdown_connections 2023-11-24T06:30:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.520+0000 7f5866528640 1 --2- 172.21.15.16:0/441160696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 0x7f58600cc440 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.520+0000 7f5866528640 1 --2- 172.21.15.16:0/441160696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 0x7f5860070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.520+0000 7f5866528640 1 -- 172.21.15.16:0/441160696 >> 172.21.15.16:0/441160696 conn(0x7f586006a7d0 msgr2=0x7f586006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.520+0000 7f5866528640 1 -- 172.21.15.16:0/441160696 shutdown_connections 2023-11-24T06:30:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.520+0000 7f5866528640 1 -- 172.21.15.16:0/441160696 wait complete. 2023-11-24T06:30:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.522+0000 7f5866528640 1 Processor -- start 2023-11-24T06:30:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.522+0000 7f5866528640 1 -- start start 2023-11-24T06:30:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.523+0000 7f5866528640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 0x7f586013e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.523+0000 7f5866528640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58600cc980 0x7f586013ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.523+0000 7f5866528640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 0x7f586013a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.523+0000 7f5866528640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586013e380 con 0x7f586015bf10 2023-11-24T06:30:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.523+0000 7f5866528640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586013d700 con 0x7f58600cc980 2023-11-24T06:30:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.523+0000 7f5866528640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586013d9e0 con 0x7f58600adf20 2023-11-24T06:30:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.523+0000 7f5865d27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 0x7f586013a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.523+0000 7f5865d27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 0x7f586013a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36398/0 (socket says 172.21.15.16:36398) 2023-11-24T06:30:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.523+0000 7f5865d27640 1 -- 172.21.15.16:0/2807261869 learned_addr learned my addr 172.21.15.16:0/2807261869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5864d25640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58600cc980 0x7f586013ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5865526640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 0x7f586013e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5865d27640 1 -- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 msgr2=0x7f586013e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5865d27640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 0x7f586013e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5865d27640 1 -- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58600cc980 msgr2=0x7f586013ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5865d27640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58600cc980 0x7f586013ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5865d27640 1 -- 172.21.15.16:0/2807261869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5850009400 con 0x7f586015bf10 2023-11-24T06:30:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5865d27640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 0x7f586013a0e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f585800caa0 tx=0x7f585800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5855ffb640 1 -- 172.21.15.16:0/2807261869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585800ac00 con 0x7f586015bf10 2023-11-24T06:30:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5855ffb640 1 -- 172.21.15.16:0/2807261869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585800ad60 con 0x7f586015bf10 2023-11-24T06:30:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5855ffb640 1 -- 172.21.15.16:0/2807261869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858012620 con 0x7f586015bf10 2023-11-24T06:30:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586013dcc0 con 0x7f586015bf10 2023-11-24T06:30:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.524+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586013c530 con 0x7f586015bf10 2023-11-24T06:30:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.526+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586013e4c0 con 0x7f586015bf10 2023-11-24T06:30:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.527+0000 7f5855ffb640 1 -- 172.21.15.16:0/2807261869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5858007070 con 0x7f586015bf10 2023-11-24T06:30:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.528+0000 7f5855ffb640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f584406b230 0x7f584406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.528+0000 7f5855ffb640 1 -- 172.21.15.16:0/2807261869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5858089cf0 con 0x7f586015bf10 2023-11-24T06:30:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.529+0000 7f5865526640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f584406b230 0x7f584406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.530+0000 7f5865526640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f584406b230 0x7f584406d6f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f585c0096d0 tx=0x7f585c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.532+0000 7f5855ffb640 1 -- 172.21.15.16:0/2807261869 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f586013e4c0 con 0x7f586015bf10 2023-11-24T06:30:36.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:36.768+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f586013e4c0 con 0x7f586015bf10 2023-11-24T06:30:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.274+0000 7f5855ffb640 1 -- 172.21.15.16:0/2807261869 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f586013e4c0 con 0x7f586015bf10 2023-11-24T06:30:37.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.277+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f584406b230 msgr2=0x7f584406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:37.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.277+0000 7f5866528640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f584406b230 0x7f584406d6f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f585c0096d0 tx=0x7f585c008040 comp rx=0 tx=0).stop 2023-11-24T06:30:37.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.277+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 msgr2=0x7f586013a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:37.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.277+0000 7f5866528640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 0x7f586013a0e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f585800caa0 tx=0x7f585800cf60 comp rx=0 tx=0).stop 2023-11-24T06:30:37.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.278+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 shutdown_connections 2023-11-24T06:30:37.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.278+0000 7f5866528640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f584406b230 0x7f584406d6f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.278+0000 7f5866528640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f586015bf10 0x7f586013a0e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.278+0000 7f5866528640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58600cc980 0x7f586013ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.278+0000 7f5866528640 1 --2- 172.21.15.16:0/2807261869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58600adf20 0x7f586013e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.278+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 >> 172.21.15.16:0/2807261869 conn(0x7f586006a7d0 msgr2=0x7f586006daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:37.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.278+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 shutdown_connections 2023-11-24T06:30:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.278+0000 7f5866528640 1 -- 172.21.15.16:0/2807261869 wait complete. 2023-11-24T06:30:37.304 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:30:37.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:30:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.529+0000 7ff3d61bc640 1 Processor -- start 2023-11-24T06:30:37.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.530+0000 7ff3d61bc640 1 -- start start 2023-11-24T06:30:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.531+0000 7ff3d61bc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3d0162810 0x7ff3d0071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.531+0000 7ff3d61bc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d0071780 0x7ff3d006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.531+0000 7ff3d61bc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d0071c10 con 0x7ff3d0162430 2023-11-24T06:30:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.531+0000 7ff3d61bc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d00702e0 con 0x7ff3d0162810 2023-11-24T06:30:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.531+0000 7ff3d61bc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d00705c0 con 0x7ff3d0071780 2023-11-24T06:30:37.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3d51ba640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff3d0162430 0x7ff3d00b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44738/0 (socket says 172.21.15.16:44738) 2023-11-24T06:30:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3d51ba640 1 -- 172.21.15.16:0/2528960866 learned_addr learned my addr 172.21.15.16:0/2528960866 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:37.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3d49b9640 1 --2- 172.21.15.16:0/2528960866 >> v2:172.21.15.16:3300/0 conn(0x7ff3d0162810 0x7ff3d0071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3c7fff640 1 -- 172.21.15.16:0/2528960866 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758755111 0 0) 0x7ff3d0071c10 con 0x7ff3d0162430 2023-11-24T06:30:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3d59bb640 1 --2- 172.21.15.16:0/2528960866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d0071780 0x7ff3d006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3c7fff640 1 -- 172.21.15.16:0/2528960866 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3bc003520 con 0x7ff3d0162430 2023-11-24T06:30:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3d49b9640 1 -- 172.21.15.16:0/2528960866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d0071780 msgr2=0x7ff3d006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3d49b9640 1 --2- 172.21.15.16:0/2528960866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d0071780 0x7ff3d006c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3d49b9640 1 -- 172.21.15.16:0/2528960866 >> v1:172.21.15.16:6789/0 conn(0x7ff3d0162430 legacy=0x7ff3d00b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3d49b9640 1 -- 172.21.15.16:0/2528960866 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d00708a0 con 0x7ff3d0162810 2023-11-24T06:30:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.532+0000 7ff3d59bb640 1 --2- 172.21.15.16:0/2528960866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d0071780 0x7ff3d006c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:37.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.534+0000 7ff3d49b9640 1 --2- 172.21.15.16:0/2528960866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162810 0x7ff3d0071240 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00097a0 tx=0x7ff3c002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631cfced7bd05310 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.534+0000 7ff3c7fff640 1 -- 172.21.15.16:0/2528960866 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c002d040 con 0x7ff3d0162810 2023-11-24T06:30:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.534+0000 7ff3c7fff640 1 -- 172.21.15.16:0/2528960866 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c0031810 con 0x7ff3d0162810 2023-11-24T06:30:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.534+0000 7ff3c7fff640 1 -- 172.21.15.16:0/2528960866 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0031ba0 con 0x7ff3d0162810 2023-11-24T06:30:37.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.535+0000 7ff3d61bc640 1 -- 172.21.15.16:0/2528960866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162810 msgr2=0x7ff3d0071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.535+0000 7ff3d61bc640 1 --2- 172.21.15.16:0/2528960866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162810 0x7ff3d0071240 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00097a0 tx=0x7ff3c002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.535+0000 7ff3d61bc640 1 -- 172.21.15.16:0/2528960866 shutdown_connections 2023-11-24T06:30:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.535+0000 7ff3d61bc640 1 --2- 172.21.15.16:0/2528960866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d0071780 0x7ff3d006c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.535+0000 7ff3d61bc640 1 --2- 172.21.15.16:0/2528960866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162810 0x7ff3d0071240 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.535+0000 7ff3d61bc640 1 -- 172.21.15.16:0/2528960866 >> 172.21.15.16:0/2528960866 conn(0x7ff3d006a800 msgr2=0x7ff3d006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.535+0000 7ff3d61bc640 1 -- 172.21.15.16:0/2528960866 shutdown_connections 2023-11-24T06:30:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.535+0000 7ff3d61bc640 1 -- 172.21.15.16:0/2528960866 wait complete. 2023-11-24T06:30:37.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.536+0000 7ff3d61bc640 1 Processor -- start 2023-11-24T06:30:37.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.537+0000 7ff3d61bc640 1 -- start start 2023-11-24T06:30:37.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d61bc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3d0071780 0x7ff3d01576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:37.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d61bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162430 0x7ff3d0154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:37.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d61bc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d015a660 0x7ff3d015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:37.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d61bc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d015b050 con 0x7ff3d0162430 2023-11-24T06:30:37.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d61bc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d01406a0 con 0x7ff3d0071780 2023-11-24T06:30:37.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d61bc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d0156960 con 0x7ff3d015a660 2023-11-24T06:30:37.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d49b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162430 0x7ff3d0154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:37.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d49b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162430 0x7ff3d0154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36418/0 (socket says 172.21.15.16:36418) 2023-11-24T06:30:37.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d49b9640 1 -- 172.21.15.16:0/1371126260 learned_addr learned my addr 172.21.15.16:0/1371126260 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d49b9640 1 -- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d015a660 msgr2=0x7ff3d015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d49b9640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d015a660 0x7ff3d015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d49b9640 1 -- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3d0071780 msgr2=0x7ff3d01576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d49b9640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3d0071780 0x7ff3d01576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.538+0000 7ff3d49b9640 1 -- 172.21.15.16:0/1371126260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c0009400 con 0x7ff3d0162430 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.539+0000 7ff3d49b9640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162430 0x7ff3d0154e60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0000c00 tx=0x7ff3c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.539+0000 7ff3c5ffb640 1 -- 172.21.15.16:0/1371126260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0006e10 con 0x7ff3d0162430 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.539+0000 7ff3c5ffb640 1 -- 172.21.15.16:0/1371126260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c003ad60 con 0x7ff3d0162430 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.539+0000 7ff3c5ffb640 1 -- 172.21.15.16:0/1371126260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0044630 con 0x7ff3d0162430 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.539+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d0157be0 con 0x7ff3d0162430 2023-11-24T06:30:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.539+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d01f7c30 con 0x7ff3d0162430 2023-11-24T06:30:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.540+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d0067450 con 0x7ff3d0162430 2023-11-24T06:30:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.541+0000 7ff3c5ffb640 1 -- 172.21.15.16:0/1371126260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff3c004d450 con 0x7ff3d0162430 2023-11-24T06:30:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.542+0000 7ff3c5ffb640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff3b406b230 0x7ff3b406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.542+0000 7ff3c5ffb640 1 -- 172.21.15.16:0/1371126260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3c00ac0e0 con 0x7ff3d0162430 2023-11-24T06:30:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.543+0000 7ff3d51ba640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff3b406b230 0x7ff3b406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:37.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.544+0000 7ff3d51ba640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff3b406b230 0x7ff3b406d6f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff3d01fa3e0 tx=0x7ff3cc007930 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.547+0000 7ff3c5ffb640 1 -- 172.21.15.16:0/1371126260 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3c0076a60 con 0x7ff3d0162430 2023-11-24T06:30:37.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.777+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3d00b2ef0 con 0x7ff3d0162430 2023-11-24T06:30:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.778+0000 7ff3c5ffb640 1 -- 172.21.15.16:0/1371126260 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4918 (secure 0 0 0) 0x7ff3c007a080 con 0x7ff3d0162430 2023-11-24T06:30:37.781 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:37.781 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1164611025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1164611025},{"type":"v1","addr":"172.21.15.16:6834","nonce":1164611025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5650,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1157106761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1157106761},{"type":"v1","addr":"172.21.15.173:6836","nonce":1157106761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/151946729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":151946729},{"type":"v1","addr":"172.21.15.173:6837","nonce":151946729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:30:33.707795+0000","modified":"2023-11-24T06:30:36.770163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6693":{"gid":6693,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2360619406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2360619406},{"type":"v1","addr":"172.21.15.16:6835","nonce":2360619406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:30:37.782 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2023-11-24T06:30:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff3b406b230 msgr2=0x7ff3b406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff3b406b230 0x7ff3b406d6f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff3d01fa3e0 tx=0x7ff3cc007930 comp rx=0 tx=0).stop 2023-11-24T06:30:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162430 msgr2=0x7ff3d0154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162430 0x7ff3d0154e60 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0000c00 tx=0x7ff3c00046c0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 shutdown_connections 2023-11-24T06:30:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff3b406b230 0x7ff3b406d6f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3d015a660 0x7ff3d015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3d0162430 0x7ff3d0154e60 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 --2- 172.21.15.16:0/1371126260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3d0071780 0x7ff3d01576a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 >> 172.21.15.16:0/1371126260 conn(0x7ff3d006a800 msgr2=0x7ff3d00cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 shutdown_connections 2023-11-24T06:30:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:37.781+0000 7ff3d61bc640 1 -- 172.21.15.16:0/1371126260 wait complete. 2023-11-24T06:30:37.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.028+0000 7f29e1def640 1 Processor -- start 2023-11-24T06:30:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.028+0000 7f29e1def640 1 -- start start 2023-11-24T06:30:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.030+0000 7f29e1def640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29dc0cc990 0x7f29dc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.030+0000 7f29e1def640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc070e50 con 0x7f29dc0cc500 2023-11-24T06:30:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.030+0000 7f29e1def640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc144020 con 0x7f29dc143c40 2023-11-24T06:30:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.030+0000 7f29e1def640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29dc0723c0 con 0x7f29dc0cc990 2023-11-24T06:30:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.030+0000 7f29e15ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29dc0cc990 0x7f29dc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.030+0000 7f29dbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f29dc0cc500 0x7f29dc06d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44744/0 (socket says 172.21.15.16:44744) 2023-11-24T06:30:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.030+0000 7f29e15ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29dc0cc990 0x7f29dc06d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36426/0 (socket says 172.21.15.16:36426) 2023-11-24T06:30:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.030+0000 7f29dbfff640 1 -- 172.21.15.16:0/1003479861 learned_addr learned my addr 172.21.15.16:0/1003479861 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:38.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.031+0000 7f29e15ee640 1 -- 172.21.15.16:0/1003479861 >> v1:172.21.15.173:6789/0 conn(0x7f29dc143c40 legacy=0x7f29dc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.031+0000 7f29e15ee640 1 -- 172.21.15.16:0/1003479861 >> v1:172.21.15.16:6789/0 conn(0x7f29dc0cc500 legacy=0x7f29dc06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.031+0000 7f29e15ee640 1 -- 172.21.15.16:0/1003479861 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc071e00 con 0x7f29dc0cc990 2023-11-24T06:30:38.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.031+0000 7f29db7fe640 1 -- 172.21.15.16:0/1003479861 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368325459 0 0) 0x7f29dc070e50 con 0x7f29dc0cc500 2023-11-24T06:30:38.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.032+0000 7f29e15ee640 1 --2- 172.21.15.16:0/1003479861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc990 0x7f29dc06d870 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f29d40097a0 tx=0x7f29d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3e2a7ecdbb1ca3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:38.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.032+0000 7f29db7fe640 1 -- 172.21.15.16:0/1003479861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4031830 con 0x7f29dc0cc990 2023-11-24T06:30:38.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.032+0000 7f29db7fe640 1 -- 172.21.15.16:0/1003479861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d4031990 con 0x7f29dc0cc990 2023-11-24T06:30:38.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.032+0000 7f29db7fe640 1 -- 172.21.15.16:0/1003479861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4031d20 con 0x7f29dc0cc990 2023-11-24T06:30:38.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.033+0000 7f29e1def640 1 -- 172.21.15.16:0/1003479861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc990 msgr2=0x7f29dc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.033+0000 7f29e1def640 1 --2- 172.21.15.16:0/1003479861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc990 0x7f29dc06d870 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f29d40097a0 tx=0x7f29d402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.033+0000 7f29e1def640 1 -- 172.21.15.16:0/1003479861 shutdown_connections 2023-11-24T06:30:38.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.033+0000 7f29e1def640 1 --2- 172.21.15.16:0/1003479861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc990 0x7f29dc06d870 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.033+0000 7f29e1def640 1 -- 172.21.15.16:0/1003479861 >> 172.21.15.16:0/1003479861 conn(0x7f29dc06a800 msgr2=0x7f29dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:38.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.033+0000 7f29e1def640 1 -- 172.21.15.16:0/1003479861 shutdown_connections 2023-11-24T06:30:38.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.033+0000 7f29e1def640 1 -- 172.21.15.16:0/1003479861 wait complete. 2023-11-24T06:30:38.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.035+0000 7f29e1def640 1 Processor -- start 2023-11-24T06:30:38.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.035+0000 7f29e1def640 1 -- start start 2023-11-24T06:30:38.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.036+0000 7f29e1def640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc500 0x7f29dc1f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.036+0000 7f29e1def640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0cc990 0x7f29dc1371b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.036+0000 7f29e1def640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc143c40 0x7f29dc1376f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.036+0000 7f29e1def640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc1f1020 con 0x7f29dc0cc500 2023-11-24T06:30:38.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.036+0000 7f29e1def640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc137c30 con 0x7f29dc0cc990 2023-11-24T06:30:38.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.036+0000 7f29e1def640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc13a790 con 0x7f29dc143c40 2023-11-24T06:30:38.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.036+0000 7f29e0ded640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc500 0x7f29dc1f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29e0ded640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc500 0x7f29dc1f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36434/0 (socket says 172.21.15.16:36434) 2023-11-24T06:30:38.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29e0ded640 1 -- 172.21.15.16:0/377788579 learned_addr learned my addr 172.21.15.16:0/377788579 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:38.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29dbfff640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0cc990 0x7f29dc1371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29e15ee640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc143c40 0x7f29dc1376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29e0ded640 1 -- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc143c40 msgr2=0x7f29dc1376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29e0ded640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc143c40 0x7f29dc1376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29e0ded640 1 -- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0cc990 msgr2=0x7f29dc1371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29e0ded640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0cc990 0x7f29dc1371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29e0ded640 1 -- 172.21.15.16:0/377788579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4009400 con 0x7f29dc0cc500 2023-11-24T06:30:38.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.037+0000 7f29e0ded640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc500 0x7f29dc1f4790 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c730 tx=0x7f29cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:38.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.038+0000 7f29d97fa640 1 -- 172.21.15.16:0/377788579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc003b70 con 0x7f29dc0cc500 2023-11-24T06:30:38.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.038+0000 7f29d97fa640 1 -- 172.21.15.16:0/377788579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc003cf0 con 0x7f29dc0cc500 2023-11-24T06:30:38.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.038+0000 7f29d97fa640 1 -- 172.21.15.16:0/377788579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc00aa90 con 0x7f29dc0cc500 2023-11-24T06:30:38.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.038+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc13aa70 con 0x7f29dc0cc500 2023-11-24T06:30:38.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.038+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc13b030 con 0x7f29dc0cc500 2023-11-24T06:30:38.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.039+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc13b250 con 0x7f29dc0cc500 2023-11-24T06:30:38.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.040+0000 7f29d97fa640 1 -- 172.21.15.16:0/377788579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f29cc00abf0 con 0x7f29dc0cc500 2023-11-24T06:30:38.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.041+0000 7f29d97fa640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f29bc06b230 0x7f29bc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.041+0000 7f29d97fa640 1 -- 172.21.15.16:0/377788579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29cc089b40 con 0x7f29dc0cc500 2023-11-24T06:30:38.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.041+0000 7f29dbfff640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f29bc06b230 0x7f29bc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.042+0000 7f29dbfff640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f29bc06b230 0x7f29bc06d6f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0cc370 tx=0x7f29c4007b90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:38.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.045+0000 7f29d97fa640 1 -- 172.21.15.16:0/377788579 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29cc054660 con 0x7f29dc0cc500 2023-11-24T06:30:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.239+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29dc1432d0 con 0x7f29dc0cc500 2023-11-24T06:30:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.242+0000 7f29d97fa640 1 -- 172.21.15.16:0/377788579 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16568 (secure 0 0 0) 0x7f29cc003e60 con 0x7f29dc0cc500 2023-11-24T06:30:38.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:38.246 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":71,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:35.240884+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:30:31.631710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:30:32.732622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.244+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f29bc06b230 msgr2=0x7f29bc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.244+0000 7f29e1def640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f29bc06b230 0x7f29bc06d6f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0cc370 tx=0x7f29c4007b90 comp rx=0 tx=0).stop 2023-11-24T06:30:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.244+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc500 msgr2=0x7f29dc1f4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.244+0000 7f29e1def640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc500 0x7f29dc1f4790 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00c730 tx=0x7f29cc00cbf0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.245+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 shutdown_connections 2023-11-24T06:30:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.245+0000 7f29e1def640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f29bc06b230 0x7f29bc06d6f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.245+0000 7f29e1def640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29dc143c40 0x7f29dc1376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.245+0000 7f29e1def640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29dc0cc990 0x7f29dc1371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.245+0000 7f29e1def640 1 --2- 172.21.15.16:0/377788579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29dc0cc500 0x7f29dc1f4790 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.245+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 >> 172.21.15.16:0/377788579 conn(0x7f29dc06a800 msgr2=0x7f29dc0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.245+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 shutdown_connections 2023-11-24T06:30:38.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.245+0000 7f29e1def640 1 -- 172.21.15.16:0/377788579 wait complete. 2023-11-24T06:30:38.265 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:30:38.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:30:38.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.482+0000 7f09397db640 1 Processor -- start 2023-11-24T06:30:38.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.483+0000 7f09397db640 1 -- start start 2023-11-24T06:30:38.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.484+0000 7f09397db640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0934160c70 0x7f09340d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.484+0000 7f09397db640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f09340c6be0 0x7f09340cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.484+0000 7f09397db640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340cf6a0 con 0x7f0934160890 2023-11-24T06:30:38.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.484+0000 7f09397db640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340d1430 con 0x7f09340c6be0 2023-11-24T06:30:38.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.484+0000 7f09397db640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340b4370 con 0x7f0934160c70 2023-11-24T06:30:38.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.485+0000 7f0938fda640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f09340c6be0 0x7f09340cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.485+0000 7f0938fda640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f09340c6be0 0x7f09340cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36440/0 (socket says 172.21.15.16:36440) 2023-11-24T06:30:38.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.485+0000 7f0938fda640 1 -- 172.21.15.16:0/251211304 learned_addr learned my addr 172.21.15.16:0/251211304 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:38.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.485+0000 7f092bfff640 1 --2- 172.21.15.16:0/251211304 >> v2:172.21.15.173:3300/0 conn(0x7f0934160c70 0x7f09340d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.485+0000 7f0938fda640 1 -- 172.21.15.16:0/251211304 >> v2:172.21.15.173:3300/0 conn(0x7f0934160c70 msgr2=0x7f09340d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:38.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.485+0000 7f0938fda640 1 --2- 172.21.15.16:0/251211304 >> v2:172.21.15.173:3300/0 conn(0x7f0934160c70 0x7f09340d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.485+0000 7f0938fda640 1 -- 172.21.15.16:0/251211304 >> v1:172.21.15.173:6789/0 conn(0x7f0934160890 legacy=0x7f09340d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:38.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.485+0000 7f0938fda640 1 -- 172.21.15.16:0/251211304 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340ce7c0 con 0x7f09340c6be0 2023-11-24T06:30:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.487+0000 7f0938fda640 1 --2- 172.21.15.16:0/251211304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 0x7f09340cfb30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f09340b5c80 tx=0x7f092c02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd1b8e01c3d203e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.487+0000 7f09337fe640 1 -- 172.21.15.16:0/251211304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c02c960 con 0x7f09340c6be0 2023-11-24T06:30:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.487+0000 7f09337fe640 1 -- 172.21.15.16:0/251211304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c02cac0 con 0x7f09340c6be0 2023-11-24T06:30:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.487+0000 7f09337fe640 1 -- 172.21.15.16:0/251211304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c02ce50 con 0x7f09340c6be0 2023-11-24T06:30:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.488+0000 7f09397db640 1 -- 172.21.15.16:0/251211304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 msgr2=0x7f09340cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.488+0000 7f09397db640 1 --2- 172.21.15.16:0/251211304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 0x7f09340cfb30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f09340b5c80 tx=0x7f092c02bdc0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.488+0000 7f09397db640 1 -- 172.21.15.16:0/251211304 shutdown_connections 2023-11-24T06:30:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.488+0000 7f09397db640 1 --2- 172.21.15.16:0/251211304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 0x7f09340cfb30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.488+0000 7f09397db640 1 --2- 172.21.15.16:0/251211304 >> v2:172.21.15.173:3300/0 conn(0x7f0934160c70 0x7f09340d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.488+0000 7f09397db640 1 -- 172.21.15.16:0/251211304 >> 172.21.15.16:0/251211304 conn(0x7f09340d3090 msgr2=0x7f09340d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.488+0000 7f09397db640 1 -- 172.21.15.16:0/251211304 shutdown_connections 2023-11-24T06:30:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.488+0000 7f09397db640 1 -- 172.21.15.16:0/251211304 wait complete. 2023-11-24T06:30:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.490+0000 7f09397db640 1 Processor -- start 2023-11-24T06:30:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.490+0000 7f09397db640 1 -- start start 2023-11-24T06:30:38.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.491+0000 7f09397db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 0x7f09340b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.491+0000 7f09397db640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0934160890 0x7f09341564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.491+0000 7f09397db640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0934160c70 0x7f0934152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.491+0000 7f09397db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934156350 con 0x7f09340c6be0 2023-11-24T06:30:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.491+0000 7f09397db640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340b61c0 con 0x7f0934160890 2023-11-24T06:30:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.491+0000 7f09397db640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340b7c90 con 0x7f0934160c70 2023-11-24T06:30:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.491+0000 7f0933fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 0x7f09340b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.491+0000 7f0933fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 0x7f09340b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36444/0 (socket says 172.21.15.16:36444) 2023-11-24T06:30:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.491+0000 7f0933fff640 1 -- 172.21.15.16:0/2214353010 learned_addr learned my addr 172.21.15.16:0/2214353010 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f0938fda640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0934160c70 0x7f0934152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f0933fff640 1 -- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0934160c70 msgr2=0x7f0934152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f092bfff640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0934160890 0x7f09341564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f0933fff640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0934160c70 0x7f0934152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f0933fff640 1 -- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0934160890 msgr2=0x7f09341564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f0933fff640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0934160890 0x7f09341564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f0933fff640 1 -- 172.21.15.16:0/2214353010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c009b30 con 0x7f09340c6be0 2023-11-24T06:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f0938fda640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0934160c70 0x7f0934152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f0933fff640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 0x7f09340b5c80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f092400b420 tx=0x7f092400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f09317fa640 1 -- 172.21.15.16:0/2214353010 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924011830 con 0x7f09340c6be0 2023-11-24T06:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f09317fa640 1 -- 172.21.15.16:0/2214353010 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0924011990 con 0x7f09340c6be0 2023-11-24T06:30:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f09317fa640 1 -- 172.21.15.16:0/2214353010 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924011d40 con 0x7f09340c6be0 2023-11-24T06:30:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.492+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09340b63f0 con 0x7f09340c6be0 2023-11-24T06:30:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.493+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340b69b0 con 0x7f09340c6be0 2023-11-24T06:30:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.494+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0934067450 con 0x7f09340c6be0 2023-11-24T06:30:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.495+0000 7f09317fa640 1 -- 172.21.15.16:0/2214353010 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f092401b460 con 0x7f09340c6be0 2023-11-24T06:30:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.496+0000 7f09317fa640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f091406b230 0x7f091406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:38.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.496+0000 7f09317fa640 1 -- 172.21.15.16:0/2214353010 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0924089870 con 0x7f09340c6be0 2023-11-24T06:30:38.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.497+0000 7f092bfff640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f091406b230 0x7f091406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:38.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.498+0000 7f092bfff640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f091406b230 0x7f091406d6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f09340ce560 tx=0x7f091c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.500+0000 7f09317fa640 1 -- 172.21.15.16:0/2214353010 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0924054390 con 0x7f09340c6be0 2023-11-24T06:30:38.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.691+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09340ced80 con 0x7f091406b230 2023-11-24T06:30:38.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.716+0000 7f09317fa640 1 -- 172.21.15.16:0/2214353010 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281025 (secure 0 0 0) 0x7f091c009d60 con 0x7f091406b230 2023-11-24T06:30:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:38.725 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:30:38.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.721+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f091406b230 msgr2=0x7f091406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.721+0000 7f09397db640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f091406b230 0x7f091406d6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f09340ce560 tx=0x7f091c008040 comp rx=0 tx=0).stop 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.721+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 msgr2=0x7f09340b5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.721+0000 7f09397db640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 0x7f09340b5c80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f092400b420 tx=0x7f092400b8e0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.722+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 shutdown_connections 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.722+0000 7f09397db640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f091406b230 0x7f091406d6f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.722+0000 7f09397db640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0934160c70 0x7f0934152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.722+0000 7f09397db640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0934160890 0x7f09341564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.722+0000 7f09397db640 1 --2- 172.21.15.16:0/2214353010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09340c6be0 0x7f09340b5c80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.722+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 >> 172.21.15.16:0/2214353010 conn(0x7f09340d3090 msgr2=0x7f093406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.722+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 shutdown_connections 2023-11-24T06:30:38.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:38.722+0000 7f09397db640 1 -- 172.21.15.16:0/2214353010 wait complete. 2023-11-24T06:30:38.745 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2023-11-24T06:30:38.178434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253000,"kb_used_data":7720,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721720,"statfs":{"total":773094113280,"available":772835041280,"internally_reserved":0,"allocated":7905280,"data_stored":3997072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.964995"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.244535+0000","last_change":"2023-11-24T06:30:33.244535+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.244535+0000","last_undegraded":"2023-11-24T06:30:33.244535+0000","last_fullsized":"2023-11-24T06:30:33.244535+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.259035+0000","last_change":"2023-11-24T06:30:33.259035+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.259035+0000","last_undegraded":"2023-11-24T06:30:33.259035+0000","last_fullsized":"2023-11-24T06:30:33.259035+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.243192+0000","last_change":"2023-11-24T06:30:33.243192+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.243192+0000","last_undegraded":"2023-11-24T06:30:33.243192+0000","last_fullsized":"2023-11-24T06:30:33.243192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.236477+0000","last_change":"2023-11-24T06:30:33.236477+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.236477+0000","last_undegraded":"2023-11-24T06:30:33.236477+0000","last_fullsized":"2023-11-24T06:30:33.236477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239321+0000","last_change":"2023-11-24T06:30:33.239321+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239321+0000","last_undegraded":"2023-11-24T06:30:33.239321+0000","last_fullsized":"2023-11-24T06:30:33.239321+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.250847+0000","last_change":"2023-11-24T06:30:33.250847+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.250847+0000","last_undegraded":"2023-11-24T06:30:33.250847+0000","last_fullsized":"2023-11-24T06:30:33.250847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.254897+0000","last_change":"2023-11-24T06:30:33.254897+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.254897+0000","last_undegraded":"2023-11-24T06:30:33.254897+0000","last_fullsized":"2023-11-24T06:30:33.254897+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.252954+0000","last_change":"2023-11-24T06:30:33.252954+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.252954+0000","last_undegraded":"2023-11-24T06:30:33.252954+0000","last_fullsized":"2023-11-24T06:30:33.252954+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.238247+0000","last_change":"2023-11-24T06:30:33.238247+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.238247+0000","last_undegraded":"2023-11-24T06:30:33.238247+0000","last_fullsized":"2023-11-24T06:30:33.238247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.244235+0000","last_change":"2023-11-24T06:30:33.244235+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.244235+0000","last_undegraded":"2023-11-24T06:30:33.244235+0000","last_fullsized":"2023-11-24T06:30:33.244235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239947+0000","last_change":"2023-11-24T06:30:33.239947+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239947+0000","last_undegraded":"2023-11-24T06:30:33.239947+0000","last_fullsized":"2023-11-24T06:30:33.239947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235872+0000","last_change":"2023-11-24T06:30:33.235872+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235872+0000","last_undegraded":"2023-11-24T06:30:33.235872+0000","last_fullsized":"2023-11-24T06:30:33.235872+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247170+0000","last_change":"2023-11-24T06:30:33.247170+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247170+0000","last_undegraded":"2023-11-24T06:30:33.247170+0000","last_fullsized":"2023-11-24T06:30:33.247170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234680+0000","last_change":"2023-11-24T06:30:33.234680+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234680+0000","last_undegraded":"2023-11-24T06:30:33.234680+0000","last_fullsized":"2023-11-24T06:30:33.234680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247698+0000","last_change":"2023-11-24T06:30:33.247698+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247698+0000","last_undegraded":"2023-11-24T06:30:33.247698+0000","last_fullsized":"2023-11-24T06:30:33.247698+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237309+0000","last_change":"2023-11-24T06:30:33.237309+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237309+0000","last_undegraded":"2023-11-24T06:30:33.237309+0000","last_fullsized":"2023-11-24T06:30:33.237309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235467+0000","last_change":"2023-11-24T06:30:33.235467+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235467+0000","last_undegraded":"2023-11-24T06:30:33.235467+0000","last_fullsized":"2023-11-24T06:30:33.235467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.236341+0000","last_change":"2023-11-24T06:30:33.236341+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.236341+0000","last_undegraded":"2023-11-24T06:30:33.236341+0000","last_fullsized":"2023-11-24T06:30:33.236341+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.258019+0000","last_change":"2023-11-24T06:30:33.258019+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.258019+0000","last_undegraded":"2023-11-24T06:30:33.258019+0000","last_fullsized":"2023-11-24T06:30:33.258019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.251109+0000","last_change":"2023-11-24T06:30:33.251109+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.251109+0000","last_undegraded":"2023-11-24T06:30:33.251109+0000","last_fullsized":"2023-11-24T06:30:33.251109+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242204+0000","last_change":"2023-11-24T06:30:33.242204+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242204+0000","last_undegraded":"2023-11-24T06:30:33.242204+0000","last_fullsized":"2023-11-24T06:30:33.242204+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239064+0000","last_change":"2023-11-24T06:30:33.239064+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239064+0000","last_undegraded":"2023-11-24T06:30:33.239064+0000","last_fullsized":"2023-11-24T06:30:33.239064+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.240220+0000","last_change":"2023-11-24T06:30:33.240220+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.240220+0000","last_undegraded":"2023-11-24T06:30:33.240220+0000","last_fullsized":"2023-11-24T06:30:33.240220+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234550+0000","last_change":"2023-11-24T06:30:33.234550+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234550+0000","last_undegraded":"2023-11-24T06:30:33.234550+0000","last_fullsized":"2023-11-24T06:30:33.234550+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.241508+0000","last_change":"2023-11-24T06:30:33.241508+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.241508+0000","last_undegraded":"2023-11-24T06:30:33.241508+0000","last_fullsized":"2023-11-24T06:30:33.241508+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247042+0000","last_change":"2023-11-24T06:30:33.247042+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247042+0000","last_undegraded":"2023-11-24T06:30:33.247042+0000","last_fullsized":"2023-11-24T06:30:33.247042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242692+0000","last_change":"2023-11-24T06:30:33.242692+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242692+0000","last_undegraded":"2023-11-24T06:30:33.242692+0000","last_fullsized":"2023-11-24T06:30:33.242692+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237560+0000","last_change":"2023-11-24T06:30:33.237560+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237560+0000","last_undegraded":"2023-11-24T06:30:33.237560+0000","last_fullsized":"2023-11-24T06:30:33.237560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246556+0000","last_change":"2023-11-24T06:30:33.246556+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246556+0000","last_undegraded":"2023-11-24T06:30:33.246556+0000","last_fullsized":"2023-11-24T06:30:33.246556+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247176+0000","last_change":"2023-11-24T06:30:33.247176+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247176+0000","last_undegraded":"2023-11-24T06:30:33.247176+0000","last_fullsized":"2023-11-24T06:30:33.247176+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.248346+0000","last_change":"2023-11-24T06:30:33.248346+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.248346+0000","last_undegraded":"2023-11-24T06:30:33.248346+0000","last_fullsized":"2023-11-24T06:30:33.248346+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.252179+0000","last_change":"2023-11-24T06:30:33.252179+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.252179+0000","last_undegraded":"2023-11-24T06:30:33.252179+0000","last_fullsized":"2023-11-24T06:30:33.252179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.241747+0000","last_change":"2023-11-24T06:30:33.241747+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.241747+0000","last_undegraded":"2023-11-24T06:30:33.241747+0000","last_fullsized":"2023-11-24T06:30:33.241747+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.263585+0000","last_change":"2023-11-24T06:30:33.258485+0000","last_active":"2023-11-24T06:30:33.263585+0000","last_peered":"2023-11-24T06:30:33.263585+0000","last_clean":"2023-11-24T06:30:33.263585+0000","last_became_active":"2023-11-24T06:30:33.258091+0000","last_became_peered":"2023-11-24T06:30:33.258091+0000","last_unstale":"2023-11-24T06:30:33.263585+0000","last_undegraded":"2023-11-24T06:30:33.263585+0000","last_fullsized":"2023-11-24T06:30:33.263585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:39.081828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234860+0000","last_change":"2023-11-24T06:30:33.234860+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234860+0000","last_undegraded":"2023-11-24T06:30:33.234860+0000","last_fullsized":"2023-11-24T06:30:33.234860+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.257894+0000","last_change":"2023-11-24T06:30:33.253586+0000","last_active":"2023-11-24T06:30:33.257894+0000","last_peered":"2023-11-24T06:30:33.257894+0000","last_clean":"2023-11-24T06:30:33.257894+0000","last_became_active":"2023-11-24T06:30:33.252371+0000","last_became_peered":"2023-11-24T06:30:33.252371+0000","last_unstale":"2023-11-24T06:30:33.257894+0000","last_undegraded":"2023-11-24T06:30:33.257894+0000","last_fullsized":"2023-11-24T06:30:33.257894+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:42:17.431240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.252752+0000","last_change":"2023-11-24T06:30:33.252752+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.252752+0000","last_undegraded":"2023-11-24T06:30:33.252752+0000","last_fullsized":"2023-11-24T06:30:33.252752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267927+0000","last_change":"2023-11-24T06:30:33.266741+0000","last_active":"2023-11-24T06:30:33.267927+0000","last_peered":"2023-11-24T06:30:33.267927+0000","last_clean":"2023-11-24T06:30:33.267927+0000","last_became_active":"2023-11-24T06:30:33.266055+0000","last_became_peered":"2023-11-24T06:30:33.266055+0000","last_unstale":"2023-11-24T06:30:33.267927+0000","last_undegraded":"2023-11-24T06:30:33.267927+0000","last_fullsized":"2023-11-24T06:30:33.267927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:46.553265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.238829+0000","last_change":"2023-11-24T06:30:33.238829+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.238829+0000","last_undegraded":"2023-11-24T06:30:33.238829+0000","last_fullsized":"2023-11-24T06:30:33.238829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267672+0000","last_change":"2023-11-24T06:30:33.260247+0000","last_active":"2023-11-24T06:30:33.267672+0000","last_peered":"2023-11-24T06:30:33.267672+0000","last_clean":"2023-11-24T06:30:33.267672+0000","last_became_active":"2023-11-24T06:30:33.259986+0000","last_became_peered":"2023-11-24T06:30:33.259986+0000","last_unstale":"2023-11-24T06:30:33.267672+0000","last_undegraded":"2023-11-24T06:30:33.267672+0000","last_fullsized":"2023-11-24T06:30:33.267672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:25.530564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246365+0000","last_change":"2023-11-24T06:30:33.246365+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246365+0000","last_undegraded":"2023-11-24T06:30:33.246365+0000","last_fullsized":"2023-11-24T06:30:33.246365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.257173+0000","last_change":"2023-11-24T06:30:33.253225+0000","last_active":"2023-11-24T06:30:33.257173+0000","last_peered":"2023-11-24T06:30:33.257173+0000","last_clean":"2023-11-24T06:30:33.257173+0000","last_became_active":"2023-11-24T06:30:33.252940+0000","last_became_peered":"2023-11-24T06:30:33.252940+0000","last_unstale":"2023-11-24T06:30:33.257173+0000","last_undegraded":"2023-11-24T06:30:33.257173+0000","last_fullsized":"2023-11-24T06:30:33.257173+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:03.692439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.240042+0000","last_change":"2023-11-24T06:30:33.240042+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.240042+0000","last_undegraded":"2023-11-24T06:30:33.240042+0000","last_fullsized":"2023-11-24T06:30:33.240042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267861+0000","last_change":"2023-11-24T06:30:33.263099+0000","last_active":"2023-11-24T06:30:33.267861+0000","last_peered":"2023-11-24T06:30:33.267861+0000","last_clean":"2023-11-24T06:30:33.267861+0000","last_became_active":"2023-11-24T06:30:33.261288+0000","last_became_peered":"2023-11-24T06:30:33.261288+0000","last_unstale":"2023-11-24T06:30:33.267861+0000","last_undegraded":"2023-11-24T06:30:33.267861+0000","last_fullsized":"2023-11-24T06:30:33.267861+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:38.745762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234536+0000","last_change":"2023-11-24T06:30:33.234536+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234536+0000","last_undegraded":"2023-11-24T06:30:33.234536+0000","last_fullsized":"2023-11-24T06:30:33.234536+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.254629+0000","last_change":"2023-11-24T06:30:33.250035+0000","last_active":"2023-11-24T06:30:33.254629+0000","last_peered":"2023-11-24T06:30:33.254629+0000","last_clean":"2023-11-24T06:30:33.254629+0000","last_became_active":"2023-11-24T06:30:33.249747+0000","last_became_peered":"2023-11-24T06:30:33.249747+0000","last_unstale":"2023-11-24T06:30:33.254629+0000","last_undegraded":"2023-11-24T06:30:33.254629+0000","last_fullsized":"2023-11-24T06:30:33.254629+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:28.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246986+0000","last_change":"2023-11-24T06:30:33.246986+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246986+0000","last_undegraded":"2023-11-24T06:30:33.246986+0000","last_fullsized":"2023-11-24T06:30:33.246986+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.245386+0000","last_change":"2023-11-24T06:30:33.240278+0000","last_active":"2023-11-24T06:30:33.245386+0000","last_peered":"2023-11-24T06:30:33.245386+0000","last_clean":"2023-11-24T06:30:33.245386+0000","last_became_active":"2023-11-24T06:30:33.238103+0000","last_became_peered":"2023-11-24T06:30:33.238103+0000","last_unstale":"2023-11-24T06:30:33.245386+0000","last_undegraded":"2023-11-24T06:30:33.245386+0000","last_fullsized":"2023-11-24T06:30:33.245386+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:01:20.140182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246380+0000","last_change":"2023-11-24T06:30:33.246380+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246380+0000","last_undegraded":"2023-11-24T06:30:33.246380+0000","last_fullsized":"2023-11-24T06:30:33.246380+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261289+0000","last_change":"2023-11-24T06:30:33.257372+0000","last_active":"2023-11-24T06:30:33.261289+0000","last_peered":"2023-11-24T06:30:33.261289+0000","last_clean":"2023-11-24T06:30:33.261289+0000","last_became_active":"2023-11-24T06:30:33.257063+0000","last_became_peered":"2023-11-24T06:30:33.257063+0000","last_unstale":"2023-11-24T06:30:33.261289+0000","last_undegraded":"2023-11-24T06:30:33.261289+0000","last_fullsized":"2023-11-24T06:30:33.261289+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:51.884745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242375+0000","last_change":"2023-11-24T06:30:33.242375+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242375+0000","last_undegraded":"2023-11-24T06:30:33.242375+0000","last_fullsized":"2023-11-24T06:30:33.242375+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269341+0000","last_change":"2023-11-24T06:30:33.266155+0000","last_active":"2023-11-24T06:30:33.269341+0000","last_peered":"2023-11-24T06:30:33.269341+0000","last_clean":"2023-11-24T06:30:33.269341+0000","last_became_active":"2023-11-24T06:30:33.265258+0000","last_became_peered":"2023-11-24T06:30:33.265258+0000","last_unstale":"2023-11-24T06:30:33.269341+0000","last_undegraded":"2023-11-24T06:30:33.269341+0000","last_fullsized":"2023-11-24T06:30:33.269341+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:31.950626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.248625+0000","last_change":"2023-11-24T06:30:33.248625+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.248625+0000","last_undegraded":"2023-11-24T06:30:33.248625+0000","last_fullsized":"2023-11-24T06:30:33.248625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265546+0000","last_change":"2023-11-24T06:30:33.264658+0000","last_active":"2023-11-24T06:30:33.265546+0000","last_peered":"2023-11-24T06:30:33.265546+0000","last_clean":"2023-11-24T06:30:33.265546+0000","last_became_active":"2023-11-24T06:30:33.264320+0000","last_became_peered":"2023-11-24T06:30:33.264320+0000","last_unstale":"2023-11-24T06:30:33.265546+0000","last_undegraded":"2023-11-24T06:30:33.265546+0000","last_fullsized":"2023-11-24T06:30:33.265546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:41.422715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235171+0000","last_change":"2023-11-24T06:30:33.235171+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235171+0000","last_undegraded":"2023-11-24T06:30:33.235171+0000","last_fullsized":"2023-11-24T06:30:33.235171+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.242786+0000","last_change":"2023-11-24T06:30:33.239974+0000","last_active":"2023-11-24T06:30:33.242786+0000","last_peered":"2023-11-24T06:30:33.242786+0000","last_clean":"2023-11-24T06:30:33.242786+0000","last_became_active":"2023-11-24T06:30:33.238273+0000","last_became_peered":"2023-11-24T06:30:33.238273+0000","last_unstale":"2023-11-24T06:30:33.242786+0000","last_undegraded":"2023-11-24T06:30:33.242786+0000","last_fullsized":"2023-11-24T06:30:33.242786+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:12.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.238368+0000","last_change":"2023-11-24T06:30:33.238368+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.238368+0000","last_undegraded":"2023-11-24T06:30:33.238368+0000","last_fullsized":"2023-11-24T06:30:33.238368+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.253699+0000","last_change":"2023-11-24T06:30:33.252044+0000","last_active":"2023-11-24T06:30:33.253699+0000","last_peered":"2023-11-24T06:30:33.253699+0000","last_clean":"2023-11-24T06:30:33.253699+0000","last_became_active":"2023-11-24T06:30:33.251774+0000","last_became_peered":"2023-11-24T06:30:33.251774+0000","last_unstale":"2023-11-24T06:30:33.253699+0000","last_undegraded":"2023-11-24T06:30:33.253699+0000","last_fullsized":"2023-11-24T06:30:33.253699+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:38.745762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.248362+0000","last_change":"2023-11-24T06:30:33.248362+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.248362+0000","last_undegraded":"2023-11-24T06:30:33.248362+0000","last_fullsized":"2023-11-24T06:30:33.248362+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267841+0000","last_change":"2023-11-24T06:30:33.261613+0000","last_active":"2023-11-24T06:30:33.267841+0000","last_peered":"2023-11-24T06:30:33.267841+0000","last_clean":"2023-11-24T06:30:33.267841+0000","last_became_active":"2023-11-24T06:30:33.260523+0000","last_became_peered":"2023-11-24T06:30:33.260523+0000","last_unstale":"2023-11-24T06:30:33.267841+0000","last_undegraded":"2023-11-24T06:30:33.267841+0000","last_fullsized":"2023-11-24T06:30:33.267841+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:59:27.319046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.249240+0000","last_change":"2023-11-24T06:30:33.249240+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.249240+0000","last_undegraded":"2023-11-24T06:30:33.249240+0000","last_fullsized":"2023-11-24T06:30:33.249240+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265229+0000","last_change":"2023-11-24T06:30:33.263590+0000","last_active":"2023-11-24T06:30:33.265229+0000","last_peered":"2023-11-24T06:30:33.265229+0000","last_clean":"2023-11-24T06:30:33.265229+0000","last_became_active":"2023-11-24T06:30:33.262788+0000","last_became_peered":"2023-11-24T06:30:33.262788+0000","last_unstale":"2023-11-24T06:30:33.265229+0000","last_undegraded":"2023-11-24T06:30:33.265229+0000","last_fullsized":"2023-11-24T06:30:33.265229+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:10.481218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.233300+0000","last_change":"2023-11-24T06:30:33.233300+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.233300+0000","last_undegraded":"2023-11-24T06:30:33.233300+0000","last_fullsized":"2023-11-24T06:30:33.233300+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261672+0000","last_change":"2023-11-24T06:30:33.259288+0000","last_active":"2023-11-24T06:30:33.261672+0000","last_peered":"2023-11-24T06:30:33.261672+0000","last_clean":"2023-11-24T06:30:33.261672+0000","last_became_active":"2023-11-24T06:30:33.254396+0000","last_became_peered":"2023-11-24T06:30:33.254396+0000","last_unstale":"2023-11-24T06:30:33.261672+0000","last_undegraded":"2023-11-24T06:30:33.261672+0000","last_fullsized":"2023-11-24T06:30:33.261672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:30.689902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237823+0000","last_change":"2023-11-24T06:30:33.237823+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237823+0000","last_undegraded":"2023-11-24T06:30:33.237823+0000","last_fullsized":"2023-11-24T06:30:33.237823+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269298+0000","last_change":"2023-11-24T06:30:33.265823+0000","last_active":"2023-11-24T06:30:33.269298+0000","last_peered":"2023-11-24T06:30:33.269298+0000","last_clean":"2023-11-24T06:30:33.269298+0000","last_became_active":"2023-11-24T06:30:33.265509+0000","last_became_peered":"2023-11-24T06:30:33.265509+0000","last_unstale":"2023-11-24T06:30:33.269298+0000","last_undegraded":"2023-11-24T06:30:33.269298+0000","last_fullsized":"2023-11-24T06:30:33.269298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:21.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.241198+0000","last_change":"2023-11-24T06:30:33.241198+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.241198+0000","last_undegraded":"2023-11-24T06:30:33.241198+0000","last_fullsized":"2023-11-24T06:30:33.241198+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269221+0000","last_change":"2023-11-24T06:30:33.268072+0000","last_active":"2023-11-24T06:30:33.269221+0000","last_peered":"2023-11-24T06:30:33.269221+0000","last_clean":"2023-11-24T06:30:33.269221+0000","last_became_active":"2023-11-24T06:30:33.267757+0000","last_became_peered":"2023-11-24T06:30:33.267757+0000","last_unstale":"2023-11-24T06:30:33.269221+0000","last_undegraded":"2023-11-24T06:30:33.269221+0000","last_fullsized":"2023-11-24T06:30:33.269221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:18.630896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235215+0000","last_change":"2023-11-24T06:30:33.235215+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235215+0000","last_undegraded":"2023-11-24T06:30:33.235215+0000","last_fullsized":"2023-11-24T06:30:33.235215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269306+0000","last_change":"2023-11-24T06:30:33.265903+0000","last_active":"2023-11-24T06:30:33.269306+0000","last_peered":"2023-11-24T06:30:33.269306+0000","last_clean":"2023-11-24T06:30:33.269306+0000","last_became_active":"2023-11-24T06:30:33.263286+0000","last_became_peered":"2023-11-24T06:30:33.263286+0000","last_unstale":"2023-11-24T06:30:33.269306+0000","last_undegraded":"2023-11-24T06:30:33.269306+0000","last_fullsized":"2023-11-24T06:30:33.269306+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:49:20.198853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.233222+0000","last_change":"2023-11-24T06:30:33.233222+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.233222+0000","last_undegraded":"2023-11-24T06:30:33.233222+0000","last_fullsized":"2023-11-24T06:30:33.233222+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.263521+0000","last_change":"2023-11-24T06:30:33.259236+0000","last_active":"2023-11-24T06:30:33.263521+0000","last_peered":"2023-11-24T06:30:33.263521+0000","last_clean":"2023-11-24T06:30:33.263521+0000","last_became_active":"2023-11-24T06:30:33.258200+0000","last_became_peered":"2023-11-24T06:30:33.258200+0000","last_unstale":"2023-11-24T06:30:33.263521+0000","last_undegraded":"2023-11-24T06:30:33.263521+0000","last_fullsized":"2023-11-24T06:30:33.263521+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:08.963743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.243611+0000","last_change":"2023-11-24T06:30:33.243611+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.243611+0000","last_undegraded":"2023-11-24T06:30:33.243611+0000","last_fullsized":"2023-11-24T06:30:33.243611+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267605+0000","last_change":"2023-11-24T06:30:33.259814+0000","last_active":"2023-11-24T06:30:33.267605+0000","last_peered":"2023-11-24T06:30:33.267605+0000","last_clean":"2023-11-24T06:30:33.267605+0000","last_became_active":"2023-11-24T06:30:33.259551+0000","last_became_peered":"2023-11-24T06:30:33.259551+0000","last_unstale":"2023-11-24T06:30:33.267605+0000","last_undegraded":"2023-11-24T06:30:33.267605+0000","last_fullsized":"2023-11-24T06:30:33.267605+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:28.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242279+0000","last_change":"2023-11-24T06:30:33.242279+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242279+0000","last_undegraded":"2023-11-24T06:30:33.242279+0000","last_fullsized":"2023-11-24T06:30:33.242279+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265212+0000","last_change":"2023-11-24T06:30:33.263043+0000","last_active":"2023-11-24T06:30:33.265212+0000","last_peered":"2023-11-24T06:30:33.265212+0000","last_clean":"2023-11-24T06:30:33.265212+0000","last_became_active":"2023-11-24T06:30:33.262727+0000","last_became_peered":"2023-11-24T06:30:33.262727+0000","last_unstale":"2023-11-24T06:30:33.265212+0000","last_undegraded":"2023-11-24T06:30:33.265212+0000","last_fullsized":"2023-11-24T06:30:33.265212+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:33.658163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242662+0000","last_change":"2023-11-24T06:30:33.242662+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242662+0000","last_undegraded":"2023-11-24T06:30:33.242662+0000","last_fullsized":"2023-11-24T06:30:33.242662+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.262428+0000","last_change":"2023-11-24T06:30:33.259539+0000","last_active":"2023-11-24T06:30:33.262428+0000","last_peered":"2023-11-24T06:30:33.262428+0000","last_clean":"2023-11-24T06:30:33.262428+0000","last_became_active":"2023-11-24T06:30:33.258994+0000","last_became_peered":"2023-11-24T06:30:33.258994+0000","last_unstale":"2023-11-24T06:30:33.262428+0000","last_undegraded":"2023-11-24T06:30:33.262428+0000","last_fullsized":"2023-11-24T06:30:33.262428+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:40.643302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.249084+0000","last_change":"2023-11-24T06:30:33.249084+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.249084+0000","last_undegraded":"2023-11-24T06:30:33.249084+0000","last_fullsized":"2023-11-24T06:30:33.249084+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267896+0000","last_change":"2023-11-24T06:30:33.264626+0000","last_active":"2023-11-24T06:30:33.267896+0000","last_peered":"2023-11-24T06:30:33.267896+0000","last_clean":"2023-11-24T06:30:33.267896+0000","last_became_active":"2023-11-24T06:30:33.264323+0000","last_became_peered":"2023-11-24T06:30:33.264323+0000","last_unstale":"2023-11-24T06:30:33.267896+0000","last_undegraded":"2023-11-24T06:30:33.267896+0000","last_fullsized":"2023-11-24T06:30:33.267896+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:44.399558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.254177+0000","last_change":"2023-11-24T06:30:33.254177+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.254177+0000","last_undegraded":"2023-11-24T06:30:33.254177+0000","last_fullsized":"2023-11-24T06:30:33.254177+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265269+0000","last_change":"2023-11-24T06:30:33.262138+0000","last_active":"2023-11-24T06:30:33.265269+0000","last_peered":"2023-11-24T06:30:33.265269+0000","last_clean":"2023-11-24T06:30:33.265269+0000","last_became_active":"2023-11-24T06:30:33.261687+0000","last_became_peered":"2023-11-24T06:30:33.261687+0000","last_unstale":"2023-11-24T06:30:33.265269+0000","last_undegraded":"2023-11-24T06:30:33.265269+0000","last_fullsized":"2023-11-24T06:30:33.265269+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:05.740113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239053+0000","last_change":"2023-11-24T06:30:33.239053+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239053+0000","last_undegraded":"2023-11-24T06:30:33.239053+0000","last_fullsized":"2023-11-24T06:30:33.239053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.257585+0000","last_change":"2023-11-24T06:30:33.255156+0000","last_active":"2023-11-24T06:30:33.257585+0000","last_peered":"2023-11-24T06:30:33.257585+0000","last_clean":"2023-11-24T06:30:33.257585+0000","last_became_active":"2023-11-24T06:30:33.254878+0000","last_became_peered":"2023-11-24T06:30:33.254878+0000","last_unstale":"2023-11-24T06:30:33.257585+0000","last_undegraded":"2023-11-24T06:30:33.257585+0000","last_fullsized":"2023-11-24T06:30:33.257585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:38.180557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237564+0000","last_change":"2023-11-24T06:30:33.237564+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237564+0000","last_undegraded":"2023-11-24T06:30:33.237564+0000","last_fullsized":"2023-11-24T06:30:33.237564+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.264428+0000","last_change":"2023-11-24T06:30:33.263337+0000","last_active":"2023-11-24T06:30:33.264428+0000","last_peered":"2023-11-24T06:30:33.264428+0000","last_clean":"2023-11-24T06:30:33.264428+0000","last_became_active":"2023-11-24T06:30:33.262997+0000","last_became_peered":"2023-11-24T06:30:33.262997+0000","last_unstale":"2023-11-24T06:30:33.264428+0000","last_undegraded":"2023-11-24T06:30:33.264428+0000","last_fullsized":"2023-11-24T06:30:33.264428+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:00.154357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.232644+0000","last_change":"2023-11-24T06:30:33.232644+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.232644+0000","last_undegraded":"2023-11-24T06:30:33.232644+0000","last_fullsized":"2023-11-24T06:30:33.232644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261315+0000","last_change":"2023-11-24T06:30:33.258636+0000","last_active":"2023-11-24T06:30:33.261315+0000","last_peered":"2023-11-24T06:30:33.261315+0000","last_clean":"2023-11-24T06:30:33.261315+0000","last_became_active":"2023-11-24T06:30:33.258193+0000","last_became_peered":"2023-11-24T06:30:33.258193+0000","last_unstale":"2023-11-24T06:30:33.261315+0000","last_undegraded":"2023-11-24T06:30:33.261315+0000","last_fullsized":"2023-11-24T06:30:33.261315+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.507654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.249296+0000","last_change":"2023-11-24T06:30:33.249296+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.249296+0000","last_undegraded":"2023-11-24T06:30:33.249296+0000","last_fullsized":"2023-11-24T06:30:33.249296+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.256814+0000","last_change":"2023-11-24T06:30:33.253808+0000","last_active":"2023-11-24T06:30:33.256814+0000","last_peered":"2023-11-24T06:30:33.256814+0000","last_clean":"2023-11-24T06:30:33.256814+0000","last_became_active":"2023-11-24T06:30:33.250888+0000","last_became_peered":"2023-11-24T06:30:33.250888+0000","last_unstale":"2023-11-24T06:30:33.256814+0000","last_undegraded":"2023-11-24T06:30:33.256814+0000","last_fullsized":"2023-11-24T06:30:33.256814+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:59:27.319046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234698+0000","last_change":"2023-11-24T06:30:33.234698+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234698+0000","last_undegraded":"2023-11-24T06:30:33.234698+0000","last_fullsized":"2023-11-24T06:30:33.234698+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261935+0000","last_change":"2023-11-24T06:30:33.257088+0000","last_active":"2023-11-24T06:30:33.261935+0000","last_peered":"2023-11-24T06:30:33.261935+0000","last_clean":"2023-11-24T06:30:33.261935+0000","last_became_active":"2023-11-24T06:30:33.256733+0000","last_became_peered":"2023-11-24T06:30:33.256733+0000","last_unstale":"2023-11-24T06:30:33.261935+0000","last_undegraded":"2023-11-24T06:30:33.261935+0000","last_fullsized":"2023-11-24T06:30:33.261935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:52.534613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.251567+0000","last_change":"2023-11-24T06:30:33.251567+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.251567+0000","last_undegraded":"2023-11-24T06:30:33.251567+0000","last_fullsized":"2023-11-24T06:30:33.251567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269279+0000","last_change":"2023-11-24T06:30:33.265710+0000","last_active":"2023-11-24T06:30:33.269279+0000","last_peered":"2023-11-24T06:30:33.269279+0000","last_clean":"2023-11-24T06:30:33.269279+0000","last_became_active":"2023-11-24T06:30:33.265397+0000","last_became_peered":"2023-11-24T06:30:33.265397+0000","last_unstale":"2023-11-24T06:30:33.269279+0000","last_undegraded":"2023-11-24T06:30:33.269279+0000","last_fullsized":"2023-11-24T06:30:33.269279+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:30.859810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242319+0000","last_change":"2023-11-24T06:30:33.242319+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242319+0000","last_undegraded":"2023-11-24T06:30:33.242319+0000","last_fullsized":"2023-11-24T06:30:33.242319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267727+0000","last_change":"2023-11-24T06:30:33.260898+0000","last_active":"2023-11-24T06:30:33.267727+0000","last_peered":"2023-11-24T06:30:33.267727+0000","last_clean":"2023-11-24T06:30:33.267727+0000","last_became_active":"2023-11-24T06:30:33.260528+0000","last_became_peered":"2023-11-24T06:30:33.260528+0000","last_unstale":"2023-11-24T06:30:33.267727+0000","last_undegraded":"2023-11-24T06:30:33.267727+0000","last_fullsized":"2023-11-24T06:30:33.267727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:12.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-24T06:30:32.217624+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:30:32.217624+0000","last_peered":"2023-11-24T06:30:32.217624+0000","last_clean":"2023-11-24T06:30:32.217624+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:30:32.217624+0000","last_undegraded":"2023-11-24T06:30:32.217624+0000","last_fullsized":"2023-11-24T06:30:32.217624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":942410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.252,"15min":1.3160000000000001},"min":{"1min":0.68100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.0599999999999996,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.123},{"interface":"front","average":{"1min":1.389,"5min":1.296,"15min":1.3009999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":4.556,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.524}]},{"osd":1,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0249999999999999,"15min":1.012},"min":{"1min":0.68200000000000005,"5min":0.629,"15min":0.629},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.343,"15min":1.377},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":3.944,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.4419999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.2729999999999999,"15min":1.288},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.5609999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.3539999999999992,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":1.306},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.204},"min":{"1min":0.79300000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.8200000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.714}]},{"osd":4,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.4870000000000001,"15min":1.5669999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.1059999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.331},{"interface":"front","average":{"1min":1.498,"5min":1.5369999999999999,"15min":1.579},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.6000000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.3580000000000001,"15min":1.399},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.3119999999999998,"5min":1.8240000000000001,"15min":1.794},"min":{"1min":0.69799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":17.582999999999998,"5min":17.582999999999998,"15min":17.582999999999998},"last":1.6739999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.736,"15min":1.8480000000000001},"min":{"1min":0.73099999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.8140000000000001,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.744},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.6240000000000001,"15min":1.694},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":6.2210000000000001,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.3999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0980000000000001,"15min":1.1020000000000001},"min":{"1min":0.73499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7390000000000001,"5min":1.829,"15min":1.829},"last":1.03},{"interface":"front","average":{"1min":1.155,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.76800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.623,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.234}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2210000000000001,"15min":1.2130000000000001},"min":{"1min":0.71099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6299999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.573},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.2010000000000001,"15min":1.2070000000000001},"min":{"1min":0.88900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.494,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.323}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.159},"min":{"1min":0.75700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.609,"5min":3.597,"15min":3.597},"last":1.704},{"interface":"front","average":{"1min":1.304,"5min":1.292,"15min":1.3140000000000001},"min":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6359999999999999,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.6579999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3129999999999999,"15min":1.3180000000000001},"min":{"1min":0.78100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5620000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.2969999999999999,"15min":1.325},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5960000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.52}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.8919999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.385,"15min":1.397},"min":{"1min":0.79600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7729999999999999,"5min":3.04,"15min":3.04},"last":1.091}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3720000000000001,"15min":1.377},"min":{"1min":0.97699999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7330000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.552,"15min":1.6020000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.851,"5min":3.774,"15min":3.774},"last":1.397}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5449999999999999,"15min":1.5589999999999999},"min":{"1min":1.0029999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7689999999999999,"5min":3.403,"15min":3.403},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.512},"min":{"1min":0.875,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.792,"5min":3.649,"15min":3.649},"last":1.2649999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.516,"5min":1.903,"15min":1.903},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.288,"15min":1.3140000000000001},"min":{"1min":0.91700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.915,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.5349999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.145,"15min":1.1399999999999999},"min":{"1min":0.78900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.147,"15min":1.1140000000000001},"min":{"1min":0.89500000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.347}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.804},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3109999999999999,"15min":1.2969999999999999},"min":{"1min":0.99199999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1669999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.2969999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3360000000000001,"15min":1.375},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.952,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.405},{"interface":"front","average":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3899999999999999},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.5840000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3180000000000001,"15min":1.3340000000000001},"min":{"1min":0.878,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.839,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.5029999999999999,"15min":1.5369999999999999},"min":{"1min":0.82299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0099999999999998,"5min":3.085,"15min":3.085},"last":1.641}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4690000000000001,"15min":1.4450000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.464},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.474,"15min":1.468},"min":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0270000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.5069999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.651,"5min":1.6140000000000001,"15min":1.6240000000000001},"min":{"1min":1.046,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.141,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.714,"5min":1.6919999999999999,"15min":1.716},"min":{"1min":1.0129999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3159999999999998,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.673}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.073,"15min":1.073},"min":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.982,"15min":1.982},"last":1.222},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.089,"15min":1.1120000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.468,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.262}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.147,"15min":1.113},"min":{"1min":0.84199999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.26,"15min":1.3049999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5840000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.0980000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8129999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.27,"5min":1.3140000000000001,"15min":1.3320000000000001},"min":{"1min":0.79800000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7250000000000001,"5min":2.254,"15min":2.254},"last":1.3}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.294,"15min":1.333},"min":{"1min":0.76100000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.833,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3779999999999999,"15min":1.3720000000000001},"min":{"1min":0.753,"5min":0.746,"15min":0.746},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.071}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.482,"15min":1.5149999999999999},"min":{"1min":0.874,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8109999999999999,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.522,"15min":1.5609999999999999},"min":{"1min":0.85799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.923,"5min":3.52,"15min":3.52},"last":1.179}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.474,"15min":1.482},"min":{"1min":0.79200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.542,"15min":1.5740000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.859,"5min":2.452,"15min":2.452},"last":1.359}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.595,"15min":1.6559999999999999},"min":{"1min":0.64000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.7729999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.615,"15min":1.613},"min":{"1min":0.84799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.0099999999999998,"5min":2.363,"15min":2.363},"last":1.2030000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.3080000000000001,"5min":1.355,"15min":1.355},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.66600000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.96399999999999997},"min":{"1min":0.51800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.077,"5min":1.091,"15min":1.113},"min":{"1min":0.61499999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.91900000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.0350000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.58499999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4550000000000001,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.133}]},{"osd":4,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3420000000000001,"15min":1.3879999999999999},"min":{"1min":0.91300000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.696,"5min":3.081,"15min":3.081},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.3340000000000001,"15min":1.339},"min":{"1min":0.74299999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.712,"5min":2.758,"15min":2.758},"last":1.7230000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.373,"15min":1.403},"min":{"1min":0.873,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.1200000000000001,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.462},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.6200000000000001,"15min":1.748},"min":{"1min":0.86299999999999999,"5min":0.745,"15min":0.745},"max":{"1min":1.657,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.786}]},{"osd":6,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3759999999999999,"15min":1.407},"min":{"1min":0.82999999999999996,"5min":0.753,"15min":0.753},"max":{"1min":1.79,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.534},{"interface":"front","average":{"1min":1.524,"5min":1.53,"15min":1.544},"min":{"1min":1.0760000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.464,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.4790000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.5760000000000001,"15min":1.609},"min":{"1min":0.84499999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.589,"5min":4.641,"15min":4.641},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.546,"15min":1.5740000000000001},"min":{"1min":0.88200000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.3439999999999999,"5min":3.798,"15min":3.798},"last":1.7490000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2729999999999999,"5min":1.427,"15min":1.427},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.619,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.59699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.48499999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.78800000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0840000000000001,"15min":1.1040000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.784,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.095,"15min":1.117},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6990000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.575}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.456,"15min":1.508},"min":{"1min":0.67900000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.6869999999999998,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.454,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.5800000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4650000000000001,"15min":1.468},"min":{"1min":1.0029999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.036,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.4690000000000001,"15min":1.5009999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.841,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.2769999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.423,"15min":1.4359999999999999},"min":{"1min":0.66900000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.968,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.004},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.522,"15min":1.54},"min":{"1min":1.0029999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.165,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.444}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5149999999999999,"15min":1.5489999999999999},"min":{"1min":0.78200000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.913,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":1.097,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.423,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.9359999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.002,"15min":1.032},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.696,"5min":2.593,"15min":2.593},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.008,"5min":1.02,"15min":1.032},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5980000000000001,"5min":1.702,"15min":1.702},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.212,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":1,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7669999999999999,"5min":1.829,"15min":1.829},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1040000000000001,"15min":1.1200000000000001},"min":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.603,"5min":1.766,"15min":1.766},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0289999999999999,"15min":1.069},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2250000000000001,"5min":1.798,"15min":1.798},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.246,"15min":1.2529999999999999},"min":{"1min":0.89000000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.621,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.359,"5min":1.417,"15min":1.4450000000000001},"min":{"1min":1.0509999999999999,"5min":1.032,"15min":1.032},"max":{"1min":1.837,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":0.96299999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.294,"15min":1.302},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.514,"5min":2.016,"15min":2.016},"last":1.304},{"interface":"front","average":{"1min":1.391,"5min":1.389,"15min":1.399},"min":{"1min":0.91700000000000004,"5min":0.877,"15min":0.877},"max":{"1min":1.7410000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.071}]},{"osd":6,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.391,"15min":1.393},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.847,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4339999999999999,"15min":1.45},"min":{"1min":0.96399999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.6499999999999999,"5min":2.371,"15min":2.371},"last":1.2509999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.444,"15min":1.448},"min":{"1min":0.89300000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9730000000000001,"5min":2.246,"15min":2.246},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4950000000000001,"15min":1.5189999999999999},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.7629999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.94499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":942410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.872,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.92400000000000004,"15min":0.95899999999999996},"min":{"1min":0.39200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.516,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.88400000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.004,"15min":1},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6619999999999999,"5min":1.784,"15min":1.784},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.1060000000000001,"15min":1.1319999999999999},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.639,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.94899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4750000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.077,"15min":1.087},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.506,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.204}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.23,"15min":1.226},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7070000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.296},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3129999999999999},"min":{"1min":1.0149999999999999,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.714,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.496}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.329,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9610000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.415},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.2709999999999999,"15min":1.278},"min":{"1min":0.86099999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.865,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1060000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.4570000000000001,"15min":1.4830000000000001},"min":{"1min":0.84199999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8500000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.353},{"interface":"front","average":{"1min":1.429,"5min":1.4710000000000001,"15min":1.5109999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.903,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.262}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.448,"15min":1.456},"min":{"1min":0.95999999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8180000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.329},{"interface":"front","average":{"1min":1.52,"5min":1.417,"15min":1.425},"min":{"1min":0.84399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0630000000000002,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.446}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:30:38.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:30:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.013+0000 7fd1725b3640 1 Processor -- start 2023-11-24T06:30:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.013+0000 7fd1725b3640 1 -- start start 2023-11-24T06:30:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.015+0000 7fd1725b3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd16c143c40 0x7fd16c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.015+0000 7fd1725b3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 0x7fd16c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.015+0000 7fd1725b3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16c070e50 con 0x7fd16c0717d0 2023-11-24T06:30:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.015+0000 7fd1725b3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16c0723c0 con 0x7fd16c143c40 2023-11-24T06:30:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.015+0000 7fd1725b3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c144020 con 0x7fd16c0ccc10 2023-11-24T06:30:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd170db0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd16c0717d0 0x7fd16c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44748/0 (socket says 172.21.15.16:44748) 2023-11-24T06:30:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd170db0640 1 -- 172.21.15.16:0/568308883 learned_addr learned my addr 172.21.15.16:0/568308883 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd1715b1640 1 --2- 172.21.15.16:0/568308883 >> v2:172.21.15.16:3300/0 conn(0x7fd16c143c40 0x7fd16c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd171db2640 1 --2- 172.21.15.16:0/568308883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 0x7fd16c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd163fff640 1 -- 172.21.15.16:0/568308883 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473341122 0 0) 0x7fd16c070e50 con 0x7fd16c0717d0 2023-11-24T06:30:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd163fff640 1 -- 172.21.15.16:0/568308883 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd158003520 con 0x7fd16c0717d0 2023-11-24T06:30:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd1715b1640 1 -- 172.21.15.16:0/568308883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 msgr2=0x7fd16c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd1715b1640 1 --2- 172.21.15.16:0/568308883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 0x7fd16c06d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd1715b1640 1 -- 172.21.15.16:0/568308883 >> v1:172.21.15.16:6789/0 conn(0x7fd16c0717d0 legacy=0x7fd16c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.016+0000 7fd1715b1640 1 -- 172.21.15.16:0/568308883 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c071e00 con 0x7fd16c143c40 2023-11-24T06:30:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.017+0000 7fd1715b1640 1 --2- 172.21.15.16:0/568308883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c143c40 0x7fd16c071290 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd1680097a0 tx=0x7fd16802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83215f107b4d3e52 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.018+0000 7fd163fff640 1 -- 172.21.15.16:0/568308883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16802e070 con 0x7fd16c143c40 2023-11-24T06:30:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.018+0000 7fd163fff640 1 -- 172.21.15.16:0/568308883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1680315a0 con 0x7fd16c143c40 2023-11-24T06:30:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.018+0000 7fd163fff640 1 -- 172.21.15.16:0/568308883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168031930 con 0x7fd16c143c40 2023-11-24T06:30:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.018+0000 7fd1725b3640 1 -- 172.21.15.16:0/568308883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c143c40 msgr2=0x7fd16c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.018+0000 7fd1725b3640 1 --2- 172.21.15.16:0/568308883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c143c40 0x7fd16c071290 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd1680097a0 tx=0x7fd16802b840 comp rx=0 tx=0).stop 2023-11-24T06:30:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.019+0000 7fd1725b3640 1 -- 172.21.15.16:0/568308883 shutdown_connections 2023-11-24T06:30:39.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.019+0000 7fd1725b3640 1 --2- 172.21.15.16:0/568308883 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 0x7fd16c06d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.019+0000 7fd1725b3640 1 --2- 172.21.15.16:0/568308883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c143c40 0x7fd16c071290 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.019+0000 7fd1725b3640 1 -- 172.21.15.16:0/568308883 >> 172.21.15.16:0/568308883 conn(0x7fd16c06a800 msgr2=0x7fd16c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.019+0000 7fd1725b3640 1 -- 172.21.15.16:0/568308883 shutdown_connections 2023-11-24T06:30:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.019+0000 7fd1725b3640 1 -- 172.21.15.16:0/568308883 wait complete. 2023-11-24T06:30:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.021+0000 7fd1725b3640 1 Processor -- start 2023-11-24T06:30:39.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.021+0000 7fd1725b3640 1 -- start start 2023-11-24T06:30:39.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1725b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c0717d0 0x7fd16c0c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1725b3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 0x7fd16c0c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1725b3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd16c143c40 0x7fd16c0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1725b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0c58f0 con 0x7fd16c0717d0 2023-11-24T06:30:39.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1725b3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0c64c0 con 0x7fd16c143c40 2023-11-24T06:30:39.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1725b3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0c4310 con 0x7fd16c0ccc10 2023-11-24T06:30:39.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1715b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c0717d0 0x7fd16c0c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1715b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c0717d0 0x7fd16c0c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36452/0 (socket says 172.21.15.16:36452) 2023-11-24T06:30:39.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1715b1640 1 -- 172.21.15.16:0/2435898857 learned_addr learned my addr 172.21.15.16:0/2435898857 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:39.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd170db0640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 0x7fd16c0c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1715b1640 1 -- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 msgr2=0x7fd16c0c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1715b1640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 0x7fd16c0c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1715b1640 1 -- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd16c143c40 msgr2=0x7fd16c0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1715b1640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd16c143c40 0x7fd16c0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.023+0000 7fd1715b1640 1 -- 172.21.15.16:0/2435898857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009400 con 0x7fd16c0717d0 2023-11-24T06:30:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.024+0000 7fd1715b1640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c0717d0 0x7fd16c0c53b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd168034040 tx=0x7fd1680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.024+0000 7fd161ffb640 1 -- 172.21.15.16:0/2435898857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168032840 con 0x7fd16c0717d0 2023-11-24T06:30:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.024+0000 7fd161ffb640 1 -- 172.21.15.16:0/2435898857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168031f00 con 0x7fd16c0717d0 2023-11-24T06:30:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.024+0000 7fd161ffb640 1 -- 172.21.15.16:0/2435898857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168032be0 con 0x7fd16c0717d0 2023-11-24T06:30:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.024+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c0c45f0 con 0x7fd16c0717d0 2023-11-24T06:30:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.024+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c0c4bb0 con 0x7fd16c0717d0 2023-11-24T06:30:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.025+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c067450 con 0x7fd16c0717d0 2023-11-24T06:30:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.026+0000 7fd161ffb640 1 -- 172.21.15.16:0/2435898857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd168032d40 con 0x7fd16c0717d0 2023-11-24T06:30:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.027+0000 7fd161ffb640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15006b230 0x7fd15006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.027+0000 7fd161ffb640 1 -- 172.21.15.16:0/2435898857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1680abdd0 con 0x7fd16c0717d0 2023-11-24T06:30:39.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.028+0000 7fd170db0640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15006b230 0x7fd15006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.029+0000 7fd170db0640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15006b230 0x7fd15006d6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0c8970 tx=0x7fd15c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:39.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.033+0000 7fd161ffb640 1 -- 172.21.15.16:0/2435898857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1680767e0 con 0x7fd16c0717d0 2023-11-24T06:30:39.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.226+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd16c06cc40 con 0x7fd15006b230 2023-11-24T06:30:39.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.247+0000 7fd161ffb640 1 -- 172.21.15.16:0/2435898857 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281025 (secure 0 0 0) 0x7fd15c005890 con 0x7fd15006b230 2023-11-24T06:30:39.254 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:30:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15006b230 msgr2=0x7fd15006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15006b230 0x7fd15006d6f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0c8970 tx=0x7fd15c009040 comp rx=0 tx=0).stop 2023-11-24T06:30:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c0717d0 msgr2=0x7fd16c0c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c0717d0 0x7fd16c0c53b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd168034040 tx=0x7fd1680046c0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 shutdown_connections 2023-11-24T06:30:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15006b230 0x7fd15006d6f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd16c143c40 0x7fd16c0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd16c0ccc10 0x7fd16c0c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 --2- 172.21.15.16:0/2435898857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd16c0717d0 0x7fd16c0c53b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 >> 172.21.15.16:0/2435898857 conn(0x7fd16c06a800 msgr2=0x7fd16c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 shutdown_connections 2023-11-24T06:30:39.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.253+0000 7fd1725b3640 1 -- 172.21.15.16:0/2435898857 wait complete. 2023-11-24T06:30:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2023-11-24T06:30:38.178434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253000,"kb_used_data":7720,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721720,"statfs":{"total":773094113280,"available":772835041280,"internally_reserved":0,"allocated":7905280,"data_stored":3997072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.964995"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.244535+0000","last_change":"2023-11-24T06:30:33.244535+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.244535+0000","last_undegraded":"2023-11-24T06:30:33.244535+0000","last_fullsized":"2023-11-24T06:30:33.244535+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.259035+0000","last_change":"2023-11-24T06:30:33.259035+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.259035+0000","last_undegraded":"2023-11-24T06:30:33.259035+0000","last_fullsized":"2023-11-24T06:30:33.259035+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.243192+0000","last_change":"2023-11-24T06:30:33.243192+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.243192+0000","last_undegraded":"2023-11-24T06:30:33.243192+0000","last_fullsized":"2023-11-24T06:30:33.243192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.236477+0000","last_change":"2023-11-24T06:30:33.236477+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.236477+0000","last_undegraded":"2023-11-24T06:30:33.236477+0000","last_fullsized":"2023-11-24T06:30:33.236477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239321+0000","last_change":"2023-11-24T06:30:33.239321+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239321+0000","last_undegraded":"2023-11-24T06:30:33.239321+0000","last_fullsized":"2023-11-24T06:30:33.239321+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.250847+0000","last_change":"2023-11-24T06:30:33.250847+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.250847+0000","last_undegraded":"2023-11-24T06:30:33.250847+0000","last_fullsized":"2023-11-24T06:30:33.250847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.254897+0000","last_change":"2023-11-24T06:30:33.254897+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.254897+0000","last_undegraded":"2023-11-24T06:30:33.254897+0000","last_fullsized":"2023-11-24T06:30:33.254897+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.252954+0000","last_change":"2023-11-24T06:30:33.252954+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.252954+0000","last_undegraded":"2023-11-24T06:30:33.252954+0000","last_fullsized":"2023-11-24T06:30:33.252954+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.238247+0000","last_change":"2023-11-24T06:30:33.238247+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.238247+0000","last_undegraded":"2023-11-24T06:30:33.238247+0000","last_fullsized":"2023-11-24T06:30:33.238247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.244235+0000","last_change":"2023-11-24T06:30:33.244235+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.244235+0000","last_undegraded":"2023-11-24T06:30:33.244235+0000","last_fullsized":"2023-11-24T06:30:33.244235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239947+0000","last_change":"2023-11-24T06:30:33.239947+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239947+0000","last_undegraded":"2023-11-24T06:30:33.239947+0000","last_fullsized":"2023-11-24T06:30:33.239947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235872+0000","last_change":"2023-11-24T06:30:33.235872+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235872+0000","last_undegraded":"2023-11-24T06:30:33.235872+0000","last_fullsized":"2023-11-24T06:30:33.235872+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247170+0000","last_change":"2023-11-24T06:30:33.247170+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247170+0000","last_undegraded":"2023-11-24T06:30:33.247170+0000","last_fullsized":"2023-11-24T06:30:33.247170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234680+0000","last_change":"2023-11-24T06:30:33.234680+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234680+0000","last_undegraded":"2023-11-24T06:30:33.234680+0000","last_fullsized":"2023-11-24T06:30:33.234680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247698+0000","last_change":"2023-11-24T06:30:33.247698+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247698+0000","last_undegraded":"2023-11-24T06:30:33.247698+0000","last_fullsized":"2023-11-24T06:30:33.247698+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237309+0000","last_change":"2023-11-24T06:30:33.237309+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237309+0000","last_undegraded":"2023-11-24T06:30:33.237309+0000","last_fullsized":"2023-11-24T06:30:33.237309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235467+0000","last_change":"2023-11-24T06:30:33.235467+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235467+0000","last_undegraded":"2023-11-24T06:30:33.235467+0000","last_fullsized":"2023-11-24T06:30:33.235467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.236341+0000","last_change":"2023-11-24T06:30:33.236341+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.236341+0000","last_undegraded":"2023-11-24T06:30:33.236341+0000","last_fullsized":"2023-11-24T06:30:33.236341+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.258019+0000","last_change":"2023-11-24T06:30:33.258019+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.258019+0000","last_undegraded":"2023-11-24T06:30:33.258019+0000","last_fullsized":"2023-11-24T06:30:33.258019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.251109+0000","last_change":"2023-11-24T06:30:33.251109+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.251109+0000","last_undegraded":"2023-11-24T06:30:33.251109+0000","last_fullsized":"2023-11-24T06:30:33.251109+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242204+0000","last_change":"2023-11-24T06:30:33.242204+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242204+0000","last_undegraded":"2023-11-24T06:30:33.242204+0000","last_fullsized":"2023-11-24T06:30:33.242204+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239064+0000","last_change":"2023-11-24T06:30:33.239064+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239064+0000","last_undegraded":"2023-11-24T06:30:33.239064+0000","last_fullsized":"2023-11-24T06:30:33.239064+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.240220+0000","last_change":"2023-11-24T06:30:33.240220+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.240220+0000","last_undegraded":"2023-11-24T06:30:33.240220+0000","last_fullsized":"2023-11-24T06:30:33.240220+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234550+0000","last_change":"2023-11-24T06:30:33.234550+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234550+0000","last_undegraded":"2023-11-24T06:30:33.234550+0000","last_fullsized":"2023-11-24T06:30:33.234550+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.241508+0000","last_change":"2023-11-24T06:30:33.241508+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.241508+0000","last_undegraded":"2023-11-24T06:30:33.241508+0000","last_fullsized":"2023-11-24T06:30:33.241508+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247042+0000","last_change":"2023-11-24T06:30:33.247042+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247042+0000","last_undegraded":"2023-11-24T06:30:33.247042+0000","last_fullsized":"2023-11-24T06:30:33.247042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242692+0000","last_change":"2023-11-24T06:30:33.242692+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242692+0000","last_undegraded":"2023-11-24T06:30:33.242692+0000","last_fullsized":"2023-11-24T06:30:33.242692+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237560+0000","last_change":"2023-11-24T06:30:33.237560+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237560+0000","last_undegraded":"2023-11-24T06:30:33.237560+0000","last_fullsized":"2023-11-24T06:30:33.237560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246556+0000","last_change":"2023-11-24T06:30:33.246556+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246556+0000","last_undegraded":"2023-11-24T06:30:33.246556+0000","last_fullsized":"2023-11-24T06:30:33.246556+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247176+0000","last_change":"2023-11-24T06:30:33.247176+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247176+0000","last_undegraded":"2023-11-24T06:30:33.247176+0000","last_fullsized":"2023-11-24T06:30:33.247176+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.248346+0000","last_change":"2023-11-24T06:30:33.248346+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.248346+0000","last_undegraded":"2023-11-24T06:30:33.248346+0000","last_fullsized":"2023-11-24T06:30:33.248346+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.252179+0000","last_change":"2023-11-24T06:30:33.252179+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.252179+0000","last_undegraded":"2023-11-24T06:30:33.252179+0000","last_fullsized":"2023-11-24T06:30:33.252179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.241747+0000","last_change":"2023-11-24T06:30:33.241747+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.241747+0000","last_undegraded":"2023-11-24T06:30:33.241747+0000","last_fullsized":"2023-11-24T06:30:33.241747+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.263585+0000","last_change":"2023-11-24T06:30:33.258485+0000","last_active":"2023-11-24T06:30:33.263585+0000","last_peered":"2023-11-24T06:30:33.263585+0000","last_clean":"2023-11-24T06:30:33.263585+0000","last_became_active":"2023-11-24T06:30:33.258091+0000","last_became_peered":"2023-11-24T06:30:33.258091+0000","last_unstale":"2023-11-24T06:30:33.263585+0000","last_undegraded":"2023-11-24T06:30:33.263585+0000","last_fullsized":"2023-11-24T06:30:33.263585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:39.081828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234860+0000","last_change":"2023-11-24T06:30:33.234860+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234860+0000","last_undegraded":"2023-11-24T06:30:33.234860+0000","last_fullsized":"2023-11-24T06:30:33.234860+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.257894+0000","last_change":"2023-11-24T06:30:33.253586+0000","last_active":"2023-11-24T06:30:33.257894+0000","last_peered":"2023-11-24T06:30:33.257894+0000","last_clean":"2023-11-24T06:30:33.257894+0000","last_became_active":"2023-11-24T06:30:33.252371+0000","last_became_peered":"2023-11-24T06:30:33.252371+0000","last_unstale":"2023-11-24T06:30:33.257894+0000","last_undegraded":"2023-11-24T06:30:33.257894+0000","last_fullsized":"2023-11-24T06:30:33.257894+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:42:17.431240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.252752+0000","last_change":"2023-11-24T06:30:33.252752+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.252752+0000","last_undegraded":"2023-11-24T06:30:33.252752+0000","last_fullsized":"2023-11-24T06:30:33.252752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267927+0000","last_change":"2023-11-24T06:30:33.266741+0000","last_active":"2023-11-24T06:30:33.267927+0000","last_peered":"2023-11-24T06:30:33.267927+0000","last_clean":"2023-11-24T06:30:33.267927+0000","last_became_active":"2023-11-24T06:30:33.266055+0000","last_became_peered":"2023-11-24T06:30:33.266055+0000","last_unstale":"2023-11-24T06:30:33.267927+0000","last_undegraded":"2023-11-24T06:30:33.267927+0000","last_fullsized":"2023-11-24T06:30:33.267927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:46.553265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.238829+0000","last_change":"2023-11-24T06:30:33.238829+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.238829+0000","last_undegraded":"2023-11-24T06:30:33.238829+0000","last_fullsized":"2023-11-24T06:30:33.238829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267672+0000","last_change":"2023-11-24T06:30:33.260247+0000","last_active":"2023-11-24T06:30:33.267672+0000","last_peered":"2023-11-24T06:30:33.267672+0000","last_clean":"2023-11-24T06:30:33.267672+0000","last_became_active":"2023-11-24T06:30:33.259986+0000","last_became_peered":"2023-11-24T06:30:33.259986+0000","last_unstale":"2023-11-24T06:30:33.267672+0000","last_undegraded":"2023-11-24T06:30:33.267672+0000","last_fullsized":"2023-11-24T06:30:33.267672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:25.530564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246365+0000","last_change":"2023-11-24T06:30:33.246365+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246365+0000","last_undegraded":"2023-11-24T06:30:33.246365+0000","last_fullsized":"2023-11-24T06:30:33.246365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.257173+0000","last_change":"2023-11-24T06:30:33.253225+0000","last_active":"2023-11-24T06:30:33.257173+0000","last_peered":"2023-11-24T06:30:33.257173+0000","last_clean":"2023-11-24T06:30:33.257173+0000","last_became_active":"2023-11-24T06:30:33.252940+0000","last_became_peered":"2023-11-24T06:30:33.252940+0000","last_unstale":"2023-11-24T06:30:33.257173+0000","last_undegraded":"2023-11-24T06:30:33.257173+0000","last_fullsized":"2023-11-24T06:30:33.257173+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:03.692439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.240042+0000","last_change":"2023-11-24T06:30:33.240042+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.240042+0000","last_undegraded":"2023-11-24T06:30:33.240042+0000","last_fullsized":"2023-11-24T06:30:33.240042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267861+0000","last_change":"2023-11-24T06:30:33.263099+0000","last_active":"2023-11-24T06:30:33.267861+0000","last_peered":"2023-11-24T06:30:33.267861+0000","last_clean":"2023-11-24T06:30:33.267861+0000","last_became_active":"2023-11-24T06:30:33.261288+0000","last_became_peered":"2023-11-24T06:30:33.261288+0000","last_unstale":"2023-11-24T06:30:33.267861+0000","last_undegraded":"2023-11-24T06:30:33.267861+0000","last_fullsized":"2023-11-24T06:30:33.267861+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:38.745762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234536+0000","last_change":"2023-11-24T06:30:33.234536+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234536+0000","last_undegraded":"2023-11-24T06:30:33.234536+0000","last_fullsized":"2023-11-24T06:30:33.234536+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.254629+0000","last_change":"2023-11-24T06:30:33.250035+0000","last_active":"2023-11-24T06:30:33.254629+0000","last_peered":"2023-11-24T06:30:33.254629+0000","last_clean":"2023-11-24T06:30:33.254629+0000","last_became_active":"2023-11-24T06:30:33.249747+0000","last_became_peered":"2023-11-24T06:30:33.249747+0000","last_unstale":"2023-11-24T06:30:33.254629+0000","last_undegraded":"2023-11-24T06:30:33.254629+0000","last_fullsized":"2023-11-24T06:30:33.254629+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:28.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246986+0000","last_change":"2023-11-24T06:30:33.246986+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246986+0000","last_undegraded":"2023-11-24T06:30:33.246986+0000","last_fullsized":"2023-11-24T06:30:33.246986+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.245386+0000","last_change":"2023-11-24T06:30:33.240278+0000","last_active":"2023-11-24T06:30:33.245386+0000","last_peered":"2023-11-24T06:30:33.245386+0000","last_clean":"2023-11-24T06:30:33.245386+0000","last_became_active":"2023-11-24T06:30:33.238103+0000","last_became_peered":"2023-11-24T06:30:33.238103+0000","last_unstale":"2023-11-24T06:30:33.245386+0000","last_undegraded":"2023-11-24T06:30:33.245386+0000","last_fullsized":"2023-11-24T06:30:33.245386+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:01:20.140182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246380+0000","last_change":"2023-11-24T06:30:33.246380+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246380+0000","last_undegraded":"2023-11-24T06:30:33.246380+0000","last_fullsized":"2023-11-24T06:30:33.246380+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261289+0000","last_change":"2023-11-24T06:30:33.257372+0000","last_active":"2023-11-24T06:30:33.261289+0000","last_peered":"2023-11-24T06:30:33.261289+0000","last_clean":"2023-11-24T06:30:33.261289+0000","last_became_active":"2023-11-24T06:30:33.257063+0000","last_became_peered":"2023-11-24T06:30:33.257063+0000","last_unstale":"2023-11-24T06:30:33.261289+0000","last_undegraded":"2023-11-24T06:30:33.261289+0000","last_fullsized":"2023-11-24T06:30:33.261289+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:51.884745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242375+0000","last_change":"2023-11-24T06:30:33.242375+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242375+0000","last_undegraded":"2023-11-24T06:30:33.242375+0000","last_fullsized":"2023-11-24T06:30:33.242375+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269341+0000","last_change":"2023-11-24T06:30:33.266155+0000","last_active":"2023-11-24T06:30:33.269341+0000","last_peered":"2023-11-24T06:30:33.269341+0000","last_clean":"2023-11-24T06:30:33.269341+0000","last_became_active":"2023-11-24T06:30:33.265258+0000","last_became_peered":"2023-11-24T06:30:33.265258+0000","last_unstale":"2023-11-24T06:30:33.269341+0000","last_undegraded":"2023-11-24T06:30:33.269341+0000","last_fullsized":"2023-11-24T06:30:33.269341+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:31.950626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.248625+0000","last_change":"2023-11-24T06:30:33.248625+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.248625+0000","last_undegraded":"2023-11-24T06:30:33.248625+0000","last_fullsized":"2023-11-24T06:30:33.248625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265546+0000","last_change":"2023-11-24T06:30:33.264658+0000","last_active":"2023-11-24T06:30:33.265546+0000","last_peered":"2023-11-24T06:30:33.265546+0000","last_clean":"2023-11-24T06:30:33.265546+0000","last_became_active":"2023-11-24T06:30:33.264320+0000","last_became_peered":"2023-11-24T06:30:33.264320+0000","last_unstale":"2023-11-24T06:30:33.265546+0000","last_undegraded":"2023-11-24T06:30:33.265546+0000","last_fullsized":"2023-11-24T06:30:33.265546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:41.422715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235171+0000","last_change":"2023-11-24T06:30:33.235171+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235171+0000","last_undegraded":"2023-11-24T06:30:33.235171+0000","last_fullsized":"2023-11-24T06:30:33.235171+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.242786+0000","last_change":"2023-11-24T06:30:33.239974+0000","last_active":"2023-11-24T06:30:33.242786+0000","last_peered":"2023-11-24T06:30:33.242786+0000","last_clean":"2023-11-24T06:30:33.242786+0000","last_became_active":"2023-11-24T06:30:33.238273+0000","last_became_peered":"2023-11-24T06:30:33.238273+0000","last_unstale":"2023-11-24T06:30:33.242786+0000","last_undegraded":"2023-11-24T06:30:33.242786+0000","last_fullsized":"2023-11-24T06:30:33.242786+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:12.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.238368+0000","last_change":"2023-11-24T06:30:33.238368+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.238368+0000","last_undegraded":"2023-11-24T06:30:33.238368+0000","last_fullsized":"2023-11-24T06:30:33.238368+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.253699+0000","last_change":"2023-11-24T06:30:33.252044+0000","last_active":"2023-11-24T06:30:33.253699+0000","last_peered":"2023-11-24T06:30:33.253699+0000","last_clean":"2023-11-24T06:30:33.253699+0000","last_became_active":"2023-11-24T06:30:33.251774+0000","last_became_peered":"2023-11-24T06:30:33.251774+0000","last_unstale":"2023-11-24T06:30:33.253699+0000","last_undegraded":"2023-11-24T06:30:33.253699+0000","last_fullsized":"2023-11-24T06:30:33.253699+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:38.745762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.248362+0000","last_change":"2023-11-24T06:30:33.248362+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.248362+0000","last_undegraded":"2023-11-24T06:30:33.248362+0000","last_fullsized":"2023-11-24T06:30:33.248362+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267841+0000","last_change":"2023-11-24T06:30:33.261613+0000","last_active":"2023-11-24T06:30:33.267841+0000","last_peered":"2023-11-24T06:30:33.267841+0000","last_clean":"2023-11-24T06:30:33.267841+0000","last_became_active":"2023-11-24T06:30:33.260523+0000","last_became_peered":"2023-11-24T06:30:33.260523+0000","last_unstale":"2023-11-24T06:30:33.267841+0000","last_undegraded":"2023-11-24T06:30:33.267841+0000","last_fullsized":"2023-11-24T06:30:33.267841+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:59:27.319046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.249240+0000","last_change":"2023-11-24T06:30:33.249240+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.249240+0000","last_undegraded":"2023-11-24T06:30:33.249240+0000","last_fullsized":"2023-11-24T06:30:33.249240+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265229+0000","last_change":"2023-11-24T06:30:33.263590+0000","last_active":"2023-11-24T06:30:33.265229+0000","last_peered":"2023-11-24T06:30:33.265229+0000","last_clean":"2023-11-24T06:30:33.265229+0000","last_became_active":"2023-11-24T06:30:33.262788+0000","last_became_peered":"2023-11-24T06:30:33.262788+0000","last_unstale":"2023-11-24T06:30:33.265229+0000","last_undegraded":"2023-11-24T06:30:33.265229+0000","last_fullsized":"2023-11-24T06:30:33.265229+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:10.481218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.233300+0000","last_change":"2023-11-24T06:30:33.233300+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.233300+0000","last_undegraded":"2023-11-24T06:30:33.233300+0000","last_fullsized":"2023-11-24T06:30:33.233300+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261672+0000","last_change":"2023-11-24T06:30:33.259288+0000","last_active":"2023-11-24T06:30:33.261672+0000","last_peered":"2023-11-24T06:30:33.261672+0000","last_clean":"2023-11-24T06:30:33.261672+0000","last_became_active":"2023-11-24T06:30:33.254396+0000","last_became_peered":"2023-11-24T06:30:33.254396+0000","last_unstale":"2023-11-24T06:30:33.261672+0000","last_undegraded":"2023-11-24T06:30:33.261672+0000","last_fullsized":"2023-11-24T06:30:33.261672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:30.689902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237823+0000","last_change":"2023-11-24T06:30:33.237823+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237823+0000","last_undegraded":"2023-11-24T06:30:33.237823+0000","last_fullsized":"2023-11-24T06:30:33.237823+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269298+0000","last_change":"2023-11-24T06:30:33.265823+0000","last_active":"2023-11-24T06:30:33.269298+0000","last_peered":"2023-11-24T06:30:33.269298+0000","last_clean":"2023-11-24T06:30:33.269298+0000","last_became_active":"2023-11-24T06:30:33.265509+0000","last_became_peered":"2023-11-24T06:30:33.265509+0000","last_unstale":"2023-11-24T06:30:33.269298+0000","last_undegraded":"2023-11-24T06:30:33.269298+0000","last_fullsized":"2023-11-24T06:30:33.269298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:21.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.241198+0000","last_change":"2023-11-24T06:30:33.241198+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.241198+0000","last_undegraded":"2023-11-24T06:30:33.241198+0000","last_fullsized":"2023-11-24T06:30:33.241198+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269221+0000","last_change":"2023-11-24T06:30:33.268072+0000","last_active":"2023-11-24T06:30:33.269221+0000","last_peered":"2023-11-24T06:30:33.269221+0000","last_clean":"2023-11-24T06:30:33.269221+0000","last_became_active":"2023-11-24T06:30:33.267757+0000","last_became_peered":"2023-11-24T06:30:33.267757+0000","last_unstale":"2023-11-24T06:30:33.269221+0000","last_undegraded":"2023-11-24T06:30:33.269221+0000","last_fullsized":"2023-11-24T06:30:33.269221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:18.630896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235215+0000","last_change":"2023-11-24T06:30:33.235215+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235215+0000","last_undegraded":"2023-11-24T06:30:33.235215+0000","last_fullsized":"2023-11-24T06:30:33.235215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269306+0000","last_change":"2023-11-24T06:30:33.265903+0000","last_active":"2023-11-24T06:30:33.269306+0000","last_peered":"2023-11-24T06:30:33.269306+0000","last_clean":"2023-11-24T06:30:33.269306+0000","last_became_active":"2023-11-24T06:30:33.263286+0000","last_became_peered":"2023-11-24T06:30:33.263286+0000","last_unstale":"2023-11-24T06:30:33.269306+0000","last_undegraded":"2023-11-24T06:30:33.269306+0000","last_fullsized":"2023-11-24T06:30:33.269306+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:49:20.198853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.233222+0000","last_change":"2023-11-24T06:30:33.233222+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.233222+0000","last_undegraded":"2023-11-24T06:30:33.233222+0000","last_fullsized":"2023-11-24T06:30:33.233222+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.263521+0000","last_change":"2023-11-24T06:30:33.259236+0000","last_active":"2023-11-24T06:30:33.263521+0000","last_peered":"2023-11-24T06:30:33.263521+0000","last_clean":"2023-11-24T06:30:33.263521+0000","last_became_active":"2023-11-24T06:30:33.258200+0000","last_became_peered":"2023-11-24T06:30:33.258200+0000","last_unstale":"2023-11-24T06:30:33.263521+0000","last_undegraded":"2023-11-24T06:30:33.263521+0000","last_fullsized":"2023-11-24T06:30:33.263521+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:08.963743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.243611+0000","last_change":"2023-11-24T06:30:33.243611+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.243611+0000","last_undegraded":"2023-11-24T06:30:33.243611+0000","last_fullsized":"2023-11-24T06:30:33.243611+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267605+0000","last_change":"2023-11-24T06:30:33.259814+0000","last_active":"2023-11-24T06:30:33.267605+0000","last_peered":"2023-11-24T06:30:33.267605+0000","last_clean":"2023-11-24T06:30:33.267605+0000","last_became_active":"2023-11-24T06:30:33.259551+0000","last_became_peered":"2023-11-24T06:30:33.259551+0000","last_unstale":"2023-11-24T06:30:33.267605+0000","last_undegraded":"2023-11-24T06:30:33.267605+0000","last_fullsized":"2023-11-24T06:30:33.267605+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:28.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242279+0000","last_change":"2023-11-24T06:30:33.242279+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242279+0000","last_undegraded":"2023-11-24T06:30:33.242279+0000","last_fullsized":"2023-11-24T06:30:33.242279+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265212+0000","last_change":"2023-11-24T06:30:33.263043+0000","last_active":"2023-11-24T06:30:33.265212+0000","last_peered":"2023-11-24T06:30:33.265212+0000","last_clean":"2023-11-24T06:30:33.265212+0000","last_became_active":"2023-11-24T06:30:33.262727+0000","last_became_peered":"2023-11-24T06:30:33.262727+0000","last_unstale":"2023-11-24T06:30:33.265212+0000","last_undegraded":"2023-11-24T06:30:33.265212+0000","last_fullsized":"2023-11-24T06:30:33.265212+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:33.658163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242662+0000","last_change":"2023-11-24T06:30:33.242662+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242662+0000","last_undegraded":"2023-11-24T06:30:33.242662+0000","last_fullsized":"2023-11-24T06:30:33.242662+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.262428+0000","last_change":"2023-11-24T06:30:33.259539+0000","last_active":"2023-11-24T06:30:33.262428+0000","last_peered":"2023-11-24T06:30:33.262428+0000","last_clean":"2023-11-24T06:30:33.262428+0000","last_became_active":"2023-11-24T06:30:33.258994+0000","last_became_peered":"2023-11-24T06:30:33.258994+0000","last_unstale":"2023-11-24T06:30:33.262428+0000","last_undegraded":"2023-11-24T06:30:33.262428+0000","last_fullsized":"2023-11-24T06:30:33.262428+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:40.643302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.249084+0000","last_change":"2023-11-24T06:30:33.249084+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.249084+0000","last_undegraded":"2023-11-24T06:30:33.249084+0000","last_fullsized":"2023-11-24T06:30:33.249084+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267896+0000","last_change":"2023-11-24T06:30:33.264626+0000","last_active":"2023-11-24T06:30:33.267896+0000","last_peered":"2023-11-24T06:30:33.267896+0000","last_clean":"2023-11-24T06:30:33.267896+0000","last_became_active":"2023-11-24T06:30:33.264323+0000","last_became_peered":"2023-11-24T06:30:33.264323+0000","last_unstale":"2023-11-24T06:30:33.267896+0000","last_undegraded":"2023-11-24T06:30:33.267896+0000","last_fullsized":"2023-11-24T06:30:33.267896+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:44.399558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.254177+0000","last_change":"2023-11-24T06:30:33.254177+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.254177+0000","last_undegraded":"2023-11-24T06:30:33.254177+0000","last_fullsized":"2023-11-24T06:30:33.254177+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265269+0000","last_change":"2023-11-24T06:30:33.262138+0000","last_active":"2023-11-24T06:30:33.265269+0000","last_peered":"2023-11-24T06:30:33.265269+0000","last_clean":"2023-11-24T06:30:33.265269+0000","last_became_active":"2023-11-24T06:30:33.261687+0000","last_became_peered":"2023-11-24T06:30:33.261687+0000","last_unstale":"2023-11-24T06:30:33.265269+0000","last_undegraded":"2023-11-24T06:30:33.265269+0000","last_fullsized":"2023-11-24T06:30:33.265269+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:05.740113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239053+0000","last_change":"2023-11-24T06:30:33.239053+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239053+0000","last_undegraded":"2023-11-24T06:30:33.239053+0000","last_fullsized":"2023-11-24T06:30:33.239053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.257585+0000","last_change":"2023-11-24T06:30:33.255156+0000","last_active":"2023-11-24T06:30:33.257585+0000","last_peered":"2023-11-24T06:30:33.257585+0000","last_clean":"2023-11-24T06:30:33.257585+0000","last_became_active":"2023-11-24T06:30:33.254878+0000","last_became_peered":"2023-11-24T06:30:33.254878+0000","last_unstale":"2023-11-24T06:30:33.257585+0000","last_undegraded":"2023-11-24T06:30:33.257585+0000","last_fullsized":"2023-11-24T06:30:33.257585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:38.180557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237564+0000","last_change":"2023-11-24T06:30:33.237564+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237564+0000","last_undegraded":"2023-11-24T06:30:33.237564+0000","last_fullsized":"2023-11-24T06:30:33.237564+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.264428+0000","last_change":"2023-11-24T06:30:33.263337+0000","last_active":"2023-11-24T06:30:33.264428+0000","last_peered":"2023-11-24T06:30:33.264428+0000","last_clean":"2023-11-24T06:30:33.264428+0000","last_became_active":"2023-11-24T06:30:33.262997+0000","last_became_peered":"2023-11-24T06:30:33.262997+0000","last_unstale":"2023-11-24T06:30:33.264428+0000","last_undegraded":"2023-11-24T06:30:33.264428+0000","last_fullsized":"2023-11-24T06:30:33.264428+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:00.154357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.232644+0000","last_change":"2023-11-24T06:30:33.232644+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.232644+0000","last_undegraded":"2023-11-24T06:30:33.232644+0000","last_fullsized":"2023-11-24T06:30:33.232644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261315+0000","last_change":"2023-11-24T06:30:33.258636+0000","last_active":"2023-11-24T06:30:33.261315+0000","last_peered":"2023-11-24T06:30:33.261315+0000","last_clean":"2023-11-24T06:30:33.261315+0000","last_became_active":"2023-11-24T06:30:33.258193+0000","last_became_peered":"2023-11-24T06:30:33.258193+0000","last_unstale":"2023-11-24T06:30:33.261315+0000","last_undegraded":"2023-11-24T06:30:33.261315+0000","last_fullsized":"2023-11-24T06:30:33.261315+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.507654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.249296+0000","last_change":"2023-11-24T06:30:33.249296+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.249296+0000","last_undegraded":"2023-11-24T06:30:33.249296+0000","last_fullsized":"2023-11-24T06:30:33.249296+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.256814+0000","last_change":"2023-11-24T06:30:33.253808+0000","last_active":"2023-11-24T06:30:33.256814+0000","last_peered":"2023-11-24T06:30:33.256814+0000","last_clean":"2023-11-24T06:30:33.256814+0000","last_became_active":"2023-11-24T06:30:33.250888+0000","last_became_peered":"2023-11-24T06:30:33.250888+0000","last_unstale":"2023-11-24T06:30:33.256814+0000","last_undegraded":"2023-11-24T06:30:33.256814+0000","last_fullsized":"2023-11-24T06:30:33.256814+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:59:27.319046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234698+0000","last_change":"2023-11-24T06:30:33.234698+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234698+0000","last_undegraded":"2023-11-24T06:30:33.234698+0000","last_fullsized":"2023-11-24T06:30:33.234698+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261935+0000","last_change":"2023-11-24T06:30:33.257088+0000","last_active":"2023-11-24T06:30:33.261935+0000","last_peered":"2023-11-24T06:30:33.261935+0000","last_clean":"2023-11-24T06:30:33.261935+0000","last_became_active":"2023-11-24T06:30:33.256733+0000","last_became_peered":"2023-11-24T06:30:33.256733+0000","last_unstale":"2023-11-24T06:30:33.261935+0000","last_undegraded":"2023-11-24T06:30:33.261935+0000","last_fullsized":"2023-11-24T06:30:33.261935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:52.534613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.251567+0000","last_change":"2023-11-24T06:30:33.251567+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.251567+0000","last_undegraded":"2023-11-24T06:30:33.251567+0000","last_fullsized":"2023-11-24T06:30:33.251567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269279+0000","last_change":"2023-11-24T06:30:33.265710+0000","last_active":"2023-11-24T06:30:33.269279+0000","last_peered":"2023-11-24T06:30:33.269279+0000","last_clean":"2023-11-24T06:30:33.269279+0000","last_became_active":"2023-11-24T06:30:33.265397+0000","last_became_peered":"2023-11-24T06:30:33.265397+0000","last_unstale":"2023-11-24T06:30:33.269279+0000","last_undegraded":"2023-11-24T06:30:33.269279+0000","last_fullsized":"2023-11-24T06:30:33.269279+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:30.859810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242319+0000","last_change":"2023-11-24T06:30:33.242319+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242319+0000","last_undegraded":"2023-11-24T06:30:33.242319+0000","last_fullsized":"2023-11-24T06:30:33.242319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267727+0000","last_change":"2023-11-24T06:30:33.260898+0000","last_active":"2023-11-24T06:30:33.267727+0000","last_peered":"2023-11-24T06:30:33.267727+0000","last_clean":"2023-11-24T06:30:33.267727+0000","last_became_active":"2023-11-24T06:30:33.260528+0000","last_became_peered":"2023-11-24T06:30:33.260528+0000","last_unstale":"2023-11-24T06:30:33.267727+0000","last_undegraded":"2023-11-24T06:30:33.267727+0000","last_fullsized":"2023-11-24T06:30:33.267727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:12.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-24T06:30:32.217624+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:30:32.217624+0000","last_peered":"2023-11-24T06:30:32.217624+0000","last_clean":"2023-11-24T06:30:32.217624+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:30:32.217624+0000","last_undegraded":"2023-11-24T06:30:32.217624+0000","last_fullsized":"2023-11-24T06:30:32.217624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":942410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.252,"15min":1.3160000000000001},"min":{"1min":0.68100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.0599999999999996,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.123},{"interface":"front","average":{"1min":1.389,"5min":1.296,"15min":1.3009999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":4.556,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.524}]},{"osd":1,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0249999999999999,"15min":1.012},"min":{"1min":0.68200000000000005,"5min":0.629,"15min":0.629},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.343,"15min":1.377},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":3.944,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.4419999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.2729999999999999,"15min":1.288},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.5609999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.3539999999999992,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":1.306},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.204},"min":{"1min":0.79300000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.8200000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.714}]},{"osd":4,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.4870000000000001,"15min":1.5669999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.1059999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.331},{"interface":"front","average":{"1min":1.498,"5min":1.5369999999999999,"15min":1.579},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.6000000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.3580000000000001,"15min":1.399},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.3119999999999998,"5min":1.8240000000000001,"15min":1.794},"min":{"1min":0.69799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":17.582999999999998,"5min":17.582999999999998,"15min":17.582999999999998},"last":1.6739999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.736,"15min":1.8480000000000001},"min":{"1min":0.73099999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.8140000000000001,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.744},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.6240000000000001,"15min":1.694},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":6.2210000000000001,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.3999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0980000000000001,"15min":1.1020000000000001},"min":{"1min":0.73499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7390000000000001,"5min":1.829,"15min":1.829},"last":1.03},{"interface":"front","average":{"1min":1.155,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.76800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.623,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.234}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2210000000000001,"15min":1.2130000000000001},"min":{"1min":0.71099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6299999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.573},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.2010000000000001,"15min":1.2070000000000001},"min":{"1min":0.88900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.494,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.323}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.159},"min":{"1min":0.75700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.609,"5min":3.597,"15min":3.597},"last":1.704},{"interface":"front","average":{"1min":1.304,"5min":1.292,"15min":1.3140000000000001},"min":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6359999999999999,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.6579999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3129999999999999,"15min":1.3180000000000001},"min":{"1min":0.78100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5620000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.2969999999999999,"15min":1.325},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5960000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.52}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.8919999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.385,"15min":1.397},"min":{"1min":0.79600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7729999999999999,"5min":3.04,"15min":3.04},"last":1.091}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3720000000000001,"15min":1.377},"min":{"1min":0.97699999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7330000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.552,"15min":1.6020000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.851,"5min":3.774,"15min":3.774},"last":1.397}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5449999999999999,"15min":1.5589999999999999},"min":{"1min":1.0029999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7689999999999999,"5min":3.403,"15min":3.403},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.512},"min":{"1min":0.875,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.792,"5min":3.649,"15min":3.649},"last":1.2649999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.516,"5min":1.903,"15min":1.903},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.288,"15min":1.3140000000000001},"min":{"1min":0.91700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.915,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.5349999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.145,"15min":1.1399999999999999},"min":{"1min":0.78900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.147,"15min":1.1140000000000001},"min":{"1min":0.89500000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.347}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.804},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3109999999999999,"15min":1.2969999999999999},"min":{"1min":0.99199999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1669999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.2969999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3360000000000001,"15min":1.375},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.952,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.405},{"interface":"front","average":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3899999999999999},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.5840000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3180000000000001,"15min":1.3340000000000001},"min":{"1min":0.878,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.839,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.5029999999999999,"15min":1.5369999999999999},"min":{"1min":0.82299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0099999999999998,"5min":3.085,"15min":3.085},"last":1.641}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4690000000000001,"15min":1.4450000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.464},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.474,"15min":1.468},"min":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0270000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.5069999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.651,"5min":1.6140000000000001,"15min":1.6240000000000001},"min":{"1min":1.046,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.141,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.714,"5min":1.6919999999999999,"15min":1.716},"min":{"1min":1.0129999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3159999999999998,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.673}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.073,"15min":1.073},"min":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.982,"15min":1.982},"last":1.222},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.089,"15min":1.1120000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.468,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.262}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.147,"15min":1.113},"min":{"1min":0.84199999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.26,"15min":1.3049999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5840000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.0980000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8129999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.27,"5min":1.3140000000000001,"15min":1.3320000000000001},"min":{"1min":0.79800000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7250000000000001,"5min":2.254,"15min":2.254},"last":1.3}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.294,"15min":1.333},"min":{"1min":0.76100000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.833,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3779999999999999,"15min":1.3720000000000001},"min":{"1min":0.753,"5min":0.746,"15min":0.746},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.071}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.482,"15min":1.5149999999999999},"min":{"1min":0.874,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8109999999999999,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.522,"15min":1.5609999999999999},"min":{"1min":0.85799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.923,"5min":3.52,"15min":3.52},"last":1.179}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.474,"15min":1.482},"min":{"1min":0.79200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.542,"15min":1.5740000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.859,"5min":2.452,"15min":2.452},"last":1.359}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.595,"15min":1.6559999999999999},"min":{"1min":0.64000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.7729999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.615,"15min":1.613},"min":{"1min":0.84799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.0099999999999998,"5min":2.363,"15min":2.363},"last":1.2030000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.3080000000000001,"5min":1.355,"15min":1.355},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.66600000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.96399999999999997},"min":{"1min":0.51800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.077,"5min":1.091,"15min":1.113},"min":{"1min":0.61499999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.91900000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.0350000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.58499999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4550000000000001,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.133}]},{"osd":4,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3420000000000001,"15min":1.3879999999999999},"min":{"1min":0.91300000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.696,"5min":3.081,"15min":3.081},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.3340000000000001,"15min":1.339},"min":{"1min":0.74299999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.712,"5min":2.758,"15min":2.758},"last":1.7230000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.373,"15min":1.403},"min":{"1min":0.873,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.1200000000000001,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.462},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.6200000000000001,"15min":1.748},"min":{"1min":0.86299999999999999,"5min":0.745,"15min":0.745},"max":{"1min":1.657,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.786}]},{"osd":6,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3759999999999999,"15min":1.407},"min":{"1min":0.82999999999999996,"5min":0.753,"15min":0.753},"max":{"1min":1.79,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.534},{"interface":"front","average":{"1min":1.524,"5min":1.53,"15min":1.544},"min":{"1min":1.0760000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.464,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.4790000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.5760000000000001,"15min":1.609},"min":{"1min":0.84499999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.589,"5min":4.641,"15min":4.641},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.546,"15min":1.5740000000000001},"min":{"1min":0.88200000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.3439999999999999,"5min":3.798,"15min":3.798},"last":1.7490000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2729999999999999,"5min":1.427,"15min":1.427},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.619,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.59699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.48499999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.78800000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0840000000000001,"15min":1.1040000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.784,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.095,"15min":1.117},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6990000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.575}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.456,"15min":1.508},"min":{"1min":0.67900000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.6869999999999998,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.454,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.5800000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4650000000000001,"15min":1.468},"min":{"1min":1.0029999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.036,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.4690000000000001,"15min":1.5009999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.841,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.2769999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.423,"15min":1.4359999999999999},"min":{"1min":0.66900000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.968,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.004},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.522,"15min":1.54},"min":{"1min":1.0029999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.165,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.444}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5149999999999999,"15min":1.5489999999999999},"min":{"1min":0.78200000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.913,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":1.097,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.423,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.9359999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.002,"15min":1.032},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.696,"5min":2.593,"15min":2.593},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.008,"5min":1.02,"15min":1.032},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5980000000000001,"5min":1.702,"15min":1.702},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.212,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":1,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7669999999999999,"5min":1.829,"15min":1.829},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1040000000000001,"15min":1.1200000000000001},"min":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.603,"5min":1.766,"15min":1.766},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0289999999999999,"15min":1.069},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2250000000000001,"5min":1.798,"15min":1.798},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.246,"15min":1.2529999999999999},"min":{"1min":0.89000000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.621,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.359,"5min":1.417,"15min":1.4450000000000001},"min":{"1min":1.0509999999999999,"5min":1.032,"15min":1.032},"max":{"1min":1.837,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":0.96299999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.294,"15min":1.302},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.514,"5min":2.016,"15min":2.016},"last":1.304},{"interface":"front","average":{"1min":1.391,"5min":1.389,"15min":1.399},"min":{"1min":0.91700000000000004,"5min":0.877,"15min":0.877},"max":{"1min":1.7410000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.071}]},{"osd":6,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.391,"15min":1.393},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.847,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4339999999999999,"15min":1.45},"min":{"1min":0.96399999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.6499999999999999,"5min":2.371,"15min":2.371},"last":1.2509999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.444,"15min":1.448},"min":{"1min":0.89300000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9730000000000001,"5min":2.246,"15min":2.246},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4950000000000001,"15min":1.5189999999999999},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.7629999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.94499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":942410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.872,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.92400000000000004,"15min":0.95899999999999996},"min":{"1min":0.39200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.516,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.88400000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.004,"15min":1},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6619999999999999,"5min":1.784,"15min":1.784},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.1060000000000001,"15min":1.1319999999999999},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.639,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.94899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4750000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.077,"15min":1.087},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.506,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.204}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.23,"15min":1.226},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7070000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.296},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3129999999999999},"min":{"1min":1.0149999999999999,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.714,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.496}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.329,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9610000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.415},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.2709999999999999,"15min":1.278},"min":{"1min":0.86099999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.865,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1060000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.4570000000000001,"15min":1.4830000000000001},"min":{"1min":0.84199999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8500000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.353},{"interface":"front","average":{"1min":1.429,"5min":1.4710000000000001,"15min":1.5109999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.903,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.262}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.448,"15min":1.456},"min":{"1min":0.95999999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8180000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.329},{"interface":"front","average":{"1min":1.52,"5min":1.417,"15min":1.425},"min":{"1min":0.84399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0630000000000002,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.446}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:30:39.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:30:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.545+0000 7f8c227fa640 1 Processor -- start 2023-11-24T06:30:39.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.545+0000 7f8c227fa640 1 -- start start 2023-11-24T06:30:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.546+0000 7f8c227fa640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c15afb0 0x7f8c1c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.546+0000 7f8c227fa640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c13ff00 con 0x7f8c1c13fae0 2023-11-24T06:30:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.546+0000 7f8c227fa640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c0cdc40 con 0x7f8c1c13cfa0 2023-11-24T06:30:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.546+0000 7f8c227fa640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0cdec0 con 0x7f8c1c15afb0 2023-11-24T06:30:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.547+0000 7f8c21ff9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c1c13fae0 0x7f8c1c0b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44758/0 (socket says 172.21.15.16:44758) 2023-11-24T06:30:39.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.547+0000 7f8c21ff9640 1 -- 172.21.15.16:0/526193589 learned_addr learned my addr 172.21.15.16:0/526193589 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.547+0000 7f8c20ff7640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c15afb0 0x7f8c1c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.547+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905790459 0 0) 0x7f8c1c13ff00 con 0x7f8c1c13fae0 2023-11-24T06:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.547+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c08003520 con 0x7f8c1c13fae0 2023-11-24T06:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 309901284 0 0) 0x7f8c08003520 con 0x7f8c1c13fae0 2023-11-24T06:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c15afb0 msgr2=0x7f8c1c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c15afb0 0x7f8c1c13f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 >> v1:172.21.15.173:6789/0 conn(0x7f8c1c13cfa0 legacy=0x7f8c1c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0ce1a0 con 0x7f8c1c13fae0 2023-11-24T06:30:39.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f8c140019f0 con 0x7f8c1c13fae0 2023-11-24T06:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 >> v1:172.21.15.16:6789/0 conn(0x7f8c1c13fae0 legacy=0x7f8c1c0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c15afb0 0x7f8c080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c08003f20 0x7f8c080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c08004d00 0x7f8c08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0cdec0 con 0x7f8c08003f20 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0800dd20 con 0x7f8c08004d00 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0800dee0 con 0x7f8c1c15afb0 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.548+0000 7f8c20ff7640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c08003f20 0x7f8c080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.549+0000 7f8c21ff9640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c08004d00 0x7f8c08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.549+0000 7f8c20ff7640 1 -- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c15afb0 msgr2=0x7f8c080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.549+0000 7f8c217f8640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c15afb0 0x7f8c080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.549+0000 7f8c20ff7640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c1c15afb0 0x7f8c080039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.549+0000 7f8c20ff7640 1 -- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c08004d00 msgr2=0x7f8c08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.549+0000 7f8c20ff7640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c08004d00 0x7f8c08005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.549+0000 7f8c20ff7640 1 -- 172.21.15.16:0/526193589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08003520 con 0x7f8c08003f20 2023-11-24T06:30:39.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.550+0000 7f8c20ff7640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c08003f20 0x7f8c080047c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009ac0 tx=0x7f8c080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1985084475e689 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:39.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.550+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c02c6b0 con 0x7f8c08003f20 2023-11-24T06:30:39.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.550+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c02c810 con 0x7f8c08003f20 2023-11-24T06:30:39.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.550+0000 7f8c13fff640 1 -- 172.21.15.16:0/526193589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c02cba0 con 0x7f8c08003f20 2023-11-24T06:30:39.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.551+0000 7f8c227fa640 1 -- 172.21.15.16:0/526193589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c08003f20 msgr2=0x7f8c080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.551+0000 7f8c227fa640 1 --2- 172.21.15.16:0/526193589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c08003f20 0x7f8c080047c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009ac0 tx=0x7f8c080057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.551+0000 7f8c21ff9640 1 -- 172.21.15.16:0/526193589 reap_dead start 2023-11-24T06:30:39.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.551+0000 7f8c227fa640 1 -- 172.21.15.16:0/526193589 shutdown_connections 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.551+0000 7f8c227fa640 1 -- 172.21.15.16:0/526193589 >> 172.21.15.16:0/526193589 conn(0x7f8c1c06a800 msgr2=0x7f8c1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.551+0000 7f8c227fa640 1 -- 172.21.15.16:0/526193589 shutdown_connections 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.551+0000 7f8c227fa640 1 -- 172.21.15.16:0/526193589 wait complete. 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.553+0000 7f8c227fa640 1 Processor -- start 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.553+0000 7f8c227fa640 1 -- start start 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c227fa640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c08003f20 0x7f8c1c1f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c227fa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1c0acfa0 0x7f8c1c0adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c227fa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c152440 0x7f8c1c071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c227fa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0ad780 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c227fa640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c152fc0 con 0x7f8c1c152440 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c227fa640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c1f6950 con 0x7f8c08003f20 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c20ff7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1c0acfa0 0x7f8c1c0adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c20ff7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1c0acfa0 0x7f8c1c0adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36470/0 (socket says 172.21.15.16:36470) 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c20ff7640 1 -- 172.21.15.16:0/1760236432 learned_addr learned my addr 172.21.15.16:0/1760236432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c217f8640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c08003f20 0x7f8c1c1f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.554+0000 7f8c21ff9640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c152440 0x7f8c1c071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c20ff7640 1 -- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c08003f20 msgr2=0x7f8c1c1f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c20ff7640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c08003f20 0x7f8c1c1f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c20ff7640 1 -- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c152440 msgr2=0x7f8c1c071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c20ff7640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c152440 0x7f8c1c071300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c20ff7640 1 -- 172.21.15.16:0/1760236432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0ad4a0 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c21ff9640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c152440 0x7f8c1c071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c217f8640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c08003f20 0x7f8c1c1f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c20ff7640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1c0acfa0 0x7f8c1c0adf60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c037040 tx=0x7f8c0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c11ffb640 1 -- 172.21.15.16:0/1760236432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c003810 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c11ffb640 1 -- 172.21.15.16:0/1760236432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c003970 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c11ffb640 1 -- 172.21.15.16:0/1760236432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c003d00 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c1f6bf0 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.555+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c1528f0 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.557+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c067450 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.558+0000 7f8c11ffb640 1 -- 172.21.15.16:0/1760236432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f8c0c047450 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.559+0000 7f8c11ffb640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8c0006b280 0x7f8c0006d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.559+0000 7f8c11ffb640 1 -- 172.21.15.16:0/1760236432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c0c0abf70 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.559+0000 7f8c217f8640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8c0006b280 0x7f8c0006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:39.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.560+0000 7f8c217f8640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8c0006b280 0x7f8c0006d740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8c180016f0 tx=0x7f8c18009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:39.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.563+0000 7f8c11ffb640 1 -- 172.21.15.16:0/1760236432 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c0c076bc0 con 0x7f8c1c0acfa0 2023-11-24T06:30:39.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.752+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c1c152b70 con 0x7f8c0006b280 2023-11-24T06:30:39.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.774+0000 7f8c11ffb640 1 -- 172.21.15.16:0/1760236432 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281025 (secure 0 0 0) 0x7f8c1c152b70 con 0x7f8c0006b280 2023-11-24T06:30:39.781 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:30:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.780+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8c0006b280 msgr2=0x7f8c0006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.780+0000 7f8c227fa640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8c0006b280 0x7f8c0006d740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8c180016f0 tx=0x7f8c18009040 comp rx=0 tx=0).stop 2023-11-24T06:30:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.780+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1c0acfa0 msgr2=0x7f8c1c0adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.780+0000 7f8c227fa640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1c0acfa0 0x7f8c1c0adf60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c037040 tx=0x7f8c0c004730 comp rx=0 tx=0).stop 2023-11-24T06:30:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.781+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 shutdown_connections 2023-11-24T06:30:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.781+0000 7f8c227fa640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8c0006b280 0x7f8c0006d740 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.781+0000 7f8c227fa640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c1c152440 0x7f8c1c071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.781+0000 7f8c227fa640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1c0acfa0 0x7f8c1c0adf60 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.781+0000 7f8c227fa640 1 --2- 172.21.15.16:0/1760236432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c08003f20 0x7f8c1c1f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.781+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 >> 172.21.15.16:0/1760236432 conn(0x7f8c1c06a800 msgr2=0x7f8c1c153c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.781+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 shutdown_connections 2023-11-24T06:30:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:39.781+0000 7f8c227fa640 1 -- 172.21.15.16:0/1760236432 wait complete. 2023-11-24T06:30:39.806 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2023-11-24T06:30:38.178434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253000,"kb_used_data":7720,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721720,"statfs":{"total":773094113280,"available":772835041280,"internally_reserved":0,"allocated":7905280,"data_stored":3997072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.964995"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.244535+0000","last_change":"2023-11-24T06:30:33.244535+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.244535+0000","last_undegraded":"2023-11-24T06:30:33.244535+0000","last_fullsized":"2023-11-24T06:30:33.244535+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.259035+0000","last_change":"2023-11-24T06:30:33.259035+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.259035+0000","last_undegraded":"2023-11-24T06:30:33.259035+0000","last_fullsized":"2023-11-24T06:30:33.259035+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.243192+0000","last_change":"2023-11-24T06:30:33.243192+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.243192+0000","last_undegraded":"2023-11-24T06:30:33.243192+0000","last_fullsized":"2023-11-24T06:30:33.243192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.236477+0000","last_change":"2023-11-24T06:30:33.236477+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.236477+0000","last_undegraded":"2023-11-24T06:30:33.236477+0000","last_fullsized":"2023-11-24T06:30:33.236477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239321+0000","last_change":"2023-11-24T06:30:33.239321+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239321+0000","last_undegraded":"2023-11-24T06:30:33.239321+0000","last_fullsized":"2023-11-24T06:30:33.239321+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.250847+0000","last_change":"2023-11-24T06:30:33.250847+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.250847+0000","last_undegraded":"2023-11-24T06:30:33.250847+0000","last_fullsized":"2023-11-24T06:30:33.250847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.254897+0000","last_change":"2023-11-24T06:30:33.254897+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.254897+0000","last_undegraded":"2023-11-24T06:30:33.254897+0000","last_fullsized":"2023-11-24T06:30:33.254897+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.252954+0000","last_change":"2023-11-24T06:30:33.252954+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.252954+0000","last_undegraded":"2023-11-24T06:30:33.252954+0000","last_fullsized":"2023-11-24T06:30:33.252954+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.238247+0000","last_change":"2023-11-24T06:30:33.238247+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.238247+0000","last_undegraded":"2023-11-24T06:30:33.238247+0000","last_fullsized":"2023-11-24T06:30:33.238247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.244235+0000","last_change":"2023-11-24T06:30:33.244235+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.244235+0000","last_undegraded":"2023-11-24T06:30:33.244235+0000","last_fullsized":"2023-11-24T06:30:33.244235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239947+0000","last_change":"2023-11-24T06:30:33.239947+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239947+0000","last_undegraded":"2023-11-24T06:30:33.239947+0000","last_fullsized":"2023-11-24T06:30:33.239947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235872+0000","last_change":"2023-11-24T06:30:33.235872+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235872+0000","last_undegraded":"2023-11-24T06:30:33.235872+0000","last_fullsized":"2023-11-24T06:30:33.235872+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247170+0000","last_change":"2023-11-24T06:30:33.247170+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247170+0000","last_undegraded":"2023-11-24T06:30:33.247170+0000","last_fullsized":"2023-11-24T06:30:33.247170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234680+0000","last_change":"2023-11-24T06:30:33.234680+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234680+0000","last_undegraded":"2023-11-24T06:30:33.234680+0000","last_fullsized":"2023-11-24T06:30:33.234680+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247698+0000","last_change":"2023-11-24T06:30:33.247698+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247698+0000","last_undegraded":"2023-11-24T06:30:33.247698+0000","last_fullsized":"2023-11-24T06:30:33.247698+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237309+0000","last_change":"2023-11-24T06:30:33.237309+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237309+0000","last_undegraded":"2023-11-24T06:30:33.237309+0000","last_fullsized":"2023-11-24T06:30:33.237309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235467+0000","last_change":"2023-11-24T06:30:33.235467+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235467+0000","last_undegraded":"2023-11-24T06:30:33.235467+0000","last_fullsized":"2023-11-24T06:30:33.235467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.236341+0000","last_change":"2023-11-24T06:30:33.236341+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.236341+0000","last_undegraded":"2023-11-24T06:30:33.236341+0000","last_fullsized":"2023-11-24T06:30:33.236341+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.258019+0000","last_change":"2023-11-24T06:30:33.258019+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.258019+0000","last_undegraded":"2023-11-24T06:30:33.258019+0000","last_fullsized":"2023-11-24T06:30:33.258019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.251109+0000","last_change":"2023-11-24T06:30:33.251109+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.251109+0000","last_undegraded":"2023-11-24T06:30:33.251109+0000","last_fullsized":"2023-11-24T06:30:33.251109+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242204+0000","last_change":"2023-11-24T06:30:33.242204+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242204+0000","last_undegraded":"2023-11-24T06:30:33.242204+0000","last_fullsized":"2023-11-24T06:30:33.242204+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239064+0000","last_change":"2023-11-24T06:30:33.239064+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239064+0000","last_undegraded":"2023-11-24T06:30:33.239064+0000","last_fullsized":"2023-11-24T06:30:33.239064+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.240220+0000","last_change":"2023-11-24T06:30:33.240220+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.240220+0000","last_undegraded":"2023-11-24T06:30:33.240220+0000","last_fullsized":"2023-11-24T06:30:33.240220+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234550+0000","last_change":"2023-11-24T06:30:33.234550+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234550+0000","last_undegraded":"2023-11-24T06:30:33.234550+0000","last_fullsized":"2023-11-24T06:30:33.234550+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.241508+0000","last_change":"2023-11-24T06:30:33.241508+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.241508+0000","last_undegraded":"2023-11-24T06:30:33.241508+0000","last_fullsized":"2023-11-24T06:30:33.241508+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247042+0000","last_change":"2023-11-24T06:30:33.247042+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247042+0000","last_undegraded":"2023-11-24T06:30:33.247042+0000","last_fullsized":"2023-11-24T06:30:33.247042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242692+0000","last_change":"2023-11-24T06:30:33.242692+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242692+0000","last_undegraded":"2023-11-24T06:30:33.242692+0000","last_fullsized":"2023-11-24T06:30:33.242692+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237560+0000","last_change":"2023-11-24T06:30:33.237560+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237560+0000","last_undegraded":"2023-11-24T06:30:33.237560+0000","last_fullsized":"2023-11-24T06:30:33.237560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246556+0000","last_change":"2023-11-24T06:30:33.246556+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246556+0000","last_undegraded":"2023-11-24T06:30:33.246556+0000","last_fullsized":"2023-11-24T06:30:33.246556+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.247176+0000","last_change":"2023-11-24T06:30:33.247176+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.247176+0000","last_undegraded":"2023-11-24T06:30:33.247176+0000","last_fullsized":"2023-11-24T06:30:33.247176+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.248346+0000","last_change":"2023-11-24T06:30:33.248346+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.248346+0000","last_undegraded":"2023-11-24T06:30:33.248346+0000","last_fullsized":"2023-11-24T06:30:33.248346+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.252179+0000","last_change":"2023-11-24T06:30:33.252179+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.252179+0000","last_undegraded":"2023-11-24T06:30:33.252179+0000","last_fullsized":"2023-11-24T06:30:33.252179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.241747+0000","last_change":"2023-11-24T06:30:33.241747+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.241747+0000","last_undegraded":"2023-11-24T06:30:33.241747+0000","last_fullsized":"2023-11-24T06:30:33.241747+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.263585+0000","last_change":"2023-11-24T06:30:33.258485+0000","last_active":"2023-11-24T06:30:33.263585+0000","last_peered":"2023-11-24T06:30:33.263585+0000","last_clean":"2023-11-24T06:30:33.263585+0000","last_became_active":"2023-11-24T06:30:33.258091+0000","last_became_peered":"2023-11-24T06:30:33.258091+0000","last_unstale":"2023-11-24T06:30:33.263585+0000","last_undegraded":"2023-11-24T06:30:33.263585+0000","last_fullsized":"2023-11-24T06:30:33.263585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:39.081828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234860+0000","last_change":"2023-11-24T06:30:33.234860+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234860+0000","last_undegraded":"2023-11-24T06:30:33.234860+0000","last_fullsized":"2023-11-24T06:30:33.234860+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.257894+0000","last_change":"2023-11-24T06:30:33.253586+0000","last_active":"2023-11-24T06:30:33.257894+0000","last_peered":"2023-11-24T06:30:33.257894+0000","last_clean":"2023-11-24T06:30:33.257894+0000","last_became_active":"2023-11-24T06:30:33.252371+0000","last_became_peered":"2023-11-24T06:30:33.252371+0000","last_unstale":"2023-11-24T06:30:33.257894+0000","last_undegraded":"2023-11-24T06:30:33.257894+0000","last_fullsized":"2023-11-24T06:30:33.257894+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:42:17.431240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.252752+0000","last_change":"2023-11-24T06:30:33.252752+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.252752+0000","last_undegraded":"2023-11-24T06:30:33.252752+0000","last_fullsized":"2023-11-24T06:30:33.252752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267927+0000","last_change":"2023-11-24T06:30:33.266741+0000","last_active":"2023-11-24T06:30:33.267927+0000","last_peered":"2023-11-24T06:30:33.267927+0000","last_clean":"2023-11-24T06:30:33.267927+0000","last_became_active":"2023-11-24T06:30:33.266055+0000","last_became_peered":"2023-11-24T06:30:33.266055+0000","last_unstale":"2023-11-24T06:30:33.267927+0000","last_undegraded":"2023-11-24T06:30:33.267927+0000","last_fullsized":"2023-11-24T06:30:33.267927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:46.553265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.238829+0000","last_change":"2023-11-24T06:30:33.238829+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.238829+0000","last_undegraded":"2023-11-24T06:30:33.238829+0000","last_fullsized":"2023-11-24T06:30:33.238829+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267672+0000","last_change":"2023-11-24T06:30:33.260247+0000","last_active":"2023-11-24T06:30:33.267672+0000","last_peered":"2023-11-24T06:30:33.267672+0000","last_clean":"2023-11-24T06:30:33.267672+0000","last_became_active":"2023-11-24T06:30:33.259986+0000","last_became_peered":"2023-11-24T06:30:33.259986+0000","last_unstale":"2023-11-24T06:30:33.267672+0000","last_undegraded":"2023-11-24T06:30:33.267672+0000","last_fullsized":"2023-11-24T06:30:33.267672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:25.530564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246365+0000","last_change":"2023-11-24T06:30:33.246365+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246365+0000","last_undegraded":"2023-11-24T06:30:33.246365+0000","last_fullsized":"2023-11-24T06:30:33.246365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.257173+0000","last_change":"2023-11-24T06:30:33.253225+0000","last_active":"2023-11-24T06:30:33.257173+0000","last_peered":"2023-11-24T06:30:33.257173+0000","last_clean":"2023-11-24T06:30:33.257173+0000","last_became_active":"2023-11-24T06:30:33.252940+0000","last_became_peered":"2023-11-24T06:30:33.252940+0000","last_unstale":"2023-11-24T06:30:33.257173+0000","last_undegraded":"2023-11-24T06:30:33.257173+0000","last_fullsized":"2023-11-24T06:30:33.257173+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:03.692439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.240042+0000","last_change":"2023-11-24T06:30:33.240042+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.240042+0000","last_undegraded":"2023-11-24T06:30:33.240042+0000","last_fullsized":"2023-11-24T06:30:33.240042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267861+0000","last_change":"2023-11-24T06:30:33.263099+0000","last_active":"2023-11-24T06:30:33.267861+0000","last_peered":"2023-11-24T06:30:33.267861+0000","last_clean":"2023-11-24T06:30:33.267861+0000","last_became_active":"2023-11-24T06:30:33.261288+0000","last_became_peered":"2023-11-24T06:30:33.261288+0000","last_unstale":"2023-11-24T06:30:33.267861+0000","last_undegraded":"2023-11-24T06:30:33.267861+0000","last_fullsized":"2023-11-24T06:30:33.267861+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:38.745762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234536+0000","last_change":"2023-11-24T06:30:33.234536+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234536+0000","last_undegraded":"2023-11-24T06:30:33.234536+0000","last_fullsized":"2023-11-24T06:30:33.234536+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.254629+0000","last_change":"2023-11-24T06:30:33.250035+0000","last_active":"2023-11-24T06:30:33.254629+0000","last_peered":"2023-11-24T06:30:33.254629+0000","last_clean":"2023-11-24T06:30:33.254629+0000","last_became_active":"2023-11-24T06:30:33.249747+0000","last_became_peered":"2023-11-24T06:30:33.249747+0000","last_unstale":"2023-11-24T06:30:33.254629+0000","last_undegraded":"2023-11-24T06:30:33.254629+0000","last_fullsized":"2023-11-24T06:30:33.254629+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:28.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246986+0000","last_change":"2023-11-24T06:30:33.246986+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246986+0000","last_undegraded":"2023-11-24T06:30:33.246986+0000","last_fullsized":"2023-11-24T06:30:33.246986+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.245386+0000","last_change":"2023-11-24T06:30:33.240278+0000","last_active":"2023-11-24T06:30:33.245386+0000","last_peered":"2023-11-24T06:30:33.245386+0000","last_clean":"2023-11-24T06:30:33.245386+0000","last_became_active":"2023-11-24T06:30:33.238103+0000","last_became_peered":"2023-11-24T06:30:33.238103+0000","last_unstale":"2023-11-24T06:30:33.245386+0000","last_undegraded":"2023-11-24T06:30:33.245386+0000","last_fullsized":"2023-11-24T06:30:33.245386+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:01:20.140182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.246380+0000","last_change":"2023-11-24T06:30:33.246380+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.246380+0000","last_undegraded":"2023-11-24T06:30:33.246380+0000","last_fullsized":"2023-11-24T06:30:33.246380+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261289+0000","last_change":"2023-11-24T06:30:33.257372+0000","last_active":"2023-11-24T06:30:33.261289+0000","last_peered":"2023-11-24T06:30:33.261289+0000","last_clean":"2023-11-24T06:30:33.261289+0000","last_became_active":"2023-11-24T06:30:33.257063+0000","last_became_peered":"2023-11-24T06:30:33.257063+0000","last_unstale":"2023-11-24T06:30:33.261289+0000","last_undegraded":"2023-11-24T06:30:33.261289+0000","last_fullsized":"2023-11-24T06:30:33.261289+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:51.884745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242375+0000","last_change":"2023-11-24T06:30:33.242375+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242375+0000","last_undegraded":"2023-11-24T06:30:33.242375+0000","last_fullsized":"2023-11-24T06:30:33.242375+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269341+0000","last_change":"2023-11-24T06:30:33.266155+0000","last_active":"2023-11-24T06:30:33.269341+0000","last_peered":"2023-11-24T06:30:33.269341+0000","last_clean":"2023-11-24T06:30:33.269341+0000","last_became_active":"2023-11-24T06:30:33.265258+0000","last_became_peered":"2023-11-24T06:30:33.265258+0000","last_unstale":"2023-11-24T06:30:33.269341+0000","last_undegraded":"2023-11-24T06:30:33.269341+0000","last_fullsized":"2023-11-24T06:30:33.269341+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:31.950626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.248625+0000","last_change":"2023-11-24T06:30:33.248625+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.248625+0000","last_undegraded":"2023-11-24T06:30:33.248625+0000","last_fullsized":"2023-11-24T06:30:33.248625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265546+0000","last_change":"2023-11-24T06:30:33.264658+0000","last_active":"2023-11-24T06:30:33.265546+0000","last_peered":"2023-11-24T06:30:33.265546+0000","last_clean":"2023-11-24T06:30:33.265546+0000","last_became_active":"2023-11-24T06:30:33.264320+0000","last_became_peered":"2023-11-24T06:30:33.264320+0000","last_unstale":"2023-11-24T06:30:33.265546+0000","last_undegraded":"2023-11-24T06:30:33.265546+0000","last_fullsized":"2023-11-24T06:30:33.265546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:41.422715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235171+0000","last_change":"2023-11-24T06:30:33.235171+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235171+0000","last_undegraded":"2023-11-24T06:30:33.235171+0000","last_fullsized":"2023-11-24T06:30:33.235171+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.242786+0000","last_change":"2023-11-24T06:30:33.239974+0000","last_active":"2023-11-24T06:30:33.242786+0000","last_peered":"2023-11-24T06:30:33.242786+0000","last_clean":"2023-11-24T06:30:33.242786+0000","last_became_active":"2023-11-24T06:30:33.238273+0000","last_became_peered":"2023-11-24T06:30:33.238273+0000","last_unstale":"2023-11-24T06:30:33.242786+0000","last_undegraded":"2023-11-24T06:30:33.242786+0000","last_fullsized":"2023-11-24T06:30:33.242786+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:12.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.238368+0000","last_change":"2023-11-24T06:30:33.238368+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.238368+0000","last_undegraded":"2023-11-24T06:30:33.238368+0000","last_fullsized":"2023-11-24T06:30:33.238368+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.253699+0000","last_change":"2023-11-24T06:30:33.252044+0000","last_active":"2023-11-24T06:30:33.253699+0000","last_peered":"2023-11-24T06:30:33.253699+0000","last_clean":"2023-11-24T06:30:33.253699+0000","last_became_active":"2023-11-24T06:30:33.251774+0000","last_became_peered":"2023-11-24T06:30:33.251774+0000","last_unstale":"2023-11-24T06:30:33.253699+0000","last_undegraded":"2023-11-24T06:30:33.253699+0000","last_fullsized":"2023-11-24T06:30:33.253699+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:38.745762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.248362+0000","last_change":"2023-11-24T06:30:33.248362+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.248362+0000","last_undegraded":"2023-11-24T06:30:33.248362+0000","last_fullsized":"2023-11-24T06:30:33.248362+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267841+0000","last_change":"2023-11-24T06:30:33.261613+0000","last_active":"2023-11-24T06:30:33.267841+0000","last_peered":"2023-11-24T06:30:33.267841+0000","last_clean":"2023-11-24T06:30:33.267841+0000","last_became_active":"2023-11-24T06:30:33.260523+0000","last_became_peered":"2023-11-24T06:30:33.260523+0000","last_unstale":"2023-11-24T06:30:33.267841+0000","last_undegraded":"2023-11-24T06:30:33.267841+0000","last_fullsized":"2023-11-24T06:30:33.267841+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:59:27.319046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.249240+0000","last_change":"2023-11-24T06:30:33.249240+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.249240+0000","last_undegraded":"2023-11-24T06:30:33.249240+0000","last_fullsized":"2023-11-24T06:30:33.249240+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265229+0000","last_change":"2023-11-24T06:30:33.263590+0000","last_active":"2023-11-24T06:30:33.265229+0000","last_peered":"2023-11-24T06:30:33.265229+0000","last_clean":"2023-11-24T06:30:33.265229+0000","last_became_active":"2023-11-24T06:30:33.262788+0000","last_became_peered":"2023-11-24T06:30:33.262788+0000","last_unstale":"2023-11-24T06:30:33.265229+0000","last_undegraded":"2023-11-24T06:30:33.265229+0000","last_fullsized":"2023-11-24T06:30:33.265229+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:10.481218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.233300+0000","last_change":"2023-11-24T06:30:33.233300+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.233300+0000","last_undegraded":"2023-11-24T06:30:33.233300+0000","last_fullsized":"2023-11-24T06:30:33.233300+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261672+0000","last_change":"2023-11-24T06:30:33.259288+0000","last_active":"2023-11-24T06:30:33.261672+0000","last_peered":"2023-11-24T06:30:33.261672+0000","last_clean":"2023-11-24T06:30:33.261672+0000","last_became_active":"2023-11-24T06:30:33.254396+0000","last_became_peered":"2023-11-24T06:30:33.254396+0000","last_unstale":"2023-11-24T06:30:33.261672+0000","last_undegraded":"2023-11-24T06:30:33.261672+0000","last_fullsized":"2023-11-24T06:30:33.261672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:30.689902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237823+0000","last_change":"2023-11-24T06:30:33.237823+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237823+0000","last_undegraded":"2023-11-24T06:30:33.237823+0000","last_fullsized":"2023-11-24T06:30:33.237823+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269298+0000","last_change":"2023-11-24T06:30:33.265823+0000","last_active":"2023-11-24T06:30:33.269298+0000","last_peered":"2023-11-24T06:30:33.269298+0000","last_clean":"2023-11-24T06:30:33.269298+0000","last_became_active":"2023-11-24T06:30:33.265509+0000","last_became_peered":"2023-11-24T06:30:33.265509+0000","last_unstale":"2023-11-24T06:30:33.269298+0000","last_undegraded":"2023-11-24T06:30:33.269298+0000","last_fullsized":"2023-11-24T06:30:33.269298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:21.340229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.241198+0000","last_change":"2023-11-24T06:30:33.241198+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.241198+0000","last_undegraded":"2023-11-24T06:30:33.241198+0000","last_fullsized":"2023-11-24T06:30:33.241198+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269221+0000","last_change":"2023-11-24T06:30:33.268072+0000","last_active":"2023-11-24T06:30:33.269221+0000","last_peered":"2023-11-24T06:30:33.269221+0000","last_clean":"2023-11-24T06:30:33.269221+0000","last_became_active":"2023-11-24T06:30:33.267757+0000","last_became_peered":"2023-11-24T06:30:33.267757+0000","last_unstale":"2023-11-24T06:30:33.269221+0000","last_undegraded":"2023-11-24T06:30:33.269221+0000","last_fullsized":"2023-11-24T06:30:33.269221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:18.630896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.235215+0000","last_change":"2023-11-24T06:30:33.235215+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.235215+0000","last_undegraded":"2023-11-24T06:30:33.235215+0000","last_fullsized":"2023-11-24T06:30:33.235215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269306+0000","last_change":"2023-11-24T06:30:33.265903+0000","last_active":"2023-11-24T06:30:33.269306+0000","last_peered":"2023-11-24T06:30:33.269306+0000","last_clean":"2023-11-24T06:30:33.269306+0000","last_became_active":"2023-11-24T06:30:33.263286+0000","last_became_peered":"2023-11-24T06:30:33.263286+0000","last_unstale":"2023-11-24T06:30:33.269306+0000","last_undegraded":"2023-11-24T06:30:33.269306+0000","last_fullsized":"2023-11-24T06:30:33.269306+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:49:20.198853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.233222+0000","last_change":"2023-11-24T06:30:33.233222+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.233222+0000","last_undegraded":"2023-11-24T06:30:33.233222+0000","last_fullsized":"2023-11-24T06:30:33.233222+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.263521+0000","last_change":"2023-11-24T06:30:33.259236+0000","last_active":"2023-11-24T06:30:33.263521+0000","last_peered":"2023-11-24T06:30:33.263521+0000","last_clean":"2023-11-24T06:30:33.263521+0000","last_became_active":"2023-11-24T06:30:33.258200+0000","last_became_peered":"2023-11-24T06:30:33.258200+0000","last_unstale":"2023-11-24T06:30:33.263521+0000","last_undegraded":"2023-11-24T06:30:33.263521+0000","last_fullsized":"2023-11-24T06:30:33.263521+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:08.963743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.243611+0000","last_change":"2023-11-24T06:30:33.243611+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.243611+0000","last_undegraded":"2023-11-24T06:30:33.243611+0000","last_fullsized":"2023-11-24T06:30:33.243611+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267605+0000","last_change":"2023-11-24T06:30:33.259814+0000","last_active":"2023-11-24T06:30:33.267605+0000","last_peered":"2023-11-24T06:30:33.267605+0000","last_clean":"2023-11-24T06:30:33.267605+0000","last_became_active":"2023-11-24T06:30:33.259551+0000","last_became_peered":"2023-11-24T06:30:33.259551+0000","last_unstale":"2023-11-24T06:30:33.267605+0000","last_undegraded":"2023-11-24T06:30:33.267605+0000","last_fullsized":"2023-11-24T06:30:33.267605+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:28.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242279+0000","last_change":"2023-11-24T06:30:33.242279+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242279+0000","last_undegraded":"2023-11-24T06:30:33.242279+0000","last_fullsized":"2023-11-24T06:30:33.242279+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265212+0000","last_change":"2023-11-24T06:30:33.263043+0000","last_active":"2023-11-24T06:30:33.265212+0000","last_peered":"2023-11-24T06:30:33.265212+0000","last_clean":"2023-11-24T06:30:33.265212+0000","last_became_active":"2023-11-24T06:30:33.262727+0000","last_became_peered":"2023-11-24T06:30:33.262727+0000","last_unstale":"2023-11-24T06:30:33.265212+0000","last_undegraded":"2023-11-24T06:30:33.265212+0000","last_fullsized":"2023-11-24T06:30:33.265212+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:33.658163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242662+0000","last_change":"2023-11-24T06:30:33.242662+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242662+0000","last_undegraded":"2023-11-24T06:30:33.242662+0000","last_fullsized":"2023-11-24T06:30:33.242662+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.262428+0000","last_change":"2023-11-24T06:30:33.259539+0000","last_active":"2023-11-24T06:30:33.262428+0000","last_peered":"2023-11-24T06:30:33.262428+0000","last_clean":"2023-11-24T06:30:33.262428+0000","last_became_active":"2023-11-24T06:30:33.258994+0000","last_became_peered":"2023-11-24T06:30:33.258994+0000","last_unstale":"2023-11-24T06:30:33.262428+0000","last_undegraded":"2023-11-24T06:30:33.262428+0000","last_fullsized":"2023-11-24T06:30:33.262428+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:40.643302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.249084+0000","last_change":"2023-11-24T06:30:33.249084+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.249084+0000","last_undegraded":"2023-11-24T06:30:33.249084+0000","last_fullsized":"2023-11-24T06:30:33.249084+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267896+0000","last_change":"2023-11-24T06:30:33.264626+0000","last_active":"2023-11-24T06:30:33.267896+0000","last_peered":"2023-11-24T06:30:33.267896+0000","last_clean":"2023-11-24T06:30:33.267896+0000","last_became_active":"2023-11-24T06:30:33.264323+0000","last_became_peered":"2023-11-24T06:30:33.264323+0000","last_unstale":"2023-11-24T06:30:33.267896+0000","last_undegraded":"2023-11-24T06:30:33.267896+0000","last_fullsized":"2023-11-24T06:30:33.267896+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:44.399558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.254177+0000","last_change":"2023-11-24T06:30:33.254177+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.254177+0000","last_undegraded":"2023-11-24T06:30:33.254177+0000","last_fullsized":"2023-11-24T06:30:33.254177+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.265269+0000","last_change":"2023-11-24T06:30:33.262138+0000","last_active":"2023-11-24T06:30:33.265269+0000","last_peered":"2023-11-24T06:30:33.265269+0000","last_clean":"2023-11-24T06:30:33.265269+0000","last_became_active":"2023-11-24T06:30:33.261687+0000","last_became_peered":"2023-11-24T06:30:33.261687+0000","last_unstale":"2023-11-24T06:30:33.265269+0000","last_undegraded":"2023-11-24T06:30:33.265269+0000","last_fullsized":"2023-11-24T06:30:33.265269+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:05.740113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.239053+0000","last_change":"2023-11-24T06:30:33.239053+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.239053+0000","last_undegraded":"2023-11-24T06:30:33.239053+0000","last_fullsized":"2023-11-24T06:30:33.239053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.257585+0000","last_change":"2023-11-24T06:30:33.255156+0000","last_active":"2023-11-24T06:30:33.257585+0000","last_peered":"2023-11-24T06:30:33.257585+0000","last_clean":"2023-11-24T06:30:33.257585+0000","last_became_active":"2023-11-24T06:30:33.254878+0000","last_became_peered":"2023-11-24T06:30:33.254878+0000","last_unstale":"2023-11-24T06:30:33.257585+0000","last_undegraded":"2023-11-24T06:30:33.257585+0000","last_fullsized":"2023-11-24T06:30:33.257585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:38.180557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.237564+0000","last_change":"2023-11-24T06:30:33.237564+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.237564+0000","last_undegraded":"2023-11-24T06:30:33.237564+0000","last_fullsized":"2023-11-24T06:30:33.237564+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.264428+0000","last_change":"2023-11-24T06:30:33.263337+0000","last_active":"2023-11-24T06:30:33.264428+0000","last_peered":"2023-11-24T06:30:33.264428+0000","last_clean":"2023-11-24T06:30:33.264428+0000","last_became_active":"2023-11-24T06:30:33.262997+0000","last_became_peered":"2023-11-24T06:30:33.262997+0000","last_unstale":"2023-11-24T06:30:33.264428+0000","last_undegraded":"2023-11-24T06:30:33.264428+0000","last_fullsized":"2023-11-24T06:30:33.264428+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:00.154357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.232644+0000","last_change":"2023-11-24T06:30:33.232644+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.232644+0000","last_undegraded":"2023-11-24T06:30:33.232644+0000","last_fullsized":"2023-11-24T06:30:33.232644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261315+0000","last_change":"2023-11-24T06:30:33.258636+0000","last_active":"2023-11-24T06:30:33.261315+0000","last_peered":"2023-11-24T06:30:33.261315+0000","last_clean":"2023-11-24T06:30:33.261315+0000","last_became_active":"2023-11-24T06:30:33.258193+0000","last_became_peered":"2023-11-24T06:30:33.258193+0000","last_unstale":"2023-11-24T06:30:33.261315+0000","last_undegraded":"2023-11-24T06:30:33.261315+0000","last_fullsized":"2023-11-24T06:30:33.261315+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:39:24.507654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.249296+0000","last_change":"2023-11-24T06:30:33.249296+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.249296+0000","last_undegraded":"2023-11-24T06:30:33.249296+0000","last_fullsized":"2023-11-24T06:30:33.249296+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.256814+0000","last_change":"2023-11-24T06:30:33.253808+0000","last_active":"2023-11-24T06:30:33.256814+0000","last_peered":"2023-11-24T06:30:33.256814+0000","last_clean":"2023-11-24T06:30:33.256814+0000","last_became_active":"2023-11-24T06:30:33.250888+0000","last_became_peered":"2023-11-24T06:30:33.250888+0000","last_unstale":"2023-11-24T06:30:33.256814+0000","last_undegraded":"2023-11-24T06:30:33.256814+0000","last_fullsized":"2023-11-24T06:30:33.256814+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:59:27.319046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.234698+0000","last_change":"2023-11-24T06:30:33.234698+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.234698+0000","last_undegraded":"2023-11-24T06:30:33.234698+0000","last_fullsized":"2023-11-24T06:30:33.234698+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.261935+0000","last_change":"2023-11-24T06:30:33.257088+0000","last_active":"2023-11-24T06:30:33.261935+0000","last_peered":"2023-11-24T06:30:33.261935+0000","last_clean":"2023-11-24T06:30:33.261935+0000","last_became_active":"2023-11-24T06:30:33.256733+0000","last_became_peered":"2023-11-24T06:30:33.256733+0000","last_unstale":"2023-11-24T06:30:33.261935+0000","last_undegraded":"2023-11-24T06:30:33.261935+0000","last_fullsized":"2023-11-24T06:30:33.261935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:52.534613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.251567+0000","last_change":"2023-11-24T06:30:33.251567+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.251567+0000","last_undegraded":"2023-11-24T06:30:33.251567+0000","last_fullsized":"2023-11-24T06:30:33.251567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.269279+0000","last_change":"2023-11-24T06:30:33.265710+0000","last_active":"2023-11-24T06:30:33.269279+0000","last_peered":"2023-11-24T06:30:33.269279+0000","last_clean":"2023-11-24T06:30:33.269279+0000","last_became_active":"2023-11-24T06:30:33.265397+0000","last_became_peered":"2023-11-24T06:30:33.265397+0000","last_unstale":"2023-11-24T06:30:33.269279+0000","last_undegraded":"2023-11-24T06:30:33.269279+0000","last_fullsized":"2023-11-24T06:30:33.269279+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:30.859810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:30:33.242319+0000","last_change":"2023-11-24T06:30:33.242319+0000","last_active":"2023-11-24T06:30:33.211156+0000","last_peered":"2023-11-24T06:30:33.211156+0000","last_clean":"2023-11-24T06:30:33.211156+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:30:33.242319+0000","last_undegraded":"2023-11-24T06:30:33.242319+0000","last_fullsized":"2023-11-24T06:30:33.242319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2023-11-24T06:30:33.267727+0000","last_change":"2023-11-24T06:30:33.260898+0000","last_active":"2023-11-24T06:30:33.267727+0000","last_peered":"2023-11-24T06:30:33.267727+0000","last_clean":"2023-11-24T06:30:33.267727+0000","last_became_active":"2023-11-24T06:30:33.260528+0000","last_became_peered":"2023-11-24T06:30:33.260528+0000","last_unstale":"2023-11-24T06:30:33.267727+0000","last_undegraded":"2023-11-24T06:30:33.267727+0000","last_fullsized":"2023-11-24T06:30:33.267727+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:12.375112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":215,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-24T06:30:32.217624+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:30:32.217624+0000","last_peered":"2023-11-24T06:30:32.217624+0000","last_clean":"2023-11-24T06:30:32.217624+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:30:32.217624+0000","last_undegraded":"2023-11-24T06:30:32.217624+0000","last_fullsized":"2023-11-24T06:30:32.217624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":942410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.252,"15min":1.3160000000000001},"min":{"1min":0.68100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.0599999999999996,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.123},{"interface":"front","average":{"1min":1.389,"5min":1.296,"15min":1.3009999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":4.556,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.524}]},{"osd":1,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0249999999999999,"15min":1.012},"min":{"1min":0.68200000000000005,"5min":0.629,"15min":0.629},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.343,"15min":1.377},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":3.944,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.4419999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.2729999999999999,"15min":1.288},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.5609999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.3539999999999992,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":1.306},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.204},"min":{"1min":0.79300000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.8200000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.714}]},{"osd":4,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.4870000000000001,"15min":1.5669999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.1059999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.331},{"interface":"front","average":{"1min":1.498,"5min":1.5369999999999999,"15min":1.579},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.6000000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.3580000000000001,"15min":1.399},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.3119999999999998,"5min":1.8240000000000001,"15min":1.794},"min":{"1min":0.69799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":17.582999999999998,"5min":17.582999999999998,"15min":17.582999999999998},"last":1.6739999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.736,"15min":1.8480000000000001},"min":{"1min":0.73099999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.8140000000000001,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.744},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.6240000000000001,"15min":1.694},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":6.2210000000000001,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.3999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0980000000000001,"15min":1.1020000000000001},"min":{"1min":0.73499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7390000000000001,"5min":1.829,"15min":1.829},"last":1.03},{"interface":"front","average":{"1min":1.155,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.76800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.623,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.234}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2210000000000001,"15min":1.2130000000000001},"min":{"1min":0.71099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6299999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.573},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.2010000000000001,"15min":1.2070000000000001},"min":{"1min":0.88900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.494,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.323}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.159},"min":{"1min":0.75700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.609,"5min":3.597,"15min":3.597},"last":1.704},{"interface":"front","average":{"1min":1.304,"5min":1.292,"15min":1.3140000000000001},"min":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6359999999999999,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.6579999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3129999999999999,"15min":1.3180000000000001},"min":{"1min":0.78100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5620000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.2969999999999999,"15min":1.325},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5960000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.52}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.8919999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.385,"15min":1.397},"min":{"1min":0.79600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7729999999999999,"5min":3.04,"15min":3.04},"last":1.091}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3720000000000001,"15min":1.377},"min":{"1min":0.97699999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7330000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.552,"15min":1.6020000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.851,"5min":3.774,"15min":3.774},"last":1.397}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5449999999999999,"15min":1.5589999999999999},"min":{"1min":1.0029999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7689999999999999,"5min":3.403,"15min":3.403},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.512},"min":{"1min":0.875,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.792,"5min":3.649,"15min":3.649},"last":1.2649999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.516,"5min":1.903,"15min":1.903},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.288,"15min":1.3140000000000001},"min":{"1min":0.91700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.915,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.5349999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.145,"15min":1.1399999999999999},"min":{"1min":0.78900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.147,"15min":1.1140000000000001},"min":{"1min":0.89500000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.347}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.804},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3109999999999999,"15min":1.2969999999999999},"min":{"1min":0.99199999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1669999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.2969999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3360000000000001,"15min":1.375},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.952,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.405},{"interface":"front","average":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3899999999999999},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.5840000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3180000000000001,"15min":1.3340000000000001},"min":{"1min":0.878,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.839,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.5029999999999999,"15min":1.5369999999999999},"min":{"1min":0.82299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0099999999999998,"5min":3.085,"15min":3.085},"last":1.641}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4690000000000001,"15min":1.4450000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.464},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.474,"15min":1.468},"min":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0270000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.5069999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.651,"5min":1.6140000000000001,"15min":1.6240000000000001},"min":{"1min":1.046,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.141,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.714,"5min":1.6919999999999999,"15min":1.716},"min":{"1min":1.0129999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3159999999999998,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.673}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.073,"15min":1.073},"min":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.982,"15min":1.982},"last":1.222},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.089,"15min":1.1120000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.468,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.262}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.147,"15min":1.113},"min":{"1min":0.84199999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.26,"15min":1.3049999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5840000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.0980000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8129999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.27,"5min":1.3140000000000001,"15min":1.3320000000000001},"min":{"1min":0.79800000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7250000000000001,"5min":2.254,"15min":2.254},"last":1.3}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.294,"15min":1.333},"min":{"1min":0.76100000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.833,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3779999999999999,"15min":1.3720000000000001},"min":{"1min":0.753,"5min":0.746,"15min":0.746},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.071}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.482,"15min":1.5149999999999999},"min":{"1min":0.874,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8109999999999999,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.522,"15min":1.5609999999999999},"min":{"1min":0.85799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.923,"5min":3.52,"15min":3.52},"last":1.179}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.474,"15min":1.482},"min":{"1min":0.79200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.542,"15min":1.5740000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.859,"5min":2.452,"15min":2.452},"last":1.359}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.595,"15min":1.6559999999999999},"min":{"1min":0.64000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.7729999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.615,"15min":1.613},"min":{"1min":0.84799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.0099999999999998,"5min":2.363,"15min":2.363},"last":1.2030000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.3080000000000001,"5min":1.355,"15min":1.355},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.66600000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.96399999999999997},"min":{"1min":0.51800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.077,"5min":1.091,"15min":1.113},"min":{"1min":0.61499999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.91900000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.0350000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.58499999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4550000000000001,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.133}]},{"osd":4,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3420000000000001,"15min":1.3879999999999999},"min":{"1min":0.91300000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.696,"5min":3.081,"15min":3.081},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.3340000000000001,"15min":1.339},"min":{"1min":0.74299999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.712,"5min":2.758,"15min":2.758},"last":1.7230000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.373,"15min":1.403},"min":{"1min":0.873,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.1200000000000001,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.462},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.6200000000000001,"15min":1.748},"min":{"1min":0.86299999999999999,"5min":0.745,"15min":0.745},"max":{"1min":1.657,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.786}]},{"osd":6,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3759999999999999,"15min":1.407},"min":{"1min":0.82999999999999996,"5min":0.753,"15min":0.753},"max":{"1min":1.79,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.534},{"interface":"front","average":{"1min":1.524,"5min":1.53,"15min":1.544},"min":{"1min":1.0760000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.464,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.4790000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.5760000000000001,"15min":1.609},"min":{"1min":0.84499999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.589,"5min":4.641,"15min":4.641},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.546,"15min":1.5740000000000001},"min":{"1min":0.88200000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.3439999999999999,"5min":3.798,"15min":3.798},"last":1.7490000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2729999999999999,"5min":1.427,"15min":1.427},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.619,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.59699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.48499999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.78800000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0840000000000001,"15min":1.1040000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.784,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.095,"15min":1.117},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6990000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.575}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.456,"15min":1.508},"min":{"1min":0.67900000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.6869999999999998,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.454,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.5800000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4650000000000001,"15min":1.468},"min":{"1min":1.0029999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.036,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.4690000000000001,"15min":1.5009999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.841,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.2769999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.423,"15min":1.4359999999999999},"min":{"1min":0.66900000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.968,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.004},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.522,"15min":1.54},"min":{"1min":1.0029999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.165,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.444}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5149999999999999,"15min":1.5489999999999999},"min":{"1min":0.78200000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.913,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":1.097,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.423,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.9359999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":352042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.002,"15min":1.032},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.696,"5min":2.593,"15min":2.593},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.008,"5min":1.02,"15min":1.032},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5980000000000001,"5min":1.702,"15min":1.702},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.212,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":1,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7669999999999999,"5min":1.829,"15min":1.829},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1040000000000001,"15min":1.1200000000000001},"min":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.603,"5min":1.766,"15min":1.766},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0289999999999999,"15min":1.069},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2250000000000001,"5min":1.798,"15min":1.798},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.246,"15min":1.2529999999999999},"min":{"1min":0.89000000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.621,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.359,"5min":1.417,"15min":1.4450000000000001},"min":{"1min":1.0509999999999999,"5min":1.032,"15min":1.032},"max":{"1min":1.837,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":0.96299999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.294,"15min":1.302},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.514,"5min":2.016,"15min":2.016},"last":1.304},{"interface":"front","average":{"1min":1.391,"5min":1.389,"15min":1.399},"min":{"1min":0.91700000000000004,"5min":0.877,"15min":0.877},"max":{"1min":1.7410000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.071}]},{"osd":6,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.391,"15min":1.393},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.847,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4339999999999999,"15min":1.45},"min":{"1min":0.96399999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.6499999999999999,"5min":2.371,"15min":2.371},"last":1.2509999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.444,"15min":1.448},"min":{"1min":0.89300000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9730000000000001,"5min":2.246,"15min":2.246},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4950000000000001,"15min":1.5189999999999999},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.7629999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.94499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1433600,"data_stored":942410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.872,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.92400000000000004,"15min":0.95899999999999996},"min":{"1min":0.39200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.516,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.88400000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.004,"15min":1},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6619999999999999,"5min":1.784,"15min":1.784},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.1060000000000001,"15min":1.1319999999999999},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.639,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.1479999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.94899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4750000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.052,"5min":1.077,"15min":1.087},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.506,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.204}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.23,"15min":1.226},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7070000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.296},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3129999999999999},"min":{"1min":1.0149999999999999,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.714,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.496}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.329,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9610000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.415},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.2709999999999999,"15min":1.278},"min":{"1min":0.86099999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.865,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1060000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.4570000000000001,"15min":1.4830000000000001},"min":{"1min":0.84199999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8500000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.353},{"interface":"front","average":{"1min":1.429,"5min":1.4710000000000001,"15min":1.5109999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.903,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.262}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.448,"15min":1.456},"min":{"1min":0.95999999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8180000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.329},{"interface":"front","average":{"1min":1.52,"5min":1.417,"15min":1.425},"min":{"1min":0.84399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0630000000000002,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.446}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:30:39.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-11-24T06:30:39.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.244535+0000', 'last_change': '2023-11-24T06:30:33.244535+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.244535+0000', 'last_undegraded': '2023-11-24T06:30:33.244535+0000', 'last_fullsized': '2023-11-24T06:30:33.244535+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-11-24T06:30:39.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.259035+0000', 'last_change': '2023-11-24T06:30:33.259035+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.259035+0000', 'last_undegraded': '2023-11-24T06:30:33.259035+0000', 'last_fullsized': '2023-11-24T06:30:33.259035+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:30:39.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-11-24T06:30:39.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.243192+0000', 'last_change': '2023-11-24T06:30:33.243192+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.243192+0000', 'last_undegraded': '2023-11-24T06:30:33.243192+0000', 'last_fullsized': '2023-11-24T06:30:33.243192+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:30:39.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-11-24T06:30:39.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.236477+0000', 'last_change': '2023-11-24T06:30:33.236477+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.236477+0000', 'last_undegraded': '2023-11-24T06:30:33.236477+0000', 'last_fullsized': '2023-11-24T06:30:33.236477+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:30:39.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-11-24T06:30:39.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.239321+0000', 'last_change': '2023-11-24T06:30:33.239321+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.239321+0000', 'last_undegraded': '2023-11-24T06:30:33.239321+0000', 'last_fullsized': '2023-11-24T06:30:33.239321+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-11-24T06:30:39.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.250847+0000', 'last_change': '2023-11-24T06:30:33.250847+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.250847+0000', 'last_undegraded': '2023-11-24T06:30:33.250847+0000', 'last_fullsized': '2023-11-24T06:30:33.250847+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:30:39.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-11-24T06:30:39.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.254897+0000', 'last_change': '2023-11-24T06:30:33.254897+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.254897+0000', 'last_undegraded': '2023-11-24T06:30:33.254897+0000', 'last_fullsized': '2023-11-24T06:30:33.254897+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:30:39.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-11-24T06:30:39.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.252954+0000', 'last_change': '2023-11-24T06:30:33.252954+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.252954+0000', 'last_undegraded': '2023-11-24T06:30:33.252954+0000', 'last_fullsized': '2023-11-24T06:30:33.252954+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:30:39.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-11-24T06:30:39.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.238247+0000', 'last_change': '2023-11-24T06:30:33.238247+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.238247+0000', 'last_undegraded': '2023-11-24T06:30:33.238247+0000', 'last_fullsized': '2023-11-24T06:30:33.238247+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.244235+0000', 'last_change': '2023-11-24T06:30:33.244235+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.244235+0000', 'last_undegraded': '2023-11-24T06:30:33.244235+0000', 'last_fullsized': '2023-11-24T06:30:33.244235+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.239947+0000', 'last_change': '2023-11-24T06:30:33.239947+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.239947+0000', 'last_undegraded': '2023-11-24T06:30:33.239947+0000', 'last_fullsized': '2023-11-24T06:30:33.239947+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.235872+0000', 'last_change': '2023-11-24T06:30:33.235872+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.235872+0000', 'last_undegraded': '2023-11-24T06:30:33.235872+0000', 'last_fullsized': '2023-11-24T06:30:33.235872+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.247170+0000', 'last_change': '2023-11-24T06:30:33.247170+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.247170+0000', 'last_undegraded': '2023-11-24T06:30:33.247170+0000', 'last_fullsized': '2023-11-24T06:30:33.247170+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-11-24T06:30:39.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.234680+0000', 'last_change': '2023-11-24T06:30:33.234680+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.234680+0000', 'last_undegraded': '2023-11-24T06:30:33.234680+0000', 'last_fullsized': '2023-11-24T06:30:33.234680+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.247698+0000', 'last_change': '2023-11-24T06:30:33.247698+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.247698+0000', 'last_undegraded': '2023-11-24T06:30:33.247698+0000', 'last_fullsized': '2023-11-24T06:30:33.247698+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.237309+0000', 'last_change': '2023-11-24T06:30:33.237309+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.237309+0000', 'last_undegraded': '2023-11-24T06:30:33.237309+0000', 'last_fullsized': '2023-11-24T06:30:33.237309+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.235467+0000', 'last_change': '2023-11-24T06:30:33.235467+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.235467+0000', 'last_undegraded': '2023-11-24T06:30:33.235467+0000', 'last_fullsized': '2023-11-24T06:30:33.235467+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.236341+0000', 'last_change': '2023-11-24T06:30:33.236341+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.236341+0000', 'last_undegraded': '2023-11-24T06:30:33.236341+0000', 'last_fullsized': '2023-11-24T06:30:33.236341+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.258019+0000', 'last_change': '2023-11-24T06:30:33.258019+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.258019+0000', 'last_undegraded': '2023-11-24T06:30:33.258019+0000', 'last_fullsized': '2023-11-24T06:30:33.258019+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:30:39.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-11-24T06:30:39.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.251109+0000', 'last_change': '2023-11-24T06:30:33.251109+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.251109+0000', 'last_undegraded': '2023-11-24T06:30:33.251109+0000', 'last_fullsized': '2023-11-24T06:30:33.251109+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:30:39.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-11-24T06:30:39.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.242204+0000', 'last_change': '2023-11-24T06:30:33.242204+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.242204+0000', 'last_undegraded': '2023-11-24T06:30:33.242204+0000', 'last_fullsized': '2023-11-24T06:30:33.242204+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:30:39.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-11-24T06:30:39.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.239064+0000', 'last_change': '2023-11-24T06:30:33.239064+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.239064+0000', 'last_undegraded': '2023-11-24T06:30:33.239064+0000', 'last_fullsized': '2023-11-24T06:30:33.239064+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:30:39.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-11-24T06:30:39.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.240220+0000', 'last_change': '2023-11-24T06:30:33.240220+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.240220+0000', 'last_undegraded': '2023-11-24T06:30:33.240220+0000', 'last_fullsized': '2023-11-24T06:30:33.240220+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.234550+0000', 'last_change': '2023-11-24T06:30:33.234550+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.234550+0000', 'last_undegraded': '2023-11-24T06:30:33.234550+0000', 'last_fullsized': '2023-11-24T06:30:33.234550+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.241508+0000', 'last_change': '2023-11-24T06:30:33.241508+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.241508+0000', 'last_undegraded': '2023-11-24T06:30:33.241508+0000', 'last_fullsized': '2023-11-24T06:30:33.241508+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.247042+0000', 'last_change': '2023-11-24T06:30:33.247042+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.247042+0000', 'last_undegraded': '2023-11-24T06:30:33.247042+0000', 'last_fullsized': '2023-11-24T06:30:33.247042+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.242692+0000', 'last_change': '2023-11-24T06:30:33.242692+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.242692+0000', 'last_undegraded': '2023-11-24T06:30:33.242692+0000', 'last_fullsized': '2023-11-24T06:30:33.242692+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-11-24T06:30:39.819 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.237560+0000', 'last_change': '2023-11-24T06:30:33.237560+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.237560+0000', 'last_undegraded': '2023-11-24T06:30:33.237560+0000', 'last_fullsized': '2023-11-24T06:30:33.237560+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.246556+0000', 'last_change': '2023-11-24T06:30:33.246556+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.246556+0000', 'last_undegraded': '2023-11-24T06:30:33.246556+0000', 'last_fullsized': '2023-11-24T06:30:33.246556+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.247176+0000', 'last_change': '2023-11-24T06:30:33.247176+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.247176+0000', 'last_undegraded': '2023-11-24T06:30:33.247176+0000', 'last_fullsized': '2023-11-24T06:30:33.247176+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.248346+0000', 'last_change': '2023-11-24T06:30:33.248346+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.248346+0000', 'last_undegraded': '2023-11-24T06:30:33.248346+0000', 'last_fullsized': '2023-11-24T06:30:33.248346+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.252179+0000', 'last_change': '2023-11-24T06:30:33.252179+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.252179+0000', 'last_undegraded': '2023-11-24T06:30:33.252179+0000', 'last_fullsized': '2023-11-24T06:30:33.252179+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-11-24T06:30:39.820 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.241747+0000', 'last_change': '2023-11-24T06:30:33.241747+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.241747+0000', 'last_undegraded': '2023-11-24T06:30:33.241747+0000', 'last_fullsized': '2023-11-24T06:30:33.241747+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.234860+0000', 'last_change': '2023-11-24T06:30:33.234860+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.234860+0000', 'last_undegraded': '2023-11-24T06:30:33.234860+0000', 'last_fullsized': '2023-11-24T06:30:33.234860+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.252752+0000', 'last_change': '2023-11-24T06:30:33.252752+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.252752+0000', 'last_undegraded': '2023-11-24T06:30:33.252752+0000', 'last_fullsized': '2023-11-24T06:30:33.252752+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.238829+0000', 'last_change': '2023-11-24T06:30:33.238829+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.238829+0000', 'last_undegraded': '2023-11-24T06:30:33.238829+0000', 'last_fullsized': '2023-11-24T06:30:33.238829+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.246365+0000', 'last_change': '2023-11-24T06:30:33.246365+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.246365+0000', 'last_undegraded': '2023-11-24T06:30:33.246365+0000', 'last_fullsized': '2023-11-24T06:30:33.246365+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-11-24T06:30:39.821 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.240042+0000', 'last_change': '2023-11-24T06:30:33.240042+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.240042+0000', 'last_undegraded': '2023-11-24T06:30:33.240042+0000', 'last_fullsized': '2023-11-24T06:30:33.240042+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.234536+0000', 'last_change': '2023-11-24T06:30:33.234536+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.234536+0000', 'last_undegraded': '2023-11-24T06:30:33.234536+0000', 'last_fullsized': '2023-11-24T06:30:33.234536+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.246986+0000', 'last_change': '2023-11-24T06:30:33.246986+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.246986+0000', 'last_undegraded': '2023-11-24T06:30:33.246986+0000', 'last_fullsized': '2023-11-24T06:30:33.246986+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.246380+0000', 'last_change': '2023-11-24T06:30:33.246380+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.246380+0000', 'last_undegraded': '2023-11-24T06:30:33.246380+0000', 'last_fullsized': '2023-11-24T06:30:33.246380+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.242375+0000', 'last_change': '2023-11-24T06:30:33.242375+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.242375+0000', 'last_undegraded': '2023-11-24T06:30:33.242375+0000', 'last_fullsized': '2023-11-24T06:30:33.242375+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-11-24T06:30:39.822 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.248625+0000', 'last_change': '2023-11-24T06:30:33.248625+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.248625+0000', 'last_undegraded': '2023-11-24T06:30:33.248625+0000', 'last_fullsized': '2023-11-24T06:30:33.248625+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.235171+0000', 'last_change': '2023-11-24T06:30:33.235171+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.235171+0000', 'last_undegraded': '2023-11-24T06:30:33.235171+0000', 'last_fullsized': '2023-11-24T06:30:33.235171+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.238368+0000', 'last_change': '2023-11-24T06:30:33.238368+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.238368+0000', 'last_undegraded': '2023-11-24T06:30:33.238368+0000', 'last_fullsized': '2023-11-24T06:30:33.238368+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.248362+0000', 'last_change': '2023-11-24T06:30:33.248362+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.248362+0000', 'last_undegraded': '2023-11-24T06:30:33.248362+0000', 'last_fullsized': '2023-11-24T06:30:33.248362+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.249240+0000', 'last_change': '2023-11-24T06:30:33.249240+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.249240+0000', 'last_undegraded': '2023-11-24T06:30:33.249240+0000', 'last_fullsized': '2023-11-24T06:30:33.249240+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-11-24T06:30:39.823 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.233300+0000', 'last_change': '2023-11-24T06:30:33.233300+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.233300+0000', 'last_undegraded': '2023-11-24T06:30:33.233300+0000', 'last_fullsized': '2023-11-24T06:30:33.233300+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-11-24T06:30:39.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.237823+0000', 'last_change': '2023-11-24T06:30:33.237823+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.237823+0000', 'last_undegraded': '2023-11-24T06:30:33.237823+0000', 'last_fullsized': '2023-11-24T06:30:33.237823+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T06:30:39.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-11-24T06:30:39.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.241198+0000', 'last_change': '2023-11-24T06:30:33.241198+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.241198+0000', 'last_undegraded': '2023-11-24T06:30:33.241198+0000', 'last_fullsized': '2023-11-24T06:30:33.241198+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-11-24T06:30:39.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.235215+0000', 'last_change': '2023-11-24T06:30:33.235215+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.235215+0000', 'last_undegraded': '2023-11-24T06:30:33.235215+0000', 'last_fullsized': '2023-11-24T06:30:33.235215+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:30:39.824 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-11-24T06:30:39.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.233222+0000', 'last_change': '2023-11-24T06:30:33.233222+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.233222+0000', 'last_undegraded': '2023-11-24T06:30:33.233222+0000', 'last_fullsized': '2023-11-24T06:30:33.233222+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-11-24T06:30:39.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.243611+0000', 'last_change': '2023-11-24T06:30:33.243611+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.243611+0000', 'last_undegraded': '2023-11-24T06:30:33.243611+0000', 'last_fullsized': '2023-11-24T06:30:33.243611+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:30:39.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-11-24T06:30:39.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.242279+0000', 'last_change': '2023-11-24T06:30:33.242279+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.242279+0000', 'last_undegraded': '2023-11-24T06:30:33.242279+0000', 'last_fullsized': '2023-11-24T06:30:33.242279+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:30:39.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-11-24T06:30:39.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.242662+0000', 'last_change': '2023-11-24T06:30:33.242662+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.242662+0000', 'last_undegraded': '2023-11-24T06:30:33.242662+0000', 'last_fullsized': '2023-11-24T06:30:33.242662+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:30:39.825 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-11-24T06:30:39.825 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.249084+0000', 'last_change': '2023-11-24T06:30:33.249084+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.249084+0000', 'last_undegraded': '2023-11-24T06:30:33.249084+0000', 'last_fullsized': '2023-11-24T06:30:33.249084+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.254177+0000', 'last_change': '2023-11-24T06:30:33.254177+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.254177+0000', 'last_undegraded': '2023-11-24T06:30:33.254177+0000', 'last_fullsized': '2023-11-24T06:30:33.254177+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.239053+0000', 'last_change': '2023-11-24T06:30:33.239053+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.239053+0000', 'last_undegraded': '2023-11-24T06:30:33.239053+0000', 'last_fullsized': '2023-11-24T06:30:33.239053+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.237564+0000', 'last_change': '2023-11-24T06:30:33.237564+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.237564+0000', 'last_undegraded': '2023-11-24T06:30:33.237564+0000', 'last_fullsized': '2023-11-24T06:30:33.237564+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.232644+0000', 'last_change': '2023-11-24T06:30:33.232644+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.232644+0000', 'last_undegraded': '2023-11-24T06:30:33.232644+0000', 'last_fullsized': '2023-11-24T06:30:33.232644+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-11-24T06:30:39.826 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.249296+0000', 'last_change': '2023-11-24T06:30:33.249296+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.249296+0000', 'last_undegraded': '2023-11-24T06:30:33.249296+0000', 'last_fullsized': '2023-11-24T06:30:33.249296+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:30:39.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-11-24T06:30:39.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.234698+0000', 'last_change': '2023-11-24T06:30:33.234698+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.234698+0000', 'last_undegraded': '2023-11-24T06:30:33.234698+0000', 'last_fullsized': '2023-11-24T06:30:33.234698+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T06:30:39.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-11-24T06:30:39.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.251567+0000', 'last_change': '2023-11-24T06:30:33.251567+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.251567+0000', 'last_undegraded': '2023-11-24T06:30:33.251567+0000', 'last_fullsized': '2023-11-24T06:30:33.251567+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T06:30:39.827 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-11-24T06:30:39.827 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2023-11-24T06:30:33.242319+0000', 'last_change': '2023-11-24T06:30:33.242319+0000', 'last_active': '2023-11-24T06:30:33.211156+0000', 'last_peered': '2023-11-24T06:30:33.211156+0000', 'last_clean': '2023-11-24T06:30:33.211156+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T06:30:33.242319+0000', 'last_undegraded': '2023-11-24T06:30:33.242319+0000', 'last_fullsized': '2023-11-24T06:30:33.242319+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'last_clean_scrub_stamp': '2023-11-24T06:30:33.211156+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T06:30:39.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T06:30:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.073+0000 7f234bfff640 1 Processor -- start 2023-11-24T06:30:40.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.073+0000 7f234bfff640 1 -- start start 2023-11-24T06:30:40.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.075+0000 7f234bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c0ce310 0x7f234c0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.075+0000 7f234bfff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234c0b1430 con 0x7f234c0d0640 2023-11-24T06:30:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.075+0000 7f234bfff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234c0d0c40 con 0x7f234c0cd820 2023-11-24T06:30:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.075+0000 7f234bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234c0d0f20 con 0x7f234c0ce310 2023-11-24T06:30:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.075+0000 7f234b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f234c0d0640 0x7f234c166f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44774/0 (socket says 172.21.15.16:44774) 2023-11-24T06:30:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.075+0000 7f234b7fe640 1 -- 172.21.15.16:0/1470985778 learned_addr learned my addr 172.21.15.16:0/1470985778 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:40.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f234a7fc640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c0ce310 0x7f234c0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305078390 0 0) 0x7f234c0b1430 con 0x7f234c0d0640 2023-11-24T06:30:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2334003520 con 0x7f234c0d0640 2023-11-24T06:30:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827467246 0 0) 0x7f234c0d0c40 con 0x7f234c0cd820 2023-11-24T06:30:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f234c0b1430 con 0x7f234c0cd820 2023-11-24T06:30:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232137350 0 0) 0x7f2334003520 con 0x7f234c0d0640 2023-11-24T06:30:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c0ce310 msgr2=0x7f234c0ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c0ce310 0x7f234c0ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 >> v1:172.21.15.173:6789/0 conn(0x7f234c0cd820 legacy=0x7f234c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c0d1200 con 0x7f234c0d0640 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f23440019f0 con 0x7f234c0d0640 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.076+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 >> v1:172.21.15.16:6789/0 conn(0x7f234c0d0640 legacy=0x7f234c166f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f2349ffb640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334003b30 0x7f2334003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f2349ffb640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c0ce310 0x7f23340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f2349ffb640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2334004d00 0x7f2334005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c0d0f20 con 0x7f2334003b30 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233400ba30 con 0x7f2334004d00 2023-11-24T06:30:40.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233400bba0 con 0x7f234c0ce310 2023-11-24T06:30:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f234affd640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334003b30 0x7f2334003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f234a7fc640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c0ce310 0x7f23340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f234b7fe640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2334004d00 0x7f2334005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f234affd640 1 -- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c0ce310 msgr2=0x7f23340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f234affd640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c0ce310 0x7f23340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f234affd640 1 -- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2334004d00 msgr2=0x7f2334005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f234affd640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2334004d00 0x7f2334005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.077+0000 7f234affd640 1 -- 172.21.15.16:0/1470985778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334003520 con 0x7f2334003b30 2023-11-24T06:30:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.079+0000 7f234affd640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334003b30 0x7f2334003f30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f233c0018d0 tx=0x7f23340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6962fc82214824 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.079+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0052d0 con 0x7f2334003b30 2023-11-24T06:30:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.079+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c005430 con 0x7f2334003b30 2023-11-24T06:30:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.079+0000 7f2349ffb640 1 -- 172.21.15.16:0/1470985778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c003810 con 0x7f2334003b30 2023-11-24T06:30:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.079+0000 7f234bfff640 1 -- 172.21.15.16:0/1470985778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334003b30 msgr2=0x7f2334003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.079+0000 7f234bfff640 1 --2- 172.21.15.16:0/1470985778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334003b30 0x7f2334003f30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f233c0018d0 tx=0x7f23340057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.079+0000 7f234b7fe640 1 -- 172.21.15.16:0/1470985778 reap_dead start 2023-11-24T06:30:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.079+0000 7f234bfff640 1 -- 172.21.15.16:0/1470985778 shutdown_connections 2023-11-24T06:30:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.079+0000 7f234bfff640 1 -- 172.21.15.16:0/1470985778 >> 172.21.15.16:0/1470985778 conn(0x7f234c099a60 msgr2=0x7f234c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.080+0000 7f234bfff640 1 -- 172.21.15.16:0/1470985778 shutdown_connections 2023-11-24T06:30:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.080+0000 7f234bfff640 1 -- 172.21.15.16:0/1470985778 wait complete. 2023-11-24T06:30:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.081+0000 7f234bfff640 1 Processor -- start 2023-11-24T06:30:40.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.082+0000 7f234bfff640 1 -- start start 2023-11-24T06:30:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.082+0000 7f234bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2334003b30 0x7f234c0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.082+0000 7f234bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234c0adca0 0x7f234c156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.082+0000 7f234bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c1570e0 0x7f234c146770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c157560 con 0x7f234c0adca0 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234bfff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c0aed40 con 0x7f2334003b30 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234c0af710 con 0x7f234c1570e0 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234c0adca0 0x7f234c156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234c0adca0 0x7f234c156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36488/0 (socket says 172.21.15.16:36488) 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234a7fc640 1 -- 172.21.15.16:0/2472885641 learned_addr learned my addr 172.21.15.16:0/2472885641 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234affd640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2334003b30 0x7f234c0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234b7fe640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c1570e0 0x7f234c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234a7fc640 1 -- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c1570e0 msgr2=0x7f234c146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234a7fc640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c1570e0 0x7f234c146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234a7fc640 1 -- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2334003b30 msgr2=0x7f234c0af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234a7fc640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2334003b30 0x7f234c0af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.083+0000 7f234a7fc640 1 -- 172.21.15.16:0/2472885641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c0ae130 con 0x7f234c0adca0 2023-11-24T06:30:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.084+0000 7f234a7fc640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234c0adca0 0x7f234c156ba0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2340011630 tx=0x7f2340011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.084+0000 7f2333fff640 1 -- 172.21.15.16:0/2472885641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000f920 con 0x7f234c0adca0 2023-11-24T06:30:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.084+0000 7f2333fff640 1 -- 172.21.15.16:0/2472885641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234000fa80 con 0x7f234c0adca0 2023-11-24T06:30:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.084+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c0ae410 con 0x7f234c0adca0 2023-11-24T06:30:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.084+0000 7f2333fff640 1 -- 172.21.15.16:0/2472885641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000fdf0 con 0x7f234c0adca0 2023-11-24T06:30:40.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.084+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c0ae9d0 con 0x7f234c0adca0 2023-11-24T06:30:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.086+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c0d0220 con 0x7f234c0adca0 2023-11-24T06:30:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.086+0000 7f2333fff640 1 -- 172.21.15.16:0/2472885641 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f2340015460 con 0x7f234c0adca0 2023-11-24T06:30:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.088+0000 7f2333fff640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f233806b230 0x7f233806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.088+0000 7f2333fff640 1 -- 172.21.15.16:0/2472885641 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f234008a690 con 0x7f234c0adca0 2023-11-24T06:30:40.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.088+0000 7f234affd640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f233806b230 0x7f233806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.090+0000 7f234affd640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f233806b230 0x7f233806d6f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f233c034000 tx=0x7f233c031000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:40.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.092+0000 7f2333fff640 1 -- 172.21.15.16:0/2472885641 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2340055160 con 0x7f234c0adca0 2023-11-24T06:30:40.210 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-11-24T06:30:40.205+0000 7f590b1bd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:30:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.367+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f234c067450 con 0x7f234c0adca0 2023-11-24T06:30:40.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.369+0000 7f2333fff640 1 -- 172.21.15.16:0/2472885641 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1431 (secure 0 0 0) 0x7f2340058780 con 0x7f234c0adca0 2023-11-24T06:30:40.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:40.373 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":243,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":71,"num_osds":8,"num_up_osds":8,"osd_up_since":1700807208,"num_in_osds":8,"osd_in_since":1700807200,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":259399680,"bytes_avail":772834713600,"bytes_total":773094113280,"write_bytes_sec":2062,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":44,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6693}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-11-24T06:30:20.173234+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-24T06:30:40.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.371+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f233806b230 msgr2=0x7f233806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.371+0000 7f234bfff640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f233806b230 0x7f233806d6f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f233c034000 tx=0x7f233c031000 comp rx=0 tx=0).stop 2023-11-24T06:30:40.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234c0adca0 msgr2=0x7f234c156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234c0adca0 0x7f234c156ba0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2340011630 tx=0x7f2340011af0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 shutdown_connections 2023-11-24T06:30:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f233806b230 0x7f233806d6f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f234c1570e0 0x7f234c146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234c0adca0 0x7f234c156ba0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 --2- 172.21.15.16:0/2472885641 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2334003b30 0x7f234c0af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 >> 172.21.15.16:0/2472885641 conn(0x7f234c099a60 msgr2=0x7f234c0b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 shutdown_connections 2023-11-24T06:30:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.372+0000 7f234bfff640 1 -- 172.21.15.16:0/2472885641 wait complete. 2023-11-24T06:30:40.395 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T06:30:40.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:30:40.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.622+0000 7fbc7241a640 1 Processor -- start 2023-11-24T06:30:40.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.622+0000 7fbc7241a640 1 -- start start 2023-11-24T06:30:40.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.624+0000 7fbc7241a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc6c143c40 0x7fbc6c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.624+0000 7fbc7241a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbc6c0717d0 0x7fbc6c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.624+0000 7fbc7241a640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6c070e50 con 0x7fbc6c0cca40 2023-11-24T06:30:40.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.624+0000 7fbc7241a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6c0723c0 con 0x7fbc6c143c40 2023-11-24T06:30:40.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.624+0000 7fbc7241a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6c0cce40 con 0x7fbc6c0717d0 2023-11-24T06:30:40.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.624+0000 7fbc71418640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc6c143c40 0x7fbc6c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.624+0000 7fbc71418640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc6c143c40 0x7fbc6c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36496/0 (socket says 172.21.15.16:36496) 2023-11-24T06:30:40.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.624+0000 7fbc71418640 1 -- 172.21.15.16:0/3404794367 learned_addr learned my addr 172.21.15.16:0/3404794367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:40.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.625+0000 7fbc71418640 1 -- 172.21.15.16:0/3404794367 >> v2:172.21.15.173:3300/0 conn(0x7fbc6c0717d0 msgr2=0x7fbc6c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:40.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.625+0000 7fbc71418640 1 --2- 172.21.15.16:0/3404794367 >> v2:172.21.15.173:3300/0 conn(0x7fbc6c0717d0 0x7fbc6c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.625+0000 7fbc71418640 1 -- 172.21.15.16:0/3404794367 >> v1:172.21.15.173:6789/0 conn(0x7fbc6c0cca40 legacy=0x7fbc6c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:40.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.625+0000 7fbc71418640 1 -- 172.21.15.16:0/3404794367 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c06e170 con 0x7fbc6c143c40 2023-11-24T06:30:40.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.626+0000 7fbc71418640 1 --2- 172.21.15.16:0/3404794367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c143c40 0x7fbc6c071290 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0b44a0 tx=0x7fbc6802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ec439f66e8808c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:40.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.626+0000 7fbc63fff640 1 -- 172.21.15.16:0/3404794367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc680317e0 con 0x7fbc6c143c40 2023-11-24T06:30:40.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.626+0000 7fbc63fff640 1 -- 172.21.15.16:0/3404794367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68031940 con 0x7fbc6c143c40 2023-11-24T06:30:40.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.626+0000 7fbc63fff640 1 -- 172.21.15.16:0/3404794367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68031cd0 con 0x7fbc6c143c40 2023-11-24T06:30:40.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.627+0000 7fbc7241a640 1 -- 172.21.15.16:0/3404794367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c143c40 msgr2=0x7fbc6c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.627+0000 7fbc7241a640 1 --2- 172.21.15.16:0/3404794367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c143c40 0x7fbc6c071290 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0b44a0 tx=0x7fbc6802b880 comp rx=0 tx=0).stop 2023-11-24T06:30:40.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.627+0000 7fbc7241a640 1 -- 172.21.15.16:0/3404794367 shutdown_connections 2023-11-24T06:30:40.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.627+0000 7fbc7241a640 1 --2- 172.21.15.16:0/3404794367 >> v2:172.21.15.173:3300/0 conn(0x7fbc6c0717d0 0x7fbc6c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.627+0000 7fbc7241a640 1 --2- 172.21.15.16:0/3404794367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c143c40 0x7fbc6c071290 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.627+0000 7fbc7241a640 1 -- 172.21.15.16:0/3404794367 >> 172.21.15.16:0/3404794367 conn(0x7fbc6c06a800 msgr2=0x7fbc6c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:40.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.627+0000 7fbc7241a640 1 -- 172.21.15.16:0/3404794367 shutdown_connections 2023-11-24T06:30:40.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.627+0000 7fbc7241a640 1 -- 172.21.15.16:0/3404794367 wait complete. 2023-11-24T06:30:40.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.629+0000 7fbc7241a640 1 Processor -- start 2023-11-24T06:30:40.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.629+0000 7fbc7241a640 1 -- start start 2023-11-24T06:30:40.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.630+0000 7fbc7241a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c0717d0 0x7fbc6c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.630+0000 7fbc7241a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c0cca40 0x7fbc6c0ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.630+0000 7fbc7241a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc6c143c40 0x7fbc6c0adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.630+0000 7fbc7241a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c1f7b10 con 0x7fbc6c0717d0 2023-11-24T06:30:40.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.630+0000 7fbc7241a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c130460 con 0x7fbc6c0cca40 2023-11-24T06:30:40.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.630+0000 7fbc7241a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c130740 con 0x7fbc6c143c40 2023-11-24T06:30:40.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc71418640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c0717d0 0x7fbc6c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc71418640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c0717d0 0x7fbc6c15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36512/0 (socket says 172.21.15.16:36512) 2023-11-24T06:30:40.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc71418640 1 -- 172.21.15.16:0/3860254212 learned_addr learned my addr 172.21.15.16:0/3860254212 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:40.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc71418640 1 -- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc6c143c40 msgr2=0x7fbc6c0adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc71418640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc6c143c40 0x7fbc6c0adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc71418640 1 -- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c0cca40 msgr2=0x7fbc6c0ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc70c17640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c0cca40 0x7fbc6c0ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc71418640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c0cca40 0x7fbc6c0ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc71418640 1 -- 172.21.15.16:0/3860254212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc680094d0 con 0x7fbc6c0717d0 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.631+0000 7fbc70c17640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c0cca40 0x7fbc6c0ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.632+0000 7fbc71418640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c0717d0 0x7fbc6c15a5c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fbc680090b0 tx=0x7fbc680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.632+0000 7fbc61ffb640 1 -- 172.21.15.16:0/3860254212 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68032980 con 0x7fbc6c0717d0 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.632+0000 7fbc61ffb640 1 -- 172.21.15.16:0/3860254212 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68032b00 con 0x7fbc6c0717d0 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.632+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6c130a20 con 0x7fbc6c0717d0 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.632+0000 7fbc61ffb640 1 -- 172.21.15.16:0/3860254212 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc680453f0 con 0x7fbc6c0717d0 2023-11-24T06:30:40.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.632+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c1f6de0 con 0x7fbc6c0717d0 2023-11-24T06:30:40.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.633+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6c130c30 con 0x7fbc6c0717d0 2023-11-24T06:30:40.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.634+0000 7fbc61ffb640 1 -- 172.21.15.16:0/3860254212 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fbc6804d3f0 con 0x7fbc6c0717d0 2023-11-24T06:30:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.636+0000 7fbc61ffb640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbc5006b230 0x7fbc5006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.636+0000 7fbc61ffb640 1 -- 172.21.15.16:0/3860254212 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc680abc60 con 0x7fbc6c0717d0 2023-11-24T06:30:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.636+0000 7fbc70c17640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbc5006b230 0x7fbc5006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:40.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.637+0000 7fbc70c17640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbc5006b230 0x7fbc5006d6f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0cb620 tx=0x7fbc5c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.640+0000 7fbc61ffb640 1 -- 172.21.15.16:0/3860254212 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc68076760 con 0x7fbc6c0717d0 2023-11-24T06:30:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.833+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc6c1302e0 con 0x7fbc5006b230 2023-11-24T06:30:40.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.855+0000 7fbc61ffb640 1 -- 172.21.15.16:0/3860254212 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286431 (secure 0 0 0) 0x7fbc5c00ae40 con 0x7fbc5006b230 2023-11-24T06:30:40.863 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:30:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbc5006b230 msgr2=0x7fbc5006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbc5006b230 0x7fbc5006d6f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0cb620 tx=0x7fbc5c008040 comp rx=0 tx=0).stop 2023-11-24T06:30:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c0717d0 msgr2=0x7fbc6c15a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c0717d0 0x7fbc6c15a5c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fbc680090b0 tx=0x7fbc680046c0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 shutdown_connections 2023-11-24T06:30:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbc5006b230 0x7fbc5006d6f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc6c143c40 0x7fbc6c0adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6c0cca40 0x7fbc6c0ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 --2- 172.21.15.16:0/3860254212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc6c0717d0 0x7fbc6c15a5c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 >> 172.21.15.16:0/3860254212 conn(0x7fbc6c06a800 msgr2=0x7fbc6c1411d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 shutdown_connections 2023-11-24T06:30:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:40.861+0000 7fbc7241a640 1 -- 172.21.15.16:0/3860254212 wait complete. 2023-11-24T06:30:40.883 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2023-11-24T06:30:40.179015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253320,"kb_used_data":7976,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721400,"statfs":{"total":773094113280,"available":772834713600,"internally_reserved":0,"allocated":8167424,"data_stored":4106492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.951534"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259642+0000","last_change":"2023-11-24T06:30:34.252293+0000","last_active":"2023-11-24T06:30:35.259642+0000","last_peered":"2023-11-24T06:30:35.259642+0000","last_clean":"2023-11-24T06:30:35.259642+0000","last_became_active":"2023-11-24T06:30:34.251526+0000","last_became_peered":"2023-11-24T06:30:34.251526+0000","last_unstale":"2023-11-24T06:30:35.259642+0000","last_undegraded":"2023-11-24T06:30:35.259642+0000","last_fullsized":"2023-11-24T06:30:35.259642+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:40.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.348535+0000","last_change":"2023-11-24T06:30:34.283473+0000","last_active":"2023-11-24T06:30:35.348535+0000","last_peered":"2023-11-24T06:30:35.348535+0000","last_clean":"2023-11-24T06:30:35.348535+0000","last_became_active":"2023-11-24T06:30:34.283122+0000","last_became_peered":"2023-11-24T06:30:34.283122+0000","last_unstale":"2023-11-24T06:30:35.348535+0000","last_undegraded":"2023-11-24T06:30:35.348535+0000","last_fullsized":"2023-11-24T06:30:35.348535+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:54.612687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259923+0000","last_change":"2023-11-24T06:30:34.259042+0000","last_active":"2023-11-24T06:30:35.259923+0000","last_peered":"2023-11-24T06:30:35.259923+0000","last_clean":"2023-11-24T06:30:35.259923+0000","last_became_active":"2023-11-24T06:30:34.258435+0000","last_became_peered":"2023-11-24T06:30:34.258435+0000","last_unstale":"2023-11-24T06:30:35.259923+0000","last_undegraded":"2023-11-24T06:30:35.259923+0000","last_fullsized":"2023-11-24T06:30:35.259923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:44.108280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256215+0000","last_change":"2023-11-24T06:30:34.280435+0000","last_active":"2023-11-24T06:30:35.256215+0000","last_peered":"2023-11-24T06:30:35.256215+0000","last_clean":"2023-11-24T06:30:35.256215+0000","last_became_active":"2023-11-24T06:30:34.280097+0000","last_became_peered":"2023-11-24T06:30:34.280097+0000","last_unstale":"2023-11-24T06:30:35.256215+0000","last_undegraded":"2023-11-24T06:30:35.256215+0000","last_fullsized":"2023-11-24T06:30:35.256215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:53.546580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343653+0000","last_change":"2023-11-24T06:30:34.261050+0000","last_active":"2023-11-24T06:30:35.343653+0000","last_peered":"2023-11-24T06:30:35.343653+0000","last_clean":"2023-11-24T06:30:35.343653+0000","last_became_active":"2023-11-24T06:30:34.260798+0000","last_became_peered":"2023-11-24T06:30:34.260798+0000","last_unstale":"2023-11-24T06:30:35.343653+0000","last_undegraded":"2023-11-24T06:30:35.343653+0000","last_fullsized":"2023-11-24T06:30:35.343653+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:29.283116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347309+0000","last_change":"2023-11-24T06:30:34.283412+0000","last_active":"2023-11-24T06:30:35.347309+0000","last_peered":"2023-11-24T06:30:35.347309+0000","last_clean":"2023-11-24T06:30:35.347309+0000","last_became_active":"2023-11-24T06:30:34.283062+0000","last_became_peered":"2023-11-24T06:30:34.283062+0000","last_unstale":"2023-11-24T06:30:35.347309+0000","last_undegraded":"2023-11-24T06:30:35.347309+0000","last_fullsized":"2023-11-24T06:30:35.347309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:46.928025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343469+0000","last_change":"2023-11-24T06:30:34.262673+0000","last_active":"2023-11-24T06:30:35.343469+0000","last_peered":"2023-11-24T06:30:35.343469+0000","last_clean":"2023-11-24T06:30:35.343469+0000","last_became_active":"2023-11-24T06:30:34.262372+0000","last_became_peered":"2023-11-24T06:30:34.262372+0000","last_unstale":"2023-11-24T06:30:35.343469+0000","last_undegraded":"2023-11-24T06:30:35.343469+0000","last_fullsized":"2023-11-24T06:30:35.343469+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:58.107784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346741+0000","last_change":"2023-11-24T06:30:34.271256+0000","last_active":"2023-11-24T06:30:35.346741+0000","last_peered":"2023-11-24T06:30:35.346741+0000","last_clean":"2023-11-24T06:30:35.346741+0000","last_became_active":"2023-11-24T06:30:34.270954+0000","last_became_peered":"2023-11-24T06:30:34.270954+0000","last_unstale":"2023-11-24T06:30:35.346741+0000","last_undegraded":"2023-11-24T06:30:35.346741+0000","last_fullsized":"2023-11-24T06:30:35.346741+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:06.752081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344712+0000","last_change":"2023-11-24T06:30:34.255460+0000","last_active":"2023-11-24T06:30:35.344712+0000","last_peered":"2023-11-24T06:30:35.344712+0000","last_clean":"2023-11-24T06:30:35.344712+0000","last_became_active":"2023-11-24T06:30:34.254633+0000","last_became_peered":"2023-11-24T06:30:34.254633+0000","last_unstale":"2023-11-24T06:30:35.344712+0000","last_undegraded":"2023-11-24T06:30:35.344712+0000","last_fullsized":"2023-11-24T06:30:35.344712+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:49:21.210820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345026+0000","last_change":"2023-11-24T06:30:34.275595+0000","last_active":"2023-11-24T06:30:35.345026+0000","last_peered":"2023-11-24T06:30:35.345026+0000","last_clean":"2023-11-24T06:30:35.345026+0000","last_became_active":"2023-11-24T06:30:34.275240+0000","last_became_peered":"2023-11-24T06:30:34.275240+0000","last_unstale":"2023-11-24T06:30:35.345026+0000","last_undegraded":"2023-11-24T06:30:35.345026+0000","last_fullsized":"2023-11-24T06:30:35.345026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:06.613023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259103+0000","last_change":"2023-11-24T06:30:34.248763+0000","last_active":"2023-11-24T06:30:35.259103+0000","last_peered":"2023-11-24T06:30:35.259103+0000","last_clean":"2023-11-24T06:30:35.259103+0000","last_became_active":"2023-11-24T06:30:34.247642+0000","last_became_peered":"2023-11-24T06:30:34.247642+0000","last_unstale":"2023-11-24T06:30:35.259103+0000","last_undegraded":"2023-11-24T06:30:35.259103+0000","last_fullsized":"2023-11-24T06:30:35.259103+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:43.282344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256396+0000","last_change":"2023-11-24T06:30:34.249620+0000","last_active":"2023-11-24T06:30:35.256396+0000","last_peered":"2023-11-24T06:30:35.256396+0000","last_clean":"2023-11-24T06:30:35.256396+0000","last_became_active":"2023-11-24T06:30:34.248964+0000","last_became_peered":"2023-11-24T06:30:34.248964+0000","last_unstale":"2023-11-24T06:30:35.256396+0000","last_undegraded":"2023-11-24T06:30:35.256396+0000","last_fullsized":"2023-11-24T06:30:35.256396+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:45.205158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.342771+0000","last_change":"2023-11-24T06:30:34.274593+0000","last_active":"2023-11-24T06:30:35.342771+0000","last_peered":"2023-11-24T06:30:35.342771+0000","last_clean":"2023-11-24T06:30:35.342771+0000","last_became_active":"2023-11-24T06:30:34.274328+0000","last_became_peered":"2023-11-24T06:30:34.274328+0000","last_unstale":"2023-11-24T06:30:35.342771+0000","last_undegraded":"2023-11-24T06:30:35.342771+0000","last_fullsized":"2023-11-24T06:30:35.342771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:08.454372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344254+0000","last_change":"2023-11-24T06:30:34.264008+0000","last_active":"2023-11-24T06:30:35.344254+0000","last_peered":"2023-11-24T06:30:35.344254+0000","last_clean":"2023-11-24T06:30:35.344254+0000","last_became_active":"2023-11-24T06:30:34.263740+0000","last_became_peered":"2023-11-24T06:30:34.263740+0000","last_unstale":"2023-11-24T06:30:35.344254+0000","last_undegraded":"2023-11-24T06:30:35.344254+0000","last_fullsized":"2023-11-24T06:30:35.344254+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:39.757730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.342845+0000","last_change":"2023-11-24T06:30:34.260735+0000","last_active":"2023-11-24T06:30:35.342845+0000","last_peered":"2023-11-24T06:30:35.342845+0000","last_clean":"2023-11-24T06:30:35.342845+0000","last_became_active":"2023-11-24T06:30:34.260434+0000","last_became_peered":"2023-11-24T06:30:34.260434+0000","last_unstale":"2023-11-24T06:30:35.342845+0000","last_undegraded":"2023-11-24T06:30:35.342845+0000","last_fullsized":"2023-11-24T06:30:35.342845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:43.282344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344395+0000","last_change":"2023-11-24T06:30:34.280175+0000","last_active":"2023-11-24T06:30:35.344395+0000","last_peered":"2023-11-24T06:30:35.344395+0000","last_clean":"2023-11-24T06:30:35.344395+0000","last_became_active":"2023-11-24T06:30:34.279901+0000","last_became_peered":"2023-11-24T06:30:34.279901+0000","last_unstale":"2023-11-24T06:30:35.344395+0000","last_undegraded":"2023-11-24T06:30:35.344395+0000","last_fullsized":"2023-11-24T06:30:35.344395+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:34.670131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344884+0000","last_change":"2023-11-24T06:30:34.255632+0000","last_active":"2023-11-24T06:30:35.344884+0000","last_peered":"2023-11-24T06:30:35.344884+0000","last_clean":"2023-11-24T06:30:35.344884+0000","last_became_active":"2023-11-24T06:30:34.255192+0000","last_became_peered":"2023-11-24T06:30:34.255192+0000","last_unstale":"2023-11-24T06:30:35.344884+0000","last_undegraded":"2023-11-24T06:30:35.344884+0000","last_fullsized":"2023-11-24T06:30:35.344884+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:32.962593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256834+0000","last_change":"2023-11-24T06:30:34.276012+0000","last_active":"2023-11-24T06:30:35.256834+0000","last_peered":"2023-11-24T06:30:35.256834+0000","last_clean":"2023-11-24T06:30:35.256834+0000","last_became_active":"2023-11-24T06:30:34.275542+0000","last_became_peered":"2023-11-24T06:30:34.275542+0000","last_unstale":"2023-11-24T06:30:35.256834+0000","last_undegraded":"2023-11-24T06:30:35.256834+0000","last_fullsized":"2023-11-24T06:30:35.256834+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:34:06.634469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.348044+0000","last_change":"2023-11-24T06:30:34.276206+0000","last_active":"2023-11-24T06:30:35.348044+0000","last_peered":"2023-11-24T06:30:35.348044+0000","last_clean":"2023-11-24T06:30:35.348044+0000","last_became_active":"2023-11-24T06:30:34.275859+0000","last_became_peered":"2023-11-24T06:30:34.275859+0000","last_unstale":"2023-11-24T06:30:35.348044+0000","last_undegraded":"2023-11-24T06:30:35.348044+0000","last_fullsized":"2023-11-24T06:30:35.348044+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:29.402712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344803+0000","last_change":"2023-11-24T06:30:34.265681+0000","last_active":"2023-11-24T06:30:35.344803+0000","last_peered":"2023-11-24T06:30:35.344803+0000","last_clean":"2023-11-24T06:30:35.344803+0000","last_became_active":"2023-11-24T06:30:34.265355+0000","last_became_peered":"2023-11-24T06:30:34.265355+0000","last_unstale":"2023-11-24T06:30:35.344803+0000","last_undegraded":"2023-11-24T06:30:35.344803+0000","last_fullsized":"2023-11-24T06:30:35.344803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:40.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344589+0000","last_change":"2023-11-24T06:30:34.276232+0000","last_active":"2023-11-24T06:30:35.344589+0000","last_peered":"2023-11-24T06:30:35.344589+0000","last_clean":"2023-11-24T06:30:35.344589+0000","last_became_active":"2023-11-24T06:30:34.274833+0000","last_became_peered":"2023-11-24T06:30:34.274833+0000","last_unstale":"2023-11-24T06:30:35.344589+0000","last_undegraded":"2023-11-24T06:30:35.344589+0000","last_fullsized":"2023-11-24T06:30:35.344589+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:11.493185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345343+0000","last_change":"2023-11-24T06:30:34.276782+0000","last_active":"2023-11-24T06:30:35.345343+0000","last_peered":"2023-11-24T06:30:35.345343+0000","last_clean":"2023-11-24T06:30:35.345343+0000","last_became_active":"2023-11-24T06:30:34.274573+0000","last_became_peered":"2023-11-24T06:30:34.274573+0000","last_unstale":"2023-11-24T06:30:35.345343+0000","last_undegraded":"2023-11-24T06:30:35.345343+0000","last_fullsized":"2023-11-24T06:30:35.345343+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:29.402712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.258987+0000","last_change":"2023-11-24T06:30:34.273439+0000","last_active":"2023-11-24T06:30:35.258987+0000","last_peered":"2023-11-24T06:30:35.258987+0000","last_clean":"2023-11-24T06:30:35.258987+0000","last_became_active":"2023-11-24T06:30:34.269631+0000","last_became_peered":"2023-11-24T06:30:34.269631+0000","last_unstale":"2023-11-24T06:30:35.258987+0000","last_undegraded":"2023-11-24T06:30:35.258987+0000","last_fullsized":"2023-11-24T06:30:35.258987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:08.454372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.258716+0000","last_change":"2023-11-24T06:30:34.280283+0000","last_active":"2023-11-24T06:30:35.258716+0000","last_peered":"2023-11-24T06:30:35.258716+0000","last_clean":"2023-11-24T06:30:35.258716+0000","last_became_active":"2023-11-24T06:30:34.279932+0000","last_became_peered":"2023-11-24T06:30:34.279932+0000","last_unstale":"2023-11-24T06:30:35.258716+0000","last_undegraded":"2023-11-24T06:30:35.258716+0000","last_fullsized":"2023-11-24T06:30:35.258716+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:02:53.020511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344811+0000","last_change":"2023-11-24T06:30:34.270241+0000","last_active":"2023-11-24T06:30:35.344811+0000","last_peered":"2023-11-24T06:30:35.344811+0000","last_clean":"2023-11-24T06:30:35.344811+0000","last_became_active":"2023-11-24T06:30:34.269570+0000","last_became_peered":"2023-11-24T06:30:34.269570+0000","last_unstale":"2023-11-24T06:30:35.344811+0000","last_undegraded":"2023-11-24T06:30:35.344811+0000","last_fullsized":"2023-11-24T06:30:35.344811+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:46.928025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346318+0000","last_change":"2023-11-24T06:30:34.276808+0000","last_active":"2023-11-24T06:30:35.346318+0000","last_peered":"2023-11-24T06:30:35.346318+0000","last_clean":"2023-11-24T06:30:35.346318+0000","last_became_active":"2023-11-24T06:30:34.275753+0000","last_became_peered":"2023-11-24T06:30:34.275753+0000","last_unstale":"2023-11-24T06:30:35.346318+0000","last_undegraded":"2023-11-24T06:30:35.346318+0000","last_fullsized":"2023-11-24T06:30:35.346318+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:51.866541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260023+0000","last_change":"2023-11-24T06:30:34.252551+0000","last_active":"2023-11-24T06:30:35.260023+0000","last_peered":"2023-11-24T06:30:35.260023+0000","last_clean":"2023-11-24T06:30:35.260023+0000","last_became_active":"2023-11-24T06:30:34.252113+0000","last_became_peered":"2023-11-24T06:30:34.252113+0000","last_unstale":"2023-11-24T06:30:35.260023+0000","last_undegraded":"2023-11-24T06:30:35.260023+0000","last_fullsized":"2023-11-24T06:30:35.260023+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:49:13.548874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345907+0000","last_change":"2023-11-24T06:30:34.274691+0000","last_active":"2023-11-24T06:30:35.345907+0000","last_peered":"2023-11-24T06:30:35.345907+0000","last_clean":"2023-11-24T06:30:35.345907+0000","last_became_active":"2023-11-24T06:30:34.274390+0000","last_became_peered":"2023-11-24T06:30:34.274390+0000","last_unstale":"2023-11-24T06:30:35.345907+0000","last_undegraded":"2023-11-24T06:30:35.345907+0000","last_fullsized":"2023-11-24T06:30:35.345907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:54.612687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260067+0000","last_change":"2023-11-24T06:30:34.261054+0000","last_active":"2023-11-24T06:30:35.260067+0000","last_peered":"2023-11-24T06:30:35.260067+0000","last_clean":"2023-11-24T06:30:35.260067+0000","last_became_active":"2023-11-24T06:30:34.260852+0000","last_became_peered":"2023-11-24T06:30:34.260852+0000","last_unstale":"2023-11-24T06:30:35.260067+0000","last_undegraded":"2023-11-24T06:30:35.260067+0000","last_fullsized":"2023-11-24T06:30:35.260067+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:26.614414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343765+0000","last_change":"2023-11-24T06:30:34.273145+0000","last_active":"2023-11-24T06:30:35.343765+0000","last_peered":"2023-11-24T06:30:35.343765+0000","last_clean":"2023-11-24T06:30:35.343765+0000","last_became_active":"2023-11-24T06:30:34.272820+0000","last_became_peered":"2023-11-24T06:30:34.272820+0000","last_unstale":"2023-11-24T06:30:35.343765+0000","last_undegraded":"2023-11-24T06:30:35.343765+0000","last_fullsized":"2023-11-24T06:30:35.343765+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:15:29.661247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343818+0000","last_change":"2023-11-24T06:30:34.261468+0000","last_active":"2023-11-24T06:30:35.343818+0000","last_peered":"2023-11-24T06:30:35.343818+0000","last_clean":"2023-11-24T06:30:35.343818+0000","last_became_active":"2023-11-24T06:30:34.261196+0000","last_became_peered":"2023-11-24T06:30:34.261196+0000","last_unstale":"2023-11-24T06:30:35.343818+0000","last_undegraded":"2023-11-24T06:30:35.343818+0000","last_fullsized":"2023-11-24T06:30:35.343818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:54:26.525502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347150+0000","last_change":"2023-11-24T06:30:34.274425+0000","last_active":"2023-11-24T06:30:35.347150+0000","last_peered":"2023-11-24T06:30:35.347150+0000","last_clean":"2023-11-24T06:30:35.347150+0000","last_became_active":"2023-11-24T06:30:34.271742+0000","last_became_peered":"2023-11-24T06:30:34.271742+0000","last_unstale":"2023-11-24T06:30:35.347150+0000","last_undegraded":"2023-11-24T06:30:35.347150+0000","last_fullsized":"2023-11-24T06:30:35.347150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:11.493185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259428+0000","last_change":"2023-11-24T06:30:34.276702+0000","last_active":"2023-11-24T06:30:35.259428+0000","last_peered":"2023-11-24T06:30:35.259428+0000","last_clean":"2023-11-24T06:30:35.259428+0000","last_became_active":"2023-11-24T06:30:34.276392+0000","last_became_peered":"2023-11-24T06:30:34.276392+0000","last_unstale":"2023-11-24T06:30:35.259428+0000","last_undegraded":"2023-11-24T06:30:35.259428+0000","last_fullsized":"2023-11-24T06:30:35.259428+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:31.302540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.276566+0000","last_change":"2023-11-24T06:30:33.258485+0000","last_active":"2023-11-24T06:30:34.276566+0000","last_peered":"2023-11-24T06:30:34.276566+0000","last_clean":"2023-11-24T06:30:34.276566+0000","last_became_active":"2023-11-24T06:30:33.258091+0000","last_became_peered":"2023-11-24T06:30:33.258091+0000","last_unstale":"2023-11-24T06:30:34.276566+0000","last_undegraded":"2023-11-24T06:30:34.276566+0000","last_fullsized":"2023-11-24T06:30:34.276566+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:34.021503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345043+0000","last_change":"2023-11-24T06:30:34.254341+0000","last_active":"2023-11-24T06:30:35.345043+0000","last_peered":"2023-11-24T06:30:35.345043+0000","last_clean":"2023-11-24T06:30:35.345043+0000","last_became_active":"2023-11-24T06:30:34.253349+0000","last_became_peered":"2023-11-24T06:30:34.253349+0000","last_unstale":"2023-11-24T06:30:35.345043+0000","last_undegraded":"2023-11-24T06:30:35.345043+0000","last_fullsized":"2023-11-24T06:30:35.345043+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:45.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.264537+0000","last_change":"2023-11-24T06:30:33.253586+0000","last_active":"2023-11-24T06:30:34.264537+0000","last_peered":"2023-11-24T06:30:34.264537+0000","last_clean":"2023-11-24T06:30:34.264537+0000","last_became_active":"2023-11-24T06:30:33.252371+0000","last_became_peered":"2023-11-24T06:30:33.252371+0000","last_unstale":"2023-11-24T06:30:34.264537+0000","last_undegraded":"2023-11-24T06:30:34.264537+0000","last_fullsized":"2023-11-24T06:30:34.264537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:11:40.811402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347857+0000","last_change":"2023-11-24T06:30:34.279290+0000","last_active":"2023-11-24T06:30:35.347857+0000","last_peered":"2023-11-24T06:30:35.347857+0000","last_clean":"2023-11-24T06:30:35.347857+0000","last_became_active":"2023-11-24T06:30:34.278957+0000","last_became_peered":"2023-11-24T06:30:34.278957+0000","last_unstale":"2023-11-24T06:30:35.347857+0000","last_undegraded":"2023-11-24T06:30:35.347857+0000","last_fullsized":"2023-11-24T06:30:35.347857+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:06.613023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.281834+0000","last_change":"2023-11-24T06:30:33.266741+0000","last_active":"2023-11-24T06:30:34.281834+0000","last_peered":"2023-11-24T06:30:34.281834+0000","last_clean":"2023-11-24T06:30:34.281834+0000","last_became_active":"2023-11-24T06:30:33.266055+0000","last_became_peered":"2023-11-24T06:30:33.266055+0000","last_unstale":"2023-11-24T06:30:34.281834+0000","last_undegraded":"2023-11-24T06:30:34.281834+0000","last_fullsized":"2023-11-24T06:30:34.281834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:20.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259596+0000","last_change":"2023-11-24T06:30:34.259166+0000","last_active":"2023-11-24T06:30:35.259596+0000","last_peered":"2023-11-24T06:30:35.259596+0000","last_clean":"2023-11-24T06:30:35.259596+0000","last_became_active":"2023-11-24T06:30:34.258558+0000","last_became_peered":"2023-11-24T06:30:34.258558+0000","last_unstale":"2023-11-24T06:30:35.259596+0000","last_undegraded":"2023-11-24T06:30:35.259596+0000","last_fullsized":"2023-11-24T06:30:35.259596+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:22.171241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.278031+0000","last_change":"2023-11-24T06:30:33.260247+0000","last_active":"2023-11-24T06:30:34.278031+0000","last_peered":"2023-11-24T06:30:34.278031+0000","last_clean":"2023-11-24T06:30:34.278031+0000","last_became_active":"2023-11-24T06:30:33.259986+0000","last_became_peered":"2023-11-24T06:30:33.259986+0000","last_unstale":"2023-11-24T06:30:34.278031+0000","last_undegraded":"2023-11-24T06:30:34.278031+0000","last_fullsized":"2023-11-24T06:30:34.278031+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:48.718422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344557+0000","last_change":"2023-11-24T06:30:34.260492+0000","last_active":"2023-11-24T06:30:35.344557+0000","last_peered":"2023-11-24T06:30:35.344557+0000","last_clean":"2023-11-24T06:30:35.344557+0000","last_became_active":"2023-11-24T06:30:34.260161+0000","last_became_peered":"2023-11-24T06:30:34.260161+0000","last_unstale":"2023-11-24T06:30:35.344557+0000","last_undegraded":"2023-11-24T06:30:35.344557+0000","last_fullsized":"2023-11-24T06:30:35.344557+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:42:18.443208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.259370+0000","last_change":"2023-11-24T06:30:33.253225+0000","last_active":"2023-11-24T06:30:34.259370+0000","last_peered":"2023-11-24T06:30:34.259370+0000","last_clean":"2023-11-24T06:30:34.259370+0000","last_became_active":"2023-11-24T06:30:33.252940+0000","last_became_peered":"2023-11-24T06:30:33.252940+0000","last_unstale":"2023-11-24T06:30:34.259370+0000","last_undegraded":"2023-11-24T06:30:34.259370+0000","last_fullsized":"2023-11-24T06:30:34.259370+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:44.581758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345011+0000","last_change":"2023-11-24T06:30:34.276484+0000","last_active":"2023-11-24T06:30:35.345011+0000","last_peered":"2023-11-24T06:30:35.345011+0000","last_clean":"2023-11-24T06:30:35.345011+0000","last_became_active":"2023-11-24T06:30:34.275957+0000","last_became_peered":"2023-11-24T06:30:34.275957+0000","last_unstale":"2023-11-24T06:30:35.345011+0000","last_undegraded":"2023-11-24T06:30:35.345011+0000","last_fullsized":"2023-11-24T06:30:35.345011+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:30.742091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.273608+0000","last_change":"2023-11-24T06:30:33.263099+0000","last_active":"2023-11-24T06:30:34.273608+0000","last_peered":"2023-11-24T06:30:34.273608+0000","last_clean":"2023-11-24T06:30:34.273608+0000","last_became_active":"2023-11-24T06:30:33.261288+0000","last_became_peered":"2023-11-24T06:30:33.261288+0000","last_unstale":"2023-11-24T06:30:34.273608+0000","last_undegraded":"2023-11-24T06:30:34.273608+0000","last_fullsized":"2023-11-24T06:30:34.273608+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:49.881712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343158+0000","last_change":"2023-11-24T06:30:34.268549+0000","last_active":"2023-11-24T06:30:35.343158+0000","last_peered":"2023-11-24T06:30:35.343158+0000","last_clean":"2023-11-24T06:30:35.343158+0000","last_became_active":"2023-11-24T06:30:34.268031+0000","last_became_peered":"2023-11-24T06:30:34.268031+0000","last_unstale":"2023-11-24T06:30:35.343158+0000","last_undegraded":"2023-11-24T06:30:35.343158+0000","last_fullsized":"2023-11-24T06:30:35.343158+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:27.880252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.233442+0000","last_change":"2023-11-24T06:30:33.250035+0000","last_active":"2023-11-24T06:30:34.233442+0000","last_peered":"2023-11-24T06:30:34.233442+0000","last_clean":"2023-11-24T06:30:34.233442+0000","last_became_active":"2023-11-24T06:30:33.249747+0000","last_became_peered":"2023-11-24T06:30:33.249747+0000","last_unstale":"2023-11-24T06:30:34.233442+0000","last_undegraded":"2023-11-24T06:30:34.233442+0000","last_fullsized":"2023-11-24T06:30:34.233442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:30.859810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343000+0000","last_change":"2023-11-24T06:30:34.261260+0000","last_active":"2023-11-24T06:30:35.343000+0000","last_peered":"2023-11-24T06:30:35.343000+0000","last_clean":"2023-11-24T06:30:35.343000+0000","last_became_active":"2023-11-24T06:30:34.260987+0000","last_became_peered":"2023-11-24T06:30:34.260987+0000","last_unstale":"2023-11-24T06:30:35.343000+0000","last_undegraded":"2023-11-24T06:30:35.343000+0000","last_fullsized":"2023-11-24T06:30:35.343000+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:31.302540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.243672+0000","last_change":"2023-11-24T06:30:33.240278+0000","last_active":"2023-11-24T06:30:34.243672+0000","last_peered":"2023-11-24T06:30:34.243672+0000","last_clean":"2023-11-24T06:30:34.243672+0000","last_became_active":"2023-11-24T06:30:33.238103+0000","last_became_peered":"2023-11-24T06:30:33.238103+0000","last_unstale":"2023-11-24T06:30:34.243672+0000","last_undegraded":"2023-11-24T06:30:34.243672+0000","last_fullsized":"2023-11-24T06:30:34.243672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:22.469854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346408+0000","last_change":"2023-11-24T06:30:34.261721+0000","last_active":"2023-11-24T06:30:35.346408+0000","last_peered":"2023-11-24T06:30:35.346408+0000","last_clean":"2023-11-24T06:30:35.346408+0000","last_became_active":"2023-11-24T06:30:34.261436+0000","last_became_peered":"2023-11-24T06:30:34.261436+0000","last_unstale":"2023-11-24T06:30:35.346408+0000","last_undegraded":"2023-11-24T06:30:35.346408+0000","last_fullsized":"2023-11-24T06:30:35.346408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:15.717290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.257406+0000","last_change":"2023-11-24T06:30:33.257372+0000","last_active":"2023-11-24T06:30:34.257406+0000","last_peered":"2023-11-24T06:30:34.257406+0000","last_clean":"2023-11-24T06:30:34.257406+0000","last_became_active":"2023-11-24T06:30:33.257063+0000","last_became_peered":"2023-11-24T06:30:33.257063+0000","last_unstale":"2023-11-24T06:30:34.257406+0000","last_undegraded":"2023-11-24T06:30:34.257406+0000","last_fullsized":"2023-11-24T06:30:34.257406+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:56.795055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259851+0000","last_change":"2023-11-24T06:30:34.281867+0000","last_active":"2023-11-24T06:30:35.259851+0000","last_peered":"2023-11-24T06:30:35.259851+0000","last_clean":"2023-11-24T06:30:35.259851+0000","last_became_active":"2023-11-24T06:30:34.280414+0000","last_became_peered":"2023-11-24T06:30:34.280414+0000","last_unstale":"2023-11-24T06:30:35.259851+0000","last_undegraded":"2023-11-24T06:30:35.259851+0000","last_fullsized":"2023-11-24T06:30:35.259851+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:42.434683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.269098+0000","last_change":"2023-11-24T06:30:33.266155+0000","last_active":"2023-11-24T06:30:34.269098+0000","last_peered":"2023-11-24T06:30:34.269098+0000","last_clean":"2023-11-24T06:30:34.269098+0000","last_became_active":"2023-11-24T06:30:33.265258+0000","last_became_peered":"2023-11-24T06:30:33.265258+0000","last_unstale":"2023-11-24T06:30:34.269098+0000","last_undegraded":"2023-11-24T06:30:34.269098+0000","last_fullsized":"2023-11-24T06:30:34.269098+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:20.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347962+0000","last_change":"2023-11-24T06:30:34.275278+0000","last_active":"2023-11-24T06:30:35.347962+0000","last_peered":"2023-11-24T06:30:35.347962+0000","last_clean":"2023-11-24T06:30:35.347962+0000","last_became_active":"2023-11-24T06:30:34.274913+0000","last_became_peered":"2023-11-24T06:30:34.274913+0000","last_unstale":"2023-11-24T06:30:35.347962+0000","last_undegraded":"2023-11-24T06:30:35.347962+0000","last_fullsized":"2023-11-24T06:30:35.347962+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:53.474598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.266907+0000","last_change":"2023-11-24T06:30:33.264658+0000","last_active":"2023-11-24T06:30:34.266907+0000","last_peered":"2023-11-24T06:30:34.266907+0000","last_clean":"2023-11-24T06:30:34.266907+0000","last_became_active":"2023-11-24T06:30:33.264320+0000","last_became_peered":"2023-11-24T06:30:33.264320+0000","last_unstale":"2023-11-24T06:30:34.266907+0000","last_undegraded":"2023-11-24T06:30:34.266907+0000","last_fullsized":"2023-11-24T06:30:34.266907+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:34.652219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343595+0000","last_change":"2023-11-24T06:30:34.255933+0000","last_active":"2023-11-24T06:30:35.343595+0000","last_peered":"2023-11-24T06:30:35.343595+0000","last_clean":"2023-11-24T06:30:35.343595+0000","last_became_active":"2023-11-24T06:30:34.255686+0000","last_became_peered":"2023-11-24T06:30:34.255686+0000","last_unstale":"2023-11-24T06:30:35.343595+0000","last_undegraded":"2023-11-24T06:30:35.343595+0000","last_fullsized":"2023-11-24T06:30:35.343595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:13.387080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.270954+0000","last_change":"2023-11-24T06:30:33.239974+0000","last_active":"2023-11-24T06:30:34.270954+0000","last_peered":"2023-11-24T06:30:34.270954+0000","last_clean":"2023-11-24T06:30:34.270954+0000","last_became_active":"2023-11-24T06:30:33.238273+0000","last_became_peered":"2023-11-24T06:30:33.238273+0000","last_unstale":"2023-11-24T06:30:34.270954+0000","last_undegraded":"2023-11-24T06:30:34.270954+0000","last_fullsized":"2023-11-24T06:30:34.270954+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:46.553265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.258671+0000","last_change":"2023-11-24T06:30:34.254999+0000","last_active":"2023-11-24T06:30:35.258671+0000","last_peered":"2023-11-24T06:30:35.258671+0000","last_clean":"2023-11-24T06:30:35.258671+0000","last_became_active":"2023-11-24T06:30:34.254110+0000","last_became_peered":"2023-11-24T06:30:34.254110+0000","last_unstale":"2023-11-24T06:30:35.258671+0000","last_undegraded":"2023-11-24T06:30:35.258671+0000","last_fullsized":"2023-11-24T06:30:35.258671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:11:41.823369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.280676+0000","last_change":"2023-11-24T06:30:33.252044+0000","last_active":"2023-11-24T06:30:34.280676+0000","last_peered":"2023-11-24T06:30:34.280676+0000","last_clean":"2023-11-24T06:30:34.280676+0000","last_became_active":"2023-11-24T06:30:33.251774+0000","last_became_peered":"2023-11-24T06:30:33.251774+0000","last_unstale":"2023-11-24T06:30:34.280676+0000","last_undegraded":"2023-11-24T06:30:34.280676+0000","last_fullsized":"2023-11-24T06:30:34.280676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:44.399558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345848+0000","last_change":"2023-11-24T06:30:34.262809+0000","last_active":"2023-11-24T06:30:35.345848+0000","last_peered":"2023-11-24T06:30:35.345848+0000","last_clean":"2023-11-24T06:30:35.345848+0000","last_became_active":"2023-11-24T06:30:34.261525+0000","last_became_peered":"2023-11-24T06:30:34.261525+0000","last_unstale":"2023-11-24T06:30:35.345848+0000","last_undegraded":"2023-11-24T06:30:35.345848+0000","last_fullsized":"2023-11-24T06:30:35.345848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:41.655269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.266966+0000","last_change":"2023-11-24T06:30:33.261613+0000","last_active":"2023-11-24T06:30:34.266966+0000","last_peered":"2023-11-24T06:30:34.266966+0000","last_clean":"2023-11-24T06:30:34.266966+0000","last_became_active":"2023-11-24T06:30:33.260523+0000","last_became_peered":"2023-11-24T06:30:33.260523+0000","last_unstale":"2023-11-24T06:30:34.266966+0000","last_undegraded":"2023-11-24T06:30:34.266966+0000","last_fullsized":"2023-11-24T06:30:34.266966+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:58.846905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.342583+0000","last_change":"2023-11-24T06:30:34.278490+0000","last_active":"2023-11-24T06:30:35.342583+0000","last_peered":"2023-11-24T06:30:35.342583+0000","last_clean":"2023-11-24T06:30:35.342583+0000","last_became_active":"2023-11-24T06:30:34.278203+0000","last_became_peered":"2023-11-24T06:30:34.278203+0000","last_unstale":"2023-11-24T06:30:35.342583+0000","last_undegraded":"2023-11-24T06:30:35.342583+0000","last_fullsized":"2023-11-24T06:30:35.342583+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:02:53.020511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.250622+0000","last_change":"2023-11-24T06:30:33.263590+0000","last_active":"2023-11-24T06:30:34.250622+0000","last_peered":"2023-11-24T06:30:34.250622+0000","last_clean":"2023-11-24T06:30:34.250622+0000","last_became_active":"2023-11-24T06:30:33.262788+0000","last_became_peered":"2023-11-24T06:30:33.262788+0000","last_unstale":"2023-11-24T06:30:34.250622+0000","last_undegraded":"2023-11-24T06:30:34.250622+0000","last_fullsized":"2023-11-24T06:30:34.250622+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:08.963743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343441+0000","last_change":"2023-11-24T06:30:34.257935+0000","last_active":"2023-11-24T06:30:35.343441+0000","last_peered":"2023-11-24T06:30:35.343441+0000","last_clean":"2023-11-24T06:30:35.343441+0000","last_became_active":"2023-11-24T06:30:34.253623+0000","last_became_peered":"2023-11-24T06:30:34.253623+0000","last_unstale":"2023-11-24T06:30:35.343441+0000","last_undegraded":"2023-11-24T06:30:35.343441+0000","last_fullsized":"2023-11-24T06:30:35.343441+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:26.542532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.279648+0000","last_change":"2023-11-24T06:30:33.259288+0000","last_active":"2023-11-24T06:30:34.279648+0000","last_peered":"2023-11-24T06:30:34.279648+0000","last_clean":"2023-11-24T06:30:34.279648+0000","last_became_active":"2023-11-24T06:30:33.254396+0000","last_became_peered":"2023-11-24T06:30:33.254396+0000","last_unstale":"2023-11-24T06:30:34.279648+0000","last_undegraded":"2023-11-24T06:30:34.279648+0000","last_fullsized":"2023-11-24T06:30:34.279648+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:59.955909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345179+0000","last_change":"2023-11-24T06:30:34.276782+0000","last_active":"2023-11-24T06:30:35.345179+0000","last_peered":"2023-11-24T06:30:35.345179+0000","last_clean":"2023-11-24T06:30:35.345179+0000","last_became_active":"2023-11-24T06:30:34.268740+0000","last_became_peered":"2023-11-24T06:30:34.268740+0000","last_unstale":"2023-11-24T06:30:35.345179+0000","last_undegraded":"2023-11-24T06:30:35.345179+0000","last_fullsized":"2023-11-24T06:30:35.345179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:53.474598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.283976+0000","last_change":"2023-11-24T06:30:33.265823+0000","last_active":"2023-11-24T06:30:34.283976+0000","last_peered":"2023-11-24T06:30:34.283976+0000","last_clean":"2023-11-24T06:30:34.283976+0000","last_became_active":"2023-11-24T06:30:33.265509+0000","last_became_peered":"2023-11-24T06:30:33.265509+0000","last_unstale":"2023-11-24T06:30:34.283976+0000","last_undegraded":"2023-11-24T06:30:34.283976+0000","last_fullsized":"2023-11-24T06:30:34.283976+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:38.989068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260089+0000","last_change":"2023-11-24T06:30:34.252661+0000","last_active":"2023-11-24T06:30:35.260089+0000","last_peered":"2023-11-24T06:30:35.260089+0000","last_clean":"2023-11-24T06:30:35.260089+0000","last_became_active":"2023-11-24T06:30:34.251657+0000","last_became_peered":"2023-11-24T06:30:34.251657+0000","last_unstale":"2023-11-24T06:30:35.260089+0000","last_undegraded":"2023-11-24T06:30:35.260089+0000","last_fullsized":"2023-11-24T06:30:35.260089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:15:26.105937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.280390+0000","last_change":"2023-11-24T06:30:33.268072+0000","last_active":"2023-11-24T06:30:34.280390+0000","last_peered":"2023-11-24T06:30:34.280390+0000","last_clean":"2023-11-24T06:30:34.280390+0000","last_became_active":"2023-11-24T06:30:33.267757+0000","last_became_peered":"2023-11-24T06:30:33.267757+0000","last_unstale":"2023-11-24T06:30:34.280390+0000","last_undegraded":"2023-11-24T06:30:34.280390+0000","last_fullsized":"2023-11-24T06:30:34.280390+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:17.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256567+0000","last_change":"2023-11-24T06:30:34.254912+0000","last_active":"2023-11-24T06:30:35.256567+0000","last_peered":"2023-11-24T06:30:35.256567+0000","last_clean":"2023-11-24T06:30:35.256567+0000","last_became_active":"2023-11-24T06:30:34.249336+0000","last_became_peered":"2023-11-24T06:30:34.249336+0000","last_unstale":"2023-11-24T06:30:35.256567+0000","last_undegraded":"2023-11-24T06:30:35.256567+0000","last_fullsized":"2023-11-24T06:30:35.256567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:53:07.851979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.282975+0000","last_change":"2023-11-24T06:30:33.265903+0000","last_active":"2023-11-24T06:30:34.282975+0000","last_peered":"2023-11-24T06:30:34.282975+0000","last_clean":"2023-11-24T06:30:34.282975+0000","last_became_active":"2023-11-24T06:30:33.263286+0000","last_became_peered":"2023-11-24T06:30:33.263286+0000","last_unstale":"2023-11-24T06:30:34.282975+0000","last_undegraded":"2023-11-24T06:30:34.282975+0000","last_fullsized":"2023-11-24T06:30:34.282975+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:56.795055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343942+0000","last_change":"2023-11-24T06:30:34.262996+0000","last_active":"2023-11-24T06:30:35.343942+0000","last_peered":"2023-11-24T06:30:35.343942+0000","last_clean":"2023-11-24T06:30:35.343942+0000","last_became_active":"2023-11-24T06:30:34.262112+0000","last_became_peered":"2023-11-24T06:30:34.262112+0000","last_unstale":"2023-11-24T06:30:35.343942+0000","last_undegraded":"2023-11-24T06:30:35.343942+0000","last_fullsized":"2023-11-24T06:30:35.343942+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:59:28.331014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.239801+0000","last_change":"2023-11-24T06:30:33.259236+0000","last_active":"2023-11-24T06:30:34.239801+0000","last_peered":"2023-11-24T06:30:34.239801+0000","last_clean":"2023-11-24T06:30:34.239801+0000","last_became_active":"2023-11-24T06:30:33.258200+0000","last_became_peered":"2023-11-24T06:30:33.258200+0000","last_unstale":"2023-11-24T06:30:34.239801+0000","last_undegraded":"2023-11-24T06:30:34.239801+0000","last_fullsized":"2023-11-24T06:30:34.239801+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:42.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260075+0000","last_change":"2023-11-24T06:30:34.249019+0000","last_active":"2023-11-24T06:30:35.260075+0000","last_peered":"2023-11-24T06:30:35.260075+0000","last_clean":"2023-11-24T06:30:35.260075+0000","last_became_active":"2023-11-24T06:30:34.248779+0000","last_became_peered":"2023-11-24T06:30:34.248779+0000","last_unstale":"2023-11-24T06:30:35.260075+0000","last_undegraded":"2023-11-24T06:30:35.260075+0000","last_fullsized":"2023-11-24T06:30:35.260075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:50.735632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.267095+0000","last_change":"2023-11-24T06:30:33.259814+0000","last_active":"2023-11-24T06:30:34.267095+0000","last_peered":"2023-11-24T06:30:34.267095+0000","last_clean":"2023-11-24T06:30:34.267095+0000","last_became_active":"2023-11-24T06:30:33.259551+0000","last_became_peered":"2023-11-24T06:30:33.259551+0000","last_unstale":"2023-11-24T06:30:34.267095+0000","last_undegraded":"2023-11-24T06:30:34.267095+0000","last_fullsized":"2023-11-24T06:30:34.267095+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:59.955909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259713+0000","last_change":"2023-11-24T06:30:34.278491+0000","last_active":"2023-11-24T06:30:35.259713+0000","last_peered":"2023-11-24T06:30:35.259713+0000","last_clean":"2023-11-24T06:30:35.259713+0000","last_became_active":"2023-11-24T06:30:34.278188+0000","last_became_peered":"2023-11-24T06:30:34.278188+0000","last_unstale":"2023-11-24T06:30:35.259713+0000","last_undegraded":"2023-11-24T06:30:35.259713+0000","last_fullsized":"2023-11-24T06:30:35.259713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:46.777167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.250972+0000","last_change":"2023-11-24T06:30:33.263043+0000","last_active":"2023-11-24T06:30:34.250972+0000","last_peered":"2023-11-24T06:30:34.250972+0000","last_clean":"2023-11-24T06:30:34.250972+0000","last_became_active":"2023-11-24T06:30:33.262727+0000","last_became_peered":"2023-11-24T06:30:33.262727+0000","last_unstale":"2023-11-24T06:30:34.250972+0000","last_undegraded":"2023-11-24T06:30:34.250972+0000","last_fullsized":"2023-11-24T06:30:34.250972+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:39.081828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260056+0000","last_change":"2023-11-24T06:30:34.278437+0000","last_active":"2023-11-24T06:30:35.260056+0000","last_peered":"2023-11-24T06:30:35.260056+0000","last_clean":"2023-11-24T06:30:35.260056+0000","last_became_active":"2023-11-24T06:30:34.278058+0000","last_became_peered":"2023-11-24T06:30:34.278058+0000","last_unstale":"2023-11-24T06:30:35.260056+0000","last_undegraded":"2023-11-24T06:30:35.260056+0000","last_fullsized":"2023-11-24T06:30:35.260056+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:05.432552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.250391+0000","last_change":"2023-11-24T06:30:33.259539+0000","last_active":"2023-11-24T06:30:34.250391+0000","last_peered":"2023-11-24T06:30:34.250391+0000","last_clean":"2023-11-24T06:30:34.250391+0000","last_became_active":"2023-11-24T06:30:33.258994+0000","last_became_peered":"2023-11-24T06:30:33.258994+0000","last_unstale":"2023-11-24T06:30:34.250391+0000","last_undegraded":"2023-11-24T06:30:34.250391+0000","last_fullsized":"2023-11-24T06:30:34.250391+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:34.021503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344664+0000","last_change":"2023-11-24T06:30:34.265135+0000","last_active":"2023-11-24T06:30:35.344664+0000","last_peered":"2023-11-24T06:30:35.344664+0000","last_clean":"2023-11-24T06:30:35.344664+0000","last_became_active":"2023-11-24T06:30:34.264865+0000","last_became_peered":"2023-11-24T06:30:34.264865+0000","last_unstale":"2023-11-24T06:30:35.344664+0000","last_undegraded":"2023-11-24T06:30:35.344664+0000","last_fullsized":"2023-11-24T06:30:35.344664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:22.171241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.284256+0000","last_change":"2023-11-24T06:30:33.264626+0000","last_active":"2023-11-24T06:30:34.284256+0000","last_peered":"2023-11-24T06:30:34.284256+0000","last_clean":"2023-11-24T06:30:34.284256+0000","last_became_active":"2023-11-24T06:30:33.264323+0000","last_became_peered":"2023-11-24T06:30:33.264323+0000","last_unstale":"2023-11-24T06:30:34.284256+0000","last_undegraded":"2023-11-24T06:30:34.284256+0000","last_fullsized":"2023-11-24T06:30:34.284256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:17.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346315+0000","last_change":"2023-11-24T06:30:34.265751+0000","last_active":"2023-11-24T06:30:35.346315+0000","last_peered":"2023-11-24T06:30:35.346315+0000","last_clean":"2023-11-24T06:30:35.346315+0000","last_became_active":"2023-11-24T06:30:34.264900+0000","last_became_peered":"2023-11-24T06:30:34.264900+0000","last_unstale":"2023-11-24T06:30:35.346315+0000","last_undegraded":"2023-11-24T06:30:35.346315+0000","last_fullsized":"2023-11-24T06:30:35.346315+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:01.166325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346631+0000","last_change":"2023-11-24T06:30:33.262138+0000","last_active":"2023-11-24T06:30:35.346631+0000","last_peered":"2023-11-24T06:30:35.346631+0000","last_clean":"2023-11-24T06:30:35.346631+0000","last_became_active":"2023-11-24T06:30:33.261687+0000","last_became_peered":"2023-11-24T06:30:33.261687+0000","last_unstale":"2023-11-24T06:30:35.346631+0000","last_undegraded":"2023-11-24T06:30:35.346631+0000","last_fullsized":"2023-11-24T06:30:35.346631+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:06:42.741740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259672+0000","last_change":"2023-11-24T06:30:34.281505+0000","last_active":"2023-11-24T06:30:35.259672+0000","last_peered":"2023-11-24T06:30:35.259672+0000","last_clean":"2023-11-24T06:30:35.259672+0000","last_became_active":"2023-11-24T06:30:34.280285+0000","last_became_peered":"2023-11-24T06:30:34.280285+0000","last_unstale":"2023-11-24T06:30:35.259672+0000","last_undegraded":"2023-11-24T06:30:35.259672+0000","last_fullsized":"2023-11-24T06:30:35.259672+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:04.704406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.243962+0000","last_change":"2023-11-24T06:30:33.255156+0000","last_active":"2023-11-24T06:30:34.243962+0000","last_peered":"2023-11-24T06:30:34.243962+0000","last_clean":"2023-11-24T06:30:34.243962+0000","last_became_active":"2023-11-24T06:30:33.254878+0000","last_became_peered":"2023-11-24T06:30:33.254878+0000","last_unstale":"2023-11-24T06:30:34.243962+0000","last_undegraded":"2023-11-24T06:30:34.243962+0000","last_fullsized":"2023-11-24T06:30:34.243962+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:17.577767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259976+0000","last_change":"2023-11-24T06:30:34.267501+0000","last_active":"2023-11-24T06:30:35.259976+0000","last_peered":"2023-11-24T06:30:35.259976+0000","last_clean":"2023-11-24T06:30:35.259976+0000","last_became_active":"2023-11-24T06:30:34.267199+0000","last_became_peered":"2023-11-24T06:30:34.267199+0000","last_unstale":"2023-11-24T06:30:35.259976+0000","last_undegraded":"2023-11-24T06:30:35.259976+0000","last_fullsized":"2023-11-24T06:30:35.259976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:37.004815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.250399+0000","last_change":"2023-11-24T06:30:33.263337+0000","last_active":"2023-11-24T06:30:34.250399+0000","last_peered":"2023-11-24T06:30:34.250399+0000","last_clean":"2023-11-24T06:30:34.250399+0000","last_became_active":"2023-11-24T06:30:33.262997+0000","last_became_peered":"2023-11-24T06:30:33.262997+0000","last_unstale":"2023-11-24T06:30:34.250399+0000","last_undegraded":"2023-11-24T06:30:34.250399+0000","last_fullsized":"2023-11-24T06:30:34.250399+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:31:07.057889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344460+0000","last_change":"2023-11-24T06:30:34.250892+0000","last_active":"2023-11-24T06:30:35.344460+0000","last_peered":"2023-11-24T06:30:35.344460+0000","last_clean":"2023-11-24T06:30:35.344460+0000","last_became_active":"2023-11-24T06:30:34.250633+0000","last_became_peered":"2023-11-24T06:30:34.250633+0000","last_unstale":"2023-11-24T06:30:35.344460+0000","last_undegraded":"2023-11-24T06:30:35.344460+0000","last_fullsized":"2023-11-24T06:30:35.344460+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:31.871778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.243675+0000","last_change":"2023-11-24T06:30:33.258636+0000","last_active":"2023-11-24T06:30:34.243675+0000","last_peered":"2023-11-24T06:30:34.243675+0000","last_clean":"2023-11-24T06:30:34.243675+0000","last_became_active":"2023-11-24T06:30:33.258193+0000","last_became_peered":"2023-11-24T06:30:33.258193+0000","last_unstale":"2023-11-24T06:30:34.243675+0000","last_undegraded":"2023-11-24T06:30:34.243675+0000","last_fullsized":"2023-11-24T06:30:34.243675+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:52.553976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347590+0000","last_change":"2023-11-24T06:30:34.281782+0000","last_active":"2023-11-24T06:30:35.347590+0000","last_peered":"2023-11-24T06:30:35.347590+0000","last_clean":"2023-11-24T06:30:35.347590+0000","last_became_active":"2023-11-24T06:30:34.278975+0000","last_became_peered":"2023-11-24T06:30:34.278975+0000","last_unstale":"2023-11-24T06:30:35.347590+0000","last_undegraded":"2023-11-24T06:30:35.347590+0000","last_fullsized":"2023-11-24T06:30:35.347590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:30.742091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.234122+0000","last_change":"2023-11-24T06:30:33.253808+0000","last_active":"2023-11-24T06:30:34.234122+0000","last_peered":"2023-11-24T06:30:34.234122+0000","last_clean":"2023-11-24T06:30:34.234122+0000","last_became_active":"2023-11-24T06:30:33.250888+0000","last_became_peered":"2023-11-24T06:30:33.250888+0000","last_unstale":"2023-11-24T06:30:34.234122+0000","last_undegraded":"2023-11-24T06:30:34.234122+0000","last_fullsized":"2023-11-24T06:30:34.234122+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:31.950626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256053+0000","last_change":"2023-11-24T06:30:34.250635+0000","last_active":"2023-11-24T06:30:35.256053+0000","last_peered":"2023-11-24T06:30:35.256053+0000","last_clean":"2023-11-24T06:30:35.256053+0000","last_became_active":"2023-11-24T06:30:34.248570+0000","last_became_peered":"2023-11-24T06:30:34.248570+0000","last_unstale":"2023-11-24T06:30:35.256053+0000","last_undegraded":"2023-11-24T06:30:35.256053+0000","last_fullsized":"2023-11-24T06:30:35.256053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:19.642863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.257857+0000","last_change":"2023-11-24T06:30:33.257088+0000","last_active":"2023-11-24T06:30:34.257857+0000","last_peered":"2023-11-24T06:30:34.257857+0000","last_clean":"2023-11-24T06:30:34.257857+0000","last_became_active":"2023-11-24T06:30:33.256733+0000","last_became_peered":"2023-11-24T06:30:33.256733+0000","last_unstale":"2023-11-24T06:30:34.257857+0000","last_undegraded":"2023-11-24T06:30:34.257857+0000","last_fullsized":"2023-11-24T06:30:34.257857+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:38.989068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346922+0000","last_change":"2023-11-24T06:30:34.277209+0000","last_active":"2023-11-24T06:30:35.346922+0000","last_peered":"2023-11-24T06:30:35.346922+0000","last_clean":"2023-11-24T06:30:35.346922+0000","last_became_active":"2023-11-24T06:30:34.274316+0000","last_became_peered":"2023-11-24T06:30:34.274316+0000","last_unstale":"2023-11-24T06:30:35.346922+0000","last_undegraded":"2023-11-24T06:30:35.346922+0000","last_fullsized":"2023-11-24T06:30:35.346922+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:34.670131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.266603+0000","last_change":"2023-11-24T06:30:33.265710+0000","last_active":"2023-11-24T06:30:34.266603+0000","last_peered":"2023-11-24T06:30:34.266603+0000","last_clean":"2023-11-24T06:30:34.266603+0000","last_became_active":"2023-11-24T06:30:33.265397+0000","last_became_peered":"2023-11-24T06:30:33.265397+0000","last_unstale":"2023-11-24T06:30:34.266603+0000","last_undegraded":"2023-11-24T06:30:34.266603+0000","last_fullsized":"2023-11-24T06:30:34.266603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:06.261210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260070+0000","last_change":"2023-11-24T06:30:34.274805+0000","last_active":"2023-11-24T06:30:35.260070+0000","last_peered":"2023-11-24T06:30:35.260070+0000","last_clean":"2023-11-24T06:30:35.260070+0000","last_became_active":"2023-11-24T06:30:34.273993+0000","last_became_peered":"2023-11-24T06:30:34.273993+0000","last_unstale":"2023-11-24T06:30:35.260070+0000","last_undegraded":"2023-11-24T06:30:35.260070+0000","last_fullsized":"2023-11-24T06:30:35.260070+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:51:59.186566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.278231+0000","last_change":"2023-11-24T06:30:33.260898+0000","last_active":"2023-11-24T06:30:34.278231+0000","last_peered":"2023-11-24T06:30:34.278231+0000","last_clean":"2023-11-24T06:30:34.278231+0000","last_became_active":"2023-11-24T06:30:33.260528+0000","last_became_peered":"2023-11-24T06:30:33.260528+0000","last_unstale":"2023-11-24T06:30:34.278231+0000","last_undegraded":"2023-11-24T06:30:34.278231+0000","last_fullsized":"2023-11-24T06:30:34.278231+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:00.101462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":217,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.236920+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:30:34.236920+0000","last_peered":"2023-11-24T06:30:34.236920+0000","last_clean":"2023-11-24T06:30:34.236920+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:30:34.236920+0000","last_undegraded":"2023-11-24T06:30:34.236920+0000","last_fullsized":"2023-11-24T06:30:34.236920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1466368,"data_stored":956101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.252,"15min":1.3160000000000001},"min":{"1min":0.68100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.0599999999999996,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.296,"15min":1.3009999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":4.556,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.6539999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0249999999999999,"15min":1.012},"min":{"1min":0.68200000000000005,"5min":0.629,"15min":0.629},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.343,"15min":1.377},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":3.944,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.754}]},{"osd":2,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.2729999999999999,"15min":1.288},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.6060000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.3539999999999992,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.204},"min":{"1min":0.79300000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.8200000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.339}]},{"osd":4,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.4870000000000001,"15min":1.5669999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.1059999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.498,"5min":1.5369999999999999,"15min":1.579},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.7110000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.3580000000000001,"15min":1.399},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.79},{"interface":"front","average":{"1min":2.3119999999999998,"5min":1.8240000000000001,"15min":1.794},"min":{"1min":0.69799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":17.582999999999998,"5min":17.582999999999998,"15min":17.582999999999998},"last":2.1419999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.736,"15min":1.8480000000000001},"min":{"1min":0.73099999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.8140000000000001,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":2.0419999999999998},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.6240000000000001,"15min":1.694},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":6.2210000000000001,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.851}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":365209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0980000000000001,"15min":1.1020000000000001},"min":{"1min":0.73499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7390000000000001,"5min":1.829,"15min":1.829},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.155,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.76800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.623,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.1839999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2210000000000001,"15min":1.2130000000000001},"min":{"1min":0.71099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6299999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.2010000000000001,"15min":1.2070000000000001},"min":{"1min":0.88900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.494,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4850000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.159},"min":{"1min":0.75700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.609,"5min":3.597,"15min":3.597},"last":1.123},{"interface":"front","average":{"1min":1.304,"5min":1.292,"15min":1.3140000000000001},"min":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6359999999999999,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.837}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3129999999999999,"15min":1.3180000000000001},"min":{"1min":0.78100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5620000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.583},{"interface":"front","average":{"1min":1.25,"5min":1.2969999999999999,"15min":1.325},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5960000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.0129999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.8919999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":2.0880000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.385,"15min":1.397},"min":{"1min":0.79600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7729999999999999,"5min":3.04,"15min":3.04},"last":1.7609999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3720000000000001,"15min":1.377},"min":{"1min":0.97699999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7330000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.272},{"interface":"front","average":{"1min":1.423,"5min":1.552,"15min":1.6020000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.851,"5min":3.774,"15min":3.774},"last":1.9630000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5449999999999999,"15min":1.5589999999999999},"min":{"1min":1.0029999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7689999999999999,"5min":3.403,"15min":3.403},"last":2.0409999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.512},"min":{"1min":0.875,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.792,"5min":3.649,"15min":3.649},"last":1.431}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":365207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.516,"5min":1.903,"15min":1.903},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.288,"15min":1.3140000000000001},"min":{"1min":0.91700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.915,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.3400000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.145,"15min":1.1399999999999999},"min":{"1min":0.78900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.877},{"interface":"front","average":{"1min":1.327,"5min":1.147,"15min":1.1140000000000001},"min":{"1min":0.89500000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.119}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3109999999999999,"15min":1.2969999999999999},"min":{"1min":0.99199999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1669999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.1830000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3360000000000001,"15min":1.375},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.952,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.448},{"interface":"front","average":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3899999999999999},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.3069999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3180000000000001,"15min":1.3340000000000001},"min":{"1min":0.878,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.839,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.5029999999999999,"15min":1.5369999999999999},"min":{"1min":0.82299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0099999999999998,"5min":3.085,"15min":3.085},"last":1.5149999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4690000000000001,"15min":1.4450000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.649},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.474,"15min":1.468},"min":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0270000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.7130000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.651,"5min":1.6140000000000001,"15min":1.6240000000000001},"min":{"1min":1.046,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.141,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.675},{"interface":"front","average":{"1min":1.714,"5min":1.6919999999999999,"15min":1.716},"min":{"1min":1.0129999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3159999999999998,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.5549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":365187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.073,"15min":1.073},"min":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.982,"15min":1.982},"last":1.298},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.089,"15min":1.1120000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.468,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.0820000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.147,"15min":1.113},"min":{"1min":0.84199999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.26,"15min":1.3049999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5840000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.4930000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8129999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.3140000000000001,"15min":1.3320000000000001},"min":{"1min":0.79800000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7250000000000001,"5min":2.254,"15min":2.254},"last":1.552}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.294,"15min":1.333},"min":{"1min":0.76100000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.833,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3779999999999999,"15min":1.3720000000000001},"min":{"1min":0.753,"5min":0.746,"15min":0.746},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4570000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.482,"15min":1.5149999999999999},"min":{"1min":0.874,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8109999999999999,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.421},{"interface":"front","average":{"1min":1.389,"5min":1.522,"15min":1.5609999999999999},"min":{"1min":0.85799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.923,"5min":3.52,"15min":3.52},"last":1.633}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.474,"15min":1.482},"min":{"1min":0.79200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.542,"15min":1.5740000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.859,"5min":2.452,"15min":2.452},"last":1.6060000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.595,"15min":1.6559999999999999},"min":{"1min":0.64000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.7729999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.615,"15min":1.613},"min":{"1min":0.84799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.0099999999999998,"5min":2.363,"15min":2.363},"last":1.579}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":365229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.3080000000000001,"5min":1.355,"15min":1.355},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.66600000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.94999999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.96399999999999997},"min":{"1min":0.51800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.077,"5min":1.091,"15min":1.113},"min":{"1min":0.61499999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.0350000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.58499999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4550000000000001,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.202}]},{"osd":4,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3420000000000001,"15min":1.3879999999999999},"min":{"1min":0.91300000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.696,"5min":3.081,"15min":3.081},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.3340000000000001,"15min":1.339},"min":{"1min":0.74299999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.712,"5min":2.758,"15min":2.758},"last":1.4390000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.373,"15min":1.403},"min":{"1min":0.873,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.1200000000000001,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.6200000000000001,"15min":1.748},"min":{"1min":0.86299999999999999,"5min":0.745,"15min":0.745},"max":{"1min":1.657,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.321}]},{"osd":6,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3759999999999999,"15min":1.407},"min":{"1min":0.82999999999999996,"5min":0.753,"15min":0.753},"max":{"1min":1.79,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.052},{"interface":"front","average":{"1min":1.524,"5min":1.53,"15min":1.544},"min":{"1min":1.0760000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.464,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.681}]},{"osd":7,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.5760000000000001,"15min":1.609},"min":{"1min":0.84499999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.589,"5min":4.641,"15min":4.641},"last":1.762},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.546,"15min":1.5740000000000001},"min":{"1min":0.88200000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.3439999999999999,"5min":3.798,"15min":3.798},"last":1.4119999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":876544,"data_stored":366729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2729999999999999,"5min":1.427,"15min":1.427},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.619,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.59699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.48499999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0840000000000001,"15min":1.1040000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.784,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.089,"5min":1.095,"15min":1.117},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6990000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.456,"15min":1.508},"min":{"1min":0.67900000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.6869999999999998,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.425,"5min":1.454,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.5800000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.2869999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4650000000000001,"15min":1.468},"min":{"1min":1.0029999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.036,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.27},{"interface":"front","average":{"1min":1.413,"5min":1.4690000000000001,"15min":1.5009999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.841,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.222}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.423,"15min":1.4359999999999999},"min":{"1min":0.66900000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.968,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.522,"15min":1.54},"min":{"1min":1.0029999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.165,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.163}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5149999999999999,"15min":1.5489999999999999},"min":{"1min":0.78200000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.913,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":1.097,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.423,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.2050000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":367345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.002,"15min":1.032},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.696,"5min":2.593,"15min":2.593},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.02,"15min":1.032},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5980000000000001,"5min":1.702,"15min":1.702},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.212,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":1,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7669999999999999,"5min":1.829,"15min":1.829},"last":1.482}]},{"osd":3,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1040000000000001,"15min":1.1200000000000001},"min":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.603,"5min":1.766,"15min":1.766},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0289999999999999,"15min":1.069},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2250000000000001,"5min":1.798,"15min":1.798},"last":1.6279999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.246,"15min":1.2529999999999999},"min":{"1min":0.89000000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.621,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.256},{"interface":"front","average":{"1min":1.359,"5min":1.417,"15min":1.4450000000000001},"min":{"1min":1.0509999999999999,"5min":1.032,"15min":1.032},"max":{"1min":1.837,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.591}]},{"osd":5,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.294,"15min":1.302},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.514,"5min":2.016,"15min":2.016},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.389,"15min":1.399},"min":{"1min":0.91700000000000004,"5min":0.877,"15min":0.877},"max":{"1min":1.7410000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.367}]},{"osd":6,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.391,"15min":1.393},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.847,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4339999999999999,"15min":1.45},"min":{"1min":0.96399999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.6499999999999999,"5min":2.371,"15min":2.371},"last":1.6459999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.444,"15min":1.448},"min":{"1min":0.89300000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9730000000000001,"5min":2.246,"15min":2.246},"last":1.734},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4950000000000001,"15min":1.5189999999999999},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.7629999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.7}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":955485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.872,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.92400000000000004,"15min":0.95899999999999996},"min":{"1min":0.39200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.516,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.004,"15min":1},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6619999999999999,"5min":1.784,"15min":1.784},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.081,"5min":1.1060000000000001,"15min":1.1319999999999999},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.639,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.72099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.94899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4750000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.052,"5min":1.077,"15min":1.087},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.506,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.23,"15min":1.226},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7070000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3129999999999999},"min":{"1min":1.0149999999999999,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.714,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.046}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.329,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9610000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.2709999999999999,"15min":1.278},"min":{"1min":0.86099999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.865,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.4570000000000001,"15min":1.4830000000000001},"min":{"1min":0.84199999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8500000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.4710000000000001,"15min":1.5109999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.903,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.123}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.448,"15min":1.456},"min":{"1min":0.95999999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8180000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.081},{"interface":"front","average":{"1min":1.52,"5min":1.417,"15min":1.425},"min":{"1min":0.84399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0630000000000002,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.02}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:30:43.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:30:44.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.127+0000 7f7d03c90640 1 Processor -- start 2023-11-24T06:30:44.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.127+0000 7f7d03c90640 1 -- start start 2023-11-24T06:30:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.129+0000 7f7d03c90640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7cfc13df80 0x7f7cfc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.129+0000 7f7d03c90640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7cfc0cf9a0 0x7f7cfc0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.129+0000 7f7d03c90640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc06c470 con 0x7f7cfc13e360 2023-11-24T06:30:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.129+0000 7f7d03c90640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc06c750 con 0x7f7cfc0cf9a0 2023-11-24T06:30:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.129+0000 7f7d03c90640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc06ca30 con 0x7f7cfc13df80 2023-11-24T06:30:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.129+0000 7f7d0222d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7cfc0cf9a0 0x7f7cfc0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.129+0000 7f7d0122b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7cfc13e360 0x7f7cfc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60320/0 (socket says 172.21.15.16:60320) 2023-11-24T06:30:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.129+0000 7f7d0122b640 1 -- 172.21.15.16:0/490722685 learned_addr learned my addr 172.21.15.16:0/490722685 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.130+0000 7f7d01a2c640 1 --2- 172.21.15.16:0/490722685 >> v2:172.21.15.173:3300/0 conn(0x7f7cfc13df80 0x7f7cfc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.130+0000 7f7d0222d640 1 -- 172.21.15.16:0/490722685 >> v2:172.21.15.173:3300/0 conn(0x7f7cfc13df80 msgr2=0x7f7cfc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.130+0000 7f7d0222d640 1 --2- 172.21.15.16:0/490722685 >> v2:172.21.15.173:3300/0 conn(0x7f7cfc13df80 0x7f7cfc15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.130+0000 7f7d0222d640 1 -- 172.21.15.16:0/490722685 >> v1:172.21.15.16:6789/0 conn(0x7f7cfc13e360 legacy=0x7f7cfc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.130+0000 7f7d0222d640 1 -- 172.21.15.16:0/490722685 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc06cd10 con 0x7f7cfc0cf9a0 2023-11-24T06:30:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.131+0000 7f7d0222d640 1 --2- 172.21.15.16:0/490722685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cf9a0 0x7f7cfc0cd840 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc072770 tx=0x7f7cf402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be43e7c43f3d5ca8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.131+0000 7f7d00a2a640 1 -- 172.21.15.16:0/490722685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf402cc70 con 0x7f7cfc0cf9a0 2023-11-24T06:30:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.131+0000 7f7d00a2a640 1 -- 172.21.15.16:0/490722685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf402cdd0 con 0x7f7cfc0cf9a0 2023-11-24T06:30:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.131+0000 7f7d00a2a640 1 -- 172.21.15.16:0/490722685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4034660 con 0x7f7cfc0cf9a0 2023-11-24T06:30:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.132+0000 7f7d03c90640 1 -- 172.21.15.16:0/490722685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cf9a0 msgr2=0x7f7cfc0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.132+0000 7f7d03c90640 1 --2- 172.21.15.16:0/490722685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cf9a0 0x7f7cfc0cd840 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc072770 tx=0x7f7cf402bec0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.132+0000 7f7d03c90640 1 -- 172.21.15.16:0/490722685 shutdown_connections 2023-11-24T06:30:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.132+0000 7f7d03c90640 1 --2- 172.21.15.16:0/490722685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc0cf9a0 0x7f7cfc0cd840 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.132+0000 7f7d03c90640 1 --2- 172.21.15.16:0/490722685 >> v2:172.21.15.173:3300/0 conn(0x7f7cfc13df80 0x7f7cfc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.132+0000 7f7d03c90640 1 -- 172.21.15.16:0/490722685 >> 172.21.15.16:0/490722685 conn(0x7f7cfc06a800 msgr2=0x7f7cfc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.132+0000 7f7d03c90640 1 -- 172.21.15.16:0/490722685 shutdown_connections 2023-11-24T06:30:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.132+0000 7f7d03c90640 1 -- 172.21.15.16:0/490722685 wait complete. 2023-11-24T06:30:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.134+0000 7f7d03c90640 1 Processor -- start 2023-11-24T06:30:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.134+0000 7f7d03c90640 1 -- start start 2023-11-24T06:30:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.135+0000 7f7d03c90640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cfc0cf9a0 0x7f7cfc13d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.135+0000 7f7d03c90640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cfc13df80 0x7f7cfc153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.135+0000 7f7d03c90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc13e360 0x7f7cfc153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.135+0000 7f7d03c90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc14b130 con 0x7f7cfc13e360 2023-11-24T06:30:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.135+0000 7f7d03c90640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1515b0 con 0x7f7cfc13df80 2023-11-24T06:30:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.135+0000 7f7d03c90640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc151890 con 0x7f7cfc0cf9a0 2023-11-24T06:30:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.135+0000 7f7d0222d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc13e360 0x7f7cfc153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d0222d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc13e360 0x7f7cfc153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39930/0 (socket says 172.21.15.16:39930) 2023-11-24T06:30:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d0222d640 1 -- 172.21.15.16:0/2140238376 learned_addr learned my addr 172.21.15.16:0/2140238376 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d01a2c640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cfc0cf9a0 0x7f7cfc13d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d0122b640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cfc13df80 0x7f7cfc153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d0222d640 1 -- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cfc0cf9a0 msgr2=0x7f7cfc13d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d0222d640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cfc0cf9a0 0x7f7cfc13d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d0222d640 1 -- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cfc13df80 msgr2=0x7f7cfc153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d0222d640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cfc13df80 0x7f7cfc153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d0222d640 1 -- 172.21.15.16:0/2140238376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4009a70 con 0x7f7cfc13e360 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.136+0000 7f7d0222d640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc13e360 0x7f7cfc153680 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009e80 tx=0x7f7cf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.137+0000 7f7cf27fc640 1 -- 172.21.15.16:0/2140238376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4040650 con 0x7f7cfc13e360 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.137+0000 7f7cf27fc640 1 -- 172.21.15.16:0/2140238376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf40407d0 con 0x7f7cfc13e360 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.137+0000 7f7cf27fc640 1 -- 172.21.15.16:0/2140238376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4040b80 con 0x7f7cfc13e360 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.137+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc14b380 con 0x7f7cfc13e360 2023-11-24T06:30:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.137+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc13dcf0 con 0x7f7cfc13e360 2023-11-24T06:30:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.138+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc14b8a0 con 0x7f7cfc13e360 2023-11-24T06:30:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.139+0000 7f7cf27fc640 1 -- 172.21.15.16:0/2140238376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f7cf4040ce0 con 0x7f7cfc13e360 2023-11-24T06:30:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.140+0000 7f7cf27fc640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ce006b230 0x7f7ce006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.140+0000 7f7cf27fc640 1 -- 172.21.15.16:0/2140238376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f7cf40acc80 con 0x7f7cfc13e360 2023-11-24T06:30:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.141+0000 7f7d01a2c640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ce006b230 0x7f7ce006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.142+0000 7f7d01a2c640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ce006b230 0x7f7ce006d6f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8006fd0 tx=0x7f7cf8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:44.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.145+0000 7f7cf27fc640 1 -- 172.21.15.16:0/2140238376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7cf4077620 con 0x7f7cfc13e360 2023-11-24T06:30:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.336+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7cfc15f9c0 con 0x7f7ce006b230 2023-11-24T06:30:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.358+0000 7f7cf27fc640 1 -- 172.21.15.16:0/2140238376 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286431 (secure 0 0 0) 0x7f7cf800b200 con 0x7f7ce006b230 2023-11-24T06:30:44.364 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:30:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ce006b230 msgr2=0x7f7ce006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ce006b230 0x7f7ce006d6f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8006fd0 tx=0x7f7cf8008040 comp rx=0 tx=0).stop 2023-11-24T06:30:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc13e360 msgr2=0x7f7cfc153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc13e360 0x7f7cfc153680 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009e80 tx=0x7f7cf4004a40 comp rx=0 tx=0).stop 2023-11-24T06:30:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 shutdown_connections 2023-11-24T06:30:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7ce006b230 0x7f7ce006d6f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc13e360 0x7f7cfc153680 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cfc13df80 0x7f7cfc153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 --2- 172.21.15.16:0/2140238376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cfc0cf9a0 0x7f7cfc13d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.363+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 >> 172.21.15.16:0/2140238376 conn(0x7f7cfc06a800 msgr2=0x7f7cfc15d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.364+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 shutdown_connections 2023-11-24T06:30:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.364+0000 7f7d03c90640 1 -- 172.21.15.16:0/2140238376 wait complete. 2023-11-24T06:30:44.387 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2023-11-24T06:30:44.180621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253320,"kb_used_data":7976,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721400,"statfs":{"total":773094113280,"available":772834713600,"internally_reserved":0,"allocated":8167424,"data_stored":4106492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.926382"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259642+0000","last_change":"2023-11-24T06:30:34.252293+0000","last_active":"2023-11-24T06:30:35.259642+0000","last_peered":"2023-11-24T06:30:35.259642+0000","last_clean":"2023-11-24T06:30:35.259642+0000","last_became_active":"2023-11-24T06:30:34.251526+0000","last_became_peered":"2023-11-24T06:30:34.251526+0000","last_unstale":"2023-11-24T06:30:35.259642+0000","last_undegraded":"2023-11-24T06:30:35.259642+0000","last_fullsized":"2023-11-24T06:30:35.259642+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:40.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.348535+0000","last_change":"2023-11-24T06:30:34.283473+0000","last_active":"2023-11-24T06:30:35.348535+0000","last_peered":"2023-11-24T06:30:35.348535+0000","last_clean":"2023-11-24T06:30:35.348535+0000","last_became_active":"2023-11-24T06:30:34.283122+0000","last_became_peered":"2023-11-24T06:30:34.283122+0000","last_unstale":"2023-11-24T06:30:35.348535+0000","last_undegraded":"2023-11-24T06:30:35.348535+0000","last_fullsized":"2023-11-24T06:30:35.348535+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:54.612687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259923+0000","last_change":"2023-11-24T06:30:34.259042+0000","last_active":"2023-11-24T06:30:35.259923+0000","last_peered":"2023-11-24T06:30:35.259923+0000","last_clean":"2023-11-24T06:30:35.259923+0000","last_became_active":"2023-11-24T06:30:34.258435+0000","last_became_peered":"2023-11-24T06:30:34.258435+0000","last_unstale":"2023-11-24T06:30:35.259923+0000","last_undegraded":"2023-11-24T06:30:35.259923+0000","last_fullsized":"2023-11-24T06:30:35.259923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:44.108280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256215+0000","last_change":"2023-11-24T06:30:34.280435+0000","last_active":"2023-11-24T06:30:35.256215+0000","last_peered":"2023-11-24T06:30:35.256215+0000","last_clean":"2023-11-24T06:30:35.256215+0000","last_became_active":"2023-11-24T06:30:34.280097+0000","last_became_peered":"2023-11-24T06:30:34.280097+0000","last_unstale":"2023-11-24T06:30:35.256215+0000","last_undegraded":"2023-11-24T06:30:35.256215+0000","last_fullsized":"2023-11-24T06:30:35.256215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:13:53.546580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343653+0000","last_change":"2023-11-24T06:30:34.261050+0000","last_active":"2023-11-24T06:30:35.343653+0000","last_peered":"2023-11-24T06:30:35.343653+0000","last_clean":"2023-11-24T06:30:35.343653+0000","last_became_active":"2023-11-24T06:30:34.260798+0000","last_became_peered":"2023-11-24T06:30:34.260798+0000","last_unstale":"2023-11-24T06:30:35.343653+0000","last_undegraded":"2023-11-24T06:30:35.343653+0000","last_fullsized":"2023-11-24T06:30:35.343653+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:29.283116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347309+0000","last_change":"2023-11-24T06:30:34.283412+0000","last_active":"2023-11-24T06:30:35.347309+0000","last_peered":"2023-11-24T06:30:35.347309+0000","last_clean":"2023-11-24T06:30:35.347309+0000","last_became_active":"2023-11-24T06:30:34.283062+0000","last_became_peered":"2023-11-24T06:30:34.283062+0000","last_unstale":"2023-11-24T06:30:35.347309+0000","last_undegraded":"2023-11-24T06:30:35.347309+0000","last_fullsized":"2023-11-24T06:30:35.347309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:46.928025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343469+0000","last_change":"2023-11-24T06:30:34.262673+0000","last_active":"2023-11-24T06:30:35.343469+0000","last_peered":"2023-11-24T06:30:35.343469+0000","last_clean":"2023-11-24T06:30:35.343469+0000","last_became_active":"2023-11-24T06:30:34.262372+0000","last_became_peered":"2023-11-24T06:30:34.262372+0000","last_unstale":"2023-11-24T06:30:35.343469+0000","last_undegraded":"2023-11-24T06:30:35.343469+0000","last_fullsized":"2023-11-24T06:30:35.343469+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:58:58.107784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346741+0000","last_change":"2023-11-24T06:30:34.271256+0000","last_active":"2023-11-24T06:30:35.346741+0000","last_peered":"2023-11-24T06:30:35.346741+0000","last_clean":"2023-11-24T06:30:35.346741+0000","last_became_active":"2023-11-24T06:30:34.270954+0000","last_became_peered":"2023-11-24T06:30:34.270954+0000","last_unstale":"2023-11-24T06:30:35.346741+0000","last_undegraded":"2023-11-24T06:30:35.346741+0000","last_fullsized":"2023-11-24T06:30:35.346741+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:06.752081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344712+0000","last_change":"2023-11-24T06:30:34.255460+0000","last_active":"2023-11-24T06:30:35.344712+0000","last_peered":"2023-11-24T06:30:35.344712+0000","last_clean":"2023-11-24T06:30:35.344712+0000","last_became_active":"2023-11-24T06:30:34.254633+0000","last_became_peered":"2023-11-24T06:30:34.254633+0000","last_unstale":"2023-11-24T06:30:35.344712+0000","last_undegraded":"2023-11-24T06:30:35.344712+0000","last_fullsized":"2023-11-24T06:30:35.344712+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:49:21.210820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345026+0000","last_change":"2023-11-24T06:30:34.275595+0000","last_active":"2023-11-24T06:30:35.345026+0000","last_peered":"2023-11-24T06:30:35.345026+0000","last_clean":"2023-11-24T06:30:35.345026+0000","last_became_active":"2023-11-24T06:30:34.275240+0000","last_became_peered":"2023-11-24T06:30:34.275240+0000","last_unstale":"2023-11-24T06:30:35.345026+0000","last_undegraded":"2023-11-24T06:30:35.345026+0000","last_fullsized":"2023-11-24T06:30:35.345026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:06.613023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259103+0000","last_change":"2023-11-24T06:30:34.248763+0000","last_active":"2023-11-24T06:30:35.259103+0000","last_peered":"2023-11-24T06:30:35.259103+0000","last_clean":"2023-11-24T06:30:35.259103+0000","last_became_active":"2023-11-24T06:30:34.247642+0000","last_became_peered":"2023-11-24T06:30:34.247642+0000","last_unstale":"2023-11-24T06:30:35.259103+0000","last_undegraded":"2023-11-24T06:30:35.259103+0000","last_fullsized":"2023-11-24T06:30:35.259103+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:43.282344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256396+0000","last_change":"2023-11-24T06:30:34.249620+0000","last_active":"2023-11-24T06:30:35.256396+0000","last_peered":"2023-11-24T06:30:35.256396+0000","last_clean":"2023-11-24T06:30:35.256396+0000","last_became_active":"2023-11-24T06:30:34.248964+0000","last_became_peered":"2023-11-24T06:30:34.248964+0000","last_unstale":"2023-11-24T06:30:35.256396+0000","last_undegraded":"2023-11-24T06:30:35.256396+0000","last_fullsized":"2023-11-24T06:30:35.256396+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:45.205158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.342771+0000","last_change":"2023-11-24T06:30:34.274593+0000","last_active":"2023-11-24T06:30:35.342771+0000","last_peered":"2023-11-24T06:30:35.342771+0000","last_clean":"2023-11-24T06:30:35.342771+0000","last_became_active":"2023-11-24T06:30:34.274328+0000","last_became_peered":"2023-11-24T06:30:34.274328+0000","last_unstale":"2023-11-24T06:30:35.342771+0000","last_undegraded":"2023-11-24T06:30:35.342771+0000","last_fullsized":"2023-11-24T06:30:35.342771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:08.454372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344254+0000","last_change":"2023-11-24T06:30:34.264008+0000","last_active":"2023-11-24T06:30:35.344254+0000","last_peered":"2023-11-24T06:30:35.344254+0000","last_clean":"2023-11-24T06:30:35.344254+0000","last_became_active":"2023-11-24T06:30:34.263740+0000","last_became_peered":"2023-11-24T06:30:34.263740+0000","last_unstale":"2023-11-24T06:30:35.344254+0000","last_undegraded":"2023-11-24T06:30:35.344254+0000","last_fullsized":"2023-11-24T06:30:35.344254+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:39.757730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.342845+0000","last_change":"2023-11-24T06:30:34.260735+0000","last_active":"2023-11-24T06:30:35.342845+0000","last_peered":"2023-11-24T06:30:35.342845+0000","last_clean":"2023-11-24T06:30:35.342845+0000","last_became_active":"2023-11-24T06:30:34.260434+0000","last_became_peered":"2023-11-24T06:30:34.260434+0000","last_unstale":"2023-11-24T06:30:35.342845+0000","last_undegraded":"2023-11-24T06:30:35.342845+0000","last_fullsized":"2023-11-24T06:30:35.342845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:43.282344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344395+0000","last_change":"2023-11-24T06:30:34.280175+0000","last_active":"2023-11-24T06:30:35.344395+0000","last_peered":"2023-11-24T06:30:35.344395+0000","last_clean":"2023-11-24T06:30:35.344395+0000","last_became_active":"2023-11-24T06:30:34.279901+0000","last_became_peered":"2023-11-24T06:30:34.279901+0000","last_unstale":"2023-11-24T06:30:35.344395+0000","last_undegraded":"2023-11-24T06:30:35.344395+0000","last_fullsized":"2023-11-24T06:30:35.344395+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:34.670131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344884+0000","last_change":"2023-11-24T06:30:34.255632+0000","last_active":"2023-11-24T06:30:35.344884+0000","last_peered":"2023-11-24T06:30:35.344884+0000","last_clean":"2023-11-24T06:30:35.344884+0000","last_became_active":"2023-11-24T06:30:34.255192+0000","last_became_peered":"2023-11-24T06:30:34.255192+0000","last_unstale":"2023-11-24T06:30:35.344884+0000","last_undegraded":"2023-11-24T06:30:35.344884+0000","last_fullsized":"2023-11-24T06:30:35.344884+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:32.962593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256834+0000","last_change":"2023-11-24T06:30:34.276012+0000","last_active":"2023-11-24T06:30:35.256834+0000","last_peered":"2023-11-24T06:30:35.256834+0000","last_clean":"2023-11-24T06:30:35.256834+0000","last_became_active":"2023-11-24T06:30:34.275542+0000","last_became_peered":"2023-11-24T06:30:34.275542+0000","last_unstale":"2023-11-24T06:30:35.256834+0000","last_undegraded":"2023-11-24T06:30:35.256834+0000","last_fullsized":"2023-11-24T06:30:35.256834+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:34:06.634469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.348044+0000","last_change":"2023-11-24T06:30:34.276206+0000","last_active":"2023-11-24T06:30:35.348044+0000","last_peered":"2023-11-24T06:30:35.348044+0000","last_clean":"2023-11-24T06:30:35.348044+0000","last_became_active":"2023-11-24T06:30:34.275859+0000","last_became_peered":"2023-11-24T06:30:34.275859+0000","last_unstale":"2023-11-24T06:30:35.348044+0000","last_undegraded":"2023-11-24T06:30:35.348044+0000","last_fullsized":"2023-11-24T06:30:35.348044+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:29.402712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344803+0000","last_change":"2023-11-24T06:30:34.265681+0000","last_active":"2023-11-24T06:30:35.344803+0000","last_peered":"2023-11-24T06:30:35.344803+0000","last_clean":"2023-11-24T06:30:35.344803+0000","last_became_active":"2023-11-24T06:30:34.265355+0000","last_became_peered":"2023-11-24T06:30:34.265355+0000","last_unstale":"2023-11-24T06:30:35.344803+0000","last_undegraded":"2023-11-24T06:30:35.344803+0000","last_fullsized":"2023-11-24T06:30:35.344803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:40.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344589+0000","last_change":"2023-11-24T06:30:34.276232+0000","last_active":"2023-11-24T06:30:35.344589+0000","last_peered":"2023-11-24T06:30:35.344589+0000","last_clean":"2023-11-24T06:30:35.344589+0000","last_became_active":"2023-11-24T06:30:34.274833+0000","last_became_peered":"2023-11-24T06:30:34.274833+0000","last_unstale":"2023-11-24T06:30:35.344589+0000","last_undegraded":"2023-11-24T06:30:35.344589+0000","last_fullsized":"2023-11-24T06:30:35.344589+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:11.493185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345343+0000","last_change":"2023-11-24T06:30:34.276782+0000","last_active":"2023-11-24T06:30:35.345343+0000","last_peered":"2023-11-24T06:30:35.345343+0000","last_clean":"2023-11-24T06:30:35.345343+0000","last_became_active":"2023-11-24T06:30:34.274573+0000","last_became_peered":"2023-11-24T06:30:34.274573+0000","last_unstale":"2023-11-24T06:30:35.345343+0000","last_undegraded":"2023-11-24T06:30:35.345343+0000","last_fullsized":"2023-11-24T06:30:35.345343+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:29.402712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.258987+0000","last_change":"2023-11-24T06:30:34.273439+0000","last_active":"2023-11-24T06:30:35.258987+0000","last_peered":"2023-11-24T06:30:35.258987+0000","last_clean":"2023-11-24T06:30:35.258987+0000","last_became_active":"2023-11-24T06:30:34.269631+0000","last_became_peered":"2023-11-24T06:30:34.269631+0000","last_unstale":"2023-11-24T06:30:35.258987+0000","last_undegraded":"2023-11-24T06:30:35.258987+0000","last_fullsized":"2023-11-24T06:30:35.258987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:08.454372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.258716+0000","last_change":"2023-11-24T06:30:34.280283+0000","last_active":"2023-11-24T06:30:35.258716+0000","last_peered":"2023-11-24T06:30:35.258716+0000","last_clean":"2023-11-24T06:30:35.258716+0000","last_became_active":"2023-11-24T06:30:34.279932+0000","last_became_peered":"2023-11-24T06:30:34.279932+0000","last_unstale":"2023-11-24T06:30:35.258716+0000","last_undegraded":"2023-11-24T06:30:35.258716+0000","last_fullsized":"2023-11-24T06:30:35.258716+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:02:53.020511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344811+0000","last_change":"2023-11-24T06:30:34.270241+0000","last_active":"2023-11-24T06:30:35.344811+0000","last_peered":"2023-11-24T06:30:35.344811+0000","last_clean":"2023-11-24T06:30:35.344811+0000","last_became_active":"2023-11-24T06:30:34.269570+0000","last_became_peered":"2023-11-24T06:30:34.269570+0000","last_unstale":"2023-11-24T06:30:35.344811+0000","last_undegraded":"2023-11-24T06:30:35.344811+0000","last_fullsized":"2023-11-24T06:30:35.344811+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:58:46.928025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346318+0000","last_change":"2023-11-24T06:30:34.276808+0000","last_active":"2023-11-24T06:30:35.346318+0000","last_peered":"2023-11-24T06:30:35.346318+0000","last_clean":"2023-11-24T06:30:35.346318+0000","last_became_active":"2023-11-24T06:30:34.275753+0000","last_became_peered":"2023-11-24T06:30:34.275753+0000","last_unstale":"2023-11-24T06:30:35.346318+0000","last_undegraded":"2023-11-24T06:30:35.346318+0000","last_fullsized":"2023-11-24T06:30:35.346318+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:51.866541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260023+0000","last_change":"2023-11-24T06:30:34.252551+0000","last_active":"2023-11-24T06:30:35.260023+0000","last_peered":"2023-11-24T06:30:35.260023+0000","last_clean":"2023-11-24T06:30:35.260023+0000","last_became_active":"2023-11-24T06:30:34.252113+0000","last_became_peered":"2023-11-24T06:30:34.252113+0000","last_unstale":"2023-11-24T06:30:35.260023+0000","last_undegraded":"2023-11-24T06:30:35.260023+0000","last_fullsized":"2023-11-24T06:30:35.260023+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:49:13.548874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345907+0000","last_change":"2023-11-24T06:30:34.274691+0000","last_active":"2023-11-24T06:30:35.345907+0000","last_peered":"2023-11-24T06:30:35.345907+0000","last_clean":"2023-11-24T06:30:35.345907+0000","last_became_active":"2023-11-24T06:30:34.274390+0000","last_became_peered":"2023-11-24T06:30:34.274390+0000","last_unstale":"2023-11-24T06:30:35.345907+0000","last_undegraded":"2023-11-24T06:30:35.345907+0000","last_fullsized":"2023-11-24T06:30:35.345907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:54.612687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260067+0000","last_change":"2023-11-24T06:30:34.261054+0000","last_active":"2023-11-24T06:30:35.260067+0000","last_peered":"2023-11-24T06:30:35.260067+0000","last_clean":"2023-11-24T06:30:35.260067+0000","last_became_active":"2023-11-24T06:30:34.260852+0000","last_became_peered":"2023-11-24T06:30:34.260852+0000","last_unstale":"2023-11-24T06:30:35.260067+0000","last_undegraded":"2023-11-24T06:30:35.260067+0000","last_fullsized":"2023-11-24T06:30:35.260067+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:26.614414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343765+0000","last_change":"2023-11-24T06:30:34.273145+0000","last_active":"2023-11-24T06:30:35.343765+0000","last_peered":"2023-11-24T06:30:35.343765+0000","last_clean":"2023-11-24T06:30:35.343765+0000","last_became_active":"2023-11-24T06:30:34.272820+0000","last_became_peered":"2023-11-24T06:30:34.272820+0000","last_unstale":"2023-11-24T06:30:35.343765+0000","last_undegraded":"2023-11-24T06:30:35.343765+0000","last_fullsized":"2023-11-24T06:30:35.343765+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:15:29.661247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343818+0000","last_change":"2023-11-24T06:30:34.261468+0000","last_active":"2023-11-24T06:30:35.343818+0000","last_peered":"2023-11-24T06:30:35.343818+0000","last_clean":"2023-11-24T06:30:35.343818+0000","last_became_active":"2023-11-24T06:30:34.261196+0000","last_became_peered":"2023-11-24T06:30:34.261196+0000","last_unstale":"2023-11-24T06:30:35.343818+0000","last_undegraded":"2023-11-24T06:30:35.343818+0000","last_fullsized":"2023-11-24T06:30:35.343818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:54:26.525502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347150+0000","last_change":"2023-11-24T06:30:34.274425+0000","last_active":"2023-11-24T06:30:35.347150+0000","last_peered":"2023-11-24T06:30:35.347150+0000","last_clean":"2023-11-24T06:30:35.347150+0000","last_became_active":"2023-11-24T06:30:34.271742+0000","last_became_peered":"2023-11-24T06:30:34.271742+0000","last_unstale":"2023-11-24T06:30:35.347150+0000","last_undegraded":"2023-11-24T06:30:35.347150+0000","last_fullsized":"2023-11-24T06:30:35.347150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:11.493185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259428+0000","last_change":"2023-11-24T06:30:34.276702+0000","last_active":"2023-11-24T06:30:35.259428+0000","last_peered":"2023-11-24T06:30:35.259428+0000","last_clean":"2023-11-24T06:30:35.259428+0000","last_became_active":"2023-11-24T06:30:34.276392+0000","last_became_peered":"2023-11-24T06:30:34.276392+0000","last_unstale":"2023-11-24T06:30:35.259428+0000","last_undegraded":"2023-11-24T06:30:35.259428+0000","last_fullsized":"2023-11-24T06:30:35.259428+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:31.302540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.276566+0000","last_change":"2023-11-24T06:30:33.258485+0000","last_active":"2023-11-24T06:30:34.276566+0000","last_peered":"2023-11-24T06:30:34.276566+0000","last_clean":"2023-11-24T06:30:34.276566+0000","last_became_active":"2023-11-24T06:30:33.258091+0000","last_became_peered":"2023-11-24T06:30:33.258091+0000","last_unstale":"2023-11-24T06:30:34.276566+0000","last_undegraded":"2023-11-24T06:30:34.276566+0000","last_fullsized":"2023-11-24T06:30:34.276566+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:34.021503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345043+0000","last_change":"2023-11-24T06:30:34.254341+0000","last_active":"2023-11-24T06:30:35.345043+0000","last_peered":"2023-11-24T06:30:35.345043+0000","last_clean":"2023-11-24T06:30:35.345043+0000","last_became_active":"2023-11-24T06:30:34.253349+0000","last_became_peered":"2023-11-24T06:30:34.253349+0000","last_unstale":"2023-11-24T06:30:35.345043+0000","last_undegraded":"2023-11-24T06:30:35.345043+0000","last_fullsized":"2023-11-24T06:30:35.345043+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:45.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.264537+0000","last_change":"2023-11-24T06:30:33.253586+0000","last_active":"2023-11-24T06:30:34.264537+0000","last_peered":"2023-11-24T06:30:34.264537+0000","last_clean":"2023-11-24T06:30:34.264537+0000","last_became_active":"2023-11-24T06:30:33.252371+0000","last_became_peered":"2023-11-24T06:30:33.252371+0000","last_unstale":"2023-11-24T06:30:34.264537+0000","last_undegraded":"2023-11-24T06:30:34.264537+0000","last_fullsized":"2023-11-24T06:30:34.264537+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:11:40.811402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347857+0000","last_change":"2023-11-24T06:30:34.279290+0000","last_active":"2023-11-24T06:30:35.347857+0000","last_peered":"2023-11-24T06:30:35.347857+0000","last_clean":"2023-11-24T06:30:35.347857+0000","last_became_active":"2023-11-24T06:30:34.278957+0000","last_became_peered":"2023-11-24T06:30:34.278957+0000","last_unstale":"2023-11-24T06:30:35.347857+0000","last_undegraded":"2023-11-24T06:30:35.347857+0000","last_fullsized":"2023-11-24T06:30:35.347857+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:18:06.613023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.281834+0000","last_change":"2023-11-24T06:30:33.266741+0000","last_active":"2023-11-24T06:30:34.281834+0000","last_peered":"2023-11-24T06:30:34.281834+0000","last_clean":"2023-11-24T06:30:34.281834+0000","last_became_active":"2023-11-24T06:30:33.266055+0000","last_became_peered":"2023-11-24T06:30:33.266055+0000","last_unstale":"2023-11-24T06:30:34.281834+0000","last_undegraded":"2023-11-24T06:30:34.281834+0000","last_fullsized":"2023-11-24T06:30:34.281834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:20.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259596+0000","last_change":"2023-11-24T06:30:34.259166+0000","last_active":"2023-11-24T06:30:35.259596+0000","last_peered":"2023-11-24T06:30:35.259596+0000","last_clean":"2023-11-24T06:30:35.259596+0000","last_became_active":"2023-11-24T06:30:34.258558+0000","last_became_peered":"2023-11-24T06:30:34.258558+0000","last_unstale":"2023-11-24T06:30:35.259596+0000","last_undegraded":"2023-11-24T06:30:35.259596+0000","last_fullsized":"2023-11-24T06:30:35.259596+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:22.171241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.278031+0000","last_change":"2023-11-24T06:30:33.260247+0000","last_active":"2023-11-24T06:30:34.278031+0000","last_peered":"2023-11-24T06:30:34.278031+0000","last_clean":"2023-11-24T06:30:34.278031+0000","last_became_active":"2023-11-24T06:30:33.259986+0000","last_became_peered":"2023-11-24T06:30:33.259986+0000","last_unstale":"2023-11-24T06:30:34.278031+0000","last_undegraded":"2023-11-24T06:30:34.278031+0000","last_fullsized":"2023-11-24T06:30:34.278031+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:59:48.718422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344557+0000","last_change":"2023-11-24T06:30:34.260492+0000","last_active":"2023-11-24T06:30:35.344557+0000","last_peered":"2023-11-24T06:30:35.344557+0000","last_clean":"2023-11-24T06:30:35.344557+0000","last_became_active":"2023-11-24T06:30:34.260161+0000","last_became_peered":"2023-11-24T06:30:34.260161+0000","last_unstale":"2023-11-24T06:30:35.344557+0000","last_undegraded":"2023-11-24T06:30:35.344557+0000","last_fullsized":"2023-11-24T06:30:35.344557+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:42:18.443208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.259370+0000","last_change":"2023-11-24T06:30:33.253225+0000","last_active":"2023-11-24T06:30:34.259370+0000","last_peered":"2023-11-24T06:30:34.259370+0000","last_clean":"2023-11-24T06:30:34.259370+0000","last_became_active":"2023-11-24T06:30:33.252940+0000","last_became_peered":"2023-11-24T06:30:33.252940+0000","last_unstale":"2023-11-24T06:30:34.259370+0000","last_undegraded":"2023-11-24T06:30:34.259370+0000","last_fullsized":"2023-11-24T06:30:34.259370+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:44.581758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345011+0000","last_change":"2023-11-24T06:30:34.276484+0000","last_active":"2023-11-24T06:30:35.345011+0000","last_peered":"2023-11-24T06:30:35.345011+0000","last_clean":"2023-11-24T06:30:35.345011+0000","last_became_active":"2023-11-24T06:30:34.275957+0000","last_became_peered":"2023-11-24T06:30:34.275957+0000","last_unstale":"2023-11-24T06:30:35.345011+0000","last_undegraded":"2023-11-24T06:30:35.345011+0000","last_fullsized":"2023-11-24T06:30:35.345011+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:30.742091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.273608+0000","last_change":"2023-11-24T06:30:33.263099+0000","last_active":"2023-11-24T06:30:34.273608+0000","last_peered":"2023-11-24T06:30:34.273608+0000","last_clean":"2023-11-24T06:30:34.273608+0000","last_became_active":"2023-11-24T06:30:33.261288+0000","last_became_peered":"2023-11-24T06:30:33.261288+0000","last_unstale":"2023-11-24T06:30:34.273608+0000","last_undegraded":"2023-11-24T06:30:34.273608+0000","last_fullsized":"2023-11-24T06:30:34.273608+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:49.881712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343158+0000","last_change":"2023-11-24T06:30:34.268549+0000","last_active":"2023-11-24T06:30:35.343158+0000","last_peered":"2023-11-24T06:30:35.343158+0000","last_clean":"2023-11-24T06:30:35.343158+0000","last_became_active":"2023-11-24T06:30:34.268031+0000","last_became_peered":"2023-11-24T06:30:34.268031+0000","last_unstale":"2023-11-24T06:30:35.343158+0000","last_undegraded":"2023-11-24T06:30:35.343158+0000","last_fullsized":"2023-11-24T06:30:35.343158+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:27.880252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.233442+0000","last_change":"2023-11-24T06:30:33.250035+0000","last_active":"2023-11-24T06:30:34.233442+0000","last_peered":"2023-11-24T06:30:34.233442+0000","last_clean":"2023-11-24T06:30:34.233442+0000","last_became_active":"2023-11-24T06:30:33.249747+0000","last_became_peered":"2023-11-24T06:30:33.249747+0000","last_unstale":"2023-11-24T06:30:34.233442+0000","last_undegraded":"2023-11-24T06:30:34.233442+0000","last_fullsized":"2023-11-24T06:30:34.233442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:30.859810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343000+0000","last_change":"2023-11-24T06:30:34.261260+0000","last_active":"2023-11-24T06:30:35.343000+0000","last_peered":"2023-11-24T06:30:35.343000+0000","last_clean":"2023-11-24T06:30:35.343000+0000","last_became_active":"2023-11-24T06:30:34.260987+0000","last_became_peered":"2023-11-24T06:30:34.260987+0000","last_unstale":"2023-11-24T06:30:35.343000+0000","last_undegraded":"2023-11-24T06:30:35.343000+0000","last_fullsized":"2023-11-24T06:30:35.343000+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:31.302540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.243672+0000","last_change":"2023-11-24T06:30:33.240278+0000","last_active":"2023-11-24T06:30:34.243672+0000","last_peered":"2023-11-24T06:30:34.243672+0000","last_clean":"2023-11-24T06:30:34.243672+0000","last_became_active":"2023-11-24T06:30:33.238103+0000","last_became_peered":"2023-11-24T06:30:33.238103+0000","last_unstale":"2023-11-24T06:30:34.243672+0000","last_undegraded":"2023-11-24T06:30:34.243672+0000","last_fullsized":"2023-11-24T06:30:34.243672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:22.469854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346408+0000","last_change":"2023-11-24T06:30:34.261721+0000","last_active":"2023-11-24T06:30:35.346408+0000","last_peered":"2023-11-24T06:30:35.346408+0000","last_clean":"2023-11-24T06:30:35.346408+0000","last_became_active":"2023-11-24T06:30:34.261436+0000","last_became_peered":"2023-11-24T06:30:34.261436+0000","last_unstale":"2023-11-24T06:30:35.346408+0000","last_undegraded":"2023-11-24T06:30:35.346408+0000","last_fullsized":"2023-11-24T06:30:35.346408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:15.717290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.257406+0000","last_change":"2023-11-24T06:30:33.257372+0000","last_active":"2023-11-24T06:30:34.257406+0000","last_peered":"2023-11-24T06:30:34.257406+0000","last_clean":"2023-11-24T06:30:34.257406+0000","last_became_active":"2023-11-24T06:30:33.257063+0000","last_became_peered":"2023-11-24T06:30:33.257063+0000","last_unstale":"2023-11-24T06:30:34.257406+0000","last_undegraded":"2023-11-24T06:30:34.257406+0000","last_fullsized":"2023-11-24T06:30:34.257406+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:56.795055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259851+0000","last_change":"2023-11-24T06:30:34.281867+0000","last_active":"2023-11-24T06:30:35.259851+0000","last_peered":"2023-11-24T06:30:35.259851+0000","last_clean":"2023-11-24T06:30:35.259851+0000","last_became_active":"2023-11-24T06:30:34.280414+0000","last_became_peered":"2023-11-24T06:30:34.280414+0000","last_unstale":"2023-11-24T06:30:35.259851+0000","last_undegraded":"2023-11-24T06:30:35.259851+0000","last_fullsized":"2023-11-24T06:30:35.259851+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:42.434683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.269098+0000","last_change":"2023-11-24T06:30:33.266155+0000","last_active":"2023-11-24T06:30:34.269098+0000","last_peered":"2023-11-24T06:30:34.269098+0000","last_clean":"2023-11-24T06:30:34.269098+0000","last_became_active":"2023-11-24T06:30:33.265258+0000","last_became_peered":"2023-11-24T06:30:33.265258+0000","last_unstale":"2023-11-24T06:30:34.269098+0000","last_undegraded":"2023-11-24T06:30:34.269098+0000","last_fullsized":"2023-11-24T06:30:34.269098+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:20.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347962+0000","last_change":"2023-11-24T06:30:34.275278+0000","last_active":"2023-11-24T06:30:35.347962+0000","last_peered":"2023-11-24T06:30:35.347962+0000","last_clean":"2023-11-24T06:30:35.347962+0000","last_became_active":"2023-11-24T06:30:34.274913+0000","last_became_peered":"2023-11-24T06:30:34.274913+0000","last_unstale":"2023-11-24T06:30:35.347962+0000","last_undegraded":"2023-11-24T06:30:35.347962+0000","last_fullsized":"2023-11-24T06:30:35.347962+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:53.474598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.266907+0000","last_change":"2023-11-24T06:30:33.264658+0000","last_active":"2023-11-24T06:30:34.266907+0000","last_peered":"2023-11-24T06:30:34.266907+0000","last_clean":"2023-11-24T06:30:34.266907+0000","last_became_active":"2023-11-24T06:30:33.264320+0000","last_became_peered":"2023-11-24T06:30:33.264320+0000","last_unstale":"2023-11-24T06:30:34.266907+0000","last_undegraded":"2023-11-24T06:30:34.266907+0000","last_fullsized":"2023-11-24T06:30:34.266907+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:34.652219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343595+0000","last_change":"2023-11-24T06:30:34.255933+0000","last_active":"2023-11-24T06:30:35.343595+0000","last_peered":"2023-11-24T06:30:35.343595+0000","last_clean":"2023-11-24T06:30:35.343595+0000","last_became_active":"2023-11-24T06:30:34.255686+0000","last_became_peered":"2023-11-24T06:30:34.255686+0000","last_unstale":"2023-11-24T06:30:35.343595+0000","last_undegraded":"2023-11-24T06:30:35.343595+0000","last_fullsized":"2023-11-24T06:30:35.343595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:13.387080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.270954+0000","last_change":"2023-11-24T06:30:33.239974+0000","last_active":"2023-11-24T06:30:34.270954+0000","last_peered":"2023-11-24T06:30:34.270954+0000","last_clean":"2023-11-24T06:30:34.270954+0000","last_became_active":"2023-11-24T06:30:33.238273+0000","last_became_peered":"2023-11-24T06:30:33.238273+0000","last_unstale":"2023-11-24T06:30:34.270954+0000","last_undegraded":"2023-11-24T06:30:34.270954+0000","last_fullsized":"2023-11-24T06:30:34.270954+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:46.553265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.258671+0000","last_change":"2023-11-24T06:30:34.254999+0000","last_active":"2023-11-24T06:30:35.258671+0000","last_peered":"2023-11-24T06:30:35.258671+0000","last_clean":"2023-11-24T06:30:35.258671+0000","last_became_active":"2023-11-24T06:30:34.254110+0000","last_became_peered":"2023-11-24T06:30:34.254110+0000","last_unstale":"2023-11-24T06:30:35.258671+0000","last_undegraded":"2023-11-24T06:30:35.258671+0000","last_fullsized":"2023-11-24T06:30:35.258671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:11:41.823369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.280676+0000","last_change":"2023-11-24T06:30:33.252044+0000","last_active":"2023-11-24T06:30:34.280676+0000","last_peered":"2023-11-24T06:30:34.280676+0000","last_clean":"2023-11-24T06:30:34.280676+0000","last_became_active":"2023-11-24T06:30:33.251774+0000","last_became_peered":"2023-11-24T06:30:33.251774+0000","last_unstale":"2023-11-24T06:30:34.280676+0000","last_undegraded":"2023-11-24T06:30:34.280676+0000","last_fullsized":"2023-11-24T06:30:34.280676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:40:44.399558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345848+0000","last_change":"2023-11-24T06:30:34.262809+0000","last_active":"2023-11-24T06:30:35.345848+0000","last_peered":"2023-11-24T06:30:35.345848+0000","last_clean":"2023-11-24T06:30:35.345848+0000","last_became_active":"2023-11-24T06:30:34.261525+0000","last_became_peered":"2023-11-24T06:30:34.261525+0000","last_unstale":"2023-11-24T06:30:35.345848+0000","last_undegraded":"2023-11-24T06:30:35.345848+0000","last_fullsized":"2023-11-24T06:30:35.345848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:41.655269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.266966+0000","last_change":"2023-11-24T06:30:33.261613+0000","last_active":"2023-11-24T06:30:34.266966+0000","last_peered":"2023-11-24T06:30:34.266966+0000","last_clean":"2023-11-24T06:30:34.266966+0000","last_became_active":"2023-11-24T06:30:33.260523+0000","last_became_peered":"2023-11-24T06:30:33.260523+0000","last_unstale":"2023-11-24T06:30:34.266966+0000","last_undegraded":"2023-11-24T06:30:34.266966+0000","last_fullsized":"2023-11-24T06:30:34.266966+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:58.846905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.342583+0000","last_change":"2023-11-24T06:30:34.278490+0000","last_active":"2023-11-24T06:30:35.342583+0000","last_peered":"2023-11-24T06:30:35.342583+0000","last_clean":"2023-11-24T06:30:35.342583+0000","last_became_active":"2023-11-24T06:30:34.278203+0000","last_became_peered":"2023-11-24T06:30:34.278203+0000","last_unstale":"2023-11-24T06:30:35.342583+0000","last_undegraded":"2023-11-24T06:30:35.342583+0000","last_fullsized":"2023-11-24T06:30:35.342583+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:02:53.020511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.250622+0000","last_change":"2023-11-24T06:30:33.263590+0000","last_active":"2023-11-24T06:30:34.250622+0000","last_peered":"2023-11-24T06:30:34.250622+0000","last_clean":"2023-11-24T06:30:34.250622+0000","last_became_active":"2023-11-24T06:30:33.262788+0000","last_became_peered":"2023-11-24T06:30:33.262788+0000","last_unstale":"2023-11-24T06:30:34.250622+0000","last_undegraded":"2023-11-24T06:30:34.250622+0000","last_fullsized":"2023-11-24T06:30:34.250622+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:08.963743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343441+0000","last_change":"2023-11-24T06:30:34.257935+0000","last_active":"2023-11-24T06:30:35.343441+0000","last_peered":"2023-11-24T06:30:35.343441+0000","last_clean":"2023-11-24T06:30:35.343441+0000","last_became_active":"2023-11-24T06:30:34.253623+0000","last_became_peered":"2023-11-24T06:30:34.253623+0000","last_unstale":"2023-11-24T06:30:35.343441+0000","last_undegraded":"2023-11-24T06:30:35.343441+0000","last_fullsized":"2023-11-24T06:30:35.343441+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:26.542532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.279648+0000","last_change":"2023-11-24T06:30:33.259288+0000","last_active":"2023-11-24T06:30:34.279648+0000","last_peered":"2023-11-24T06:30:34.279648+0000","last_clean":"2023-11-24T06:30:34.279648+0000","last_became_active":"2023-11-24T06:30:33.254396+0000","last_became_peered":"2023-11-24T06:30:33.254396+0000","last_unstale":"2023-11-24T06:30:34.279648+0000","last_undegraded":"2023-11-24T06:30:34.279648+0000","last_fullsized":"2023-11-24T06:30:34.279648+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:59.955909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.345179+0000","last_change":"2023-11-24T06:30:34.276782+0000","last_active":"2023-11-24T06:30:35.345179+0000","last_peered":"2023-11-24T06:30:35.345179+0000","last_clean":"2023-11-24T06:30:35.345179+0000","last_became_active":"2023-11-24T06:30:34.268740+0000","last_became_peered":"2023-11-24T06:30:34.268740+0000","last_unstale":"2023-11-24T06:30:35.345179+0000","last_undegraded":"2023-11-24T06:30:35.345179+0000","last_fullsized":"2023-11-24T06:30:35.345179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:21:53.474598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.283976+0000","last_change":"2023-11-24T06:30:33.265823+0000","last_active":"2023-11-24T06:30:34.283976+0000","last_peered":"2023-11-24T06:30:34.283976+0000","last_clean":"2023-11-24T06:30:34.283976+0000","last_became_active":"2023-11-24T06:30:33.265509+0000","last_became_peered":"2023-11-24T06:30:33.265509+0000","last_unstale":"2023-11-24T06:30:34.283976+0000","last_undegraded":"2023-11-24T06:30:34.283976+0000","last_fullsized":"2023-11-24T06:30:34.283976+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:38.989068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260089+0000","last_change":"2023-11-24T06:30:34.252661+0000","last_active":"2023-11-24T06:30:35.260089+0000","last_peered":"2023-11-24T06:30:35.260089+0000","last_clean":"2023-11-24T06:30:35.260089+0000","last_became_active":"2023-11-24T06:30:34.251657+0000","last_became_peered":"2023-11-24T06:30:34.251657+0000","last_unstale":"2023-11-24T06:30:35.260089+0000","last_undegraded":"2023-11-24T06:30:35.260089+0000","last_fullsized":"2023-11-24T06:30:35.260089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:15:26.105937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.280390+0000","last_change":"2023-11-24T06:30:33.268072+0000","last_active":"2023-11-24T06:30:34.280390+0000","last_peered":"2023-11-24T06:30:34.280390+0000","last_clean":"2023-11-24T06:30:34.280390+0000","last_became_active":"2023-11-24T06:30:33.267757+0000","last_became_peered":"2023-11-24T06:30:33.267757+0000","last_unstale":"2023-11-24T06:30:34.280390+0000","last_undegraded":"2023-11-24T06:30:34.280390+0000","last_fullsized":"2023-11-24T06:30:34.280390+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:17.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256567+0000","last_change":"2023-11-24T06:30:34.254912+0000","last_active":"2023-11-24T06:30:35.256567+0000","last_peered":"2023-11-24T06:30:35.256567+0000","last_clean":"2023-11-24T06:30:35.256567+0000","last_became_active":"2023-11-24T06:30:34.249336+0000","last_became_peered":"2023-11-24T06:30:34.249336+0000","last_unstale":"2023-11-24T06:30:35.256567+0000","last_undegraded":"2023-11-24T06:30:35.256567+0000","last_fullsized":"2023-11-24T06:30:35.256567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:53:07.851979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.282975+0000","last_change":"2023-11-24T06:30:33.265903+0000","last_active":"2023-11-24T06:30:34.282975+0000","last_peered":"2023-11-24T06:30:34.282975+0000","last_clean":"2023-11-24T06:30:34.282975+0000","last_became_active":"2023-11-24T06:30:33.263286+0000","last_became_peered":"2023-11-24T06:30:33.263286+0000","last_unstale":"2023-11-24T06:30:34.282975+0000","last_undegraded":"2023-11-24T06:30:34.282975+0000","last_fullsized":"2023-11-24T06:30:34.282975+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:56:56.795055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.343942+0000","last_change":"2023-11-24T06:30:34.262996+0000","last_active":"2023-11-24T06:30:35.343942+0000","last_peered":"2023-11-24T06:30:35.343942+0000","last_clean":"2023-11-24T06:30:35.343942+0000","last_became_active":"2023-11-24T06:30:34.262112+0000","last_became_peered":"2023-11-24T06:30:34.262112+0000","last_unstale":"2023-11-24T06:30:35.343942+0000","last_undegraded":"2023-11-24T06:30:35.343942+0000","last_fullsized":"2023-11-24T06:30:35.343942+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:59:28.331014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.239801+0000","last_change":"2023-11-24T06:30:33.259236+0000","last_active":"2023-11-24T06:30:34.239801+0000","last_peered":"2023-11-24T06:30:34.239801+0000","last_clean":"2023-11-24T06:30:34.239801+0000","last_became_active":"2023-11-24T06:30:33.258200+0000","last_became_peered":"2023-11-24T06:30:33.258200+0000","last_unstale":"2023-11-24T06:30:34.239801+0000","last_undegraded":"2023-11-24T06:30:34.239801+0000","last_fullsized":"2023-11-24T06:30:34.239801+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:42.462023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260075+0000","last_change":"2023-11-24T06:30:34.249019+0000","last_active":"2023-11-24T06:30:35.260075+0000","last_peered":"2023-11-24T06:30:35.260075+0000","last_clean":"2023-11-24T06:30:35.260075+0000","last_became_active":"2023-11-24T06:30:34.248779+0000","last_became_peered":"2023-11-24T06:30:34.248779+0000","last_unstale":"2023-11-24T06:30:35.260075+0000","last_undegraded":"2023-11-24T06:30:35.260075+0000","last_fullsized":"2023-11-24T06:30:35.260075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:50.735632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.267095+0000","last_change":"2023-11-24T06:30:33.259814+0000","last_active":"2023-11-24T06:30:34.267095+0000","last_peered":"2023-11-24T06:30:34.267095+0000","last_clean":"2023-11-24T06:30:34.267095+0000","last_became_active":"2023-11-24T06:30:33.259551+0000","last_became_peered":"2023-11-24T06:30:33.259551+0000","last_unstale":"2023-11-24T06:30:34.267095+0000","last_undegraded":"2023-11-24T06:30:34.267095+0000","last_fullsized":"2023-11-24T06:30:34.267095+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:59.955909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259713+0000","last_change":"2023-11-24T06:30:34.278491+0000","last_active":"2023-11-24T06:30:35.259713+0000","last_peered":"2023-11-24T06:30:35.259713+0000","last_clean":"2023-11-24T06:30:35.259713+0000","last_became_active":"2023-11-24T06:30:34.278188+0000","last_became_peered":"2023-11-24T06:30:34.278188+0000","last_unstale":"2023-11-24T06:30:35.259713+0000","last_undegraded":"2023-11-24T06:30:35.259713+0000","last_fullsized":"2023-11-24T06:30:35.259713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:31:46.777167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.250972+0000","last_change":"2023-11-24T06:30:33.263043+0000","last_active":"2023-11-24T06:30:34.250972+0000","last_peered":"2023-11-24T06:30:34.250972+0000","last_clean":"2023-11-24T06:30:34.250972+0000","last_became_active":"2023-11-24T06:30:33.262727+0000","last_became_peered":"2023-11-24T06:30:33.262727+0000","last_unstale":"2023-11-24T06:30:34.250972+0000","last_undegraded":"2023-11-24T06:30:34.250972+0000","last_fullsized":"2023-11-24T06:30:34.250972+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:39.081828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260056+0000","last_change":"2023-11-24T06:30:34.278437+0000","last_active":"2023-11-24T06:30:35.260056+0000","last_peered":"2023-11-24T06:30:35.260056+0000","last_clean":"2023-11-24T06:30:35.260056+0000","last_became_active":"2023-11-24T06:30:34.278058+0000","last_became_peered":"2023-11-24T06:30:34.278058+0000","last_unstale":"2023-11-24T06:30:35.260056+0000","last_undegraded":"2023-11-24T06:30:35.260056+0000","last_fullsized":"2023-11-24T06:30:35.260056+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:05.432552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.250391+0000","last_change":"2023-11-24T06:30:33.259539+0000","last_active":"2023-11-24T06:30:34.250391+0000","last_peered":"2023-11-24T06:30:34.250391+0000","last_clean":"2023-11-24T06:30:34.250391+0000","last_became_active":"2023-11-24T06:30:33.258994+0000","last_became_peered":"2023-11-24T06:30:33.258994+0000","last_unstale":"2023-11-24T06:30:34.250391+0000","last_undegraded":"2023-11-24T06:30:34.250391+0000","last_fullsized":"2023-11-24T06:30:34.250391+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:41:34.021503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344664+0000","last_change":"2023-11-24T06:30:34.265135+0000","last_active":"2023-11-24T06:30:35.344664+0000","last_peered":"2023-11-24T06:30:35.344664+0000","last_clean":"2023-11-24T06:30:35.344664+0000","last_became_active":"2023-11-24T06:30:34.264865+0000","last_became_peered":"2023-11-24T06:30:34.264865+0000","last_unstale":"2023-11-24T06:30:35.344664+0000","last_undegraded":"2023-11-24T06:30:35.344664+0000","last_fullsized":"2023-11-24T06:30:35.344664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:22.171241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.284256+0000","last_change":"2023-11-24T06:30:33.264626+0000","last_active":"2023-11-24T06:30:34.284256+0000","last_peered":"2023-11-24T06:30:34.284256+0000","last_clean":"2023-11-24T06:30:34.284256+0000","last_became_active":"2023-11-24T06:30:33.264323+0000","last_became_peered":"2023-11-24T06:30:33.264323+0000","last_unstale":"2023-11-24T06:30:34.284256+0000","last_undegraded":"2023-11-24T06:30:34.284256+0000","last_fullsized":"2023-11-24T06:30:34.284256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:17.149862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346315+0000","last_change":"2023-11-24T06:30:34.265751+0000","last_active":"2023-11-24T06:30:35.346315+0000","last_peered":"2023-11-24T06:30:35.346315+0000","last_clean":"2023-11-24T06:30:35.346315+0000","last_became_active":"2023-11-24T06:30:34.264900+0000","last_became_peered":"2023-11-24T06:30:34.264900+0000","last_unstale":"2023-11-24T06:30:35.346315+0000","last_undegraded":"2023-11-24T06:30:35.346315+0000","last_fullsized":"2023-11-24T06:30:35.346315+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:01.166325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346631+0000","last_change":"2023-11-24T06:30:33.262138+0000","last_active":"2023-11-24T06:30:35.346631+0000","last_peered":"2023-11-24T06:30:35.346631+0000","last_clean":"2023-11-24T06:30:35.346631+0000","last_became_active":"2023-11-24T06:30:33.261687+0000","last_became_peered":"2023-11-24T06:30:33.261687+0000","last_unstale":"2023-11-24T06:30:35.346631+0000","last_undegraded":"2023-11-24T06:30:35.346631+0000","last_fullsized":"2023-11-24T06:30:35.346631+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:06:42.741740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259672+0000","last_change":"2023-11-24T06:30:34.281505+0000","last_active":"2023-11-24T06:30:35.259672+0000","last_peered":"2023-11-24T06:30:35.259672+0000","last_clean":"2023-11-24T06:30:35.259672+0000","last_became_active":"2023-11-24T06:30:34.280285+0000","last_became_peered":"2023-11-24T06:30:34.280285+0000","last_unstale":"2023-11-24T06:30:35.259672+0000","last_undegraded":"2023-11-24T06:30:35.259672+0000","last_fullsized":"2023-11-24T06:30:35.259672+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:04.704406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.243962+0000","last_change":"2023-11-24T06:30:33.255156+0000","last_active":"2023-11-24T06:30:34.243962+0000","last_peered":"2023-11-24T06:30:34.243962+0000","last_clean":"2023-11-24T06:30:34.243962+0000","last_became_active":"2023-11-24T06:30:33.254878+0000","last_became_peered":"2023-11-24T06:30:33.254878+0000","last_unstale":"2023-11-24T06:30:34.243962+0000","last_undegraded":"2023-11-24T06:30:34.243962+0000","last_fullsized":"2023-11-24T06:30:34.243962+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:17.577767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.259976+0000","last_change":"2023-11-24T06:30:34.267501+0000","last_active":"2023-11-24T06:30:35.259976+0000","last_peered":"2023-11-24T06:30:35.259976+0000","last_clean":"2023-11-24T06:30:35.259976+0000","last_became_active":"2023-11-24T06:30:34.267199+0000","last_became_peered":"2023-11-24T06:30:34.267199+0000","last_unstale":"2023-11-24T06:30:35.259976+0000","last_undegraded":"2023-11-24T06:30:35.259976+0000","last_fullsized":"2023-11-24T06:30:35.259976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:37.004815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.250399+0000","last_change":"2023-11-24T06:30:33.263337+0000","last_active":"2023-11-24T06:30:34.250399+0000","last_peered":"2023-11-24T06:30:34.250399+0000","last_clean":"2023-11-24T06:30:34.250399+0000","last_became_active":"2023-11-24T06:30:33.262997+0000","last_became_peered":"2023-11-24T06:30:33.262997+0000","last_unstale":"2023-11-24T06:30:34.250399+0000","last_undegraded":"2023-11-24T06:30:34.250399+0000","last_fullsized":"2023-11-24T06:30:34.250399+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:31:07.057889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.344460+0000","last_change":"2023-11-24T06:30:34.250892+0000","last_active":"2023-11-24T06:30:35.344460+0000","last_peered":"2023-11-24T06:30:35.344460+0000","last_clean":"2023-11-24T06:30:35.344460+0000","last_became_active":"2023-11-24T06:30:34.250633+0000","last_became_peered":"2023-11-24T06:30:34.250633+0000","last_unstale":"2023-11-24T06:30:35.344460+0000","last_undegraded":"2023-11-24T06:30:35.344460+0000","last_fullsized":"2023-11-24T06:30:35.344460+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:31.871778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.243675+0000","last_change":"2023-11-24T06:30:33.258636+0000","last_active":"2023-11-24T06:30:34.243675+0000","last_peered":"2023-11-24T06:30:34.243675+0000","last_clean":"2023-11-24T06:30:34.243675+0000","last_became_active":"2023-11-24T06:30:33.258193+0000","last_became_peered":"2023-11-24T06:30:33.258193+0000","last_unstale":"2023-11-24T06:30:34.243675+0000","last_undegraded":"2023-11-24T06:30:34.243675+0000","last_fullsized":"2023-11-24T06:30:34.243675+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:52.553976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.347590+0000","last_change":"2023-11-24T06:30:34.281782+0000","last_active":"2023-11-24T06:30:35.347590+0000","last_peered":"2023-11-24T06:30:35.347590+0000","last_clean":"2023-11-24T06:30:35.347590+0000","last_became_active":"2023-11-24T06:30:34.278975+0000","last_became_peered":"2023-11-24T06:30:34.278975+0000","last_unstale":"2023-11-24T06:30:35.347590+0000","last_undegraded":"2023-11-24T06:30:35.347590+0000","last_fullsized":"2023-11-24T06:30:35.347590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:30.742091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.234122+0000","last_change":"2023-11-24T06:30:33.253808+0000","last_active":"2023-11-24T06:30:34.234122+0000","last_peered":"2023-11-24T06:30:34.234122+0000","last_clean":"2023-11-24T06:30:34.234122+0000","last_became_active":"2023-11-24T06:30:33.250888+0000","last_became_peered":"2023-11-24T06:30:33.250888+0000","last_unstale":"2023-11-24T06:30:34.234122+0000","last_undegraded":"2023-11-24T06:30:34.234122+0000","last_fullsized":"2023-11-24T06:30:34.234122+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:10:31.950626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.256053+0000","last_change":"2023-11-24T06:30:34.250635+0000","last_active":"2023-11-24T06:30:35.256053+0000","last_peered":"2023-11-24T06:30:35.256053+0000","last_clean":"2023-11-24T06:30:35.256053+0000","last_became_active":"2023-11-24T06:30:34.248570+0000","last_became_peered":"2023-11-24T06:30:34.248570+0000","last_unstale":"2023-11-24T06:30:35.256053+0000","last_undegraded":"2023-11-24T06:30:35.256053+0000","last_fullsized":"2023-11-24T06:30:35.256053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:19.642863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.257857+0000","last_change":"2023-11-24T06:30:33.257088+0000","last_active":"2023-11-24T06:30:34.257857+0000","last_peered":"2023-11-24T06:30:34.257857+0000","last_clean":"2023-11-24T06:30:34.257857+0000","last_became_active":"2023-11-24T06:30:33.256733+0000","last_became_peered":"2023-11-24T06:30:33.256733+0000","last_unstale":"2023-11-24T06:30:34.257857+0000","last_undegraded":"2023-11-24T06:30:34.257857+0000","last_fullsized":"2023-11-24T06:30:34.257857+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:00:38.989068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.346922+0000","last_change":"2023-11-24T06:30:34.277209+0000","last_active":"2023-11-24T06:30:35.346922+0000","last_peered":"2023-11-24T06:30:35.346922+0000","last_clean":"2023-11-24T06:30:35.346922+0000","last_became_active":"2023-11-24T06:30:34.274316+0000","last_became_peered":"2023-11-24T06:30:34.274316+0000","last_unstale":"2023-11-24T06:30:35.346922+0000","last_undegraded":"2023-11-24T06:30:35.346922+0000","last_fullsized":"2023-11-24T06:30:35.346922+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:34.670131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.266603+0000","last_change":"2023-11-24T06:30:33.265710+0000","last_active":"2023-11-24T06:30:34.266603+0000","last_peered":"2023-11-24T06:30:34.266603+0000","last_clean":"2023-11-24T06:30:34.266603+0000","last_became_active":"2023-11-24T06:30:33.265397+0000","last_became_peered":"2023-11-24T06:30:33.265397+0000","last_unstale":"2023-11-24T06:30:34.266603+0000","last_undegraded":"2023-11-24T06:30:34.266603+0000","last_fullsized":"2023-11-24T06:30:34.266603+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:06.261210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:30:35.260070+0000","last_change":"2023-11-24T06:30:34.274805+0000","last_active":"2023-11-24T06:30:35.260070+0000","last_peered":"2023-11-24T06:30:35.260070+0000","last_clean":"2023-11-24T06:30:35.260070+0000","last_became_active":"2023-11-24T06:30:34.273993+0000","last_became_peered":"2023-11-24T06:30:34.273993+0000","last_unstale":"2023-11-24T06:30:35.260070+0000","last_undegraded":"2023-11-24T06:30:35.260070+0000","last_fullsized":"2023-11-24T06:30:35.260070+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:33.211156+0000","last_clean_scrub_stamp":"2023-11-24T06:30:33.211156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:51:59.186566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.278231+0000","last_change":"2023-11-24T06:30:33.260898+0000","last_active":"2023-11-24T06:30:34.278231+0000","last_peered":"2023-11-24T06:30:34.278231+0000","last_clean":"2023-11-24T06:30:34.278231+0000","last_became_active":"2023-11-24T06:30:33.260528+0000","last_became_peered":"2023-11-24T06:30:33.260528+0000","last_unstale":"2023-11-24T06:30:34.278231+0000","last_undegraded":"2023-11-24T06:30:34.278231+0000","last_fullsized":"2023-11-24T06:30:34.278231+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:30:32.199189+0000","last_clean_scrub_stamp":"2023-11-24T06:30:32.199189+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:00.101462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":217,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T06:30:34.236920+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:30:34.236920+0000","last_peered":"2023-11-24T06:30:34.236920+0000","last_clean":"2023-11-24T06:30:34.236920+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:30:34.236920+0000","last_undegraded":"2023-11-24T06:30:34.236920+0000","last_fullsized":"2023-11-24T06:30:34.236920+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1466368,"data_stored":956101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.252,"15min":1.3160000000000001},"min":{"1min":0.68100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":4.0599999999999996,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.296,"15min":1.3009999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":4.556,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.6539999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0249999999999999,"15min":1.012},"min":{"1min":0.68200000000000005,"5min":0.629,"15min":0.629},"max":{"1min":2.456,"5min":2.456,"15min":2.456},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.343,"15min":1.377},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":3.944,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.754}]},{"osd":2,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.2729999999999999,"15min":1.288},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":5.4589999999999996,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.84799999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.6060000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.3859999999999999,"15min":1.4099999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.3539999999999992,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.208,"15min":1.204},"min":{"1min":0.79300000000000004,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.8200000000000001,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.339}]},{"osd":4,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.4870000000000001,"15min":1.5669999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.1059999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.498,"5min":1.5369999999999999,"15min":1.579},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.7110000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.3580000000000001,"15min":1.399},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.79},{"interface":"front","average":{"1min":2.3119999999999998,"5min":1.8240000000000001,"15min":1.794},"min":{"1min":0.69799999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":17.582999999999998,"5min":17.582999999999998,"15min":17.582999999999998},"last":2.1419999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.736,"15min":1.8480000000000001},"min":{"1min":0.73099999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.8140000000000001,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":2.0419999999999998},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.6240000000000001,"15min":1.694},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":6.2210000000000001,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.851}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":365209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0980000000000001,"15min":1.1020000000000001},"min":{"1min":0.73499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7390000000000001,"5min":1.829,"15min":1.829},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.155,"5min":1.1180000000000001,"15min":1.1140000000000001},"min":{"1min":0.76800000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.623,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.1839999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.2210000000000001,"15min":1.2130000000000001},"min":{"1min":0.71099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6299999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.2010000000000001,"15min":1.2070000000000001},"min":{"1min":0.88900000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.494,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4850000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.159},"min":{"1min":0.75700000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.609,"5min":3.597,"15min":3.597},"last":1.123},{"interface":"front","average":{"1min":1.304,"5min":1.292,"15min":1.3140000000000001},"min":{"1min":0.74299999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6359999999999999,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.837}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3129999999999999,"15min":1.3180000000000001},"min":{"1min":0.78100000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5620000000000001,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.583},{"interface":"front","average":{"1min":1.25,"5min":1.2969999999999999,"15min":1.325},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.5960000000000001,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.0129999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.399,"15min":1.4379999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.8919999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":2.0880000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.385,"15min":1.397},"min":{"1min":0.79600000000000004,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7729999999999999,"5min":3.04,"15min":3.04},"last":1.7609999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3720000000000001,"15min":1.377},"min":{"1min":0.97699999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7330000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.272},{"interface":"front","average":{"1min":1.423,"5min":1.552,"15min":1.6020000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.851,"5min":3.774,"15min":3.774},"last":1.9630000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5449999999999999,"15min":1.5589999999999999},"min":{"1min":1.0029999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7689999999999999,"5min":3.403,"15min":3.403},"last":2.0409999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4990000000000001,"15min":1.512},"min":{"1min":0.875,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.792,"5min":3.649,"15min":3.649},"last":1.431}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":365207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.163,"15min":1.1850000000000001},"min":{"1min":0.72299999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.516,"5min":1.903,"15min":1.903},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.288,"15min":1.3140000000000001},"min":{"1min":0.91700000000000004,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.915,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.3400000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.145,"15min":1.1399999999999999},"min":{"1min":0.78900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.877},{"interface":"front","average":{"1min":1.327,"5min":1.147,"15min":1.1140000000000001},"min":{"1min":0.89500000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.119}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8640000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3109999999999999,"15min":1.2969999999999999},"min":{"1min":0.99199999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.1669999999999998,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.1830000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3360000000000001,"15min":1.375},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.952,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.448},{"interface":"front","average":{"1min":1.411,"5min":1.3819999999999999,"15min":1.3899999999999999},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.3069999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3180000000000001,"15min":1.3340000000000001},"min":{"1min":0.878,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.839,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.5029999999999999,"15min":1.5369999999999999},"min":{"1min":0.82299999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0099999999999998,"5min":3.085,"15min":3.085},"last":1.5149999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4690000000000001,"15min":1.4450000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.649},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.474,"15min":1.468},"min":{"1min":0.66400000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0270000000000001,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.7130000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.651,"5min":1.6140000000000001,"15min":1.6240000000000001},"min":{"1min":1.046,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.141,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.675},{"interface":"front","average":{"1min":1.714,"5min":1.6919999999999999,"15min":1.716},"min":{"1min":1.0129999999999999,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.3159999999999998,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.5549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":365187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.073,"15min":1.073},"min":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5920000000000001,"5min":1.982,"15min":1.982},"last":1.298},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.089,"15min":1.1120000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.468,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.0820000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.147,"15min":1.113},"min":{"1min":0.84199999999999997,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.26,"15min":1.3049999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5840000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.4930000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.8129999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.3140000000000001,"15min":1.3320000000000001},"min":{"1min":0.79800000000000004,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7250000000000001,"5min":2.254,"15min":2.254},"last":1.552}]},{"osd":3,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.294,"15min":1.333},"min":{"1min":0.76100000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.833,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.3779999999999999,"15min":1.3720000000000001},"min":{"1min":0.753,"5min":0.746,"15min":0.746},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4570000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.482,"15min":1.5149999999999999},"min":{"1min":0.874,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8109999999999999,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.421},{"interface":"front","average":{"1min":1.389,"5min":1.522,"15min":1.5609999999999999},"min":{"1min":0.85799999999999998,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.923,"5min":3.52,"15min":3.52},"last":1.633}]},{"osd":6,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.474,"15min":1.482},"min":{"1min":0.79200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6950000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.542,"15min":1.5740000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.859,"5min":2.452,"15min":2.452},"last":1.6060000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.595,"15min":1.6559999999999999},"min":{"1min":0.64000000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.7729999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.5740000000000001,"5min":1.615,"15min":1.613},"min":{"1min":0.84799999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.0099999999999998,"5min":2.363,"15min":2.363},"last":1.579}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":365229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.42799999999999999,"5min":0.378,"15min":0.378},"max":{"1min":1.3080000000000001,"5min":1.355,"15min":1.355},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.94499999999999995,"15min":0.95999999999999996},"min":{"1min":0.66600000000000004,"5min":0.372,"15min":0.372},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.94999999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.96399999999999997},"min":{"1min":0.51800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5600000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.077,"5min":1.091,"15min":1.113},"min":{"1min":0.61499999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0580000000000001,"15min":1.0740000000000001},"min":{"1min":0.495,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.0350000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0840000000000001,"15min":1.105},"min":{"1min":0.58499999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.4550000000000001,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.202}]},{"osd":4,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3420000000000001,"15min":1.3879999999999999},"min":{"1min":0.91300000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.696,"5min":3.081,"15min":3.081},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.3340000000000001,"15min":1.339},"min":{"1min":0.74299999999999999,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.712,"5min":2.758,"15min":2.758},"last":1.4390000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.373,"15min":1.403},"min":{"1min":0.873,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.1200000000000001,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.6200000000000001,"15min":1.748},"min":{"1min":0.86299999999999999,"5min":0.745,"15min":0.745},"max":{"1min":1.657,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.321}]},{"osd":6,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3759999999999999,"15min":1.407},"min":{"1min":0.82999999999999996,"5min":0.753,"15min":0.753},"max":{"1min":1.79,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.052},{"interface":"front","average":{"1min":1.524,"5min":1.53,"15min":1.544},"min":{"1min":1.0760000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.464,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.681}]},{"osd":7,"last update":"Fri Nov 24 06:29:57 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.5760000000000001,"15min":1.609},"min":{"1min":0.84499999999999997,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.589,"5min":4.641,"15min":4.641},"last":1.762},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.546,"15min":1.5740000000000001},"min":{"1min":0.88200000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.3439999999999999,"5min":3.798,"15min":3.798},"last":1.4119999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":876544,"data_stored":366729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.2729999999999999,"5min":1.427,"15min":1.427},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.97599999999999998,"15min":1.01},"min":{"1min":0.36199999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.619,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.59699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.048,"15min":1.052},"min":{"1min":0.48499999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0840000000000001,"15min":1.1040000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.784,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.089,"5min":1.095,"15min":1.117},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6990000000000001,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.456,"15min":1.508},"min":{"1min":0.67900000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.6869999999999998,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.425,"5min":1.454,"15min":1.4850000000000001},"min":{"1min":0.82399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.5800000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.2869999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4650000000000001,"15min":1.468},"min":{"1min":1.0029999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.036,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.27},{"interface":"front","average":{"1min":1.413,"5min":1.4690000000000001,"15min":1.5009999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.841,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.222}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.423,"15min":1.4359999999999999},"min":{"1min":0.66900000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.968,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.522,"15min":1.54},"min":{"1min":1.0029999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.165,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.163}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5149999999999999,"15min":1.5489999999999999},"min":{"1min":0.78200000000000003,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.913,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":1.097,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.423,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.2050000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":367345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.002,"15min":1.032},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.696,"5min":2.593,"15min":2.593},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.02,"15min":1.032},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5980000000000001,"5min":1.702,"15min":1.702},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.212,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":1,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7669999999999999,"5min":1.829,"15min":1.829},"last":1.482}]},{"osd":3,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1040000000000001,"15min":1.1200000000000001},"min":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.603,"5min":1.766,"15min":1.766},"last":1.3160000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":1.0289999999999999,"15min":1.069},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2250000000000001,"5min":1.798,"15min":1.798},"last":1.6279999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.246,"15min":1.2529999999999999},"min":{"1min":0.89000000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.621,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.256},{"interface":"front","average":{"1min":1.359,"5min":1.417,"15min":1.4450000000000001},"min":{"1min":1.0509999999999999,"5min":1.032,"15min":1.032},"max":{"1min":1.837,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.591}]},{"osd":5,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.294,"15min":1.302},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.514,"5min":2.016,"15min":2.016},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.389,"15min":1.399},"min":{"1min":0.91700000000000004,"5min":0.877,"15min":0.877},"max":{"1min":1.7410000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.367}]},{"osd":6,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.391,"15min":1.393},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.847,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4339999999999999,"15min":1.45},"min":{"1min":0.96399999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.6499999999999999,"5min":2.371,"15min":2.371},"last":1.6459999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.444,"15min":1.448},"min":{"1min":0.89300000000000002,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.9730000000000001,"5min":2.246,"15min":2.246},"last":1.734},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4950000000000001,"15min":1.5189999999999999},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":1.7629999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.7}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1458176,"data_stored":955485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.872,"15min":0.85799999999999998},"min":{"1min":0.48099999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.92400000000000004,"15min":0.95899999999999996},"min":{"1min":0.39200000000000002,"5min":0.31,"15min":0.31},"max":{"1min":1.516,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.004,"15min":1},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6619999999999999,"5min":1.784,"15min":1.784},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.081,"5min":1.1060000000000001,"15min":1.1319999999999999},"min":{"1min":0.48299999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.639,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.72099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.94899999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4750000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.052,"5min":1.077,"15min":1.087},"min":{"1min":0.53700000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.506,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.23,"15min":1.226},"min":{"1min":0.80700000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7070000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3129999999999999},"min":{"1min":1.0149999999999999,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":1.714,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.046}]},{"osd":5,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.329,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9610000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.2709999999999999,"15min":1.278},"min":{"1min":0.86099999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.865,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.4570000000000001,"15min":1.4830000000000001},"min":{"1min":0.84199999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.8500000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.4710000000000001,"15min":1.5109999999999999},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.903,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.123}]},{"osd":7,"last update":"Fri Nov 24 06:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.448,"15min":1.456},"min":{"1min":0.95999999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8180000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.081},{"interface":"front","average":{"1min":1.52,"5min":1.417,"15min":1.425},"min":{"1min":0.84399999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0630000000000002,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.02}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:30:44.392 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:30:44.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.665+0000 7f4993121640 1 Processor -- start 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.665+0000 7f4993121640 1 -- start start 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.667+0000 7f4993121640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f498c15d190 0x7f498c0d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.667+0000 7f4993121640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f498c0b1f30 0x7f498c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.667+0000 7f4993121640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c0d0050 con 0x7f498c15d570 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.667+0000 7f4993121640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c0d02a0 con 0x7f498c0b1f30 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.667+0000 7f4993121640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c0d0580 con 0x7f498c15d190 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.667+0000 7f499191e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f498c0b1f30 0x7f498c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.667+0000 7f499191e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f498c0b1f30 0x7f498c0d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39946/0 (socket says 172.21.15.16:39946) 2023-11-24T06:30:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.667+0000 7f499191e640 1 -- 172.21.15.16:0/3874689348 learned_addr learned my addr 172.21.15.16:0/3874689348 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.668+0000 7f499211f640 1 --2- 172.21.15.16:0/3874689348 >> v2:172.21.15.173:3300/0 conn(0x7f498c15d190 0x7f498c0d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.668+0000 7f499191e640 1 -- 172.21.15.16:0/3874689348 >> v2:172.21.15.173:3300/0 conn(0x7f498c15d190 msgr2=0x7f498c0d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.668+0000 7f499191e640 1 --2- 172.21.15.16:0/3874689348 >> v2:172.21.15.173:3300/0 conn(0x7f498c15d190 0x7f498c0d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.668+0000 7f499191e640 1 -- 172.21.15.16:0/3874689348 >> v1:172.21.15.173:6789/0 conn(0x7f498c15d570 legacy=0x7f498c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.668+0000 7f499191e640 1 -- 172.21.15.16:0/3874689348 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0d0860 con 0x7f498c0b1f30 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.670+0000 7f499191e640 1 --2- 172.21.15.16:0/3874689348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c0b1f30 0x7f498c0d1b30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f498c08b460 tx=0x7f497c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c061ff25c1f040e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.670+0000 7f499111d640 1 -- 172.21.15.16:0/3874689348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c031870 con 0x7f498c0b1f30 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.670+0000 7f499111d640 1 -- 172.21.15.16:0/3874689348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c0319d0 con 0x7f498c0b1f30 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.670+0000 7f499111d640 1 -- 172.21.15.16:0/3874689348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c031d60 con 0x7f498c0b1f30 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.671+0000 7f4993121640 1 -- 172.21.15.16:0/3874689348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c0b1f30 msgr2=0x7f498c0d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.671+0000 7f4993121640 1 --2- 172.21.15.16:0/3874689348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c0b1f30 0x7f498c0d1b30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f498c08b460 tx=0x7f497c02b910 comp rx=0 tx=0).stop 2023-11-24T06:30:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.671+0000 7f4993121640 1 -- 172.21.15.16:0/3874689348 shutdown_connections 2023-11-24T06:30:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.671+0000 7f4993121640 1 --2- 172.21.15.16:0/3874689348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c0b1f30 0x7f498c0d1b30 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.671+0000 7f4993121640 1 --2- 172.21.15.16:0/3874689348 >> v2:172.21.15.173:3300/0 conn(0x7f498c15d190 0x7f498c0d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.671+0000 7f4993121640 1 -- 172.21.15.16:0/3874689348 >> 172.21.15.16:0/3874689348 conn(0x7f498c0d3010 msgr2=0x7f498c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.671+0000 7f4993121640 1 -- 172.21.15.16:0/3874689348 shutdown_connections 2023-11-24T06:30:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.671+0000 7f4993121640 1 -- 172.21.15.16:0/3874689348 wait complete. 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.674+0000 7f4993121640 1 Processor -- start 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.674+0000 7f4993121640 1 -- start start 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.675+0000 7f4993121640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f498c0b1f30 0x7f498c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.675+0000 7f4993121640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c15d190 0x7f498c15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.675+0000 7f4993121640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f498c15d570 0x7f498c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.675+0000 7f4993121640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0b5c30 con 0x7f498c15d190 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.675+0000 7f4993121640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0ca850 con 0x7f498c0b1f30 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.675+0000 7f4993121640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0c99c0 con 0x7f498c15d570 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499191e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c15d190 0x7f498c15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499191e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c15d190 0x7f498c15f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39962/0 (socket says 172.21.15.16:39962) 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499191e640 1 -- 172.21.15.16:0/356519778 learned_addr learned my addr 172.21.15.16:0/356519778 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499211f640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f498c0b1f30 0x7f498c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499191e640 1 -- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f498c15d570 msgr2=0x7f498c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499191e640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f498c15d570 0x7f498c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499191e640 1 -- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f498c0b1f30 msgr2=0x7f498c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499191e640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f498c0b1f30 0x7f498c0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499191e640 1 -- 172.21.15.16:0/356519778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c0094d0 con 0x7f498c15d190 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499211f640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f498c0b1f30 0x7f498c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.676+0000 7f499191e640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c15d190 0x7f498c15f750 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f497c000c00 tx=0x7f497c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.677+0000 7f4982ffd640 1 -- 172.21.15.16:0/356519778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c03abe0 con 0x7f498c15d190 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.677+0000 7f4982ffd640 1 -- 172.21.15.16:0/356519778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c03ad40 con 0x7f498c15d190 2023-11-24T06:30:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.677+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c0b5e40 con 0x7f498c15d190 2023-11-24T06:30:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.677+0000 7f4982ffd640 1 -- 172.21.15.16:0/356519778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c044610 con 0x7f498c15d190 2023-11-24T06:30:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.677+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c0ca190 con 0x7f498c15d190 2023-11-24T06:30:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.678+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c067450 con 0x7f498c15d190 2023-11-24T06:30:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.679+0000 7f4982ffd640 1 -- 172.21.15.16:0/356519778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f497c04d450 con 0x7f498c15d190 2023-11-24T06:30:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.680+0000 7f4982ffd640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497006b230 0x7f497006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.681+0000 7f4982ffd640 1 -- 172.21.15.16:0/356519778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f497c02d040 con 0x7f498c15d190 2023-11-24T06:30:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.681+0000 7f499211f640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497006b230 0x7f497006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.682+0000 7f499211f640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497006b230 0x7f497006d6f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4988009f50 tx=0x7f498800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.687+0000 7f4982ffd640 1 -- 172.21.15.16:0/356519778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f497c076db0 con 0x7f498c15d190 2023-11-24T06:30:44.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.960+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f498c0d2df0 con 0x7f498c15d190 2023-11-24T06:30:44.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.964+0000 7f4982ffd640 1 -- 172.21.15.16:0/356519778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f497c07a3d0 con 0x7f498c15d190 2023-11-24T06:30:44.967 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-24T06:30:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA== 2023-11-24T06:30:44.967 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2023-11-24T06:30:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2023-11-24T06:30:44.968 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:30:44.968 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-11-24T06:30:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.967+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497006b230 msgr2=0x7f497006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.967+0000 7f4993121640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497006b230 0x7f497006d6f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4988009f50 tx=0x7f498800b040 comp rx=0 tx=0).stop 2023-11-24T06:30:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.967+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c15d190 msgr2=0x7f498c15f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.967+0000 7f4993121640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c15d190 0x7f498c15f750 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f497c000c00 tx=0x7f497c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.967+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 shutdown_connections 2023-11-24T06:30:44.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.967+0000 7f4993121640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497006b230 0x7f497006d6f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.967+0000 7f4993121640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f498c15d570 0x7f498c15fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.967+0000 7f4993121640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f498c15d190 0x7f498c15f750 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.967+0000 7f4993121640 1 --2- 172.21.15.16:0/356519778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f498c0b1f30 0x7f498c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:44.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.968+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 >> 172.21.15.16:0/356519778 conn(0x7f498c0d3010 msgr2=0x7f498c0b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:44.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.968+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 shutdown_connections 2023-11-24T06:30:44.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:44.968+0000 7f4993121640 1 -- 172.21.15.16:0/356519778 wait complete. 2023-11-24T06:30:44.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:30:45.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.207+0000 7fd175a39640 1 Processor -- start 2023-11-24T06:30:45.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.208+0000 7fd175a39640 1 -- start start 2023-11-24T06:30:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.209+0000 7fd175a39640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd1700d0100 0x7fd1700b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:45.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.209+0000 7fd175a39640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1700d0f40 0x7fd1700d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.209+0000 7fd175a39640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1700d0a10 con 0x7fd170160920 2023-11-24T06:30:45.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.209+0000 7fd175a39640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1700cf080 con 0x7fd1700d0100 2023-11-24T06:30:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.209+0000 7fd175a39640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700c6be0 con 0x7fd1700d0f40 2023-11-24T06:30:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.210+0000 7fd175238640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1700d0f40 0x7fd1700d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:45.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.210+0000 7fd175238640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1700d0f40 0x7fd1700d13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:33652/0 (socket says 172.21.15.16:33652) 2023-11-24T06:30:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.210+0000 7fd175238640 1 -- 172.21.15.16:0/567638434 learned_addr learned my addr 172.21.15.16:0/567638434 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.210+0000 7fd167fff640 1 --2- 172.21.15.16:0/567638434 >> v2:172.21.15.173:3300/0 conn(0x7fd1700d0100 0x7fd1700b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:45.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.211+0000 7fd167fff640 1 -- 172.21.15.16:0/567638434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1700d0f40 msgr2=0x7fd1700d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.211+0000 7fd167fff640 1 --2- 172.21.15.16:0/567638434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1700d0f40 0x7fd1700d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.211+0000 7fd167fff640 1 -- 172.21.15.16:0/567638434 >> v1:172.21.15.173:6789/0 conn(0x7fd170160920 legacy=0x7fd1700cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:45.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.211+0000 7fd167fff640 1 -- 172.21.15.16:0/567638434 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700ce7e0 con 0x7fd1700d0100 2023-11-24T06:30:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.211+0000 7fd1677fe640 1 -- 172.21.15.16:0/567638434 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073970769 0 0) 0x7fd1700d0a10 con 0x7fd170160920 2023-11-24T06:30:45.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.211+0000 7fd175238640 1 --2- 172.21.15.16:0/567638434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1700d0f40 0x7fd1700d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:30:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.212+0000 7fd167fff640 1 --2- 172.21.15.16:0/567638434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0100 0x7fd1700b3f30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd1600097a0 tx=0x7fd16002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a00c92feb64301 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.213+0000 7fd1677fe640 1 -- 172.21.15.16:0/567638434 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16002d040 con 0x7fd1700d0100 2023-11-24T06:30:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.213+0000 7fd1677fe640 1 -- 172.21.15.16:0/567638434 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd160031810 con 0x7fd1700d0100 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.213+0000 7fd1677fe640 1 -- 172.21.15.16:0/567638434 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160031ba0 con 0x7fd1700d0100 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.214+0000 7fd175a39640 1 -- 172.21.15.16:0/567638434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0100 msgr2=0x7fd1700b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.214+0000 7fd175a39640 1 --2- 172.21.15.16:0/567638434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0100 0x7fd1700b3f30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd1600097a0 tx=0x7fd16002b8b0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.214+0000 7fd175a39640 1 -- 172.21.15.16:0/567638434 shutdown_connections 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.214+0000 7fd175a39640 1 --2- 172.21.15.16:0/567638434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1700d0f40 0x7fd1700d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.214+0000 7fd175a39640 1 --2- 172.21.15.16:0/567638434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0100 0x7fd1700b3f30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.214+0000 7fd175a39640 1 -- 172.21.15.16:0/567638434 >> 172.21.15.16:0/567638434 conn(0x7fd1700d3090 msgr2=0x7fd1700d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.214+0000 7fd175a39640 1 -- 172.21.15.16:0/567638434 shutdown_connections 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.214+0000 7fd175a39640 1 -- 172.21.15.16:0/567638434 wait complete. 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.216+0000 7fd175a39640 1 Processor -- start 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.216+0000 7fd175a39640 1 -- start start 2023-11-24T06:30:45.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.217+0000 7fd175a39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1700d0100 0x7fd1701f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.217+0000 7fd175a39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0f40 0x7fd170159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.217+0000 7fd175a39640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd170160920 0x7fd1701f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.217+0000 7fd175a39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd170159fd0 con 0x7fd1700d0100 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.217+0000 7fd175a39640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700c6e80 con 0x7fd1700d0f40 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.217+0000 7fd175a39640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700c7160 con 0x7fd170160920 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.217+0000 7fd174a37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1700d0100 0x7fd1701f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.217+0000 7fd174a37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1700d0100 0x7fd1701f6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39978/0 (socket says 172.21.15.16:39978) 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.217+0000 7fd174a37640 1 -- 172.21.15.16:0/3271942281 learned_addr learned my addr 172.21.15.16:0/3271942281 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd175238640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd170160920 0x7fd1701f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd167fff640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0f40 0x7fd170159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd174a37640 1 -- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd170160920 msgr2=0x7fd1701f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd174a37640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd170160920 0x7fd1701f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd174a37640 1 -- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0f40 msgr2=0x7fd170159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd174a37640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0f40 0x7fd170159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd174a37640 1 -- 172.21.15.16:0/3271942281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009b30 con 0x7fd1700d0100 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd167fff640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0f40 0x7fd170159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd174a37640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1700d0100 0x7fd1701f6130 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00c4b0 tx=0x7fd16c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.218+0000 7fd1657fa640 1 -- 172.21.15.16:0/3271942281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c005480 con 0x7fd1700d0100 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.219+0000 7fd1657fa640 1 -- 172.21.15.16:0/3271942281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c0035f0 con 0x7fd1700d0100 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.219+0000 7fd1657fa640 1 -- 172.21.15.16:0/3271942281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0039e0 con 0x7fd1700d0100 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.219+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd160009400 con 0x7fd1700d0100 2023-11-24T06:30:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.219+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17015a2b0 con 0x7fd1700d0100 2023-11-24T06:30:45.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.220+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170067450 con 0x7fd1700d0100 2023-11-24T06:30:45.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.221+0000 7fd1657fa640 1 -- 172.21.15.16:0/3271942281 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd16c003b40 con 0x7fd1700d0100 2023-11-24T06:30:45.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.222+0000 7fd1657fa640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15c06b230 0x7fd15c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:45.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.222+0000 7fd1657fa640 1 -- 172.21.15.16:0/3271942281 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fd16c089460 con 0x7fd1700d0100 2023-11-24T06:30:45.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.223+0000 7fd167fff640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15c06b230 0x7fd15c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:45.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.224+0000 7fd167fff640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15c06b230 0x7fd15c06d6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd160036000 tx=0x7fd16003c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:45.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.226+0000 7fd1657fa640 1 -- 172.21.15.16:0/3271942281 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd16c053e10 con 0x7fd1700d0100 2023-11-24T06:30:45.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.458+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1700d3960 con 0x7fd1700d0100 2023-11-24T06:30:45.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.459+0000 7fd1657fa640 1 -- 172.21.15.16:0/3271942281 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fd16c057430 con 0x7fd1700d0100 2023-11-24T06:30:45.462 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:45.462 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1164611025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1164611025},{"type":"v1","addr":"172.21.15.16:6834","nonce":1164611025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5650,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1157106761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1157106761},{"type":"v1","addr":"172.21.15.173:6836","nonce":1157106761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/151946729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":151946729},{"type":"v1","addr":"172.21.15.173:6837","nonce":151946729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:30:33.707795+0000","modified":"2023-11-24T06:30:36.770163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6693":{"gid":6693,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2360619406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2360619406},{"type":"v1","addr":"172.21.15.16:6835","nonce":2360619406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:30:45.462 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2023-11-24T06:30:45.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15c06b230 msgr2=0x7fd15c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:45.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15c06b230 0x7fd15c06d6f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd160036000 tx=0x7fd16003c040 comp rx=0 tx=0).stop 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1700d0100 msgr2=0x7fd1701f6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1700d0100 0x7fd1701f6130 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00c4b0 tx=0x7fd16c00c970 comp rx=0 tx=0).stop 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 shutdown_connections 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd15c06b230 0x7fd15c06d6f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd170160920 0x7fd1701f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1700d0f40 0x7fd170159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 --2- 172.21.15.16:0/3271942281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1700d0100 0x7fd1701f6130 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 >> 172.21.15.16:0/3271942281 conn(0x7fd1700d3090 msgr2=0x7fd1700cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 shutdown_connections 2023-11-24T06:30:45.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:45.462+0000 7fd175a39640 1 -- 172.21.15.16:0/3271942281 wait complete. 2023-11-24T06:30:45.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:30:33.707795+0000', 'modified': '2023-11-24T06:30:36.770163+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6693}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6693': {'gid': 6693, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/2360619406', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 2360619406}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2360619406}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:30:45.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:30:45.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:30:46.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.123+0000 7f49354d4640 1 Processor -- start 2023-11-24T06:30:46.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.124+0000 7f49354d4640 1 -- start start 2023-11-24T06:30:46.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.125+0000 7f49354d4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f493013e360 0x7f49300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.125+0000 7f49354d4640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f493015f9c0 0x7f49300cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f49354d4640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493015c5f0 con 0x7f493013df80 2023-11-24T06:30:46.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f49354d4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493006c470 con 0x7f493013e360 2023-11-24T06:30:46.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f49354d4640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493006c750 con 0x7f493015f9c0 2023-11-24T06:30:46.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f492e7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f493013e360 0x7f49300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f492e7fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f493013e360 0x7f49300cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39986/0 (socket says 172.21.15.16:39986) 2023-11-24T06:30:46.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f492e7fc640 1 -- 172.21.15.16:0/2062437935 learned_addr learned my addr 172.21.15.16:0/2062437935 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:46.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f492e7fc640 1 -- 172.21.15.16:0/2062437935 >> v2:172.21.15.173:3300/0 conn(0x7f493015f9c0 msgr2=0x7f49300cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f492e7fc640 1 --2- 172.21.15.16:0/2062437935 >> v2:172.21.15.173:3300/0 conn(0x7f493015f9c0 0x7f49300cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f492e7fc640 1 -- 172.21.15.16:0/2062437935 >> v1:172.21.15.173:6789/0 conn(0x7f493013df80 legacy=0x7f493015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.126+0000 7f492e7fc640 1 -- 172.21.15.16:0/2062437935 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493006ca30 con 0x7f493013e360 2023-11-24T06:30:46.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.127+0000 7f492e7fc640 1 --2- 172.21.15.16:0/2062437935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 0x7f49300cf290 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f493006ded0 tx=0x7f491802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da937a4e263af6e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.128+0000 7f492dffb640 1 -- 172.21.15.16:0/2062437935 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918031870 con 0x7f493013e360 2023-11-24T06:30:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.128+0000 7f492dffb640 1 -- 172.21.15.16:0/2062437935 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49180319d0 con 0x7f493013e360 2023-11-24T06:30:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.128+0000 7f492dffb640 1 -- 172.21.15.16:0/2062437935 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918031d60 con 0x7f493013e360 2023-11-24T06:30:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.128+0000 7f49354d4640 1 -- 172.21.15.16:0/2062437935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 msgr2=0x7f49300cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.128+0000 7f49354d4640 1 --2- 172.21.15.16:0/2062437935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 0x7f49300cf290 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f493006ded0 tx=0x7f491802b910 comp rx=0 tx=0).stop 2023-11-24T06:30:46.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.128+0000 7f49354d4640 1 -- 172.21.15.16:0/2062437935 shutdown_connections 2023-11-24T06:30:46.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.128+0000 7f49354d4640 1 --2- 172.21.15.16:0/2062437935 >> v2:172.21.15.173:3300/0 conn(0x7f493015f9c0 0x7f49300cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.128+0000 7f49354d4640 1 --2- 172.21.15.16:0/2062437935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 0x7f49300cf290 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.128+0000 7f49354d4640 1 -- 172.21.15.16:0/2062437935 >> 172.21.15.16:0/2062437935 conn(0x7f493006a800 msgr2=0x7f493006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:46.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.129+0000 7f49354d4640 1 -- 172.21.15.16:0/2062437935 shutdown_connections 2023-11-24T06:30:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.129+0000 7f49354d4640 1 -- 172.21.15.16:0/2062437935 wait complete. 2023-11-24T06:30:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.130+0000 7f49354d4640 1 Processor -- start 2023-11-24T06:30:46.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.130+0000 7f49354d4640 1 -- start start 2023-11-24T06:30:46.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.131+0000 7f49354d4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f493013df80 0x7f49300a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.131+0000 7f49354d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 0x7f49301521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.131+0000 7f49354d4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f493015f9c0 0x7f49300ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.131+0000 7f49354d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300a9090 con 0x7f493013e360 2023-11-24T06:30:46.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.131+0000 7f49354d4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49301532f0 con 0x7f493015f9c0 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.131+0000 7f49354d4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49300a9b80 con 0x7f493013df80 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 0x7f49301521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 0x7f49301521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39994/0 (socket says 172.21.15.16:39994) 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492e7fc640 1 -- 172.21.15.16:0/2932834137 learned_addr learned my addr 172.21.15.16:0/2932834137 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492e7fc640 1 -- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f493013df80 msgr2=0x7f49300a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492e7fc640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f493013df80 0x7f49300a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492e7fc640 1 -- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f493015f9c0 msgr2=0x7f49300ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492f7fe640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f493015f9c0 0x7f49300ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492e7fc640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f493015f9c0 0x7f49300ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492e7fc640 1 -- 172.21.15.16:0/2932834137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180094d0 con 0x7f493013e360 2023-11-24T06:30:46.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.132+0000 7f492f7fe640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f493015f9c0 0x7f49300ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.133+0000 7f492e7fc640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 0x7f49301521f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4918000c00 tx=0x7f49180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.133+0000 7f49277fe640 1 -- 172.21.15.16:0/2932834137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491803ad10 con 0x7f493013e360 2023-11-24T06:30:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.133+0000 7f49277fe640 1 -- 172.21.15.16:0/2932834137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491803ae90 con 0x7f493013e360 2023-11-24T06:30:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.133+0000 7f49277fe640 1 -- 172.21.15.16:0/2932834137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918044630 con 0x7f493013e360 2023-11-24T06:30:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.133+0000 7f49354d4640 1 -- 172.21.15.16:0/2932834137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300a92e0 con 0x7f493013e360 2023-11-24T06:30:46.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.133+0000 7f49354d4640 1 -- 172.21.15.16:0/2932834137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49300a98a0 con 0x7f493013e360 2023-11-24T06:30:46.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.136+0000 7f49277fe640 1 -- 172.21.15.16:0/2932834137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f491804d450 con 0x7f493013e360 2023-11-24T06:30:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.138+0000 7f49277fe640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f491006b230 0x7f491006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.138+0000 7f49277fe640 1 -- 172.21.15.16:0/2932834137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f491802d040 con 0x7f493013e360 2023-11-24T06:30:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.138+0000 7f4938062740 1 Processor -- start 2023-11-24T06:30:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.138+0000 7f4938062740 1 -- start start 2023-11-24T06:30:46.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.138+0000 7f492effd640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f491006b230 0x7f491006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f4938062740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd42d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f4938062740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd427ba0 0x556abd42c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f4938062740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd42c5e0 0x556abd470560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f4938062740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abd38b880 con 0x556abd427ba0 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f4938062740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abd39aae0 con 0x556abd42cea0 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f4938062740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x556abd3981d0 con 0x556abd42c5e0 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f492e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd427ba0 0x556abd42c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f492e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd427ba0 0x556abd42c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39996/0 (socket says 172.21.15.16:39996) 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f492e7fc640 1 -- 172.21.15.16:0/1450217821 learned_addr learned my addr 172.21.15.16:0/1450217821 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:46.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.139+0000 7f492effd640 1 --2- 172.21.15.16:0/1450217821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd42d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f492e7fc640 1 -- 172.21.15.16:0/1450217821 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd42c5e0 msgr2=0x556abd470560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f492e7fc640 1 --2- 172.21.15.16:0/1450217821 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd42c5e0 0x556abd470560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f492e7fc640 1 -- 172.21.15.16:0/1450217821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 msgr2=0x556abd42d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f492e7fc640 1 --2- 172.21.15.16:0/1450217821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd42d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f492e7fc640 1 -- 172.21.15.16:0/1450217821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556abd35b860 con 0x556abd427ba0 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f492effd640 1 --2- 172.21.15.16:0/1450217821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd42d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f492effd640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f491006b230 0x7f491006d6f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f492000ac40 tx=0x7f4920009250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f492e7fc640 1 --2- 172.21.15.16:0/1450217821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd427ba0 0x556abd42c0a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f491807a950 tx=0x7f491807ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab240530e71c016 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f4924ff9640 1 -- 172.21.15.16:0/1450217821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918077360 con 0x556abd427ba0 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f4924ff9640 1 -- 172.21.15.16:0/1450217821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49180774c0 con 0x556abd427ba0 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.140+0000 7f4924ff9640 1 -- 172.21.15.16:0/1450217821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491807b0b0 con 0x556abd427ba0 2023-11-24T06:30:46.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.141+0000 7f4938062740 1 -- 172.21.15.16:0/1450217821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd427ba0 msgr2=0x556abd42c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.141+0000 7f4938062740 1 --2- 172.21.15.16:0/1450217821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd427ba0 0x556abd42c0a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f491807a950 tx=0x7f491807ac50 comp rx=0 tx=0).stop 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.141+0000 7f4938062740 1 -- 172.21.15.16:0/1450217821 shutdown_connections 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.141+0000 7f4938062740 1 --2- 172.21.15.16:0/1450217821 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd42c5e0 0x556abd470560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.141+0000 7f4938062740 1 --2- 172.21.15.16:0/1450217821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd427ba0 0x556abd42c0a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.141+0000 7f4938062740 1 --2- 172.21.15.16:0/1450217821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd42d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.141+0000 7f4938062740 1 -- 172.21.15.16:0/1450217821 >> 172.21.15.16:0/1450217821 conn(0x556abd3eec70 msgr2=0x556abd472990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.141+0000 7f4938062740 1 -- 172.21.15.16:0/1450217821 shutdown_connections 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.141+0000 7f4938062740 1 -- 172.21.15.16:0/1450217821 wait complete. 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.142+0000 7f4938062740 1 -- start start 2023-11-24T06:30:46.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.142+0000 7f4938062740 1 Processor -- start 2023-11-24T06:30:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f4938062740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 0x556abd44c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f4938062740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd44d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f4938062740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd44dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f4938062740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abd393ac0 con 0x556abd42c5e0 2023-11-24T06:30:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f4938062740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abd3302f0 con 0x556abd42cea0 2023-11-24T06:30:46.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f4938062740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x556abd34dc70 con 0x556abd427ba0 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f4938062740 10 client.0 _ll_register_callbacks cb 0x556abd3523b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f492e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd44d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f492e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd44d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40012/0 (socket says 172.21.15.16:40012) 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.143+0000 7f492e7fc640 1 -- 172.21.15.16:0/4173553519 learned_addr learned my addr 172.21.15.16:0/4173553519 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f492e7fc640 1 -- 172.21.15.16:0/4173553519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 msgr2=0x556abd44c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f492effd640 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 0x556abd44c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f492f7fe640 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd44dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f492e7fc640 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 0x556abd44c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f492e7fc640 1 -- 172.21.15.16:0/4173553519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 msgr2=0x556abd44dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f492e7fc640 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd44dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f492e7fc640 1 -- 172.21.15.16:0/4173553519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49300a98a0 con 0x556abd42c5e0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f492f7fe640 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd44dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f492e7fc640 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd44d7c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f491802bf20 tx=0x7f49180ac040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615b1104be09e4e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f490dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f490dffb640 1 -- 172.21.15.16:0/4173553519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918082ac0 con 0x556abd42c5e0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f490dffb640 1 -- 172.21.15.16:0/4173553519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f4918096030 con 0x556abd42c5e0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f4938062740 1 -- 172.21.15.16:0/4173553519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556abd38caa0 con 0x556abd42c5e0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.144+0000 7f490dffb640 1 -- 172.21.15.16:0/4173553519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4918082c20 con 0x556abd42c5e0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f4938062740 10 client.6852 fetch_fsmap learned FSMap version 44 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f4938062740 1 -- 172.21.15.16:0/4173553519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x556abd392140 con 0x556abd42c5e0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f490dffb640 1 -- 172.21.15.16:0/4173553519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556abd38caa0 con 0x556abd42c5e0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f490dffb640 1 -- 172.21.15.16:0/4173553519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4918099340 con 0x556abd42c5e0 2023-11-24T06:30:46.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f4938062740 10 client.6852 fetch_fsmap finished waiting for FSMap version 44 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f4938062740 10 client.6852 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f4938062740 20 client.6852 populate_metadata read hostname 'smithi016' 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f4938062740 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] conn(0x556abd470fd0 0x556abd4713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f4938062740 4 client.6852 mds_command: new command op to 6693 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f4938062740 1 -- 172.21.15.16:0/4173553519 --> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556abd38c7c0 con 0x556abd470fd0 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.145+0000 7f492effd640 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] conn(0x556abd470fd0 0x556abd4713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.146+0000 7f492effd640 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] conn(0x556abd470fd0 0x556abd4713b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977d880986e8962e server_cookie=8d8fc91d7f7c97c7 in_seq=0 out_seq=0 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.146+0000 7f490dffb640 10 client.6852 ms_handle_connect on v2:172.21.15.16:6833/2360619406 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f490dffb640 1 -- 172.21.15.16:0/4173553519 <== mds.0 v2:172.21.15.16:6833/2360619406 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556abd38c7c0 con 0x556abd470fd0 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f490dffb640 10 client.6852 handle_command_reply: tid=0 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 client.6852 shutdown 2023-11-24T06:30:46.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 -- 172.21.15.16:0/4173553519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 msgr2=0x556abd44d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd44d7c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f491802bf20 tx=0x7f49180ac040 comp rx=0 tx=0).stop 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 -- 172.21.15.16:0/4173553519 shutdown_connections 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] conn(0x556abd470fd0 0x556abd4713b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42cea0 0x556abd44dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd44d7c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 --2- 172.21.15.16:0/4173553519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 0x556abd44c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 -- 172.21.15.16:0/4173553519 >> 172.21.15.16:0/4173553519 conn(0x556abd3eec70 msgr2=0x556abd470320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f490dffb640 0 client.6852 ms_handle_reset on v2:172.21.15.16:6833/2360619406 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 -- 172.21.15.16:0/4173553519 shutdown_connections 2023-11-24T06:30:46.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.147+0000 7f4938062740 1 -- 172.21.15.16:0/4173553519 wait complete. 2023-11-24T06:30:46.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.148+0000 7f4938062740 20 client.6852 trim_cache size 0 max 16384 2023-11-24T06:30:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.165+0000 7f4938062740 1 Processor -- start 2023-11-24T06:30:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.165+0000 7f4938062740 1 -- start start 2023-11-24T06:30:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f4938062740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 0x556abd450720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f4938062740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42c5e0 0x556abd472990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f4938062740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42cea0 0x556abd472ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f4938062740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abd0f6f20 con 0x556abd42cea0 2023-11-24T06:30:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f4938062740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abd4292f0 con 0x556abd42c5e0 2023-11-24T06:30:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f4938062740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x556abd472090 con 0x556abd427ba0 2023-11-24T06:30:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42cea0 0x556abd472ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f492f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42cea0 0x556abd472ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40018/0 (socket says 172.21.15.16:40018) 2023-11-24T06:30:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f492f7fe640 1 -- 172.21.15.16:0/2957873449 learned_addr learned my addr 172.21.15.16:0/2957873449 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f492effd640 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 0x556abd450720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.166+0000 7f492e7fc640 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42c5e0 0x556abd472990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f492f7fe640 1 -- 172.21.15.16:0/2957873449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 msgr2=0x556abd450720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f492f7fe640 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 0x556abd450720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f492f7fe640 1 -- 172.21.15.16:0/2957873449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42c5e0 msgr2=0x556abd472990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f492f7fe640 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42c5e0 0x556abd472990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f492f7fe640 1 -- 172.21.15.16:0/2957873449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556abd32fb80 con 0x556abd42cea0 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f492e7fc640 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42c5e0 0x556abd472990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f492f7fe640 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42cea0 0x556abd472ed0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f493015b6f0 tx=0x7f492800bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deacb561bec1093a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f49257fa640 1 -- 172.21.15.16:0/2957873449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800dbb0 con 0x556abd42cea0 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f49257fa640 1 -- 172.21.15.16:0/2957873449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492800c040 con 0x556abd42cea0 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.167+0000 7f49257fa640 1 -- 172.21.15.16:0/2957873449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800be10 con 0x556abd42cea0 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 -- 172.21.15.16:0/2957873449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42cea0 msgr2=0x556abd472ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42cea0 0x556abd472ed0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f493015b6f0 tx=0x7f492800bbc0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 -- 172.21.15.16:0/2957873449 shutdown_connections 2023-11-24T06:30:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42cea0 0x556abd472ed0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd42c5e0 0x556abd472990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 --2- 172.21.15.16:0/2957873449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd427ba0 0x556abd450720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 -- 172.21.15.16:0/2957873449 >> 172.21.15.16:0/2957873449 conn(0x556abd3eec70 msgr2=0x556abd426c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 -- 172.21.15.16:0/2957873449 shutdown_connections 2023-11-24T06:30:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 -- 172.21.15.16:0/2957873449 wait complete. 2023-11-24T06:30:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.168+0000 7f4938062740 1 -- start start 2023-11-24T06:30:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.169+0000 7f4938062740 1 Processor -- start 2023-11-24T06:30:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f4938062740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd427ba0 0x556abd4540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f4938062740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd48b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f4938062740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd42cea0 0x556abd472990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f4938062740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abd454890 con 0x556abd42c5e0 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f4938062740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x556abd475c00 con 0x556abd427ba0 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f4938062740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x556abd42c240 con 0x556abd42cea0 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f4938062740 10 client.0 _ll_register_callbacks cb 0x7f491807a660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f492e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd48b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f492e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd48b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40030/0 (socket says 172.21.15.16:40030) 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f492e7fc640 1 -- 172.21.15.16:0/4005711290 learned_addr learned my addr 172.21.15.16:0/4005711290 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f492e7fc640 1 -- 172.21.15.16:0/4005711290 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd42cea0 msgr2=0x556abd472990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f492e7fc640 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd42cea0 0x556abd472990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f492e7fc640 1 -- 172.21.15.16:0/4005711290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd427ba0 msgr2=0x556abd4540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f492e7fc640 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd427ba0 0x556abd4540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.170+0000 7f492e7fc640 1 -- 172.21.15.16:0/4005711290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492800b390 con 0x556abd42c5e0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f492e7fc640 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd48b840 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f491807a950 tx=0x7f491807aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2065d8eb7de664 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f490dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f490dffb640 1 -- 172.21.15.16:0/4005711290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918082cc0 con 0x556abd42c5e0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f490dffb640 1 -- 172.21.15.16:0/4005711290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f4918096070 con 0x556abd42c5e0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f4938062740 1 -- 172.21.15.16:0/4005711290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49180774c0 con 0x556abd42c5e0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f490dffb640 1 -- 172.21.15.16:0/4005711290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f491807eda0 con 0x556abd42c5e0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f4938062740 10 client.6858 fetch_fsmap learned FSMap version 44 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f4938062740 1 -- 172.21.15.16:0/4005711290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x556abd355170 con 0x556abd42c5e0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f490dffb640 1 -- 172.21.15.16:0/4005711290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49180774c0 con 0x556abd42c5e0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f490dffb640 1 -- 172.21.15.16:0/4005711290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f491807fae0 con 0x556abd42c5e0 2023-11-24T06:30:46.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f4938062740 10 client.6858 fetch_fsmap finished waiting for FSMap version 44 2023-11-24T06:30:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f4938062740 10 client.6858 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:30:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.171+0000 7f4938062740 20 client.6858 populate_metadata read hostname 'smithi016' 2023-11-24T06:30:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.172+0000 7f4938062740 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] conn(0x556abd4e4920 0x556abd472ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.172+0000 7f4938062740 4 client.6858 mds_command: new command op to 6693 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.172+0000 7f4938062740 1 -- 172.21.15.16:0/4005711290 --> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556abd0946c0 con 0x556abd4e4920 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.172+0000 7f492effd640 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] conn(0x556abd4e4920 0x556abd472ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.172+0000 7f492effd640 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] conn(0x556abd4e4920 0x556abd472ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4939712f8fbcf08 server_cookie=ab91a2881e47507d in_seq=0 out_seq=0 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.172+0000 7f490dffb640 10 client.6858 ms_handle_connect on v2:172.21.15.16:6833/2360619406 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f490dffb640 1 -- 172.21.15.16:0/4005711290 <== mds.0 v2:172.21.15.16:6833/2360619406 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x556abd0946c0 con 0x556abd4e4920 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f490dffb640 10 client.6858 handle_command_reply: tid=0 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 client.6858 shutdown 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 -- 172.21.15.16:0/4005711290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 msgr2=0x556abd48b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd48b840 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f491807a950 tx=0x7f491807aad0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 -- 172.21.15.16:0/4005711290 shutdown_connections 2023-11-24T06:30:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.16:6833/2360619406,v1:172.21.15.16:6835/2360619406] conn(0x556abd4e4920 0x556abd472ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x556abd42cea0 0x556abd472990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556abd42c5e0 0x556abd48b840 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 --2- 172.21.15.16:0/4005711290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556abd427ba0 0x556abd4540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f490dffb640 0 client.6858 ms_handle_reset on v2:172.21.15.16:6833/2360619406 2023-11-24T06:30:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 -- 172.21.15.16:0/4005711290 >> 172.21.15.16:0/4005711290 conn(0x556abd3eec70 msgr2=0x556abd426c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 -- 172.21.15.16:0/4005711290 shutdown_connections 2023-11-24T06:30:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 1 -- 172.21.15.16:0/4005711290 wait complete. 2023-11-24T06:30:46.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.173+0000 7f4938062740 20 client.6858 trim_cache size 0 max 16384 2023-11-24T06:30:46.182 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:30:46.182 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6dc1668b-58b0-47dd-b8dc-4164e5a228ab", 2023-11-24T06:30:46.182 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-24T06:30:46.182 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6693, 2023-11-24T06:30:46.182 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-24T06:30:46.182 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-24T06:30:46.182 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-24T06:30:46.183 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 11.943912185, 2023-11-24T06:30:46.183 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 44, 2023-11-24T06:30:46.183 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 73, 2023-11-24T06:30:46.183 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 71, 2023-11-24T06:30:46.183 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 17.267662746999999 2023-11-24T06:30:46.183 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:30:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 -- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f491006b230 msgr2=0x7f491006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f491006b230 0x7f491006d6f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f492000ac40 tx=0x7f4920009250 comp rx=0 tx=0).stop 2023-11-24T06:30:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 -- 172.21.15.16:0/2932834137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 msgr2=0x7f49301521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 0x7f49301521f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4918000c00 tx=0x7f49180046c0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 -- 172.21.15.16:0/2932834137 shutdown_connections 2023-11-24T06:30:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f491006b230 0x7f491006d6f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f493015f9c0 0x7f49300ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f493013e360 0x7f49301521f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 --2- 172.21.15.16:0/2932834137 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f493013df80 0x7f49300a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 -- 172.21.15.16:0/2932834137 >> 172.21.15.16:0/2932834137 conn(0x7f493006a800 msgr2=0x7f49300cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 -- 172.21.15.16:0/2932834137 shutdown_connections 2023-11-24T06:30:46.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.174+0000 7f49354d4640 1 -- 172.21.15.16:0/2932834137 wait complete. 2023-11-24T06:30:46.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:30:46.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.401+0000 7f7336b89640 1 Processor -- start 2023-11-24T06:30:46.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.401+0000 7f7336b89640 1 -- start start 2023-11-24T06:30:46.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.403+0000 7f7336b89640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f733013df80 0x7f733015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.403+0000 7f7336b89640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733006c470 con 0x7f733013e360 2023-11-24T06:30:46.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.403+0000 7f7336b89640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733006c750 con 0x7f73300cf9a0 2023-11-24T06:30:46.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.403+0000 7f7336b89640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733006ca30 con 0x7f733013df80 2023-11-24T06:30:46.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.403+0000 7f732ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f733013e360 0x7f73300cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60336/0 (socket says 172.21.15.16:60336) 2023-11-24T06:30:46.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.403+0000 7f732ffff640 1 -- 172.21.15.16:0/1663096369 learned_addr learned my addr 172.21.15.16:0/1663096369 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f7334925640 1 --2- 172.21.15.16:0/1663096369 >> v2:172.21.15.173:3300/0 conn(0x7f733013df80 0x7f733015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513643356 0 0) 0x7f733006c470 con 0x7f733013e360 2023-11-24T06:30:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7318003520 con 0x7f733013e360 2023-11-24T06:30:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1763016273 0 0) 0x7f7318003520 con 0x7f733013e360 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 >> v2:172.21.15.173:3300/0 conn(0x7f733013df80 msgr2=0x7f733015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 --2- 172.21.15.16:0/1663096369 >> v2:172.21.15.173:3300/0 conn(0x7f733013df80 0x7f733015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 >> v1:172.21.15.173:6789/0 conn(0x7f73300cf9a0 legacy=0x7f73300cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733006cd10 con 0x7f733013e360 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f7320001670 con 0x7f733013e360 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 >> v1:172.21.15.16:6789/0 conn(0x7f733013e360 legacy=0x7f73300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7318003970 0x7f7318003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73180047c0 0x7f7318004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f731800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733006ca30 con 0x7f7318005180 2023-11-24T06:30:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731800c0d0 con 0x7f73180047c0 2023-11-24T06:30:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731800c290 con 0x7f7318003970 2023-11-24T06:30:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.404+0000 7f7335126640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f731800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.405+0000 7f7334925640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7318003970 0x7f7318003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.405+0000 7f732ffff640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73180047c0 0x7f7318004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.405+0000 7f7335126640 1 -- 172.21.15.16:0/1663096369 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7318003970 msgr2=0x7f7318003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.405+0000 7f7335126640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7318003970 0x7f7318003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.405+0000 7f7335126640 1 -- 172.21.15.16:0/1663096369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73180047c0 msgr2=0x7f7318004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.405+0000 7f7335126640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73180047c0 0x7f7318004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.405+0000 7f7335126640 1 -- 172.21.15.16:0/1663096369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7318003520 con 0x7f7318005180 2023-11-24T06:30:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.406+0000 7f7335126640 1 -- 172.21.15.16:0/1663096369 reap_dead start 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.406+0000 7f7335126640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f731800b980 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7328001e20 tx=0x7f73180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae1d23113d79b74 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.406+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328003b70 con 0x7f7318005180 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.406+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7328003cd0 con 0x7f7318005180 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.406+0000 7f732f7fe640 1 -- 172.21.15.16:0/1663096369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328005310 con 0x7f7318005180 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.407+0000 7f7336b89640 1 -- 172.21.15.16:0/1663096369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 msgr2=0x7f731800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.407+0000 7f7336b89640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f731800b980 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7328001e20 tx=0x7f73180057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.407+0000 7f7336b89640 1 -- 172.21.15.16:0/1663096369 shutdown_connections 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.407+0000 7f7336b89640 1 --2- 172.21.15.16:0/1663096369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f731800b980 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.407+0000 7f7336b89640 1 -- 172.21.15.16:0/1663096369 >> 172.21.15.16:0/1663096369 conn(0x7f733006a800 msgr2=0x7f733006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.407+0000 7f7336b89640 1 -- 172.21.15.16:0/1663096369 shutdown_connections 2023-11-24T06:30:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.407+0000 7f7336b89640 1 -- 172.21.15.16:0/1663096369 wait complete. 2023-11-24T06:30:46.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.409+0000 7f7336b89640 1 Processor -- start 2023-11-24T06:30:46.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.409+0000 7f7336b89640 1 -- start start 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.410+0000 7f7336b89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f73300b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.410+0000 7f7336b89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73301506b0 0x7f7330150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.410+0000 7f7336b89640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733014ed70 0x7f733014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.410+0000 7f7336b89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733014f730 con 0x7f7318005180 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.410+0000 7f7336b89640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73300b07a0 con 0x7f733014ed70 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.410+0000 7f7336b89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7330151220 con 0x7f73301506b0 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.410+0000 7f7334925640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f73300b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.410+0000 7f7334925640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f73300b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40052/0 (socket says 172.21.15.16:40052) 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.410+0000 7f7334925640 1 -- 172.21.15.16:0/3162854513 learned_addr learned my addr 172.21.15.16:0/3162854513 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:46.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7335126640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733014ed70 0x7f733014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f732ffff640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73301506b0 0x7f7330150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7334925640 1 -- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73301506b0 msgr2=0x7f7330150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7334925640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73301506b0 0x7f7330150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7334925640 1 -- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733014ed70 msgr2=0x7f733014f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7334925640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733014ed70 0x7f733014f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7334925640 1 -- 172.21.15.16:0/3162854513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7330151500 con 0x7f7318005180 2023-11-24T06:30:46.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f732ffff640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73301506b0 0x7f7330150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:46.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7335126640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733014ed70 0x7f733014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:46.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7334925640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f73300b2f60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f732400f580 tx=0x7f732400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f732d7fa640 1 -- 172.21.15.16:0/3162854513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324013810 con 0x7f7318005180 2023-11-24T06:30:46.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f732d7fa640 1 -- 172.21.15.16:0/3162854513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7324013970 con 0x7f7318005180 2023-11-24T06:30:46.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f732d7fa640 1 -- 172.21.15.16:0/3162854513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7324013ce0 con 0x7f7318005180 2023-11-24T06:30:46.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73301517e0 con 0x7f7318005180 2023-11-24T06:30:46.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.411+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7330151da0 con 0x7f7318005180 2023-11-24T06:30:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.413+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7330151f60 con 0x7f7318005180 2023-11-24T06:30:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.414+0000 7f732d7fa640 1 -- 172.21.15.16:0/3162854513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f732401f460 con 0x7f7318005180 2023-11-24T06:30:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.414+0000 7f732d7fa640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f731c06b230 0x7f731c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.415+0000 7f732d7fa640 1 -- 172.21.15.16:0/3162854513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f732408a1a0 con 0x7f7318005180 2023-11-24T06:30:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.415+0000 7f732ffff640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f731c06b230 0x7f731c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.416+0000 7f732ffff640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f731c06b230 0x7f731c06d6f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7320001200 tx=0x7f7320001190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.419+0000 7f732d7fa640 1 -- 172.21.15.16:0/3162854513 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7324054be0 con 0x7f7318005180 2023-11-24T06:30:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.649+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f73300675d0 con 0x7f7318005180 2023-11-24T06:30:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.649+0000 7f732d7fa640 1 -- 172.21.15.16:0/3162854513 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7324058200 con 0x7f7318005180 2023-11-24T06:30:46.652 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:30:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f731c06b230 msgr2=0x7f731c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f731c06b230 0x7f731c06d6f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7320001200 tx=0x7f7320001190 comp rx=0 tx=0).stop 2023-11-24T06:30:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 msgr2=0x7f73300b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f73300b2f60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f732400f580 tx=0x7f732400fa40 comp rx=0 tx=0).stop 2023-11-24T06:30:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 shutdown_connections 2023-11-24T06:30:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f731c06b230 0x7f731c06d6f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f733014ed70 0x7f733014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73301506b0 0x7f7330150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 --2- 172.21.15.16:0/3162854513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7318005180 0x7f73300b2f60 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 >> 172.21.15.16:0/3162854513 conn(0x7f733006a800 msgr2=0x7f73300cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 shutdown_connections 2023-11-24T06:30:46.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:46.652+0000 7f7336b89640 1 -- 172.21.15.16:0/3162854513 wait complete. 2023-11-24T06:30:46.674 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:30:46.674 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:30:46.674 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-24T06:30:46.674 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:30:46.674 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:46.675 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:30:46.675 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:30:46.675 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:30:46.675 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:30:46.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:46.675 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-24T06:30:46.728 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:46.729 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42097sec preferred_lft 42097sec 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:30:46.730 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:46.731 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:30:46.731 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:46.731 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8cfd:d2ff:fee1:5b18/64 scope link 2023-11-24T06:30:46.731 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:46.731 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:30:46.731 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:46.731 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8c89:e0ff:fefc:a3fb/64 scope link 2023-11-24T06:30:46.731 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:46.731 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:30:46.732 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether ea:e3:8b:37:17:ee brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:46.732 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e8e3:8bff:fe37:17ee/64 scope link 2023-11-24T06:30:46.732 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:46.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:46.732 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:30:46.824 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-24T06:30:46.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:46.829 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-24T06:30:46.887 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-24T06:30:46.887 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:30:46.887 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:30:46.887 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:30:46.887 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:30:46.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:46.888 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:30:46.888 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:30:46.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:30:46.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:30:46.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:30:46.888 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:30:47.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:47.096 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-24T06:30:47.150 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:30:47.150 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:30:47.151 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:30:47.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:47.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-24T06:30:47.251 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:30:47.251 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:30:47.251 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:47.301 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:30:47.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:30:47.302 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:47.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:30:47.443 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:30:47.502 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:30:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:30:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:30:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:30:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:30:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:30:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:30:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:30:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:30:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:30:47.504 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:47.505 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:47.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:47.506 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:30:47.561 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:30:47.562 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:30:47.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:30:47.650 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:30:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:30:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:30:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:30:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:30:47.699 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:30:47.700 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:30:47.701 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:30:47.702 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:47.702 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:47.702 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:47.702 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:47.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:47.702 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:30:47.768 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-24T06:30:47.766+0000 7fe9d948f480 -1 init, newargv = 0x556bc141f8d0 newargc=15 2023-11-24T06:30:47.768 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[111529]: starting ceph client 2023-11-24T06:30:47.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[111529]: starting fuse 2023-11-24T06:30:48.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:30:48.959 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:30:49.024 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:30:49.024 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:30:49.024 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:30:49.024 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:30:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:30:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:30:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:30:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:30:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:30:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:30:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:30:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:30:49.026 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:30:49.026 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:30:49.026 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:30:49.026 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:30:49.026 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:30:49.026 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:30:49.026 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:30:49.026 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:30:49.027 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:30:49.027 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:30:49.027 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:30:49.027 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:30:49.027 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:30:49.027 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:30:49.027 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:30:49.027 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:30:49.028 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:30:49.028 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:49.028 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:30:49.028 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:49.028 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:30:49.028 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:49.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:49.030 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:30:49.088 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-24T06:30:49.088 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:30:49.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-24T06:30:49.089 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:30:49.090 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-24T06:30:49.091 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-24T06:30:49.286 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.111529.asok 2023-11-24T06:30:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:30:49.305 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.111529.asok 2023-11-24T06:30:49.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:49.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.111529.asok status 2023-11-24T06:30:49.555 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:30:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-24T06:30:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:30:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:30:49.555 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "111529", 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6870, 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-24T06:30:49.556 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 6870 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1980291242 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1980291242 2023-11-24T06:30:49.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:30:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.6870 192.168.0.1:0/1980291242", 2023-11-24T06:30:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1980291242", 2023-11-24T06:30:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-24T06:30:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 44, 2023-11-24T06:30:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 73, 2023-11-24T06:30:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:30:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-24T06:30:49.558 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-24T06:30:49.558 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:30:49.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:49.570 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-24T06:30:49.570 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:49.630 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-24T06:30:49.631 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:49.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:49.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:49.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:30:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.960+0000 7fbfb93ba640 1 Processor -- start 2023-11-24T06:30:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.960+0000 7fbfb93ba640 1 -- start start 2023-11-24T06:30:49.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.962+0000 7fbfb93ba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbfb4071f00 0x7fbfb4146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.962+0000 7fbfb93ba640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb40b2e10 con 0x7fbfb406c7c0 2023-11-24T06:30:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.962+0000 7fbfb93ba640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb406cc10 con 0x7fbfb4071410 2023-11-24T06:30:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.962+0000 7fbfb93ba640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb406cec0 con 0x7fbfb4071f00 2023-11-24T06:30:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.962+0000 7fbfb37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbfb4071f00 0x7fbfb4146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.962+0000 7fbfb8bb9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbfb406c7c0 0x7fbfb4166cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60348/0 (socket says 172.21.15.16:60348) 2023-11-24T06:30:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.962+0000 7fbfb8bb9640 1 -- 172.21.15.16:0/664602970 learned_addr learned my addr 172.21.15.16:0/664602970 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.963+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/664602970 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167689541 0 0) 0x7fbfb40b2e10 con 0x7fbfb406c7c0 2023-11-24T06:30:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.963+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/664602970 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf9c003520 con 0x7fbfb406c7c0 2023-11-24T06:30:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.963+0000 7fbfb37fe640 1 -- 172.21.15.16:0/664602970 >> v1:172.21.15.173:6789/0 conn(0x7fbfb4071410 legacy=0x7fbfb40717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.963+0000 7fbfb37fe640 1 -- 172.21.15.16:0/664602970 >> v1:172.21.15.16:6789/0 conn(0x7fbfb406c7c0 legacy=0x7fbfb4166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.963+0000 7fbfb37fe640 1 -- 172.21.15.16:0/664602970 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb406d1a0 con 0x7fbfb4071f00 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.964+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/664602970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4071f00 0x7fbfb4146770 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80097a0 tx=0x7fbfa802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95b86f3ca8657cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.964+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/664602970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8031830 con 0x7fbfb4071f00 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.964+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/664602970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa8031990 con 0x7fbfb4071f00 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.964+0000 7fbfb2ffd640 1 -- 172.21.15.16:0/664602970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8031d20 con 0x7fbfb4071f00 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.965+0000 7fbfb93ba640 1 -- 172.21.15.16:0/664602970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4071f00 msgr2=0x7fbfb4146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.965+0000 7fbfb93ba640 1 --2- 172.21.15.16:0/664602970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4071f00 0x7fbfb4146770 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80097a0 tx=0x7fbfa802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.965+0000 7fbfb93ba640 1 -- 172.21.15.16:0/664602970 shutdown_connections 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.965+0000 7fbfb93ba640 1 --2- 172.21.15.16:0/664602970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb4071f00 0x7fbfb4146770 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.965+0000 7fbfb93ba640 1 -- 172.21.15.16:0/664602970 >> 172.21.15.16:0/664602970 conn(0x7fbfb406a800 msgr2=0x7fbfb406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.965+0000 7fbfb93ba640 1 -- 172.21.15.16:0/664602970 shutdown_connections 2023-11-24T06:30:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.965+0000 7fbfb93ba640 1 -- 172.21.15.16:0/664602970 wait complete. 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.967+0000 7fbfb93ba640 1 Processor -- start 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.967+0000 7fbfb93ba640 1 -- start start 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.968+0000 7fbfb93ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb406c7c0 0x7fbfb41345b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.968+0000 7fbfb93ba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbfb4071410 0x7fbfb4144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.968+0000 7fbfb93ba640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfb4071f00 0x7fbfb4144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.968+0000 7fbfb93ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4134af0 con 0x7fbfb406c7c0 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.968+0000 7fbfb93ba640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb41f2a70 con 0x7fbfb4071f00 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.968+0000 7fbfb93ba640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb415f440 con 0x7fbfb4071410 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.968+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb406c7c0 0x7fbfb41345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.969+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb406c7c0 0x7fbfb41345b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40062/0 (socket says 172.21.15.16:40062) 2023-11-24T06:30:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.969+0000 7fbfb3fff640 1 -- 172.21.15.16:0/1123178367 learned_addr learned my addr 172.21.15.16:0/1123178367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.969+0000 7fbfb8bb9640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfb4071f00 0x7fbfb4144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.969+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbfb4071410 0x7fbfb4144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.969+0000 7fbfb3fff640 1 -- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbfb4071410 msgr2=0x7fbfb4144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.969+0000 7fbfb3fff640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbfb4071410 0x7fbfb4144220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.969+0000 7fbfb3fff640 1 -- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfb4071f00 msgr2=0x7fbfb4144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.969+0000 7fbfb3fff640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfb4071f00 0x7fbfb4144760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.969+0000 7fbfb3fff640 1 -- 172.21.15.16:0/1123178367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8009400 con 0x7fbfb406c7c0 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.970+0000 7fbfb3fff640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb406c7c0 0x7fbfb41345b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400c450 tx=0x7fbfa400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.970+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/1123178367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40052d0 con 0x7fbfb406c7c0 2023-11-24T06:30:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.970+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/1123178367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa4005430 con 0x7fbfb406c7c0 2023-11-24T06:30:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.970+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/1123178367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4012870 con 0x7fbfb406c7c0 2023-11-24T06:30:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.970+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb415f720 con 0x7fbfb406c7c0 2023-11-24T06:30:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.970+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb415fce0 con 0x7fbfb406c7c0 2023-11-24T06:30:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.971+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb4067450 con 0x7fbfb406c7c0 2023-11-24T06:30:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.972+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/1123178367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fbfa40129d0 con 0x7fbfb406c7c0 2023-11-24T06:30:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.973+0000 7fbfb0ff9640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbfa006b230 0x7fbfa006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.973+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/1123178367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fbfa4089940 con 0x7fbfb406c7c0 2023-11-24T06:30:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.974+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbfa006b230 0x7fbfa006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.975+0000 7fbfb37fe640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbfa006b230 0x7fbfa006d6f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8036000 tx=0x7fbfa803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:49.978+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/1123178367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbfa4054370 con 0x7fbfb406c7c0 2023-11-24T06:30:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.214+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbfb406b3f0 con 0x7fbfb406c7c0 2023-11-24T06:30:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.214+0000 7fbfb0ff9640 1 -- 172.21.15.16:0/1123178367 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbfa4057990 con 0x7fbfb406c7c0 2023-11-24T06:30:50.217 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:30:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbfa006b230 msgr2=0x7fbfa006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbfa006b230 0x7fbfa006d6f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8036000 tx=0x7fbfa803c040 comp rx=0 tx=0).stop 2023-11-24T06:30:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb406c7c0 msgr2=0x7fbfb41345b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb406c7c0 0x7fbfb41345b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400c450 tx=0x7fbfa400c910 comp rx=0 tx=0).stop 2023-11-24T06:30:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 shutdown_connections 2023-11-24T06:30:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fbfa006b230 0x7fbfa006d6f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbfb4071f00 0x7fbfb4144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbfb4071410 0x7fbfb4144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 --2- 172.21.15.16:0/1123178367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfb406c7c0 0x7fbfb41345b0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 >> 172.21.15.16:0/1123178367 conn(0x7fbfb406a800 msgr2=0x7fbfb40717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 shutdown_connections 2023-11-24T06:30:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:50.217+0000 7fbfb93ba640 1 -- 172.21.15.16:0/1123178367 wait complete. 2023-11-24T06:30:50.239 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:30:50.239 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:30:50.239 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2023-11-24T06:30:50.239 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:30:50.239 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:50.239 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:30:50.239 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:30:50.240 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:30:50.240 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:30:50.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:50.240 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2023-11-24T06:30:50.303 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:30:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:30:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:30:50.303 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2023-11-24T06:30:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:50.304 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:50.304 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2023-11-24T06:30:50.304 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2023-11-24T06:30:50.305 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:30:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:50.305 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:30:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:50.305 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:30:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42153sec preferred_lft 42153sec 2023-11-24T06:30:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2023-11-24T06:30:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:50.306 INFO:teuthology.orchestra.run.smithi173.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:30:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:30:50.306 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:30:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::c4e1:59ff:feae:2f56/64 scope link 2023-11-24T06:30:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:50.307 INFO:teuthology.orchestra.run.smithi173.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:30:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ea:3b:9f:a2:9b:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:50.307 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::e83b:9fff:fea2:9ba6/64 scope link 2023-11-24T06:30:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:50.308 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:30:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::64de:a3ff:fee0:a827/64 scope link 2023-11-24T06:30:50.308 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:30:50.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:50.309 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:30:50.399 INFO:teuthology.orchestra.run.smithi173.stdout:1 2023-11-24T06:30:50.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:50.403 DEBUG:teuthology.orchestra.run.smithi173:> route 2023-11-24T06:30:50.456 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2023-11-24T06:30:50.457 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:30:50.457 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:30:50.457 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:30:50.457 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:30:50.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:50.457 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:30:50.457 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:30:50.457 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:30:50.458 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:30:50.458 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:30:50.458 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:30:50.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:50.642 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2023-11-24T06:30:50.693 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:30:50.693 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:30:50.693 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:30:50.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:50.694 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2023-11-24T06:30:50.789 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:30:50.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:30:50.790 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:50.844 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:30:50.844 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:30:50.844 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:50.896 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:30:50.991 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:30:51.046 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:30:51.046 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:30:51.046 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:30:51.046 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:30:51.047 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:30:51.047 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:30:51.047 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:30:51.048 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:30:51.049 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:30:51.050 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:51.050 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:51.050 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:51.050 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:51.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:51.051 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:30:51.107 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:30:51.107 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:30:51.115 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:30:51.196 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:30:51.240 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:30:51.240 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:30:51.240 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:30:51.240 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:30:51.240 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:30:51.240 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:30:51.240 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:30:51.240 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:30:51.241 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:51.242 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:30:51.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:51.244 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:30:51.306 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:2023-11-24T06:30:51.303+0000 7ffafda38480 -1 init, newargv = 0x559bc0749840 newargc=15 2023-11-24T06:30:51.307 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[97548]: starting ceph client 2023-11-24T06:30:51.319 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[97548]: starting fuse 2023-11-24T06:30:51.331 INFO:teuthology.orchestra.run.smithi173.stdout:39 2023-11-24T06:30:51.332 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-11-24T06:30:51.332 DEBUG:teuthology.orchestra.run.smithi173:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:30:51.332 DEBUG:teuthology.orchestra.run.smithi173:> import glob 2023-11-24T06:30:51.332 DEBUG:teuthology.orchestra.run.smithi173:> import re 2023-11-24T06:30:51.332 DEBUG:teuthology.orchestra.run.smithi173:> import os 2023-11-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi173:> import subprocess 2023-11-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi173:> def _find_admin_socket(client_name): 2023-11-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi173:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi173:> files = glob.glob(asok_path) 2023-11-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi173:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi173:> # Given a non-glob path, it better be there 2023-11-24T06:30:51.333 DEBUG:teuthology.orchestra.run.smithi173:> if "*" not in asok_path: 2023-11-24T06:30:51.334 DEBUG:teuthology.orchestra.run.smithi173:> assert(len(files) == 1) 2023-11-24T06:30:51.334 DEBUG:teuthology.orchestra.run.smithi173:> return files[0] 2023-11-24T06:30:51.334 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:30:51.334 DEBUG:teuthology.orchestra.run.smithi173:> for f in files: 2023-11-24T06:30:51.334 DEBUG:teuthology.orchestra.run.smithi173:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:30:51.334 DEBUG:teuthology.orchestra.run.smithi173:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:30:51.335 DEBUG:teuthology.orchestra.run.smithi173:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:30:51.335 DEBUG:teuthology.orchestra.run.smithi173:> contents = proc_f.read() 2023-11-24T06:30:51.336 DEBUG:teuthology.orchestra.run.smithi173:> if mountpoint in contents: 2023-11-24T06:30:51.336 DEBUG:teuthology.orchestra.run.smithi173:> return f 2023-11-24T06:30:51.336 DEBUG:teuthology.orchestra.run.smithi173:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:30:51.336 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:30:51.336 DEBUG:teuthology.orchestra.run.smithi173:> print(_find_admin_socket("client.1")) 2023-11-24T06:30:51.336 DEBUG:teuthology.orchestra.run.smithi173:> ' 2023-11-24T06:30:51.535 INFO:teuthology.orchestra.run.smithi173.stdout:/var/run/ceph/ceph-client.1.97548.asok 2023-11-24T06:30:51.539 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:30:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:30:51.552 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.97548.asok 2023-11-24T06:30:51.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:51.552 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.97548.asok status 2023-11-24T06:30:51.794 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-11-24T06:30:51.794 INFO:teuthology.orchestra.run.smithi173.stdout: "metadata": { 2023-11-24T06:30:51.794 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:30:51.794 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:30:51.794 INFO:teuthology.orchestra.run.smithi173.stdout: "entity_id": "1", 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "hostname": "smithi173", 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "pid": "97548", 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "root": "/" 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_count": 0, 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_pinned_count": 0, 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "id": 5764, 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "inst": { 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "name": { 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "client", 2023-11-24T06:30:51.795 INFO:teuthology.orchestra.run.smithi173.stdout: "num": 5764 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 203720914 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 203720914 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:30:51.796 INFO:teuthology.orchestra.run.smithi173.stdout: "inst_str": "client.5764 192.168.0.1:0/203720914", 2023-11-24T06:30:51.797 INFO:teuthology.orchestra.run.smithi173.stdout: "addr_str": "192.168.0.1:0/203720914", 2023-11-24T06:30:51.797 INFO:teuthology.orchestra.run.smithi173.stdout: "inode_count": 1, 2023-11-24T06:30:51.797 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_epoch": 44, 2023-11-24T06:30:51.797 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch": 73, 2023-11-24T06:30:51.797 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:30:51.797 INFO:teuthology.orchestra.run.smithi173.stdout: "blocklisted": false, 2023-11-24T06:30:51.797 INFO:teuthology.orchestra.run.smithi173.stdout: "fs_name": "cephfs" 2023-11-24T06:30:51.797 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-11-24T06:30:51.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:51.809 DEBUG:teuthology.orchestra.run.smithi173:> stat --file-system '--printf=%T 2023-11-24T06:30:51.810 DEBUG:teuthology.orchestra.run.smithi173:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:51.865 INFO:teuthology.orchestra.run.smithi173.stdout:fuseblk 2023-11-24T06:30:51.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:51.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:51.866 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:30:51.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:30:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.203+0000 7f618359e640 1 Processor -- start 2023-11-24T06:30:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.203+0000 7f618359e640 1 -- start start 2023-11-24T06:30:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.204+0000 7f618359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 0x7f61840d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.204+0000 7f618359e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6184160c70 0x7f61840d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.204+0000 7f618359e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61840cf6a0 con 0x7f61840c6be0 2023-11-24T06:30:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.204+0000 7f618359e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61840d1430 con 0x7f6184160c70 2023-11-24T06:30:52.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.204+0000 7f618359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61840b4370 con 0x7f6184160890 2023-11-24T06:30:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.205+0000 7f617bfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6184160c70 0x7f61840d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.205+0000 7f618259c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 0x7f61840d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.205+0000 7f617bfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6184160c70 0x7f61840d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:45674/0 (socket says 172.21.15.16:45674) 2023-11-24T06:30:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.205+0000 7f617bfff640 1 -- 172.21.15.16:0/4098848654 learned_addr learned my addr 172.21.15.16:0/4098848654 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:52.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.206+0000 7f617bfff640 1 -- 172.21.15.16:0/4098848654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 msgr2=0x7f61840d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.206+0000 7f617bfff640 1 --2- 172.21.15.16:0/4098848654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 0x7f61840d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.206+0000 7f617bfff640 1 -- 172.21.15.16:0/4098848654 >> v1:172.21.15.173:6789/0 conn(0x7f61840c6be0 legacy=0x7f6184161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.206+0000 7f617bfff640 1 -- 172.21.15.16:0/4098848654 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61840ce7c0 con 0x7f6184160c70 2023-11-24T06:30:52.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.207+0000 7f617bfff640 1 --2- 172.21.15.16:0/4098848654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6184160c70 0x7f61840d1670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f616c009870 tx=0x7f616c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900f66a185faae7f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.208+0000 7f6181d9b640 1 -- 172.21.15.16:0/4098848654 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c0317c0 con 0x7f6184160c70 2023-11-24T06:30:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.208+0000 7f6181d9b640 1 -- 172.21.15.16:0/4098848654 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c031920 con 0x7f6184160c70 2023-11-24T06:30:52.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.208+0000 7f6181d9b640 1 -- 172.21.15.16:0/4098848654 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c031cb0 con 0x7f6184160c70 2023-11-24T06:30:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.209+0000 7f618359e640 1 -- 172.21.15.16:0/4098848654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6184160c70 msgr2=0x7f61840d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.209+0000 7f618359e640 1 --2- 172.21.15.16:0/4098848654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6184160c70 0x7f61840d1670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f616c009870 tx=0x7f616c02b860 comp rx=0 tx=0).stop 2023-11-24T06:30:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.209+0000 7f618359e640 1 -- 172.21.15.16:0/4098848654 shutdown_connections 2023-11-24T06:30:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.209+0000 7f618359e640 1 --2- 172.21.15.16:0/4098848654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6184160c70 0x7f61840d1670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:52.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.209+0000 7f618359e640 1 --2- 172.21.15.16:0/4098848654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 0x7f61840d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.209+0000 7f618359e640 1 -- 172.21.15.16:0/4098848654 >> 172.21.15.16:0/4098848654 conn(0x7f61840d3090 msgr2=0x7f61840d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.209+0000 7f618359e640 1 -- 172.21.15.16:0/4098848654 shutdown_connections 2023-11-24T06:30:52.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.209+0000 7f618359e640 1 -- 172.21.15.16:0/4098848654 wait complete. 2023-11-24T06:30:52.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.211+0000 7f618359e640 1 Processor -- start 2023-11-24T06:30:52.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.212+0000 7f618359e640 1 -- start start 2023-11-24T06:30:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.213+0000 7f618359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61840c6be0 0x7f61840b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.213+0000 7f618359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 0x7f618415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.213+0000 7f618359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6184160c70 0x7f618415cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.213+0000 7f618359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61841f3210 con 0x7f6184160c70 2023-11-24T06:30:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.213+0000 7f618359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61840b72e0 con 0x7f61840c6be0 2023-11-24T06:30:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.213+0000 7f618359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61840b75c0 con 0x7f6184160890 2023-11-24T06:30:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f617bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 0x7f618415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f6182d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6184160c70 0x7f618415cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:52.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f6182d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6184160c70 0x7f618415cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40076/0 (socket says 172.21.15.16:40076) 2023-11-24T06:30:52.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f6182d9d640 1 -- 172.21.15.16:0/1074318286 learned_addr learned my addr 172.21.15.16:0/1074318286 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:52.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f618259c640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61840c6be0 0x7f61840b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:52.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f6182d9d640 1 -- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 msgr2=0x7f618415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:52.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f6182d9d640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 0x7f618415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:52.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f6182d9d640 1 -- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61840c6be0 msgr2=0x7f61840b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:52.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f6182d9d640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61840c6be0 0x7f61840b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:52.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f6182d9d640 1 -- 172.21.15.16:0/1074318286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c0094d0 con 0x7f6184160c70 2023-11-24T06:30:52.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.214+0000 7f6182d9d640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6184160c70 0x7f618415cc50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f617c00c6b0 tx=0x7f617c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:52.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.215+0000 7f617affd640 1 -- 172.21.15.16:0/1074318286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c003b70 con 0x7f6184160c70 2023-11-24T06:30:52.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.215+0000 7f617affd640 1 -- 172.21.15.16:0/1074318286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c003cf0 con 0x7f6184160c70 2023-11-24T06:30:52.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.215+0000 7f617affd640 1 -- 172.21.15.16:0/1074318286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c003ea0 con 0x7f6184160c70 2023-11-24T06:30:52.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.215+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61840b78a0 con 0x7f6184160c70 2023-11-24T06:30:52.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.215+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61840b7e60 con 0x7f6184160c70 2023-11-24T06:30:52.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.216+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6184067450 con 0x7f6184160c70 2023-11-24T06:30:52.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.217+0000 7f617affd640 1 -- 172.21.15.16:0/1074318286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f617c005310 con 0x7f6184160c70 2023-11-24T06:30:52.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.218+0000 7f617affd640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f616406b2b0 0x7f616406d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:52.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.219+0000 7f617affd640 1 -- 172.21.15.16:0/1074318286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f617c0899e0 con 0x7f6184160c70 2023-11-24T06:30:52.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.219+0000 7f618259c640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f616406b2b0 0x7f616406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:52.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.220+0000 7f618259c640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f616406b2b0 0x7f616406d770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6174009a30 tx=0x7f6174008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:52.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.224+0000 7f617affd640 1 -- 172.21.15.16:0/1074318286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f617c054520 con 0x7f6184160c70 2023-11-24T06:30:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:52.457+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f618408c7e0 con 0x7f6184160c70 2023-11-24T06:30:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.088+0000 7f617affd640 1 -- 172.21.15.16:0/1074318286 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 136+0+0 (secure 0 0 0) 0x7f617c057b40 con 0x7f6184160c70 2023-11-24T06:30:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.091+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f616406b2b0 msgr2=0x7f616406d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.091+0000 7f618359e640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f616406b2b0 0x7f616406d770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6174009a30 tx=0x7f6174008040 comp rx=0 tx=0).stop 2023-11-24T06:30:53.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6184160c70 msgr2=0x7f618415cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6184160c70 0x7f618415cc50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f617c00c6b0 tx=0x7f617c00cb70 comp rx=0 tx=0).stop 2023-11-24T06:30:53.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 shutdown_connections 2023-11-24T06:30:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f616406b2b0 0x7f616406d770 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6184160c70 0x7f618415cc50 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6184160890 0x7f618415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 --2- 172.21.15.16:0/1074318286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61840c6be0 0x7f61840b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 >> 172.21.15.16:0/1074318286 conn(0x7f61840d3090 msgr2=0x7f61840c7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 shutdown_connections 2023-11-24T06:30:53.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.092+0000 7f618359e640 1 -- 172.21.15.16:0/1074318286 wait complete. 2023-11-24T06:30:53.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:30:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.336+0000 7fabb6f61640 1 Processor -- start 2023-11-24T06:30:53.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.336+0000 7fabb6f61640 1 -- start start 2023-11-24T06:30:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.338+0000 7fabb6f61640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb00cf9a0 0x7fabb015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.338+0000 7fabb6f61640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fabb015e1e0 0x7fabb00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.338+0000 7fabb6f61640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb00b1ee0 con 0x7fabb0143ae0 2023-11-24T06:30:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.338+0000 7fabb6f61640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb00b2780 con 0x7fabb015e1e0 2023-11-24T06:30:53.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.338+0000 7fabb6f61640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb015e990 con 0x7fabb00cf9a0 2023-11-24T06:30:53.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.339+0000 7fabb54fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fabb015e1e0 0x7fabb00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.339+0000 7fabb54fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fabb015e1e0 0x7fabb00702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:42188/0 (socket says 172.21.15.16:42188) 2023-11-24T06:30:53.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.339+0000 7fabb54fe640 1 -- 172.21.15.16:0/2085969324 learned_addr learned my addr 172.21.15.16:0/2085969324 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:53.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.339+0000 7fabaffff640 1 --2- 172.21.15.16:0/2085969324 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb00cf9a0 0x7fabb015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.339+0000 7fabb54fe640 1 -- 172.21.15.16:0/2085969324 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb00cf9a0 msgr2=0x7fabb015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:53.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.340+0000 7fabb54fe640 1 --2- 172.21.15.16:0/2085969324 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb00cf9a0 0x7fabb015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.340+0000 7fabb54fe640 1 -- 172.21.15.16:0/2085969324 >> v1:172.21.15.173:6789/0 conn(0x7fabb0143ae0 legacy=0x7fabb00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.340+0000 7fabb54fe640 1 -- 172.21.15.16:0/2085969324 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb00b2130 con 0x7fabb015e1e0 2023-11-24T06:30:53.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.340+0000 7fabaf7fe640 1 -- 172.21.15.16:0/2085969324 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151825257 0 0) 0x7fabb00b1ee0 con 0x7fabb0143ae0 2023-11-24T06:30:53.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.341+0000 7fabb54fe640 1 --2- 172.21.15.16:0/2085969324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb015e1e0 0x7fabb00702c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faba8009e50 tx=0x7faba802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb82596c677746d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:53.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.342+0000 7fabaf7fe640 1 -- 172.21.15.16:0/2085969324 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba802cb00 con 0x7fabb015e1e0 2023-11-24T06:30:53.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.342+0000 7fabaf7fe640 1 -- 172.21.15.16:0/2085969324 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba802cc60 con 0x7fabb015e1e0 2023-11-24T06:30:53.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.342+0000 7fabaf7fe640 1 -- 172.21.15.16:0/2085969324 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80343f0 con 0x7fabb015e1e0 2023-11-24T06:30:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.342+0000 7fabb6f61640 1 -- 172.21.15.16:0/2085969324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb015e1e0 msgr2=0x7fabb00702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.342+0000 7fabb6f61640 1 --2- 172.21.15.16:0/2085969324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb015e1e0 0x7fabb00702c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faba8009e50 tx=0x7faba802bf60 comp rx=0 tx=0).stop 2023-11-24T06:30:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.343+0000 7fabb6f61640 1 -- 172.21.15.16:0/2085969324 shutdown_connections 2023-11-24T06:30:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.343+0000 7fabb6f61640 1 --2- 172.21.15.16:0/2085969324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb015e1e0 0x7fabb00702c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.343+0000 7fabb6f61640 1 --2- 172.21.15.16:0/2085969324 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb00cf9a0 0x7fabb015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.343+0000 7fabb6f61640 1 -- 172.21.15.16:0/2085969324 >> 172.21.15.16:0/2085969324 conn(0x7fabb006a800 msgr2=0x7fabb006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.343+0000 7fabb6f61640 1 -- 172.21.15.16:0/2085969324 shutdown_connections 2023-11-24T06:30:53.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.343+0000 7fabb6f61640 1 -- 172.21.15.16:0/2085969324 wait complete. 2023-11-24T06:30:53.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.344+0000 7fabb6f61640 1 Processor -- start 2023-11-24T06:30:53.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.345+0000 7fabb6f61640 1 -- start start 2023-11-24T06:30:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabb6f61640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb00cf9a0 0x7fabb015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabb6f61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb0143ae0 0x7fabb0141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabb6f61640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb015e1e0 0x7fabb00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabb6f61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00c5ce0 con 0x7fabb0143ae0 2023-11-24T06:30:53.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabb6f61640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00c7eb0 con 0x7fabb00cf9a0 2023-11-24T06:30:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabb6f61640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb00c5440 con 0x7fabb015e1e0 2023-11-24T06:30:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb0143ae0 0x7fabb0141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb0143ae0 0x7fabb0141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39784/0 (socket says 172.21.15.16:39784) 2023-11-24T06:30:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabaffff640 1 -- 172.21.15.16:0/3471071726 learned_addr learned my addr 172.21.15.16:0/3471071726 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabb4cfd640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb00cf9a0 0x7fabb015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.346+0000 7fabb54fe640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb015e1e0 0x7fabb00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabaffff640 1 -- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb015e1e0 msgr2=0x7fabb00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabaffff640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb015e1e0 0x7fabb00b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabaffff640 1 -- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb00cf9a0 msgr2=0x7fabb015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabaffff640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb00cf9a0 0x7fabb015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabaffff640 1 -- 172.21.15.16:0/3471071726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba8009b30 con 0x7fabb0143ae0 2023-11-24T06:30:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabaffff640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb0143ae0 0x7fabb0141930 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e9b0 tx=0x7fab9c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabad7fa640 1 -- 172.21.15.16:0/3471071726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c009670 con 0x7fabb0143ae0 2023-11-24T06:30:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabad7fa640 1 -- 172.21.15.16:0/3471071726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c00c9a0 con 0x7fabb0143ae0 2023-11-24T06:30:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb00c5720 con 0x7fabb0143ae0 2023-11-24T06:30:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabad7fa640 1 -- 172.21.15.16:0/3471071726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00cd50 con 0x7fabb0143ae0 2023-11-24T06:30:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.347+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb00c8130 con 0x7fabb0143ae0 2023-11-24T06:30:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.348+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb015ae10 con 0x7fabb0143ae0 2023-11-24T06:30:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.349+0000 7fabad7fa640 1 -- 172.21.15.16:0/3471071726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fab9c013470 con 0x7fabb0143ae0 2023-11-24T06:30:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.350+0000 7fabad7fa640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab9406b230 0x7fab9406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.350+0000 7fabad7fa640 1 -- 172.21.15.16:0/3471071726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fab9c08a0b0 con 0x7fabb0143ae0 2023-11-24T06:30:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.351+0000 7fabb4cfd640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab9406b230 0x7fab9406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.352+0000 7fabb4cfd640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab9406b230 0x7fab9406d6f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faba4000f60 tx=0x7faba4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:53.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.354+0000 7fabad7fa640 1 -- 172.21.15.16:0/3471071726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab9c0bce90 con 0x7fabb0143ae0 2023-11-24T06:30:53.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.572+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fabb015e640 con 0x7fab9406b230 2023-11-24T06:30:53.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.576+0000 7fabad7fa640 1 -- 172.21.15.16:0/3471071726 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7faba4001cf0 con 0x7fab9406b230 2023-11-24T06:30:53.578 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:30:53.579 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:30:53.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephfs" 2023-11-24T06:30:53.579 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:30:53.579 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:30:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.578+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab9406b230 msgr2=0x7fab9406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.578+0000 7fabb6f61640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab9406b230 0x7fab9406d6f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faba4000f60 tx=0x7faba4009040 comp rx=0 tx=0).stop 2023-11-24T06:30:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.578+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb0143ae0 msgr2=0x7fabb0141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.578+0000 7fabb6f61640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb0143ae0 0x7fabb0141930 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00e9b0 tx=0x7fab9c00ee70 comp rx=0 tx=0).stop 2023-11-24T06:30:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.578+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 shutdown_connections 2023-11-24T06:30:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.578+0000 7fabb6f61640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab9406b230 0x7fab9406d6f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.578+0000 7fabb6f61640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabb015e1e0 0x7fabb00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.578+0000 7fabb6f61640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb0143ae0 0x7fabb0141930 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.578+0000 7fabb6f61640 1 --2- 172.21.15.16:0/3471071726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabb00cf9a0 0x7fabb015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.579+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 >> 172.21.15.16:0/3471071726 conn(0x7fabb006a800 msgr2=0x7fabb006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.579+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 shutdown_connections 2023-11-24T06:30:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.579+0000 7fabb6f61640 1 -- 172.21.15.16:0/3471071726 wait complete. 2023-11-24T06:30:53.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:30:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.825+0000 7f59166bb640 1 Processor -- start 2023-11-24T06:30:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.825+0000 7f59166bb640 1 -- start start 2023-11-24T06:30:53.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:30:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.827+0000 7f59166bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59100ceea0 0x7f59100d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.827+0000 7f59166bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59100b2c70 con 0x7f59100d0150 2023-11-24T06:30:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.827+0000 7f59166bb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59100cf6d0 con 0x7f591015d3c0 2023-11-24T06:30:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.827+0000 7f59166bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59100cf3f0 con 0x7f59100ceea0 2023-11-24T06:30:53.833 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:30:53.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.828+0000 7f5915eba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59100ceea0 0x7f59100d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.828+0000 7f5914eb8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f59100d0150 0x7f59100b3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40386/0 (socket says 172.21.15.16:40386) 2023-11-24T06:30:53.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.828+0000 7f5914eb8640 1 -- 172.21.15.16:0/1738081649 learned_addr learned my addr 172.21.15.16:0/1738081649 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:53.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.828+0000 7f5915eba640 1 -- 172.21.15.16:0/1738081649 >> v1:172.21.15.173:6789/0 conn(0x7f591015d3c0 legacy=0x7f59100cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.828+0000 7f5915eba640 1 -- 172.21.15.16:0/1738081649 >> v1:172.21.15.16:6789/0 conn(0x7f59100d0150 legacy=0x7f59100b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.828+0000 7f5915eba640 1 -- 172.21.15.16:0/1738081649 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59100b2620 con 0x7f59100ceea0 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.830+0000 7f5915eba640 1 --2- 172.21.15.16:0/1738081649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59100ceea0 0x7f59100d1f20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5908009d90 tx=0x7f590802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d7d180e89b023b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.830+0000 7f5907fff640 1 -- 172.21.15.16:0/1738081649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590802cd00 con 0x7f59100ceea0 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.830+0000 7f5907fff640 1 -- 172.21.15.16:0/1738081649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590802ce60 con 0x7f59100ceea0 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.830+0000 7f5907fff640 1 -- 172.21.15.16:0/1738081649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908034660 con 0x7f59100ceea0 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.830+0000 7f59166bb640 1 -- 172.21.15.16:0/1738081649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59100ceea0 msgr2=0x7f59100d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.830+0000 7f59166bb640 1 --2- 172.21.15.16:0/1738081649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59100ceea0 0x7f59100d1f20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5908009d90 tx=0x7f590802bf50 comp rx=0 tx=0).stop 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.831+0000 7f59166bb640 1 -- 172.21.15.16:0/1738081649 shutdown_connections 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.831+0000 7f59166bb640 1 --2- 172.21.15.16:0/1738081649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59100ceea0 0x7f59100d1f20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.831+0000 7f59166bb640 1 -- 172.21.15.16:0/1738081649 >> 172.21.15.16:0/1738081649 conn(0x7f59100d3050 msgr2=0x7f59100d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.831+0000 7f59166bb640 1 -- 172.21.15.16:0/1738081649 shutdown_connections 2023-11-24T06:30:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.831+0000 7f59166bb640 1 -- 172.21.15.16:0/1738081649 wait complete. 2023-11-24T06:30:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.832+0000 7f59166bb640 1 Processor -- start 2023-11-24T06:30:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.833+0000 7f59166bb640 1 -- start start 2023-11-24T06:30:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f59166bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59100ceea0 0x7f59100ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f59166bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59100d0150 0x7f5910157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f59166bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f591015d3c0 0x7f591015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f59166bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59100cc5b0 con 0x7f591015d3c0 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f59166bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591015b260 con 0x7f59100d0150 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f59166bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59100ca700 con 0x7f59100ceea0 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f5915eba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f591015d3c0 0x7f591015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f5915eba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f591015d3c0 0x7f591015ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39796/0 (socket says 172.21.15.16:39796) 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f5915eba640 1 -- 172.21.15.16:0/1751155111 learned_addr learned my addr 172.21.15.16:0/1751155111 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.834+0000 7f59156b9640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59100ceea0 0x7f59100ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.835+0000 7f5914eb8640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59100d0150 0x7f5910157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.835+0000 7f5915eba640 1 -- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59100ceea0 msgr2=0x7f59100ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.835+0000 7f5915eba640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59100ceea0 0x7f59100ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.835+0000 7f5915eba640 1 -- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59100d0150 msgr2=0x7f5910157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.835+0000 7f5915eba640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59100d0150 0x7f5910157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.835+0000 7f5915eba640 1 -- 172.21.15.16:0/1751155111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5908009a70 con 0x7f591015d3c0 2023-11-24T06:30:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.835+0000 7f5914eb8640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59100d0150 0x7f5910157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.835+0000 7f5915eba640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f591015d3c0 0x7f591015ad20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f5908009d90 tx=0x7f5908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.835+0000 7f5905ffb640 1 -- 172.21.15.16:0/1751155111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590803d600 con 0x7f591015d3c0 2023-11-24T06:30:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.836+0000 7f5905ffb640 1 -- 172.21.15.16:0/1751155111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590803d780 con 0x7f591015d3c0 2023-11-24T06:30:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.836+0000 7f5905ffb640 1 -- 172.21.15.16:0/1751155111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590803db30 con 0x7f591015d3c0 2023-11-24T06:30:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.836+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59100ca9e0 con 0x7f591015d3c0 2023-11-24T06:30:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.836+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59100cafa0 con 0x7f591015d3c0 2023-11-24T06:30:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.837+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591008ba80 con 0x7f591015d3c0 2023-11-24T06:30:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.838+0000 7f5905ffb640 1 -- 172.21.15.16:0/1751155111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f590803b040 con 0x7f591015d3c0 2023-11-24T06:30:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.839+0000 7f5905ffb640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f58f406b230 0x7f58f406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.839+0000 7f5905ffb640 1 -- 172.21.15.16:0/1751155111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f59080ac500 con 0x7f591015d3c0 2023-11-24T06:30:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.840+0000 7f59156b9640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f58f406b230 0x7f58f406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.842+0000 7f59156b9640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f58f406b230 0x7f58f406d6f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f590c000f60 tx=0x7f590c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:53.844+0000 7f5905ffb640 1 -- 172.21.15.16:0/1751155111 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5908076f30 con 0x7f591015d3c0 2023-11-24T06:30:54.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.043+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5910067450 con 0x7f591015d3c0 2023-11-24T06:30:54.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.044+0000 7f5905ffb640 1 -- 172.21.15.16:0/1751155111 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f590807a550 con 0x7f591015d3c0 2023-11-24T06:30:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f58f406b230 msgr2=0x7f58f406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f58f406b230 0x7f58f406d6f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f590c000f60 tx=0x7f590c009040 comp rx=0 tx=0).stop 2023-11-24T06:30:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f591015d3c0 msgr2=0x7f591015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f591015d3c0 0x7f591015ad20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f5908009d90 tx=0x7f5908004a40 comp rx=0 tx=0).stop 2023-11-24T06:30:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 shutdown_connections 2023-11-24T06:30:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f58f406b230 0x7f58f406d6f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f591015d3c0 0x7f591015ad20 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59100d0150 0x7f5910157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 --2- 172.21.15.16:0/1751155111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59100ceea0 0x7f59100ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 >> 172.21.15.16:0/1751155111 conn(0x7f59100d3050 msgr2=0x7f591006aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 shutdown_connections 2023-11-24T06:30:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.047+0000 7f59166bb640 1 -- 172.21.15.16:0/1751155111 wait complete. 2023-11-24T06:30:54.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp5352 2023-11-24T06:30:54.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.290+0000 7fce65dfc640 1 Processor -- start 2023-11-24T06:30:54.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.291+0000 7fce65dfc640 1 -- start start 2023-11-24T06:30:54.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.293+0000 7fce65dfc640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fce600b3ae0 0x7fce6015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.293+0000 7fce65dfc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce600cf290 0x7fce600cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.293+0000 7fce65dfc640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce60070c40 con 0x7fce600ae5a0 2023-11-24T06:30:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.293+0000 7fce65dfc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce60071450 con 0x7fce600cf290 2023-11-24T06:30:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.293+0000 7fce65dfc640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce600cfc60 con 0x7fce600b3ae0 2023-11-24T06:30:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.293+0000 7fce5ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce600cf290 0x7fce600cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.293+0000 7fce5ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce600cf290 0x7fce600cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39812/0 (socket says 172.21.15.16:39812) 2023-11-24T06:30:54.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.293+0000 7fce5ffff640 1 -- 172.21.15.16:0/934243647 learned_addr learned my addr 172.21.15.16:0/934243647 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:54.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.294+0000 7fce57fff640 1 --2- 172.21.15.16:0/934243647 >> v2:172.21.15.173:3300/0 conn(0x7fce600b3ae0 0x7fce6015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.294+0000 7fce5ffff640 1 -- 172.21.15.16:0/934243647 >> v2:172.21.15.173:3300/0 conn(0x7fce600b3ae0 msgr2=0x7fce6015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:54.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.294+0000 7fce5ffff640 1 --2- 172.21.15.16:0/934243647 >> v2:172.21.15.173:3300/0 conn(0x7fce600b3ae0 0x7fce6015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.294+0000 7fce5ffff640 1 -- 172.21.15.16:0/934243647 >> v1:172.21.15.173:6789/0 conn(0x7fce600ae5a0 legacy=0x7fce6006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:54.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.294+0000 7fce5ffff640 1 -- 172.21.15.16:0/934243647 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6006c470 con 0x7fce600cf290 2023-11-24T06:30:54.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.295+0000 7fce5ffff640 1 --2- 172.21.15.16:0/934243647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 0x7fce600cf720 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fce600cfea0 tx=0x7fce5802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439489248b1e9df8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.295+0000 7fce5effd640 1 -- 172.21.15.16:0/934243647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5802cb20 con 0x7fce600cf290 2023-11-24T06:30:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.295+0000 7fce5effd640 1 -- 172.21.15.16:0/934243647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5802cc80 con 0x7fce600cf290 2023-11-24T06:30:54.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.295+0000 7fce5effd640 1 -- 172.21.15.16:0/934243647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce580343f0 con 0x7fce600cf290 2023-11-24T06:30:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.296+0000 7fce65dfc640 1 -- 172.21.15.16:0/934243647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 msgr2=0x7fce600cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.296+0000 7fce65dfc640 1 --2- 172.21.15.16:0/934243647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 0x7fce600cf720 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fce600cfea0 tx=0x7fce5802bf80 comp rx=0 tx=0).stop 2023-11-24T06:30:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.296+0000 7fce65dfc640 1 -- 172.21.15.16:0/934243647 shutdown_connections 2023-11-24T06:30:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.296+0000 7fce65dfc640 1 --2- 172.21.15.16:0/934243647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 0x7fce600cf720 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.296+0000 7fce65dfc640 1 --2- 172.21.15.16:0/934243647 >> v2:172.21.15.173:3300/0 conn(0x7fce600b3ae0 0x7fce6015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.296+0000 7fce65dfc640 1 -- 172.21.15.16:0/934243647 >> 172.21.15.16:0/934243647 conn(0x7fce6006a800 msgr2=0x7fce6006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.296+0000 7fce65dfc640 1 -- 172.21.15.16:0/934243647 shutdown_connections 2023-11-24T06:30:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.296+0000 7fce65dfc640 1 -- 172.21.15.16:0/934243647 wait complete. 2023-11-24T06:30:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.298+0000 7fce65dfc640 1 Processor -- start 2023-11-24T06:30:54.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.298+0000 7fce65dfc640 1 -- start start 2023-11-24T06:30:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.299+0000 7fce65dfc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce600ae5a0 0x7fce601494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.299+0000 7fce65dfc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce600b3ae0 0x7fce6015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.299+0000 7fce65dfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 0x7fce6015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.299+0000 7fce65dfc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601f2c30 con 0x7fce600cf290 2023-11-24T06:30:54.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.299+0000 7fce65dfc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce60160220 con 0x7fce600ae5a0 2023-11-24T06:30:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.299+0000 7fce65dfc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6015f180 con 0x7fce600b3ae0 2023-11-24T06:30:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 0x7fce6015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 0x7fce6015cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39826/0 (socket says 172.21.15.16:39826) 2023-11-24T06:30:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5ffff640 1 -- 172.21.15.16:0/2934463301 learned_addr learned my addr 172.21.15.16:0/2934463301 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:54.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce57fff640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce600b3ae0 0x7fce6015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5f7fe640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce600ae5a0 0x7fce601494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5ffff640 1 -- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce600b3ae0 msgr2=0x7fce6015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5ffff640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce600b3ae0 0x7fce6015c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5ffff640 1 -- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce600ae5a0 msgr2=0x7fce601494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5ffff640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce600ae5a0 0x7fce601494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5ffff640 1 -- 172.21.15.16:0/2934463301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58009b30 con 0x7fce600cf290 2023-11-24T06:30:54.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.300+0000 7fce5ffff640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 0x7fce6015cc60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fce58009f40 tx=0x7fce58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:54.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.301+0000 7fce5cff9640 1 -- 172.21.15.16:0/2934463301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58007e70 con 0x7fce600cf290 2023-11-24T06:30:54.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.301+0000 7fce5cff9640 1 -- 172.21.15.16:0/2934463301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce58009510 con 0x7fce600cf290 2023-11-24T06:30:54.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.301+0000 7fce5cff9640 1 -- 172.21.15.16:0/2934463301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5803e690 con 0x7fce600cf290 2023-11-24T06:30:54.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.301+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6015f460 con 0x7fce600cf290 2023-11-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.301+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6015fa20 con 0x7fce600cf290 2023-11-24T06:30:54.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.302+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce600675d0 con 0x7fce600cf290 2023-11-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.303+0000 7fce5cff9640 1 -- 172.21.15.16:0/2934463301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fce5803e7f0 con 0x7fce600cf290 2023-11-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.304+0000 7fce5cff9640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce4006b280 0x7fce4006d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.304+0000 7fce5cff9640 1 -- 172.21.15.16:0/2934463301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fce580ac8a0 con 0x7fce600cf290 2023-11-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.305+0000 7fce5f7fe640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce4006b280 0x7fce4006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.306+0000 7fce5f7fe640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce4006b280 0x7fce4006d740 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fce50003d90 tx=0x7fce50009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.309+0000 7fce5cff9640 1 -- 172.21.15.16:0/2934463301 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce580773e0 con 0x7fce600cf290 2023-11-24T06:30:54.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.521+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp5352", "target": ["mon-mgr", ""]}) v1 -- 0x7fce60071170 con 0x7fce4006b280 2023-11-24T06:30:54.538 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:54.535+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:54.538 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:54.535+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:54.538 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:54.535+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:54.538 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:54.535+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:54.538 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:54.535+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.572+0000 7fce5cff9640 1 -- 172.21.15.16:0/2934463301 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fce50003980 con 0x7fce4006b280 2023-11-24T06:30:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce4006b280 msgr2=0x7fce4006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce4006b280 0x7fce4006d740 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fce50003d90 tx=0x7fce50009040 comp rx=0 tx=0).stop 2023-11-24T06:30:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 msgr2=0x7fce6015cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 0x7fce6015cc60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fce58009f40 tx=0x7fce58004a40 comp rx=0 tx=0).stop 2023-11-24T06:30:54.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 shutdown_connections 2023-11-24T06:30:54.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce4006b280 0x7fce4006d740 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce600cf290 0x7fce6015cc60 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce600b3ae0 0x7fce6015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 --2- 172.21.15.16:0/2934463301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce600ae5a0 0x7fce601494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 >> 172.21.15.16:0/2934463301 conn(0x7fce6006a800 msgr2=0x7fce6015aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 shutdown_connections 2023-11-24T06:30:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.575+0000 7fce65dfc640 1 -- 172.21.15.16:0/2934463301 wait complete. 2023-11-24T06:30:54.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol5605 --group_name subvol_grp5352 2023-11-24T06:30:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.819+0000 7fa8a3fff640 1 Processor -- start 2023-11-24T06:30:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.819+0000 7fa8a3fff640 1 -- start start 2023-11-24T06:30:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.821+0000 7fa8a3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8a4143c40 0x7fa8a4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.821+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 0x7fa8a40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.821+0000 7fa8a3fff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a4070e50 con 0x7fa8a40cca40 2023-11-24T06:30:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.821+0000 7fa8a3fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a40723c0 con 0x7fa8a4143c40 2023-11-24T06:30:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.821+0000 7fa8a3fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40cce70 con 0x7fa8a40717d0 2023-11-24T06:30:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.821+0000 7fa8a2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8a4143c40 0x7fa8a4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.821+0000 7fa8a2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8a4143c40 0x7fa8a4071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39828/0 (socket says 172.21.15.16:39828) 2023-11-24T06:30:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.821+0000 7fa8a2ffd640 1 -- 172.21.15.16:0/2340075211 learned_addr learned my addr 172.21.15.16:0/2340075211 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.822+0000 7fa8a27fc640 1 --2- 172.21.15.16:0/2340075211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 0x7fa8a40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.822+0000 7fa8a2ffd640 1 -- 172.21.15.16:0/2340075211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 msgr2=0x7fa8a40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.822+0000 7fa8a2ffd640 1 --2- 172.21.15.16:0/2340075211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 0x7fa8a40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.822+0000 7fa8a2ffd640 1 -- 172.21.15.16:0/2340075211 >> v1:172.21.15.16:6789/0 conn(0x7fa8a40cca40 legacy=0x7fa8a415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.822+0000 7fa8a2ffd640 1 -- 172.21.15.16:0/2340075211 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a406e170 con 0x7fa8a4143c40 2023-11-24T06:30:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.823+0000 7fa8a2ffd640 1 --2- 172.21.15.16:0/2340075211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 0x7fa8a4071290 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa894009870 tx=0x7fa89402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660d5272510bdbe0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.823+0000 7fa8a1ffb640 1 -- 172.21.15.16:0/2340075211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894031640 con 0x7fa8a4143c40 2023-11-24T06:30:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.823+0000 7fa8a1ffb640 1 -- 172.21.15.16:0/2340075211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8940317a0 con 0x7fa8a4143c40 2023-11-24T06:30:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.823+0000 7fa8a1ffb640 1 -- 172.21.15.16:0/2340075211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894031b30 con 0x7fa8a4143c40 2023-11-24T06:30:54.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.824+0000 7fa8a3fff640 1 -- 172.21.15.16:0/2340075211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 msgr2=0x7fa8a4071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.824+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/2340075211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 0x7fa8a4071290 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa894009870 tx=0x7fa89402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.824+0000 7fa8a3fff640 1 -- 172.21.15.16:0/2340075211 shutdown_connections 2023-11-24T06:30:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.824+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/2340075211 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 0x7fa8a40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.824+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/2340075211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 0x7fa8a4071290 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.824+0000 7fa8a3fff640 1 -- 172.21.15.16:0/2340075211 >> 172.21.15.16:0/2340075211 conn(0x7fa8a406a800 msgr2=0x7fa8a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.824+0000 7fa8a3fff640 1 -- 172.21.15.16:0/2340075211 shutdown_connections 2023-11-24T06:30:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.824+0000 7fa8a3fff640 1 -- 172.21.15.16:0/2340075211 wait complete. 2023-11-24T06:30:54.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.826+0000 7fa8a3fff640 1 Processor -- start 2023-11-24T06:30:54.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.826+0000 7fa8a3fff640 1 -- start start 2023-11-24T06:30:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.827+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 0x7fa8a413e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.827+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8a40cca40 0x7fa8a41e0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.827+0000 7fa8a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 0x7fa8a41e0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.827+0000 7fa8a3fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a41e1590 con 0x7fa8a4143c40 2023-11-24T06:30:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.827+0000 7fa8a3fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a41415e0 con 0x7fa8a40cca40 2023-11-24T06:30:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.827+0000 7fa8a3fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a413ee90 con 0x7fa8a40717d0 2023-11-24T06:30:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.827+0000 7fa8a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 0x7fa8a41e0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.827+0000 7fa8a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 0x7fa8a41e0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39844/0 (socket says 172.21.15.16:39844) 2023-11-24T06:30:54.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.827+0000 7fa8a37fe640 1 -- 172.21.15.16:0/980555681 learned_addr learned my addr 172.21.15.16:0/980555681 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:54.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a2ffd640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 0x7fa8a413e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a37fe640 1 -- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 msgr2=0x7fa8a413e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a37fe640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 0x7fa8a413e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a37fe640 1 -- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8a40cca40 msgr2=0x7fa8a41e0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a27fc640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8a40cca40 0x7fa8a41e0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a37fe640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8a40cca40 0x7fa8a41e0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a37fe640 1 -- 172.21.15.16:0/980555681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8940094d0 con 0x7fa8a4143c40 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a2ffd640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 0x7fa8a413e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a37fe640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 0x7fa8a41e0c50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00c7a0 tx=0x7fa89c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa88bfff640 1 -- 172.21.15.16:0/980555681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c003b70 con 0x7fa8a4143c40 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa88bfff640 1 -- 172.21.15.16:0/980555681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c003cf0 con 0x7fa8a4143c40 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa88bfff640 1 -- 172.21.15.16:0/980555681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c00aa90 con 0x7fa8a4143c40 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.828+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a41409d0 con 0x7fa8a4143c40 2023-11-24T06:30:54.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.829+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a4140f90 con 0x7fa8a4143c40 2023-11-24T06:30:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.829+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a4141790 con 0x7fa8a4143c40 2023-11-24T06:30:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.831+0000 7fa88bfff640 1 -- 172.21.15.16:0/980555681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa89c00abf0 con 0x7fa8a4143c40 2023-11-24T06:30:54.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.831+0000 7fa88bfff640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa89006b230 0x7fa89006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:54.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.831+0000 7fa88bfff640 1 -- 172.21.15.16:0/980555681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fa89c089ce0 con 0x7fa8a4143c40 2023-11-24T06:30:54.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.832+0000 7fa8a2ffd640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa89006b230 0x7fa89006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:54.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.833+0000 7fa8a2ffd640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa89006b230 0x7fa89006d6f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa894036000 tx=0x7fa89403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:54.836+0000 7fa88bfff640 1 -- 172.21.15.16:0/980555681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8a4141790 con 0x7fa8a4143c40 2023-11-24T06:30:55.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.055+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol5605", "group_name": "subvol_grp5352", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8a41e2fc0 con 0x7fa89006b230 2023-11-24T06:30:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.119+0000 7fa88bfff640 1 -- 172.21.15.16:0/980555681 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa8a41e2fc0 con 0x7fa89006b230 2023-11-24T06:30:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa89006b230 msgr2=0x7fa89006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:55.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa89006b230 0x7fa89006d6f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa894036000 tx=0x7fa89403c040 comp rx=0 tx=0).stop 2023-11-24T06:30:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 msgr2=0x7fa8a41e0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 0x7fa8a41e0c50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00c7a0 tx=0x7fa89c00cc60 comp rx=0 tx=0).stop 2023-11-24T06:30:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 shutdown_connections 2023-11-24T06:30:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa89006b230 0x7fa89006d6f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8a4143c40 0x7fa8a41e0c50 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8a40cca40 0x7fa8a41e0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:55.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 --2- 172.21.15.16:0/980555681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8a40717d0 0x7fa8a413e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.122+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 >> 172.21.15.16:0/980555681 conn(0x7fa8a406a800 msgr2=0x7fa8a413f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.123+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 shutdown_connections 2023-11-24T06:30:55.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.123+0000 7fa8a3fff640 1 -- 172.21.15.16:0/980555681 wait complete. 2023-11-24T06:30:55.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:55.143 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp5352/.snap/ancestral_snap_1) 2023-11-24T06:30:55.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:55.264 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%i ./volumes/subvol_grp5352) 2023-11-24T06:30:55.322 INFO:teuthology.orchestra.run.smithi016.stdout:1099511627777 2023-11-24T06:30:55.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:55.323 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls ./volumes/subvol_grp5352/subvol5605/.snap/_ancestral_snap_1_1099511627777) 2023-11-24T06:30:55.389 INFO:teuthology.orchestra.run.smithi016.stdout:be8faffb-e3ff-4522-bf4f-c44cc16f9ece 2023-11-24T06:30:55.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol5605 _ancestral_snap_1_1099511627777 subvol_grp5352 2023-11-24T06:30:55.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.621+0000 7fb84849b640 1 Processor -- start 2023-11-24T06:30:55.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.622+0000 7fb84849b640 1 -- start start 2023-11-24T06:30:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.623+0000 7fb84849b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb840161740 0x7fb840168380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.623+0000 7fb84849b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8400cf260 con 0x7fb8400d0560 2023-11-24T06:30:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.623+0000 7fb84849b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8400cf4c0 con 0x7fb8400c9870 2023-11-24T06:30:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.623+0000 7fb84849b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8400d2ef0 con 0x7fb840161740 2023-11-24T06:30:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.623+0000 7fb846a38640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb840161740 0x7fb840168380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.623+0000 7fb846a38640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb840161740 0x7fb840168380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39846/0 (socket says 172.21.15.16:39846) 2023-11-24T06:30:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.623+0000 7fb846a38640 1 -- 172.21.15.16:0/1300928327 learned_addr learned my addr 172.21.15.16:0/1300928327 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.624+0000 7fb846a38640 1 -- 172.21.15.16:0/1300928327 >> v1:172.21.15.173:6789/0 conn(0x7fb8400c9870 legacy=0x7fb8400ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.624+0000 7fb846a38640 1 -- 172.21.15.16:0/1300928327 >> v1:172.21.15.16:6789/0 conn(0x7fb8400d0560 legacy=0x7fb840161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.624+0000 7fb846a38640 1 -- 172.21.15.16:0/1300928327 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8400cf7a0 con 0x7fb840161740 2023-11-24T06:30:55.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.625+0000 7fb846a38640 1 --2- 172.21.15.16:0/1300928327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb840161740 0x7fb840168380 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb8400b4cf0 tx=0x7fb83802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1831e78807e7dfc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:55.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.625+0000 7fb845235640 1 -- 172.21.15.16:0/1300928327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838030840 con 0x7fb840161740 2023-11-24T06:30:55.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.625+0000 7fb845235640 1 -- 172.21.15.16:0/1300928327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8380309a0 con 0x7fb840161740 2023-11-24T06:30:55.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.625+0000 7fb845235640 1 -- 172.21.15.16:0/1300928327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838030d30 con 0x7fb840161740 2023-11-24T06:30:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.626+0000 7fb84849b640 1 -- 172.21.15.16:0/1300928327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb840161740 msgr2=0x7fb840168380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.626+0000 7fb84849b640 1 --2- 172.21.15.16:0/1300928327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb840161740 0x7fb840168380 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb8400b4cf0 tx=0x7fb83802b980 comp rx=0 tx=0).stop 2023-11-24T06:30:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.626+0000 7fb84849b640 1 -- 172.21.15.16:0/1300928327 shutdown_connections 2023-11-24T06:30:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.626+0000 7fb84849b640 1 --2- 172.21.15.16:0/1300928327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb840161740 0x7fb840168380 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.626+0000 7fb84849b640 1 -- 172.21.15.16:0/1300928327 >> 172.21.15.16:0/1300928327 conn(0x7fb84009c560 msgr2=0x7fb8400d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.626+0000 7fb84849b640 1 -- 172.21.15.16:0/1300928327 shutdown_connections 2023-11-24T06:30:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.626+0000 7fb84849b640 1 -- 172.21.15.16:0/1300928327 wait complete. 2023-11-24T06:30:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.628+0000 7fb84849b640 1 Processor -- start 2023-11-24T06:30:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.628+0000 7fb84849b640 1 -- start start 2023-11-24T06:30:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.629+0000 7fb84849b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8400c9870 0x7fb8400b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:55.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.629+0000 7fb84849b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8400d0560 0x7fb84006b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:55.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.629+0000 7fb84849b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb840161740 0x7fb84006bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:55.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.629+0000 7fb84849b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400b6b10 con 0x7fb840161740 2023-11-24T06:30:55.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.629+0000 7fb84849b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb840155ac0 con 0x7fb8400c9870 2023-11-24T06:30:55.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.629+0000 7fb84849b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400b5f20 con 0x7fb8400d0560 2023-11-24T06:30:55.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.629+0000 7fb846a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb840161740 0x7fb84006bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:55.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.629+0000 7fb846a38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb840161740 0x7fb84006bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39856/0 (socket says 172.21.15.16:39856) 2023-11-24T06:30:55.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.629+0000 7fb846a38640 1 -- 172.21.15.16:0/2355802523 learned_addr learned my addr 172.21.15.16:0/2355802523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:55.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.630+0000 7fb846237640 1 --2- 172.21.15.16:0/2355802523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8400c9870 0x7fb8400b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.630+0000 7fb845a36640 1 --2- 172.21.15.16:0/2355802523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8400d0560 0x7fb84006b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.630+0000 7fb846a38640 1 -- 172.21.15.16:0/2355802523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8400d0560 msgr2=0x7fb84006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.630+0000 7fb846a38640 1 --2- 172.21.15.16:0/2355802523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8400d0560 0x7fb84006b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.630+0000 7fb846a38640 1 -- 172.21.15.16:0/2355802523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8400c9870 msgr2=0x7fb8400b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.630+0000 7fb846a38640 1 --2- 172.21.15.16:0/2355802523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8400c9870 0x7fb8400b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.630+0000 7fb846a38640 1 -- 172.21.15.16:0/2355802523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8380094d0 con 0x7fb840161740 2023-11-24T06:30:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.630+0000 7fb846a38640 1 --2- 172.21.15.16:0/2355802523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb840161740 0x7fb84006bd30 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb83802b9c0 tx=0x7fb838004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.631+0000 7fb836ffd640 1 -- 172.21.15.16:0/2355802523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838039d90 con 0x7fb840161740 2023-11-24T06:30:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.631+0000 7fb836ffd640 1 -- 172.21.15.16:0/2355802523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8380433f0 con 0x7fb840161740 2023-11-24T06:30:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.631+0000 7fb836ffd640 1 -- 172.21.15.16:0/2355802523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380437a0 con 0x7fb840161740 2023-11-24T06:30:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.631+0000 7fb84849b640 1 -- 172.21.15.16:0/2355802523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8400b6170 con 0x7fb840161740 2023-11-24T06:30:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.631+0000 7fb84849b640 1 -- 172.21.15.16:0/2355802523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8400b6730 con 0x7fb840161740 2023-11-24T06:30:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.632+0000 7fb84849b640 1 -- 172.21.15.16:0/2355802523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8400b6c50 con 0x7fb840161740 2023-11-24T06:30:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.633+0000 7fb836ffd640 1 -- 172.21.15.16:0/2355802523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fb83804c3b0 con 0x7fb840161740 2023-11-24T06:30:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.634+0000 7fb836ffd640 1 --2- 172.21.15.16:0/2355802523 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb82406b230 0x7fb82406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.634+0000 7fb836ffd640 1 -- 172.21.15.16:0/2355802523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fb8380ac600 con 0x7fb840161740 2023-11-24T06:30:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.635+0000 7fb846237640 1 --2- 172.21.15.16:0/2355802523 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb82406b230 0x7fb82406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.636+0000 7fb846237640 1 --2- 172.21.15.16:0/2355802523 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb82406b230 0x7fb82406d6f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb83c001db0 tx=0x7fb83c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:55.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.638+0000 7fb836ffd640 1 -- 172.21.15.16:0/2355802523 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8400b6c50 con 0x7fb840161740 2023-11-24T06:30:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.862+0000 7fb84849b640 1 -- 172.21.15.16:0/2355802523 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol5605", "snap_name": "_ancestral_snap_1_1099511627777", "group_name": "subvol_grp5352", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8400d38c0 con 0x7fb82406b230 2023-11-24T06:30:55.875 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:55.872+0000 7fd2b5afe640 -1 mgr.server reply reply (22) Invalid argument snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T06:30:55.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:55.873+0000 7fb836ffd640 1 -- 172.21.15.16:0/2355802523 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: -22 snapshot name '_ancestral_snap_1_1099511627777' is invalid) v1 ==== 66+0+0 (secure 0 0 0) 0x7fb8400d38c0 con 0x7fb82406b230 2023-11-24T06:30:55.876 INFO:teuthology.orchestra.run.smithi016.stderr:Error EINVAL: snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T06:30:55.886 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-24T06:30:55.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:55.886 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir ./volumes/subvol_grp5352/.snap/ancestral_snap_1) 2023-11-24T06:30:56.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol5605 --group_name subvol_grp5352 2023-11-24T06:30:56.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.250+0000 7f36d1bac640 1 Processor -- start 2023-11-24T06:30:56.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.250+0000 7f36d1bac640 1 -- start start 2023-11-24T06:30:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.252+0000 7f36d1bac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36cc143c40 0x7f36cc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.252+0000 7f36d1bac640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36cc0717d0 0x7f36cc0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.252+0000 7f36d1bac640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36cc070e50 con 0x7f36cc0cca40 2023-11-24T06:30:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.252+0000 7f36d1bac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36cc0723c0 con 0x7f36cc143c40 2023-11-24T06:30:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.252+0000 7f36d1bac640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc0cce70 con 0x7f36cc0717d0 2023-11-24T06:30:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.252+0000 7f36d0baa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36cc143c40 0x7f36cc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.253+0000 7f36d0baa640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36cc143c40 0x7f36cc071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39868/0 (socket says 172.21.15.16:39868) 2023-11-24T06:30:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.253+0000 7f36d0baa640 1 -- 172.21.15.16:0/3276932555 learned_addr learned my addr 172.21.15.16:0/3276932555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.253+0000 7f36cbfff640 1 --2- 172.21.15.16:0/3276932555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36cc0717d0 0x7f36cc0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.253+0000 7f36d0baa640 1 -- 172.21.15.16:0/3276932555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36cc0717d0 msgr2=0x7f36cc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.253+0000 7f36d0baa640 1 --2- 172.21.15.16:0/3276932555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36cc0717d0 0x7f36cc0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.253+0000 7f36d0baa640 1 -- 172.21.15.16:0/3276932555 >> v1:172.21.15.173:6789/0 conn(0x7f36cc0cca40 legacy=0x7f36cc15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:56.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.253+0000 7f36d0baa640 1 -- 172.21.15.16:0/3276932555 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc06e170 con 0x7f36cc143c40 2023-11-24T06:30:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.254+0000 7f36d0baa640 1 --2- 172.21.15.16:0/3276932555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc143c40 0x7f36cc071290 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009870 tx=0x7f36c002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fecbaf71e468be server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.255+0000 7f36cb7fe640 1 -- 172.21.15.16:0/3276932555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c00317c0 con 0x7f36cc143c40 2023-11-24T06:30:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.255+0000 7f36cb7fe640 1 -- 172.21.15.16:0/3276932555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c0031920 con 0x7f36cc143c40 2023-11-24T06:30:56.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.255+0000 7f36cb7fe640 1 -- 172.21.15.16:0/3276932555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0031cb0 con 0x7f36cc143c40 2023-11-24T06:30:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.255+0000 7f36d1bac640 1 -- 172.21.15.16:0/3276932555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc143c40 msgr2=0x7f36cc071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.255+0000 7f36d1bac640 1 --2- 172.21.15.16:0/3276932555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc143c40 0x7f36cc071290 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009870 tx=0x7f36c002b860 comp rx=0 tx=0).stop 2023-11-24T06:30:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.255+0000 7f36d1bac640 1 -- 172.21.15.16:0/3276932555 shutdown_connections 2023-11-24T06:30:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.255+0000 7f36d1bac640 1 --2- 172.21.15.16:0/3276932555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36cc0717d0 0x7f36cc0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.256+0000 7f36d1bac640 1 --2- 172.21.15.16:0/3276932555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc143c40 0x7f36cc071290 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.256+0000 7f36d1bac640 1 -- 172.21.15.16:0/3276932555 >> 172.21.15.16:0/3276932555 conn(0x7f36cc06a800 msgr2=0x7f36cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.256+0000 7f36d1bac640 1 -- 172.21.15.16:0/3276932555 shutdown_connections 2023-11-24T06:30:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.256+0000 7f36d1bac640 1 -- 172.21.15.16:0/3276932555 wait complete. 2023-11-24T06:30:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.258+0000 7f36d1bac640 1 Processor -- start 2023-11-24T06:30:56.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.258+0000 7f36d1bac640 1 -- start start 2023-11-24T06:30:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.260+0000 7f36d1bac640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36cc0717d0 0x7f36cc13e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.260+0000 7f36d1bac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc0cca40 0x7f36cc142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.260+0000 7f36d1bac640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36cc143c40 0x7f36cc142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.260+0000 7f36d1bac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc13fdf0 con 0x7f36cc0cca40 2023-11-24T06:30:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.260+0000 7f36d1bac640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc1f3820 con 0x7f36cc0717d0 2023-11-24T06:30:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.260+0000 7f36d1bac640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc1e4550 con 0x7f36cc143c40 2023-11-24T06:30:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.260+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc0cca40 0x7f36cc142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.260+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc0cca40 0x7f36cc142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39878/0 (socket says 172.21.15.16:39878) 2023-11-24T06:30:56.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.260+0000 7f36cbfff640 1 -- 172.21.15.16:0/1499133880 learned_addr learned my addr 172.21.15.16:0/1499133880 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36d0baa640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36cc0717d0 0x7f36cc13e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36cbfff640 1 -- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36cc143c40 msgr2=0x7f36cc142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36cbfff640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36cc143c40 0x7f36cc142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36cbfff640 1 -- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36cc0717d0 msgr2=0x7f36cc13e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36cbfff640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36cc0717d0 0x7f36cc13e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36cbfff640 1 -- 172.21.15.16:0/1499133880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c00094d0 con 0x7f36cc0cca40 2023-11-24T06:30:56.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36cbfff640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc0cca40 0x7f36cc142450 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f36b800d800 tx=0x7f36b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36c97fa640 1 -- 172.21.15.16:0/1499133880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b800b6c0 con 0x7f36cc0cca40 2023-11-24T06:30:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36c97fa640 1 -- 172.21.15.16:0/1499133880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b800b820 con 0x7f36cc0cca40 2023-11-24T06:30:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.261+0000 7f36c97fa640 1 -- 172.21.15.16:0/1499133880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b800bbb0 con 0x7f36cc0cca40 2023-11-24T06:30:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.262+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36cc1e4830 con 0x7f36cc0cca40 2023-11-24T06:30:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.262+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36cc1e4df0 con 0x7f36cc0cca40 2023-11-24T06:30:56.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.263+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36cc0cd930 con 0x7f36cc0cca40 2023-11-24T06:30:56.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.264+0000 7f36c97fa640 1 -- 172.21.15.16:0/1499133880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f36b800bd10 con 0x7f36cc0cca40 2023-11-24T06:30:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.265+0000 7f36c97fa640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36b006b260 0x7f36b006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.265+0000 7f36c97fa640 1 -- 172.21.15.16:0/1499133880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f36b808a280 con 0x7f36cc0cca40 2023-11-24T06:30:56.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.266+0000 7f36d0baa640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36b006b260 0x7f36b006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.267+0000 7f36d0baa640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36b006b260 0x7f36b006d720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f36c0036000 tx=0x7f36c003d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.269+0000 7f36c97fa640 1 -- 172.21.15.16:0/1499133880 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36b8054d40 con 0x7f36cc0cca40 2023-11-24T06:30:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.498+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol5605", "group_name": "subvol_grp5352", "target": ["mon-mgr", ""]}) v1 -- 0x7f36cc06de30 con 0x7f36b006b260 2023-11-24T06:30:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.536+0000 7f36c97fa640 1 -- 172.21.15.16:0/1499133880 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f36cc06de30 con 0x7f36b006b260 2023-11-24T06:30:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36b006b260 msgr2=0x7f36b006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36b006b260 0x7f36b006d720 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f36c0036000 tx=0x7f36c003d040 comp rx=0 tx=0).stop 2023-11-24T06:30:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc0cca40 msgr2=0x7f36cc142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc0cca40 0x7f36cc142450 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f36b800d800 tx=0x7f36b800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 shutdown_connections 2023-11-24T06:30:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f36b006b260 0x7f36b006d720 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36cc143c40 0x7f36cc142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36cc0cca40 0x7f36cc142450 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 --2- 172.21.15.16:0/1499133880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36cc0717d0 0x7f36cc13e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 >> 172.21.15.16:0/1499133880 conn(0x7f36cc06a800 msgr2=0x7f36cc06cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 shutdown_connections 2023-11-24T06:30:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.539+0000 7f36d1bac640 1 -- 172.21.15.16:0/1499133880 wait complete. 2023-11-24T06:30:56.550 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.548+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.550 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.548+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.551 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.548+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.551 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.548+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.551 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.548+0000 7fd2b8303640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:56.561 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T06:30:56.578 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.575+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.578 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.575+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.578 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.575+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.578 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.575+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.578 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:30:56.575+0000 7fd2b7b02640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:30:56.631 INFO:teuthology.orchestra.run.smithi016.stdout:2 2023-11-24T06:30:56.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp5352 2023-11-24T06:30:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.847+0000 7fdd65d8e640 1 Processor -- start 2023-11-24T06:30:56.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.847+0000 7fdd65d8e640 1 -- start start 2023-11-24T06:30:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.849+0000 7fdd65d8e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6013e360 0x7fdd600cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.849+0000 7fdd65d8e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6015c5f0 con 0x7fdd6015f9c0 2023-11-24T06:30:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.849+0000 7fdd65d8e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6006c470 con 0x7fdd6013df80 2023-11-24T06:30:56.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.849+0000 7fdd65d8e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6006c750 con 0x7fdd6013e360 2023-11-24T06:30:56.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.849+0000 7fdd5ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdd6015f9c0 0x7fdd600cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40418/0 (socket says 172.21.15.16:40418) 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.849+0000 7fdd5ffff640 1 -- 172.21.15.16:0/1846570221 learned_addr learned my addr 172.21.15.16:0/1846570221 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.849+0000 7fdd5effd640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6013e360 0x7fdd600cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535946425 0 0) 0x7fdd6015c5f0 con 0x7fdd6015f9c0 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd48003520 con 0x7fdd6015f9c0 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 200526990 0 0) 0x7fdd48003520 con 0x7fdd6015f9c0 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6013e360 msgr2=0x7fdd600cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6013e360 0x7fdd600cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 >> v1:172.21.15.173:6789/0 conn(0x7fdd6013df80 legacy=0x7fdd6015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6006ca30 con 0x7fdd6015f9c0 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fdd58001670 con 0x7fdd6015f9c0 2023-11-24T06:30:56.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 >> v1:172.21.15.16:6789/0 conn(0x7fdd6015f9c0 legacy=0x7fdd600cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd480039a0 0x7fdd48003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6013e360 0x7fdd480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd48004d00 0x7fdd48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6006c750 con 0x7fdd480039a0 2023-11-24T06:30:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4800ba60 con 0x7fdd48004d00 2023-11-24T06:30:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4800bc20 con 0x7fdd6013e360 2023-11-24T06:30:56.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.850+0000 7fdd5f7fe640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd480039a0 0x7fdd48003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.851+0000 7fdd5f7fe640 1 -- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6013e360 msgr2=0x7fdd480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:56.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.851+0000 7fdd5f7fe640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6013e360 0x7fdd480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.851+0000 7fdd5ffff640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd48004d00 0x7fdd48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.851+0000 7fdd5f7fe640 1 -- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd48004d00 msgr2=0x7fdd48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.851+0000 7fdd5f7fe640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd48004d00 0x7fdd48005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.851+0000 7fdd5f7fe640 1 -- 172.21.15.16:0/1846570221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48003520 con 0x7fdd480039a0 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.852+0000 7fdd5f7fe640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd480039a0 0x7fdd48003dc0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdd50001e20 tx=0x7fdd480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b665f792e7d969 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.852+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50003b70 con 0x7fdd480039a0 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.852+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50003cd0 con 0x7fdd480039a0 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.852+0000 7fdd5e7fc640 1 -- 172.21.15.16:0/1846570221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50005310 con 0x7fdd480039a0 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.853+0000 7fdd65d8e640 1 -- 172.21.15.16:0/1846570221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd480039a0 msgr2=0x7fdd48003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.853+0000 7fdd65d8e640 1 --2- 172.21.15.16:0/1846570221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd480039a0 0x7fdd48003dc0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdd50001e20 tx=0x7fdd480057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.853+0000 7fdd5ffff640 1 -- 172.21.15.16:0/1846570221 reap_dead start 2023-11-24T06:30:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.853+0000 7fdd65d8e640 1 -- 172.21.15.16:0/1846570221 shutdown_connections 2023-11-24T06:30:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.853+0000 7fdd65d8e640 1 -- 172.21.15.16:0/1846570221 >> 172.21.15.16:0/1846570221 conn(0x7fdd6006a800 msgr2=0x7fdd6006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.853+0000 7fdd65d8e640 1 -- 172.21.15.16:0/1846570221 shutdown_connections 2023-11-24T06:30:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.853+0000 7fdd65d8e640 1 -- 172.21.15.16:0/1846570221 wait complete. 2023-11-24T06:30:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.855+0000 7fdd65d8e640 1 Processor -- start 2023-11-24T06:30:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.855+0000 7fdd65d8e640 1 -- start start 2023-11-24T06:30:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.856+0000 7fdd65d8e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd480039a0 0x7fdd6013d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.856+0000 7fdd65d8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd6013dad0 0x7fdd600b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.856+0000 7fdd65d8e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd601529e0 0x7fdd60152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.856+0000 7fdd65d8e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601533a0 con 0x7fdd6013dad0 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.856+0000 7fdd65d8e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600b1ca0 con 0x7fdd480039a0 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.856+0000 7fdd65d8e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600b2b00 con 0x7fdd601529e0 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.856+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd6013dad0 0x7fdd600b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.856+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd6013dad0 0x7fdd600b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39896/0 (socket says 172.21.15.16:39896) 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.856+0000 7fdd5effd640 1 -- 172.21.15.16:0/3718254287 learned_addr learned my addr 172.21.15.16:0/3718254287 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd5f7fe640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd480039a0 0x7fdd6013d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd5effd640 1 -- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd601529e0 msgr2=0x7fdd60152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd5effd640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd601529e0 0x7fdd60152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd5effd640 1 -- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd480039a0 msgr2=0x7fdd6013d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd5effd640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd480039a0 0x7fdd6013d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd5effd640 1 -- 172.21.15.16:0/3718254287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600b1ef0 con 0x7fdd6013dad0 2023-11-24T06:30:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd5effd640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd6013dad0 0x7fdd600b3490 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400e7f0 tx=0x7fdd5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd47fff640 1 -- 172.21.15.16:0/3718254287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400c940 con 0x7fdd6013dad0 2023-11-24T06:30:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd47fff640 1 -- 172.21.15.16:0/3718254287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5400caa0 con 0x7fdd6013dad0 2023-11-24T06:30:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd47fff640 1 -- 172.21.15.16:0/3718254287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400ce30 con 0x7fdd6013dad0 2023-11-24T06:30:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd600b21d0 con 0x7fdd6013dad0 2023-11-24T06:30:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.857+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd600b2790 con 0x7fdd6013dad0 2023-11-24T06:30:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.859+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd600675d0 con 0x7fdd6013dad0 2023-11-24T06:30:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.859+0000 7fdd47fff640 1 -- 172.21.15.16:0/3718254287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fdd54013470 con 0x7fdd6013dad0 2023-11-24T06:30:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.860+0000 7fdd47fff640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd4c06b230 0x7fdd4c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.861+0000 7fdd47fff640 1 -- 172.21.15.16:0/3718254287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fdd54089da0 con 0x7fdd6013dad0 2023-11-24T06:30:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.861+0000 7fdd5f7fe640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd4c06b230 0x7fdd4c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.862+0000 7fdd5f7fe640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd4c06b230 0x7fdd4c06d6f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdd50034000 tx=0x7fdd50009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:56.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:56.865+0000 7fdd47fff640 1 -- 172.21.15.16:0/3718254287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd540547d0 con 0x7fdd6013dad0 2023-11-24T06:30:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.087+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp5352", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd60154f10 con 0x7fdd4c06b230 2023-11-24T06:30:57.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.098+0000 7fdd47fff640 1 -- 172.21.15.16:0/3718254287 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdd60154f10 con 0x7fdd4c06b230 2023-11-24T06:30:57.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd4c06b230 msgr2=0x7fdd4c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:57.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd4c06b230 0x7fdd4c06d6f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdd50034000 tx=0x7fdd50009030 comp rx=0 tx=0).stop 2023-11-24T06:30:57.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd6013dad0 msgr2=0x7fdd600b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:57.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd6013dad0 0x7fdd600b3490 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400e7f0 tx=0x7fdd5400ecb0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 shutdown_connections 2023-11-24T06:30:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdd4c06b230 0x7fdd4c06d6f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd601529e0 0x7fdd60152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd6013dad0 0x7fdd600b3490 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 --2- 172.21.15.16:0/3718254287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd480039a0 0x7fdd6013d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 >> 172.21.15.16:0/3718254287 conn(0x7fdd6006a800 msgr2=0x7fdd600b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:57.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 shutdown_connections 2023-11-24T06:30:57.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.101+0000 7fdd65d8e640 1 -- 172.21.15.16:0/3718254287 wait complete. 2023-11-24T06:30:57.123 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:30:57.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:30:57.232 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:30:57.363 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:30:57.363 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:30:57.363 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:30:57.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:30:57.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.656+0000 7f9139d81640 1 Processor -- start 2023-11-24T06:30:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.656+0000 7f9139d81640 1 -- start start 2023-11-24T06:30:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.658+0000 7f9139d81640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9134143c40 0x7f9134071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.658+0000 7f9139d81640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f91340717d0 0x7f91340cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:57.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.658+0000 7f9139d81640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9134070e50 con 0x7f91340cca40 2023-11-24T06:30:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.658+0000 7f9139d81640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340723c0 con 0x7f9134143c40 2023-11-24T06:30:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.658+0000 7f9139d81640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340cce40 con 0x7f91340717d0 2023-11-24T06:30:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.658+0000 7f9139580640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f91340cca40 0x7f913415eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40434/0 (socket says 172.21.15.16:40434) 2023-11-24T06:30:57.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.658+0000 7f9139580640 1 -- 172.21.15.16:0/1233389846 learned_addr learned my addr 172.21.15.16:0/1233389846 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.659+0000 7f9138d7f640 1 --2- 172.21.15.16:0/1233389846 >> v2:172.21.15.16:3300/0 conn(0x7f9134143c40 0x7f9134071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.659+0000 7f9133fff640 1 --2- 172.21.15.16:0/1233389846 >> v2:172.21.15.173:3300/0 conn(0x7f91340717d0 0x7f91340cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.659+0000 7f91337fe640 1 -- 172.21.15.16:0/1233389846 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219104225 0 0) 0x7f9134070e50 con 0x7f91340cca40 2023-11-24T06:30:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.659+0000 7f91337fe640 1 -- 172.21.15.16:0/1233389846 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9124003520 con 0x7f91340cca40 2023-11-24T06:30:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.659+0000 7f9138d7f640 1 -- 172.21.15.16:0/1233389846 >> v2:172.21.15.173:3300/0 conn(0x7f91340717d0 msgr2=0x7f91340cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.659+0000 7f91337fe640 1 -- 172.21.15.16:0/1233389846 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 918745354 0 0) 0x7f9124003520 con 0x7f91340cca40 2023-11-24T06:30:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.659+0000 7f9138d7f640 1 --2- 172.21.15.16:0/1233389846 >> v2:172.21.15.173:3300/0 conn(0x7f91340717d0 0x7f91340cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.659+0000 7f9138d7f640 1 -- 172.21.15.16:0/1233389846 >> v1:172.21.15.16:6789/0 conn(0x7f91340cca40 legacy=0x7f913415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:57.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.659+0000 7f9138d7f640 1 -- 172.21.15.16:0/1233389846 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913406e170 con 0x7f9134143c40 2023-11-24T06:30:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.660+0000 7f9138d7f640 1 --2- 172.21.15.16:0/1233389846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 0x7f9134071290 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f91340b44a0 tx=0x7f912802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825b316658a5cabd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.661+0000 7f91337fe640 1 -- 172.21.15.16:0/1233389846 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c0019f0 con 0x7f9134143c40 2023-11-24T06:30:57.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.661+0000 7f91337fe640 1 -- 172.21.15.16:0/1233389846 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912802e050 con 0x7f9134143c40 2023-11-24T06:30:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.661+0000 7f91337fe640 1 -- 172.21.15.16:0/1233389846 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128030730 con 0x7f9134143c40 2023-11-24T06:30:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.661+0000 7f9139d81640 1 -- 172.21.15.16:0/1233389846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 msgr2=0x7f9134071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.661+0000 7f9139d81640 1 --2- 172.21.15.16:0/1233389846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 0x7f9134071290 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f91340b44a0 tx=0x7f912802b710 comp rx=0 tx=0).stop 2023-11-24T06:30:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.661+0000 7f9139d81640 1 -- 172.21.15.16:0/1233389846 shutdown_connections 2023-11-24T06:30:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.661+0000 7f9139d81640 1 --2- 172.21.15.16:0/1233389846 >> v2:172.21.15.173:3300/0 conn(0x7f91340717d0 0x7f91340cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.661+0000 7f9139d81640 1 --2- 172.21.15.16:0/1233389846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 0x7f9134071290 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.661+0000 7f9139d81640 1 -- 172.21.15.16:0/1233389846 >> 172.21.15.16:0/1233389846 conn(0x7f913406a800 msgr2=0x7f913406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.662+0000 7f9139d81640 1 -- 172.21.15.16:0/1233389846 shutdown_connections 2023-11-24T06:30:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.662+0000 7f9139d81640 1 -- 172.21.15.16:0/1233389846 wait complete. 2023-11-24T06:30:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.663+0000 7f9139d81640 1 Processor -- start 2023-11-24T06:30:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.664+0000 7f9139d81640 1 -- start start 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9139d81640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91340717d0 0x7f91340c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9139d81640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91340cca40 0x7f91340aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9139d81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 0x7f91340af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9139d81640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340c4d40 con 0x7f9134143c40 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9139d81640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9134135430 con 0x7f91340717d0 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9139d81640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340c42b0 con 0x7f91340cca40 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9139580640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 0x7f91340af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9139580640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 0x7f91340af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39908/0 (socket says 172.21.15.16:39908) 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9139580640 1 -- 172.21.15.16:0/859933851 learned_addr learned my addr 172.21.15.16:0/859933851 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9138d7f640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91340717d0 0x7f91340c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.665+0000 7f9133fff640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91340cca40 0x7f91340aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f9139580640 1 -- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91340cca40 msgr2=0x7f91340aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f9139580640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91340cca40 0x7f91340aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f9139580640 1 -- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91340717d0 msgr2=0x7f91340c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f9139580640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91340717d0 0x7f91340c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f9139580640 1 -- 172.21.15.16:0/859933851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128009400 con 0x7f9134143c40 2023-11-24T06:30:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f9139580640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 0x7f91340af090 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f912c001f70 tx=0x7f912c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f91317fa640 1 -- 172.21.15.16:0/859933851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c012400 con 0x7f9134143c40 2023-11-24T06:30:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f91317fa640 1 -- 172.21.15.16:0/859933851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c012560 con 0x7f9134143c40 2023-11-24T06:30:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f91317fa640 1 -- 172.21.15.16:0/859933851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c012910 con 0x7f9134143c40 2023-11-24T06:30:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91340c44a0 con 0x7f9134143c40 2023-11-24T06:30:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.666+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91340c4a60 con 0x7f9134143c40 2023-11-24T06:30:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.667+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9134135190 con 0x7f9134143c40 2023-11-24T06:30:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.668+0000 7f91317fa640 1 -- 172.21.15.16:0/859933851 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f912c012a70 con 0x7f9134143c40 2023-11-24T06:30:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.669+0000 7f91317fa640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f911806b2c0 0x7f911806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.669+0000 7f91317fa640 1 -- 172.21.15.16:0/859933851 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f912c08a620 con 0x7f9134143c40 2023-11-24T06:30:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.670+0000 7f9138d7f640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f911806b2c0 0x7f911806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:57.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.671+0000 7f9138d7f640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f911806b2c0 0x7f911806d780 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9128034000 tx=0x7f9128008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.674+0000 7f91317fa640 1 -- 172.21.15.16:0/859933851 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9134135190 con 0x7f9134143c40 2023-11-24T06:30:57.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.870+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9134135190 con 0x7f9134143c40 2023-11-24T06:30:57.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.872+0000 7f91317fa640 1 -- 172.21.15.16:0/859933851 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16714 (secure 0 0 0) 0x7f9134135190 con 0x7f9134143c40 2023-11-24T06:30:57.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:30:57.875 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:45.037086+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:30:31.631710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:30:32.732622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:30:57.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f911806b2c0 msgr2=0x7f911806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f911806b2c0 0x7f911806d780 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9128034000 tx=0x7f9128008030 comp rx=0 tx=0).stop 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 msgr2=0x7f91340af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 0x7f91340af090 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f912c001f70 tx=0x7f912c00a430 comp rx=0 tx=0).stop 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 shutdown_connections 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f911806b2c0 0x7f911806d780 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9134143c40 0x7f91340af090 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91340cca40 0x7f91340aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 --2- 172.21.15.16:0/859933851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91340717d0 0x7f91340c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 >> 172.21.15.16:0/859933851 conn(0x7f913406a800 msgr2=0x7f9134142ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:57.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.875+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 shutdown_connections 2023-11-24T06:30:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:57.876+0000 7f9139d81640 1 -- 172.21.15.16:0/859933851 wait complete. 2023-11-24T06:30:57.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:30:31.631710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:30:32.732622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:30:57.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:30:58.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.126+0000 7f142e88c640 1 Processor -- start 2023-11-24T06:30:58.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.126+0000 7f142e88c640 1 -- start start 2023-11-24T06:30:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.128+0000 7f142e88c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14280ccb50 0x7f1428070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.128+0000 7f142e88c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f142815bf10 0x7f1428143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.128+0000 7f142e88c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142815f4e0 con 0x7f14280adf20 2023-11-24T06:30:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.128+0000 7f142e88c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142815f970 con 0x7f14280ccb50 2023-11-24T06:30:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.128+0000 7f142e88c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1428071130 con 0x7f142815bf10 2023-11-24T06:30:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.129+0000 7f142d88a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f14280adf20 0x7f14280cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40444/0 (socket says 172.21.15.16:40444) 2023-11-24T06:30:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.129+0000 7f142d88a640 1 -- 172.21.15.16:0/1388822184 learned_addr learned my addr 172.21.15.16:0/1388822184 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.129+0000 7f142d089640 1 --2- 172.21.15.16:0/1388822184 >> v2:172.21.15.16:3300/0 conn(0x7f14280ccb50 0x7f1428070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.129+0000 7f142e08b640 1 --2- 172.21.15.16:0/1388822184 >> v2:172.21.15.173:3300/0 conn(0x7f142815bf10 0x7f1428143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.129+0000 7f142c888640 1 -- 172.21.15.16:0/1388822184 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229644920 0 0) 0x7f142815f4e0 con 0x7f14280adf20 2023-11-24T06:30:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.129+0000 7f142d089640 1 -- 172.21.15.16:0/1388822184 >> v2:172.21.15.173:3300/0 conn(0x7f142815bf10 msgr2=0x7f1428143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.129+0000 7f142d089640 1 --2- 172.21.15.16:0/1388822184 >> v2:172.21.15.173:3300/0 conn(0x7f142815bf10 0x7f1428143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.129+0000 7f142d089640 1 -- 172.21.15.16:0/1388822184 >> v1:172.21.15.16:6789/0 conn(0x7f14280adf20 legacy=0x7f14280cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.129+0000 7f142d089640 1 -- 172.21.15.16:0/1388822184 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142815ebb0 con 0x7f14280ccb50 2023-11-24T06:30:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.130+0000 7f142d089640 1 --2- 172.21.15.16:0/1388822184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14280ccb50 0x7f1428070a30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f142806ff20 tx=0x7f141802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc38d9877ffdc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.131+0000 7f142c888640 1 -- 172.21.15.16:0/1388822184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141802e070 con 0x7f14280ccb50 2023-11-24T06:30:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.131+0000 7f142c888640 1 -- 172.21.15.16:0/1388822184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14180315a0 con 0x7f14280ccb50 2023-11-24T06:30:58.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.131+0000 7f142c888640 1 -- 172.21.15.16:0/1388822184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418031930 con 0x7f14280ccb50 2023-11-24T06:30:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.131+0000 7f142e88c640 1 -- 172.21.15.16:0/1388822184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14280ccb50 msgr2=0x7f1428070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.131+0000 7f142e88c640 1 --2- 172.21.15.16:0/1388822184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14280ccb50 0x7f1428070a30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f142806ff20 tx=0x7f141802b840 comp rx=0 tx=0).stop 2023-11-24T06:30:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.131+0000 7f142e88c640 1 -- 172.21.15.16:0/1388822184 shutdown_connections 2023-11-24T06:30:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.131+0000 7f142e88c640 1 --2- 172.21.15.16:0/1388822184 >> v2:172.21.15.173:3300/0 conn(0x7f142815bf10 0x7f1428143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.131+0000 7f142e88c640 1 --2- 172.21.15.16:0/1388822184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14280ccb50 0x7f1428070a30 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.131+0000 7f142e88c640 1 -- 172.21.15.16:0/1388822184 >> 172.21.15.16:0/1388822184 conn(0x7f142806a7d0 msgr2=0x7f142806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.132+0000 7f142e88c640 1 -- 172.21.15.16:0/1388822184 shutdown_connections 2023-11-24T06:30:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.132+0000 7f142e88c640 1 -- 172.21.15.16:0/1388822184 wait complete. 2023-11-24T06:30:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.133+0000 7f142e88c640 1 Processor -- start 2023-11-24T06:30:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.133+0000 7f142e88c640 1 -- start start 2023-11-24T06:30:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.134+0000 7f142e88c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14280adf20 0x7f142813e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.134+0000 7f142e88c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14280ccb50 0x7f14280ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.134+0000 7f142e88c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142815bf10 0x7f14280ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.134+0000 7f142e88c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14281f2d30 con 0x7f142815bf10 2023-11-24T06:30:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.134+0000 7f142e88c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142813ed90 con 0x7f14280ccb50 2023-11-24T06:30:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.134+0000 7f142e88c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1428140d30 con 0x7f14280adf20 2023-11-24T06:30:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.135+0000 7f142e08b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142815bf10 0x7f14280ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.135+0000 7f142e08b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142815bf10 0x7f14280ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39922/0 (socket says 172.21.15.16:39922) 2023-11-24T06:30:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.135+0000 7f142e08b640 1 -- 172.21.15.16:0/74953862 learned_addr learned my addr 172.21.15.16:0/74953862 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.135+0000 7f142e08b640 1 -- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14280adf20 msgr2=0x7f142813e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.135+0000 7f142e08b640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14280adf20 0x7f142813e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.135+0000 7f142e08b640 1 -- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14280ccb50 msgr2=0x7f14280ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.135+0000 7f142e08b640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14280ccb50 0x7f14280ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.135+0000 7f142e08b640 1 -- 172.21.15.16:0/74953862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1418009400 con 0x7f142815bf10 2023-11-24T06:30:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.136+0000 7f142e08b640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142815bf10 0x7f14280ace10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1420007e60 tx=0x7f142000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.136+0000 7f141e7fc640 1 -- 172.21.15.16:0/74953862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420014420 con 0x7f142815bf10 2023-11-24T06:30:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.136+0000 7f141e7fc640 1 -- 172.21.15.16:0/74953862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14200145a0 con 0x7f142815bf10 2023-11-24T06:30:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.136+0000 7f141e7fc640 1 -- 172.21.15.16:0/74953862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420014950 con 0x7f142815bf10 2023-11-24T06:30:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.136+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1428141010 con 0x7f142815bf10 2023-11-24T06:30:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.136+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14280b2210 con 0x7f142815bf10 2023-11-24T06:30:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.137+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1428067450 con 0x7f142815bf10 2023-11-24T06:30:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.138+0000 7f141e7fc640 1 -- 172.21.15.16:0/74953862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f1420014ab0 con 0x7f142815bf10 2023-11-24T06:30:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.139+0000 7f141e7fc640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f140c06b2c0 0x7f140c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.139+0000 7f141e7fc640 1 -- 172.21.15.16:0/74953862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f142008a6f0 con 0x7f142815bf10 2023-11-24T06:30:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.140+0000 7f142d88a640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f140c06b2c0 0x7f140c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.141+0000 7f142d88a640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f140c06b2c0 0x7f140c06d780 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1424007fd0 tx=0x7f1424007920 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.144+0000 7f141e7fc640 1 -- 172.21.15.16:0/74953862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14200551b0 con 0x7f142815bf10 2023-11-24T06:30:58.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.346+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1428072020 con 0x7f142815bf10 2023-11-24T06:30:58.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.347+0000 7f141e7fc640 1 -- 172.21.15.16:0/74953862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14200587d0 con 0x7f142815bf10 2023-11-24T06:30:58.349 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:30:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f140c06b2c0 msgr2=0x7f140c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f140c06b2c0 0x7f140c06d780 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1424007fd0 tx=0x7f1424007920 comp rx=0 tx=0).stop 2023-11-24T06:30:58.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142815bf10 msgr2=0x7f14280ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142815bf10 0x7f14280ace10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f1420007e60 tx=0x7f142000e4b0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 shutdown_connections 2023-11-24T06:30:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f140c06b2c0 0x7f140c06d780 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142815bf10 0x7f14280ace10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14280ccb50 0x7f14280ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 --2- 172.21.15.16:0/74953862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f14280adf20 0x7f142813e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 >> 172.21.15.16:0/74953862 conn(0x7f142806a7d0 msgr2=0x7f142815b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 shutdown_connections 2023-11-24T06:30:58.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.350+0000 7f142e88c640 1 -- 172.21.15.16:0/74953862 wait complete. 2023-11-24T06:30:58.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:30:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.601+0000 7f4d4ffff640 1 Processor -- start 2023-11-24T06:30:58.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.602+0000 7f4d4ffff640 1 -- start start 2023-11-24T06:30:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.603+0000 7f4d4ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d500c6c00 0x7f4d500d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.603+0000 7f4d4ffff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d500cf6c0 con 0x7f4d50160c90 2023-11-24T06:30:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.603+0000 7f4d4ffff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d500d1450 con 0x7f4d501608b0 2023-11-24T06:30:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.603+0000 7f4d4ffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d500b4390 con 0x7f4d500c6c00 2023-11-24T06:30:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.604+0000 7f4d4f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d500c6c00 0x7f4d500d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:58.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.604+0000 7f4d4f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d500c6c00 0x7f4d500d3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39932/0 (socket says 172.21.15.16:39932) 2023-11-24T06:30:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.604+0000 7f4d4e7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4d50160c90 0x7f4d500cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40446/0 (socket says 172.21.15.16:40446) 2023-11-24T06:30:58.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.604+0000 7f4d4f7fe640 1 -- 172.21.15.16:0/3737878270 learned_addr learned my addr 172.21.15.16:0/3737878270 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.604+0000 7f4d4f7fe640 1 -- 172.21.15.16:0/3737878270 >> v1:172.21.15.173:6789/0 conn(0x7f4d501608b0 legacy=0x7f4d500d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.604+0000 7f4d4f7fe640 1 -- 172.21.15.16:0/3737878270 >> v1:172.21.15.16:6789/0 conn(0x7f4d50160c90 legacy=0x7f4d500cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.604+0000 7f4d4f7fe640 1 -- 172.21.15.16:0/3737878270 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d500ce7e0 con 0x7f4d500c6c00 2023-11-24T06:30:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.605+0000 7f4d4f7fe640 1 --2- 172.21.15.16:0/3737878270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d500c6c00 0x7f4d500d3bd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009e50 tx=0x7f4d4802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367e0f77c25b9be2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.606+0000 7f4d4dffb640 1 -- 172.21.15.16:0/3737878270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4802c9f0 con 0x7f4d500c6c00 2023-11-24T06:30:58.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.606+0000 7f4d4dffb640 1 -- 172.21.15.16:0/3737878270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4802cb50 con 0x7f4d500c6c00 2023-11-24T06:30:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.606+0000 7f4d4dffb640 1 -- 172.21.15.16:0/3737878270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d480343f0 con 0x7f4d500c6c00 2023-11-24T06:30:58.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.606+0000 7f4d4ffff640 1 -- 172.21.15.16:0/3737878270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d500c6c00 msgr2=0x7f4d500d3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.606+0000 7f4d4ffff640 1 --2- 172.21.15.16:0/3737878270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d500c6c00 0x7f4d500d3bd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009e50 tx=0x7f4d4802be50 comp rx=0 tx=0).stop 2023-11-24T06:30:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.607+0000 7f4d4ffff640 1 -- 172.21.15.16:0/3737878270 shutdown_connections 2023-11-24T06:30:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.607+0000 7f4d4ffff640 1 --2- 172.21.15.16:0/3737878270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d500c6c00 0x7f4d500d3bd0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.607+0000 7f4d4ffff640 1 -- 172.21.15.16:0/3737878270 >> 172.21.15.16:0/3737878270 conn(0x7f4d500d30b0 msgr2=0x7f4d500d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.607+0000 7f4d4ffff640 1 -- 172.21.15.16:0/3737878270 shutdown_connections 2023-11-24T06:30:58.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.607+0000 7f4d4ffff640 1 -- 172.21.15.16:0/3737878270 wait complete. 2023-11-24T06:30:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.608+0000 7f4d4ffff640 1 Processor -- start 2023-11-24T06:30:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.609+0000 7f4d4ffff640 1 -- start start 2023-11-24T06:30:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d500c6c00 0x7f4d501f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d501608b0 0x7f4d500b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d50160c90 0x7f4d5015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d501f74d0 con 0x7f4d501608b0 2023-11-24T06:30:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4ffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500b8080 con 0x7f4d500c6c00 2023-11-24T06:30:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d500b8300 con 0x7f4d50160c90 2023-11-24T06:30:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d501608b0 0x7f4d500b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d501608b0 0x7f4d500b7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39942/0 (socket says 172.21.15.16:39942) 2023-11-24T06:30:58.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4e7fc640 1 -- 172.21.15.16:0/467088135 learned_addr learned my addr 172.21.15.16:0/467088135 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4e7fc640 1 -- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d50160c90 msgr2=0x7f4d5015f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4e7fc640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d50160c90 0x7f4d5015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4e7fc640 1 -- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d500c6c00 msgr2=0x7f4d501f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4e7fc640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d500c6c00 0x7f4d501f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.610+0000 7f4d4e7fc640 1 -- 172.21.15.16:0/467088135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d48009b30 con 0x7f4d501608b0 2023-11-24T06:30:58.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.611+0000 7f4d4e7fc640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d501608b0 0x7f4d500b7b40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c3e0 tx=0x7f4d4400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.611+0000 7f4d377fe640 1 -- 172.21.15.16:0/467088135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440074e0 con 0x7f4d501608b0 2023-11-24T06:30:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.611+0000 7f4d377fe640 1 -- 172.21.15.16:0/467088135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d44007640 con 0x7f4d501608b0 2023-11-24T06:30:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.611+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d500b85e0 con 0x7f4d501608b0 2023-11-24T06:30:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.611+0000 7f4d377fe640 1 -- 172.21.15.16:0/467088135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440049e0 con 0x7f4d501608b0 2023-11-24T06:30:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.611+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d500b8ba0 con 0x7f4d501608b0 2023-11-24T06:30:58.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.612+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d500cb1d0 con 0x7f4d501608b0 2023-11-24T06:30:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.613+0000 7f4d377fe640 1 -- 172.21.15.16:0/467088135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f4d44004b40 con 0x7f4d501608b0 2023-11-24T06:30:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.614+0000 7f4d377fe640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3c06b2c0 0x7f4d3c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:58.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.614+0000 7f4d377fe640 1 -- 172.21.15.16:0/467088135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f4d44089900 con 0x7f4d501608b0 2023-11-24T06:30:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.615+0000 7f4d4effd640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3c06b2c0 0x7f4d3c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:58.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.616+0000 7f4d4effd640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3c06b2c0 0x7f4d3c06d780 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4d40006fd0 tx=0x7f4d40006bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:58.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.618+0000 7f4d377fe640 1 -- 172.21.15.16:0/467088135 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d500cb1d0 con 0x7f4d501608b0 2023-11-24T06:30:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.812+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d500cb1d0 con 0x7f4d501608b0 2023-11-24T06:30:58.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.813+0000 7f4d377fe640 1 -- 172.21.15.16:0/467088135 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d500cb1d0 con 0x7f4d501608b0 2023-11-24T06:30:58.815 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:30:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3c06b2c0 msgr2=0x7f4d3c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3c06b2c0 0x7f4d3c06d780 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4d40006fd0 tx=0x7f4d40006bc0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d501608b0 msgr2=0x7f4d500b7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d501608b0 0x7f4d500b7b40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c3e0 tx=0x7f4d4400c8a0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 shutdown_connections 2023-11-24T06:30:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3c06b2c0 0x7f4d3c06d780 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d50160c90 0x7f4d5015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d501608b0 0x7f4d500b7b40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 --2- 172.21.15.16:0/467088135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d500c6c00 0x7f4d501f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 >> 172.21.15.16:0/467088135 conn(0x7f4d500d30b0 msgr2=0x7f4d500c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 shutdown_connections 2023-11-24T06:30:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:58.816+0000 7f4d4ffff640 1 -- 172.21.15.16:0/467088135 wait complete. 2023-11-24T06:30:58.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:30:59.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.064+0000 7f77618e7640 1 Processor -- start 2023-11-24T06:30:59.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.064+0000 7f77618e7640 1 -- start start 2023-11-24T06:30:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.066+0000 7f77618e7640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f775c13df80 0x7f775c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.066+0000 7f77618e7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c0cf9a0 0x7f775c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.066+0000 7f77618e7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f775c06c470 con 0x7f775c13e360 2023-11-24T06:30:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.066+0000 7f77618e7640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f775c06c750 con 0x7f775c13df80 2023-11-24T06:30:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.067+0000 7f77618e7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c06ca30 con 0x7f775c0cf9a0 2023-11-24T06:30:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.067+0000 7f775a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f775c13e360 0x7f775c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40458/0 (socket says 172.21.15.16:40458) 2023-11-24T06:30:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.067+0000 7f775a7fc640 1 -- 172.21.15.16:0/1533006720 learned_addr learned my addr 172.21.15.16:0/1533006720 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:30:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.067+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357805969 0 0) 0x7f775c06c470 con 0x7f775c13e360 2023-11-24T06:30:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.067+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7744003520 con 0x7f775c13e360 2023-11-24T06:30:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.067+0000 7f775b7fe640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c0cf9a0 0x7f775c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.067+0000 7f775affd640 1 --2- 172.21.15.16:0/1533006720 >> v2:172.21.15.173:3300/0 conn(0x7f775c13df80 0x7f775c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4204069248 0 0) 0x7f7744003520 con 0x7f775c13e360 2023-11-24T06:30:59.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c0cf9a0 msgr2=0x7f775c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c0cf9a0 0x7f775c0cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 >> v2:172.21.15.173:3300/0 conn(0x7f775c13df80 msgr2=0x7f775c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 --2- 172.21.15.16:0/1533006720 >> v2:172.21.15.173:3300/0 conn(0x7f775c13df80 0x7f775c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c06cd10 con 0x7f775c13e360 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f7750001670 con 0x7f775c13e360 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f775affd640 1 --2- 172.21.15.16:0/1533006720 >> v2:172.21.15.173:3300/0 conn(0x7f775c13df80 0x7f775c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 >> v1:172.21.15.16:6789/0 conn(0x7f775c13e360 legacy=0x7f775c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c0cf9a0 0x7f7744003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f77440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7744004d00 0x7f7744005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c06c750 con 0x7f7744003fa0 2023-11-24T06:30:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c06ca30 con 0x7f7744004d00 2023-11-24T06:30:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774400ba80 con 0x7f775c0cf9a0 2023-11-24T06:30:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f775a7fc640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f77440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f775affd640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c0cf9a0 0x7f7744003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.068+0000 7f775b7fe640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7744004d00 0x7f7744005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.069+0000 7f775a7fc640 1 -- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c0cf9a0 msgr2=0x7f7744003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.069+0000 7f775a7fc640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c0cf9a0 0x7f7744003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.069+0000 7f775a7fc640 1 -- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7744004d00 msgr2=0x7f7744005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.069+0000 7f775a7fc640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7744004d00 0x7f7744005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.069+0000 7f775a7fc640 1 -- 172.21.15.16:0/1533006720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744003520 con 0x7f7744003fa0 2023-11-24T06:30:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.070+0000 7f775a7fc640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f77440047c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f775000c210 tx=0x7f77440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914a3a61fb5766e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.070+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750005ad0 con 0x7f7744003fa0 2023-11-24T06:30:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.070+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7750005c30 con 0x7f7744003fa0 2023-11-24T06:30:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.070+0000 7f7759ffb640 1 -- 172.21.15.16:0/1533006720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775002caf0 con 0x7f7744003fa0 2023-11-24T06:30:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.071+0000 7f77618e7640 1 -- 172.21.15.16:0/1533006720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 msgr2=0x7f77440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.071+0000 7f77618e7640 1 --2- 172.21.15.16:0/1533006720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f77440047c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f775000c210 tx=0x7f77440057f0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.071+0000 7f775b7fe640 1 -- 172.21.15.16:0/1533006720 reap_dead start 2023-11-24T06:30:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.071+0000 7f77618e7640 1 -- 172.21.15.16:0/1533006720 shutdown_connections 2023-11-24T06:30:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.071+0000 7f77618e7640 1 -- 172.21.15.16:0/1533006720 >> 172.21.15.16:0/1533006720 conn(0x7f775c06a800 msgr2=0x7f775c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.071+0000 7f77618e7640 1 -- 172.21.15.16:0/1533006720 shutdown_connections 2023-11-24T06:30:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.071+0000 7f77618e7640 1 -- 172.21.15.16:0/1533006720 wait complete. 2023-11-24T06:30:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.073+0000 7f77618e7640 1 Processor -- start 2023-11-24T06:30:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.073+0000 7f77618e7640 1 -- start start 2023-11-24T06:30:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f77618e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f775c1e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f77618e7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f775c13cbe0 0x7f775c13d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f77618e7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c13d7b0 0x7f775c1f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f77618e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c13d060 con 0x7f7744003fa0 2023-11-24T06:30:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f77618e7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c1e5750 con 0x7f775c13cbe0 2023-11-24T06:30:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f77618e7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c1e5a30 con 0x7f775c13d7b0 2023-11-24T06:30:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f775affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f775c1e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f775affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f775c1e67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39964/0 (socket says 172.21.15.16:39964) 2023-11-24T06:30:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f775affd640 1 -- 172.21.15.16:0/126957424 learned_addr learned my addr 172.21.15.16:0/126957424 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.074+0000 7f775a7fc640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f775c13cbe0 0x7f775c13d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f775affd640 1 -- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c13d7b0 msgr2=0x7f775c1f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f775b7fe640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c13d7b0 0x7f775c1f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f775affd640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c13d7b0 0x7f775c1f0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f775affd640 1 -- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f775c13cbe0 msgr2=0x7f775c13d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f775affd640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f775c13cbe0 0x7f775c13d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f775affd640 1 -- 172.21.15.16:0/126957424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c1e5d10 con 0x7f7744003fa0 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f775b7fe640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c13d7b0 0x7f775c1f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f775affd640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f775c1e67f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f774c0107b0 tx=0x7f774c010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f7743fff640 1 -- 172.21.15.16:0/126957424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00eb60 con 0x7f7744003fa0 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f7743fff640 1 -- 172.21.15.16:0/126957424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c00c050 con 0x7f7744003fa0 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f7743fff640 1 -- 172.21.15.16:0/126957424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c010ec0 con 0x7f7744003fa0 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c1e5ff0 con 0x7f7744003fa0 2023-11-24T06:30:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.075+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c1e5390 con 0x7f7744003fa0 2023-11-24T06:30:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.077+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c0675d0 con 0x7f7744003fa0 2023-11-24T06:30:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.077+0000 7f7743fff640 1 -- 172.21.15.16:0/126957424 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f774c015470 con 0x7f7744003fa0 2023-11-24T06:30:59.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.078+0000 7f7743fff640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f774806b2c0 0x7f774806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.079+0000 7f7743fff640 1 -- 172.21.15.16:0/126957424 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f774c08a9e0 con 0x7f7744003fa0 2023-11-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.079+0000 7f775a7fc640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f774806b2c0 0x7f774806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.080+0000 7f775a7fc640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f774806b2c0 0x7f774806d780 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7750034000 tx=0x7f7750007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:59.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.083+0000 7f7743fff640 1 -- 172.21.15.16:0/126957424 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f774c055260 con 0x7f7744003fa0 2023-11-24T06:30:59.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.143+0000 7f7743fff640 1 -- 172.21.15.16:0/126957424 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f774c055630 con 0x7f7744003fa0 2023-11-24T06:30:59.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.295+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f775c0b43e0 con 0x7f7744003fa0 2023-11-24T06:30:59.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.297+0000 7f7743fff640 1 -- 172.21.15.16:0/126957424 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f774c05f0a0 con 0x7f7744003fa0 2023-11-24T06:30:59.304 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:30:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.303+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f774806b2c0 msgr2=0x7f774806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.303+0000 7f77618e7640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f774806b2c0 0x7f774806d780 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7750034000 tx=0x7f7750007000 comp rx=0 tx=0).stop 2023-11-24T06:30:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.303+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 msgr2=0x7f775c1e67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.303+0000 7f77618e7640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f775c1e67f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f774c0107b0 tx=0x7f774c010c70 comp rx=0 tx=0).stop 2023-11-24T06:30:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.304+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 shutdown_connections 2023-11-24T06:30:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.304+0000 7f77618e7640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f774806b2c0 0x7f774806d780 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.304+0000 7f77618e7640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f775c13d7b0 0x7f775c1f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.304+0000 7f77618e7640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f775c13cbe0 0x7f775c13d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.304+0000 7f77618e7640 1 --2- 172.21.15.16:0/126957424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7744003fa0 0x7f775c1e67f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.304+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 >> 172.21.15.16:0/126957424 conn(0x7f775c06a800 msgr2=0x7f775c1540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.305+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 shutdown_connections 2023-11-24T06:30:59.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.305+0000 7f77618e7640 1 -- 172.21.15.16:0/126957424 wait complete. 2023-11-24T06:30:59.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:30:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.524+0000 7f533270f640 1 Processor -- start 2023-11-24T06:30:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.524+0000 7f533270f640 1 -- start start 2023-11-24T06:30:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.526+0000 7f533270f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f532c13df80 0x7f532c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.526+0000 7f533270f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f532c13e360 0x7f532c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.526+0000 7f533270f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c06c470 con 0x7f532c15f9c0 2023-11-24T06:30:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.526+0000 7f533270f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c06c750 con 0x7f532c13df80 2023-11-24T06:30:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.526+0000 7f533270f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c06ca30 con 0x7f532c13e360 2023-11-24T06:30:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.526+0000 7f532bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f532c13df80 0x7f532c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.526+0000 7f532bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f532c13df80 0x7f532c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39968/0 (socket says 172.21.15.16:39968) 2023-11-24T06:30:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.526+0000 7f532bfff640 1 -- 172.21.15.16:0/3759317734 learned_addr learned my addr 172.21.15.16:0/3759317734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.527+0000 7f53237fe640 1 --2- 172.21.15.16:0/3759317734 >> v2:172.21.15.173:3300/0 conn(0x7f532c13e360 0x7f532c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.527+0000 7f532bfff640 1 -- 172.21.15.16:0/3759317734 >> v2:172.21.15.173:3300/0 conn(0x7f532c13e360 msgr2=0x7f532c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:30:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.527+0000 7f532bfff640 1 --2- 172.21.15.16:0/3759317734 >> v2:172.21.15.173:3300/0 conn(0x7f532c13e360 0x7f532c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.527+0000 7f532bfff640 1 -- 172.21.15.16:0/3759317734 >> v1:172.21.15.16:6789/0 conn(0x7f532c15f9c0 legacy=0x7f532c1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.527+0000 7f532bfff640 1 -- 172.21.15.16:0/3759317734 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c06cd10 con 0x7f532c13df80 2023-11-24T06:30:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.528+0000 7f532bfff640 1 --2- 172.21.15.16:0/3759317734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13df80 0x7f532c15bee0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f532c06ded0 tx=0x7f531c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7688e91ed2e001cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f532b7fe640 1 -- 172.21.15.16:0/3759317734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c030860 con 0x7f532c13df80 2023-11-24T06:30:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f532b7fe640 1 -- 172.21.15.16:0/3759317734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c0309c0 con 0x7f532c13df80 2023-11-24T06:30:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f532b7fe640 1 -- 172.21.15.16:0/3759317734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c030d50 con 0x7f532c13df80 2023-11-24T06:30:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f533270f640 1 -- 172.21.15.16:0/3759317734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13df80 msgr2=0x7f532c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f533270f640 1 --2- 172.21.15.16:0/3759317734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13df80 0x7f532c15bee0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f532c06ded0 tx=0x7f531c02b9a0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f533270f640 1 -- 172.21.15.16:0/3759317734 shutdown_connections 2023-11-24T06:30:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f533270f640 1 --2- 172.21.15.16:0/3759317734 >> v2:172.21.15.173:3300/0 conn(0x7f532c13e360 0x7f532c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f533270f640 1 --2- 172.21.15.16:0/3759317734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13df80 0x7f532c15bee0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f533270f640 1 -- 172.21.15.16:0/3759317734 >> 172.21.15.16:0/3759317734 conn(0x7f532c06a800 msgr2=0x7f532c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f533270f640 1 -- 172.21.15.16:0/3759317734 shutdown_connections 2023-11-24T06:30:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.529+0000 7f533270f640 1 -- 172.21.15.16:0/3759317734 wait complete. 2023-11-24T06:30:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.531+0000 7f533270f640 1 Processor -- start 2023-11-24T06:30:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.531+0000 7f533270f640 1 -- start start 2023-11-24T06:30:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.532+0000 7f533270f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f532c13df80 0x7f532c070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.532+0000 7f533270f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13e360 0x7f532c14aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.532+0000 7f533270f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f532c15f9c0 0x7f532c0a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.532+0000 7f533270f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c071f70 con 0x7f532c13e360 2023-11-24T06:30:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.532+0000 7f533270f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c071360 con 0x7f532c15f9c0 2023-11-24T06:30:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.532+0000 7f533270f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c071640 con 0x7f532c13df80 2023-11-24T06:30:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f53237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13e360 0x7f532c14aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f53237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13e360 0x7f532c14aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39972/0 (socket says 172.21.15.16:39972) 2023-11-24T06:30:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f53237fe640 1 -- 172.21.15.16:0/2932939217 learned_addr learned my addr 172.21.15.16:0/2932939217 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:30:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f53237fe640 1 -- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f532c13df80 msgr2=0x7f532c070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f53237fe640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f532c13df80 0x7f532c070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f53237fe640 1 -- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f532c15f9c0 msgr2=0x7f532c0a9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f5330cac640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f532c15f9c0 0x7f532c0a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f53237fe640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f532c15f9c0 0x7f532c0a9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f53237fe640 1 -- 172.21.15.16:0/2932939217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c0094d0 con 0x7f532c13e360 2023-11-24T06:30:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.533+0000 7f53237fe640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13e360 0x7f532c14aed0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5314007ec0 tx=0x7f531400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.534+0000 7f53297fa640 1 -- 172.21.15.16:0/2932939217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314003b90 con 0x7f532c13e360 2023-11-24T06:30:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.534+0000 7f53297fa640 1 -- 172.21.15.16:0/2932939217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314003cf0 con 0x7f532c13e360 2023-11-24T06:30:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.534+0000 7f53297fa640 1 -- 172.21.15.16:0/2932939217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314003ea0 con 0x7f532c13e360 2023-11-24T06:30:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.534+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c071920 con 0x7f532c13e360 2023-11-24T06:30:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.534+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c14b410 con 0x7f532c13e360 2023-11-24T06:30:59.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.535+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c0675d0 con 0x7f532c13e360 2023-11-24T06:30:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.536+0000 7f53297fa640 1 -- 172.21.15.16:0/2932939217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5314004250 con 0x7f532c13e360 2023-11-24T06:30:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.537+0000 7f53297fa640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f530c06b3e0 0x7f530c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:30:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.537+0000 7f53297fa640 1 -- 172.21.15.16:0/2932939217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f531408a4d0 con 0x7f532c13e360 2023-11-24T06:30:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.538+0000 7f532bfff640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f530c06b3e0 0x7f530c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:30:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.539+0000 7f532bfff640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f530c06b3e0 0x7f530c06d8a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f531c036000 tx=0x7f531c03b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:30:59.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.541+0000 7f53297fa640 1 -- 172.21.15.16:0/2932939217 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5314054d80 con 0x7f532c13e360 2023-11-24T06:30:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.748+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f532c0720b0 con 0x7f532c13e360 2023-11-24T06:30:59.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.749+0000 7f53297fa640 1 -- 172.21.15.16:0/2932939217 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f53140583a0 con 0x7f532c13e360 2023-11-24T06:30:59.752 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:30:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f530c06b3e0 msgr2=0x7f530c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f530c06b3e0 0x7f530c06d8a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f531c036000 tx=0x7f531c03b040 comp rx=0 tx=0).stop 2023-11-24T06:30:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13e360 msgr2=0x7f532c14aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:30:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13e360 0x7f532c14aed0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5314007ec0 tx=0x7f531400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 shutdown_connections 2023-11-24T06:30:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f530c06b3e0 0x7f530c06d8a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f532c15f9c0 0x7f532c0a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c13e360 0x7f532c14aed0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 --2- 172.21.15.16:0/2932939217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f532c13df80 0x7f532c070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:30:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.752+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 >> 172.21.15.16:0/2932939217 conn(0x7f532c06a800 msgr2=0x7f532c155620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:30:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.753+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 shutdown_connections 2023-11-24T06:30:59.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:30:59.753+0000 7f533270f640 1 -- 172.21.15.16:0/2932939217 wait complete. 2023-11-24T06:30:59.772 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-24T06:30:59.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:59.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:30:59.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[111529]: fuse finished with error 0 and tester_r 0 2023-11-24T06:30:59.878 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:30:59.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:30:59.878 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:30:59.973 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:30:59.973 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:00.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.268+0000 7f33773e9640 1 Processor -- start 2023-11-24T06:31:00.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.268+0000 7f33773e9640 1 -- start start 2023-11-24T06:31:00.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.270+0000 7f33773e9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3370162430 0x7f33700b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.270+0000 7f33773e9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3370162810 0x7f3370071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.270+0000 7f33773e9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33700702e0 con 0x7f33700b3650 2023-11-24T06:31:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.270+0000 7f33773e9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33700705c0 con 0x7f3370162430 2023-11-24T06:31:00.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.270+0000 7f33773e9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33700708a0 con 0x7f3370162810 2023-11-24T06:31:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.270+0000 7f33763e7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3370162430 0x7f33700b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.270+0000 7f3376be8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f33700b3650 0x7f33701650c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40476/0 (socket says 172.21.15.16:40476) 2023-11-24T06:31:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.270+0000 7f3376be8640 1 -- 172.21.15.16:0/1658187395 learned_addr learned my addr 172.21.15.16:0/1658187395 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.270+0000 7f3375be6640 1 --2- 172.21.15.16:0/1658187395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3370162810 0x7f3370071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.271+0000 7f33763e7640 1 -- 172.21.15.16:0/1658187395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3370162810 msgr2=0x7f3370071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.271+0000 7f33763e7640 1 --2- 172.21.15.16:0/1658187395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3370162810 0x7f3370071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.271+0000 7f33763e7640 1 -- 172.21.15.16:0/1658187395 >> v1:172.21.15.16:6789/0 conn(0x7f33700b3650 legacy=0x7f33701650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.271+0000 7f33763e7640 1 -- 172.21.15.16:0/1658187395 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3370070b80 con 0x7f3370162430 2023-11-24T06:31:00.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.272+0000 7f33763e7640 1 --2- 172.21.15.16:0/1658187395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370162430 0x7f33700b3110 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f336c0097a0 tx=0x7f336c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b01fa8a11c7c9b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.272+0000 7f33753e5640 1 -- 172.21.15.16:0/1658187395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c02d040 con 0x7f3370162430 2023-11-24T06:31:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.272+0000 7f33753e5640 1 -- 172.21.15.16:0/1658187395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c031810 con 0x7f3370162430 2023-11-24T06:31:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.272+0000 7f33753e5640 1 -- 172.21.15.16:0/1658187395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c031ba0 con 0x7f3370162430 2023-11-24T06:31:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.273+0000 7f33773e9640 1 -- 172.21.15.16:0/1658187395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370162430 msgr2=0x7f33700b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.273+0000 7f33773e9640 1 --2- 172.21.15.16:0/1658187395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370162430 0x7f33700b3110 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f336c0097a0 tx=0x7f336c02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.273+0000 7f33773e9640 1 -- 172.21.15.16:0/1658187395 shutdown_connections 2023-11-24T06:31:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.273+0000 7f33773e9640 1 --2- 172.21.15.16:0/1658187395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3370162810 0x7f3370071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.273+0000 7f33773e9640 1 --2- 172.21.15.16:0/1658187395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370162430 0x7f33700b3110 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.273+0000 7f33773e9640 1 -- 172.21.15.16:0/1658187395 >> 172.21.15.16:0/1658187395 conn(0x7f337006a800 msgr2=0x7f337006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.273+0000 7f33773e9640 1 -- 172.21.15.16:0/1658187395 shutdown_connections 2023-11-24T06:31:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.273+0000 7f33773e9640 1 -- 172.21.15.16:0/1658187395 wait complete. 2023-11-24T06:31:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.275+0000 7f33773e9640 1 Processor -- start 2023-11-24T06:31:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.275+0000 7f33773e9640 1 -- start start 2023-11-24T06:31:00.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.276+0000 7f33773e9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33700b3650 0x7f3370140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.276+0000 7f33773e9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3370162430 0x7f33701539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.276+0000 7f33773e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370142cc0 0x7f3370143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.276+0000 7f33773e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3370159600 con 0x7f3370142cc0 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.276+0000 7f33773e9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3370154620 con 0x7f3370162430 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.276+0000 7f33773e9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3370143670 con 0x7f33700b3650 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.276+0000 7f3376be8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370142cc0 0x7f3370143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.276+0000 7f3376be8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370142cc0 0x7f3370143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39984/0 (socket says 172.21.15.16:39984) 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.276+0000 7f3376be8640 1 -- 172.21.15.16:0/2797450943 learned_addr learned my addr 172.21.15.16:0/2797450943 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3375be6640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3370162430 0x7f33701539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f33763e7640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33700b3650 0x7f3370140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3376be8640 1 -- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33700b3650 msgr2=0x7f3370140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3376be8640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33700b3650 0x7f3370140c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3376be8640 1 -- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3370162430 msgr2=0x7f33701539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3376be8640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3370162430 0x7f33701539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3376be8640 1 -- 172.21.15.16:0/2797450943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c009400 con 0x7f3370142cc0 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3375be6640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3370162430 0x7f33701539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3376be8640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370142cc0 0x7f3370143130 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f336800cbf0 tx=0x7f336800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3366ffd640 1 -- 172.21.15.16:0/2797450943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336800ad40 con 0x7f3370142cc0 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.277+0000 7f3366ffd640 1 -- 172.21.15.16:0/2797450943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3368012400 con 0x7f3370142cc0 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.278+0000 7f3366ffd640 1 -- 172.21.15.16:0/2797450943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336800aea0 con 0x7f3370142cc0 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.278+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3370162f50 con 0x7f3370142cc0 2023-11-24T06:31:00.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.278+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3370159290 con 0x7f3370142cc0 2023-11-24T06:31:00.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.279+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3370067450 con 0x7f3370142cc0 2023-11-24T06:31:00.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.280+0000 7f3366ffd640 1 -- 172.21.15.16:0/2797450943 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f3368007070 con 0x7f3370142cc0 2023-11-24T06:31:00.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.281+0000 7f3366ffd640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335406b410 0x7f335406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.281+0000 7f3366ffd640 1 -- 172.21.15.16:0/2797450943 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f3368089fb0 con 0x7f3370142cc0 2023-11-24T06:31:00.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.282+0000 7f33763e7640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335406b410 0x7f335406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.283+0000 7f33763e7640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335406b410 0x7f335406d8d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f336c036000 tx=0x7f336c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:00.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.286+0000 7f3366ffd640 1 -- 172.21.15.16:0/2797450943 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33680547d0 con 0x7f3370142cc0 2023-11-24T06:31:00.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.483+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3370161610 con 0x7f3370142cc0 2023-11-24T06:31:00.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.486+0000 7f3366ffd640 1 -- 172.21.15.16:0/2797450943 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16824 (secure 0 0 0) 0x7f3368017020 con 0x7f3370142cc0 2023-11-24T06:31:00.489 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:00.489 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:59.119430+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:30:31.631710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:30:32.732622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335406b410 msgr2=0x7f335406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335406b410 0x7f335406d8d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f336c036000 tx=0x7f336c03c040 comp rx=0 tx=0).stop 2023-11-24T06:31:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370142cc0 msgr2=0x7f3370143130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370142cc0 0x7f3370143130 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f336800cbf0 tx=0x7f336800a430 comp rx=0 tx=0).stop 2023-11-24T06:31:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 shutdown_connections 2023-11-24T06:31:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f335406b410 0x7f335406d8d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3370142cc0 0x7f3370143130 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3370162430 0x7f33701539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 --2- 172.21.15.16:0/2797450943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33700b3650 0x7f3370140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.489+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 >> 172.21.15.16:0/2797450943 conn(0x7f337006a800 msgr2=0x7f33700cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:00.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.490+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 shutdown_connections 2023-11-24T06:31:00.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.490+0000 7f33773e9640 1 -- 172.21.15.16:0/2797450943 wait complete. 2023-11-24T06:31:00.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:30:31.631710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:30:32.732622+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 74, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:31:00.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:31:00.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.731+0000 7f92ba17c640 1 Processor -- start 2023-11-24T06:31:00.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.731+0000 7f92ba17c640 1 -- start start 2023-11-24T06:31:00.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.733+0000 7f92ba17c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f92b4143810 0x7f92b4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.733+0000 7f92ba17c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b40ae170 0x7f92b415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.733+0000 7f92ba17c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b406c730 con 0x7f92b415d770 2023-11-24T06:31:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.733+0000 7f92ba17c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b4143bf0 con 0x7f92b4143810 2023-11-24T06:31:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.733+0000 7f92ba17c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40ae5f0 con 0x7f92b40ae170 2023-11-24T06:31:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.734+0000 7f92b917a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f92b4143810 0x7f92b4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.734+0000 7f92b8979640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b40ae170 0x7f92b415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.734+0000 7f92b917a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f92b4143810 0x7f92b4071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:42352/0 (socket says 172.21.15.16:42352) 2023-11-24T06:31:00.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.734+0000 7f92b917a640 1 -- 172.21.15.16:0/4073677327 learned_addr learned my addr 172.21.15.16:0/4073677327 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.735+0000 7f92b917a640 1 -- 172.21.15.16:0/4073677327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b40ae170 msgr2=0x7f92b415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.735+0000 7f92b917a640 1 --2- 172.21.15.16:0/4073677327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b40ae170 0x7f92b415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.735+0000 7f92b917a640 1 -- 172.21.15.16:0/4073677327 >> v1:172.21.15.173:6789/0 conn(0x7f92b415d770 legacy=0x7f92b415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:00.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.735+0000 7f92b917a640 1 -- 172.21.15.16:0/4073677327 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40724d0 con 0x7f92b4143810 2023-11-24T06:31:00.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.736+0000 7f92b917a640 1 --2- 172.21.15.16:0/4073677327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b4143810 0x7f92b4071e40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009e50 tx=0x7f92b002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99cd11ee6576a3bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92abfff640 1 -- 172.21.15.16:0/4073677327 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b002ca70 con 0x7f92b4143810 2023-11-24T06:31:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92abfff640 1 -- 172.21.15.16:0/4073677327 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b002cbd0 con 0x7f92b4143810 2023-11-24T06:31:00.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92abfff640 1 -- 172.21.15.16:0/4073677327 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b00343f0 con 0x7f92b4143810 2023-11-24T06:31:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92ba17c640 1 -- 172.21.15.16:0/4073677327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b4143810 msgr2=0x7f92b4071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92ba17c640 1 --2- 172.21.15.16:0/4073677327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b4143810 0x7f92b4071e40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009e50 tx=0x7f92b002bed0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92ba17c640 1 -- 172.21.15.16:0/4073677327 shutdown_connections 2023-11-24T06:31:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92ba17c640 1 --2- 172.21.15.16:0/4073677327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b40ae170 0x7f92b415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92ba17c640 1 --2- 172.21.15.16:0/4073677327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b4143810 0x7f92b4071e40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92ba17c640 1 -- 172.21.15.16:0/4073677327 >> 172.21.15.16:0/4073677327 conn(0x7f92b406a7d0 msgr2=0x7f92b406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.737+0000 7f92ba17c640 1 -- 172.21.15.16:0/4073677327 shutdown_connections 2023-11-24T06:31:00.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.738+0000 7f92ba17c640 1 -- 172.21.15.16:0/4073677327 wait complete. 2023-11-24T06:31:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.739+0000 7f92ba17c640 1 Processor -- start 2023-11-24T06:31:00.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.739+0000 7f92ba17c640 1 -- start start 2023-11-24T06:31:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.740+0000 7f92ba17c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92b40ae170 0x7f92b41f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.740+0000 7f92ba17c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b4143810 0x7f92b4137c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.740+0000 7f92ba17c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b415d770 0x7f92b4145ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.740+0000 7f92ba17c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b41381b0 con 0x7f92b40ae170 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.740+0000 7f92ba17c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4146e70 con 0x7f92b4143810 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.740+0000 7f92ba17c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b41465d0 con 0x7f92b415d770 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.741+0000 7f92b917a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92b40ae170 0x7f92b41f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.741+0000 7f92b917a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92b40ae170 0x7f92b41f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39990/0 (socket says 172.21.15.16:39990) 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.741+0000 7f92b917a640 1 -- 172.21.15.16:0/4122354201 learned_addr learned my addr 172.21.15.16:0/4122354201 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.741+0000 7f92b8979640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b4143810 0x7f92b4137c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.741+0000 7f92b917a640 1 -- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b415d770 msgr2=0x7f92b4145ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.741+0000 7f92b917a640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b415d770 0x7f92b4145ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.741+0000 7f92b917a640 1 -- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b4143810 msgr2=0x7f92b4137c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.741+0000 7f92b917a640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b4143810 0x7f92b4137c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.741+0000 7f92b917a640 1 -- 172.21.15.16:0/4122354201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b0009b30 con 0x7f92b40ae170 2023-11-24T06:31:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.742+0000 7f92b917a640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92b40ae170 0x7f92b41f6d70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009e50 tx=0x7f92b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.742+0000 7f92a9ffb640 1 -- 172.21.15.16:0/4122354201 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0009510 con 0x7f92b40ae170 2023-11-24T06:31:00.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.742+0000 7f92a9ffb640 1 -- 172.21.15.16:0/4122354201 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b003e450 con 0x7f92b40ae170 2023-11-24T06:31:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.742+0000 7f92a9ffb640 1 -- 172.21.15.16:0/4122354201 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b003e820 con 0x7f92b40ae170 2023-11-24T06:31:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.742+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b41468b0 con 0x7f92b40ae170 2023-11-24T06:31:00.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.742+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b40afaf0 con 0x7f92b40ae170 2023-11-24T06:31:00.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.743+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b40afd10 con 0x7f92b40ae170 2023-11-24T06:31:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.744+0000 7f92a9ffb640 1 -- 172.21.15.16:0/4122354201 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f92b003e980 con 0x7f92b40ae170 2023-11-24T06:31:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.745+0000 7f92a9ffb640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f929806b3e0 0x7f929806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.745+0000 7f92a9ffb640 1 -- 172.21.15.16:0/4122354201 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f92b00acdd0 con 0x7f92b40ae170 2023-11-24T06:31:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.746+0000 7f92b8979640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f929806b3e0 0x7f929806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:00.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.747+0000 7f92b8979640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f929806b3e0 0x7f929806d8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f92b41613d0 tx=0x7f92a4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:00.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.749+0000 7f92a9ffb640 1 -- 172.21.15.16:0/4122354201 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92b00775d0 con 0x7f92b40ae170 2023-11-24T06:31:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.941+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f92b4067450 con 0x7f92b40ae170 2023-11-24T06:31:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.942+0000 7f92a9ffb640 1 -- 172.21.15.16:0/4122354201 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f92b0043020 con 0x7f92b40ae170 2023-11-24T06:31:00.945 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:31:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f929806b3e0 msgr2=0x7f929806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f929806b3e0 0x7f929806d8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f92b41613d0 tx=0x7f92a4008040 comp rx=0 tx=0).stop 2023-11-24T06:31:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92b40ae170 msgr2=0x7f92b41f6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92b40ae170 0x7f92b41f6d70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009e50 tx=0x7f92b0004a40 comp rx=0 tx=0).stop 2023-11-24T06:31:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 shutdown_connections 2023-11-24T06:31:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f929806b3e0 0x7f929806d8a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92b415d770 0x7f92b4145ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92b4143810 0x7f92b4137c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 --2- 172.21.15.16:0/4122354201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92b40ae170 0x7f92b41f6d70 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 >> 172.21.15.16:0/4122354201 conn(0x7f92b406a7d0 msgr2=0x7f92b40b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 shutdown_connections 2023-11-24T06:31:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:00.945+0000 7f92ba17c640 1 -- 172.21.15.16:0/4122354201 wait complete. 2023-11-24T06:31:00.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:31:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.202+0000 7f3ddac90640 1 Processor -- start 2023-11-24T06:31:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.202+0000 7f3ddac90640 1 -- start start 2023-11-24T06:31:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.204+0000 7f3ddac90640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd413e360 0x7f3dd40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.204+0000 7f3ddac90640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd415c5f0 con 0x7f3dd415f9c0 2023-11-24T06:31:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.204+0000 7f3ddac90640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd406c470 con 0x7f3dd413df80 2023-11-24T06:31:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.204+0000 7f3ddac90640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd406c750 con 0x7f3dd413e360 2023-11-24T06:31:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.204+0000 7f3dd922d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3dd415f9c0 0x7f3dd40cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40480/0 (socket says 172.21.15.16:40480) 2023-11-24T06:31:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.204+0000 7f3dd922d640 1 -- 172.21.15.16:0/4144375111 learned_addr learned my addr 172.21.15.16:0/4144375111 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd3fff640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd413e360 0x7f3dd40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587466454 0 0) 0x7f3dd415c5f0 con 0x7f3dd415f9c0 2023-11-24T06:31:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dc0003520 con 0x7f3dd415f9c0 2023-11-24T06:31:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2000513948 0 0) 0x7f3dc0003520 con 0x7f3dd415f9c0 2023-11-24T06:31:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd413e360 msgr2=0x7f3dd40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd413e360 0x7f3dd40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 >> v1:172.21.15.173:6789/0 conn(0x7f3dd413df80 legacy=0x7f3dd415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd406ca30 con 0x7f3dd415f9c0 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f3dcc001670 con 0x7f3dd415f9c0 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 >> v1:172.21.15.16:6789/0 conn(0x7f3dd415f9c0 legacy=0x7f3dd40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd413e360 0x7f3dc00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.205+0000 7f3dd37fe640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dc0003f20 0x7f3dc00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd37fe640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dc0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd406c750 con 0x7f3dc0004d00 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc000ba60 con 0x7f3dc0003f20 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc000bc20 con 0x7f3dd413e360 2023-11-24T06:31:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd922d640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dc0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:01.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd922d640 1 -- 172.21.15.16:0/4144375111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd413e360 msgr2=0x7f3dc00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd922d640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd413e360 0x7f3dc00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd922d640 1 -- 172.21.15.16:0/4144375111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dc0003f20 msgr2=0x7f3dc00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd922d640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dc0003f20 0x7f3dc00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.206+0000 7f3dd922d640 1 -- 172.21.15.16:0/4144375111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc0003520 con 0x7f3dc0004d00 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.207+0000 7f3dd922d640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dc0005180 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c240 tx=0x7f3dc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5555438dcc388f2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.208+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc005ad0 con 0x7f3dc0004d00 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.208+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc005c30 con 0x7f3dc0004d00 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.208+0000 7f3dd37fe640 1 -- 172.21.15.16:0/4144375111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc02caf0 con 0x7f3dc0004d00 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.208+0000 7f3ddac90640 1 -- 172.21.15.16:0/4144375111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 msgr2=0x7f3dc0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.208+0000 7f3ddac90640 1 --2- 172.21.15.16:0/4144375111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dc0005180 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c240 tx=0x7f3dc00057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.208+0000 7f3dd922d640 1 -- 172.21.15.16:0/4144375111 reap_dead start 2023-11-24T06:31:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.209+0000 7f3ddac90640 1 -- 172.21.15.16:0/4144375111 shutdown_connections 2023-11-24T06:31:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.209+0000 7f3ddac90640 1 -- 172.21.15.16:0/4144375111 >> 172.21.15.16:0/4144375111 conn(0x7f3dd406a800 msgr2=0x7f3dd406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.209+0000 7f3ddac90640 1 -- 172.21.15.16:0/4144375111 shutdown_connections 2023-11-24T06:31:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.209+0000 7f3ddac90640 1 -- 172.21.15.16:0/4144375111 wait complete. 2023-11-24T06:31:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.210+0000 7f3ddac90640 1 Processor -- start 2023-11-24T06:31:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.211+0000 7f3ddac90640 1 -- start start 2023-11-24T06:31:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3ddac90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dd40add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3ddac90640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd40b2cf0 0x7f3dd40b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3ddac90640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dd40a84c0 0x7f3dd40a8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3ddac90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4131e40 con 0x7f3dc0004d00 2023-11-24T06:31:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3ddac90640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd41f3c50 con 0x7f3dd40a84c0 2023-11-24T06:31:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3ddac90640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40b36b0 con 0x7f3dd40b2cf0 2023-11-24T06:31:01.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3dd8a2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dd40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3dd8a2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dd40add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40000/0 (socket says 172.21.15.16:40000) 2023-11-24T06:31:01.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3dd8a2c640 1 -- 172.21.15.16:0/46837288 learned_addr learned my addr 172.21.15.16:0/46837288 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3dd8a2c640 1 -- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd40b2cf0 msgr2=0x7f3dd40b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3dd8a2c640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd40b2cf0 0x7f3dd40b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3dd8a2c640 1 -- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dd40a84c0 msgr2=0x7f3dd40a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3dd8a2c640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dd40a84c0 0x7f3dd40a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.212+0000 7f3dd8a2c640 1 -- 172.21.15.16:0/46837288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4131250 con 0x7f3dc0004d00 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.213+0000 7f3dd8a2c640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dd40add80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c870 tx=0x7f3dc400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.213+0000 7f3dd17fa640 1 -- 172.21.15.16:0/46837288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4003b70 con 0x7f3dc0004d00 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.213+0000 7f3dd17fa640 1 -- 172.21.15.16:0/46837288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4003cd0 con 0x7f3dc0004d00 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.213+0000 7f3dd17fa640 1 -- 172.21.15.16:0/46837288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400aa30 con 0x7f3dc0004d00 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.213+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd40a8eb0 con 0x7f3dc0004d00 2023-11-24T06:31:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.213+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd4131780 con 0x7f3dc0004d00 2023-11-24T06:31:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.214+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd40675d0 con 0x7f3dc0004d00 2023-11-24T06:31:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.215+0000 7f3dd17fa640 1 -- 172.21.15.16:0/46837288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f3dc400ab90 con 0x7f3dc0004d00 2023-11-24T06:31:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.216+0000 7f3dd17fa640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3db406b410 0x7f3db406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.216+0000 7f3dd17fa640 1 -- 172.21.15.16:0/46837288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f3dc4089ef0 con 0x7f3dc0004d00 2023-11-24T06:31:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.217+0000 7f3dd3fff640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3db406b410 0x7f3db406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:01.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.218+0000 7f3dd3fff640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3db406b410 0x7f3db406d8d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3dd413df50 tx=0x7f3dbc009500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:01.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.221+0000 7f3dd17fa640 1 -- 172.21.15.16:0/46837288 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3dc40546f0 con 0x7f3dc0004d00 2023-11-24T06:31:01.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.418+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3dd406df00 con 0x7f3dc0004d00 2023-11-24T06:31:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.418+0000 7f3dd17fa640 1 -- 172.21.15.16:0/46837288 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3dc4057d10 con 0x7f3dc0004d00 2023-11-24T06:31:01.421 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3db406b410 msgr2=0x7f3db406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3db406b410 0x7f3db406d8d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3dd413df50 tx=0x7f3dbc009500 comp rx=0 tx=0).stop 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 msgr2=0x7f3dd40add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dd40add80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c870 tx=0x7f3dc400cd30 comp rx=0 tx=0).stop 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 shutdown_connections 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3db406b410 0x7f3db406d8d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dd40a84c0 0x7f3dd40a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dd40b2cf0 0x7f3dd40b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 --2- 172.21.15.16:0/46837288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc0004d00 0x7f3dd40add80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.421+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 >> 172.21.15.16:0/46837288 conn(0x7f3dd406a800 msgr2=0x7f3dd40b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.422+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 shutdown_connections 2023-11-24T06:31:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.422+0000 7f3ddac90640 1 -- 172.21.15.16:0/46837288 wait complete. 2023-11-24T06:31:01.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:31:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.663+0000 7f0ab15cb640 1 Processor -- start 2023-11-24T06:31:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.663+0000 7f0ab15cb640 1 -- start start 2023-11-24T06:31:01.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.665+0000 7f0ab15cb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0aac0adf20 0x7f0aac070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.665+0000 7f0ab15cb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac15bf10 0x7f0aac0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.665+0000 7f0ab15cb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac15f4e0 con 0x7f0aac0cc980 2023-11-24T06:31:01.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.665+0000 7f0ab15cb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aac15f970 con 0x7f0aac0adf20 2023-11-24T06:31:01.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.665+0000 7f0ab15cb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0cce10 con 0x7f0aac15bf10 2023-11-24T06:31:01.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.666+0000 7f0aabfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0aac0adf20 0x7f0aac070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.666+0000 7f0aab7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac15bf10 0x7f0aac0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.666+0000 7f0aabfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0aac0adf20 0x7f0aac070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.16:42386/0 (socket says 172.21.15.16:42386) 2023-11-24T06:31:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.666+0000 7f0aabfff640 1 -- 172.21.15.16:0/4181515338 learned_addr learned my addr 172.21.15.16:0/4181515338 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.667+0000 7f0aabfff640 1 -- 172.21.15.16:0/4181515338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac15bf10 msgr2=0x7f0aac0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.667+0000 7f0aabfff640 1 --2- 172.21.15.16:0/4181515338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac15bf10 0x7f0aac0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.667+0000 7f0aabfff640 1 -- 172.21.15.16:0/4181515338 >> v1:172.21.15.173:6789/0 conn(0x7f0aac0cc980 legacy=0x7f0aac143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.667+0000 7f0aabfff640 1 -- 172.21.15.16:0/4181515338 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac071130 con 0x7f0aac0adf20 2023-11-24T06:31:01.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.667+0000 7f0aab7fe640 1 --2- 172.21.15.16:0/4181515338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac15bf10 0x7f0aac0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:31:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.668+0000 7f0aabfff640 1 --2- 172.21.15.16:0/4181515338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aac0adf20 0x7f0aac070a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009870 tx=0x7f0a9c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cc52bc2d6642097 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.669+0000 7f0aaaffd640 1 -- 172.21.15.16:0/4181515338 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c031640 con 0x7f0aac0adf20 2023-11-24T06:31:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.669+0000 7f0aaaffd640 1 -- 172.21.15.16:0/4181515338 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c0317a0 con 0x7f0aac0adf20 2023-11-24T06:31:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.669+0000 7f0aaaffd640 1 -- 172.21.15.16:0/4181515338 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c031b30 con 0x7f0aac0adf20 2023-11-24T06:31:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.669+0000 7f0ab15cb640 1 -- 172.21.15.16:0/4181515338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aac0adf20 msgr2=0x7f0aac070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.669+0000 7f0ab15cb640 1 --2- 172.21.15.16:0/4181515338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aac0adf20 0x7f0aac070a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009870 tx=0x7f0a9c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.670+0000 7f0ab15cb640 1 -- 172.21.15.16:0/4181515338 shutdown_connections 2023-11-24T06:31:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.670+0000 7f0ab15cb640 1 --2- 172.21.15.16:0/4181515338 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac15bf10 0x7f0aac0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.670+0000 7f0ab15cb640 1 --2- 172.21.15.16:0/4181515338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aac0adf20 0x7f0aac070a30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.670+0000 7f0ab15cb640 1 -- 172.21.15.16:0/4181515338 >> 172.21.15.16:0/4181515338 conn(0x7f0aac06a7d0 msgr2=0x7f0aac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.670+0000 7f0ab15cb640 1 -- 172.21.15.16:0/4181515338 shutdown_connections 2023-11-24T06:31:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.670+0000 7f0ab15cb640 1 -- 172.21.15.16:0/4181515338 wait complete. 2023-11-24T06:31:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.672+0000 7f0ab15cb640 1 Processor -- start 2023-11-24T06:31:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.672+0000 7f0ab15cb640 1 -- start start 2023-11-24T06:31:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.674+0000 7f0ab15cb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac0adf20 0x7f0aac0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.674+0000 7f0ab15cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0cc980 0x7f0aac13bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.674+0000 7f0ab15cb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aac15bf10 0x7f0aac15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.674+0000 7f0ab15cb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0b1e90 con 0x7f0aac0cc980 2023-11-24T06:31:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.674+0000 7f0ab15cb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac13ac90 con 0x7f0aac15bf10 2023-11-24T06:31:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.674+0000 7f0ab15cb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac13af70 con 0x7f0aac0adf20 2023-11-24T06:31:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.674+0000 7f0aab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0cc980 0x7f0aac13bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.674+0000 7f0aab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0cc980 0x7f0aac13bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40014/0 (socket says 172.21.15.16:40014) 2023-11-24T06:31:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.674+0000 7f0aab7fe640 1 -- 172.21.15.16:0/2551469757 learned_addr learned my addr 172.21.15.16:0/2551469757 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0aab7fe640 1 -- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac0adf20 msgr2=0x7f0aac0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0aab7fe640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac0adf20 0x7f0aac0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0aab7fe640 1 -- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aac15bf10 msgr2=0x7f0aac15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0aab7fe640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aac15bf10 0x7f0aac15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0aab7fe640 1 -- 172.21.15.16:0/2551469757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa0009400 con 0x7f0aac0cc980 2023-11-24T06:31:01.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0aab7fe640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0cc980 0x7f0aac13bcd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000ebe0 tx=0x7f0aa000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0aa8ff9640 1 -- 172.21.15.16:0/2551469757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0009760 con 0x7f0aac0cc980 2023-11-24T06:31:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0aa8ff9640 1 -- 172.21.15.16:0/2551469757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa000cc30 con 0x7f0aac0cc980 2023-11-24T06:31:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c0094d0 con 0x7f0aac0cc980 2023-11-24T06:31:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.675+0000 7f0aa8ff9640 1 -- 172.21.15.16:0/2551469757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa000eee0 con 0x7f0aac0cc980 2023-11-24T06:31:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.676+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac0b20a0 con 0x7f0aac0cc980 2023-11-24T06:31:01.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.677+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac0ce1a0 con 0x7f0aac0cc980 2023-11-24T06:31:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.678+0000 7f0aa8ff9640 1 -- 172.21.15.16:0/2551469757 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0aa0013470 con 0x7f0aac0cc980 2023-11-24T06:31:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.679+0000 7f0aa8ff9640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a9806b3e0 0x7f0a9806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.679+0000 7f0aa8ff9640 1 -- 172.21.15.16:0/2551469757 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f0aa008a400 con 0x7f0aac0cc980 2023-11-24T06:31:01.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.679+0000 7f0aabfff640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a9806b3e0 0x7f0a9806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:01.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.681+0000 7f0aabfff640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a9806b3e0 0x7f0a9806d8a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c036000 tx=0x7f0a9c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.683+0000 7f0aa8ff9640 1 -- 172.21.15.16:0/2551469757 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0aa0054b70 con 0x7f0aac0cc980 2023-11-24T06:31:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.877+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0aac067450 con 0x7f0aac0cc980 2023-11-24T06:31:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.878+0000 7f0aa8ff9640 1 -- 172.21.15.16:0/2551469757 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0aa0058190 con 0x7f0aac0cc980 2023-11-24T06:31:01.880 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:31:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.880+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a9806b3e0 msgr2=0x7f0a9806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.880+0000 7f0ab15cb640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a9806b3e0 0x7f0a9806d8a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c036000 tx=0x7f0a9c030000 comp rx=0 tx=0).stop 2023-11-24T06:31:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.880+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0cc980 msgr2=0x7f0aac13bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.881+0000 7f0ab15cb640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0cc980 0x7f0aac13bcd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000ebe0 tx=0x7f0aa000c520 comp rx=0 tx=0).stop 2023-11-24T06:31:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.881+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 shutdown_connections 2023-11-24T06:31:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.881+0000 7f0ab15cb640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a9806b3e0 0x7f0a9806d8a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.881+0000 7f0ab15cb640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aac15bf10 0x7f0aac15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.881+0000 7f0ab15cb640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0cc980 0x7f0aac13bcd0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.881+0000 7f0ab15cb640 1 --2- 172.21.15.16:0/2551469757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aac0adf20 0x7f0aac0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.881+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 >> 172.21.15.16:0/2551469757 conn(0x7f0aac06a7d0 msgr2=0x7f0aac15da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.881+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 shutdown_connections 2023-11-24T06:31:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:01.881+0000 7f0ab15cb640 1 -- 172.21.15.16:0/2551469757 wait complete. 2023-11-24T06:31:01.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:31:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.128+0000 7f17689d5640 1 Processor -- start 2023-11-24T06:31:02.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.128+0000 7f17689d5640 1 -- start start 2023-11-24T06:31:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.130+0000 7f17689d5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f17640d0560 0x7f1764161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.130+0000 7f17689d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17640d2260 0x7f1764161570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.130+0000 7f17689d5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17640cf260 con 0x7f17640c9870 2023-11-24T06:31:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.130+0000 7f17689d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17640d2ef0 con 0x7f17640d2260 2023-11-24T06:31:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.130+0000 7f17689d5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17640c9c50 con 0x7f17640d0560 2023-11-24T06:31:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.130+0000 7f1762d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17640d2260 0x7f1764161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.130+0000 7f1762d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17640d2260 0x7f1764161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40032/0 (socket says 172.21.15.16:40032) 2023-11-24T06:31:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.130+0000 7f1762d9d640 1 -- 172.21.15.16:0/1424221818 learned_addr learned my addr 172.21.15.16:0/1424221818 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.130+0000 7f1761d9b640 1 --2- 172.21.15.16:0/1424221818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f17640d0560 0x7f1764161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.131+0000 7f1762d9d640 1 -- 172.21.15.16:0/1424221818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f17640d0560 msgr2=0x7f1764161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.131+0000 7f1762d9d640 1 --2- 172.21.15.16:0/1424221818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f17640d0560 0x7f1764161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.131+0000 7f1762d9d640 1 -- 172.21.15.16:0/1424221818 >> v1:172.21.15.173:6789/0 conn(0x7f17640c9870 legacy=0x7f17640ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.131+0000 7f1762d9d640 1 -- 172.21.15.16:0/1424221818 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17640cf4c0 con 0x7f17640d2260 2023-11-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.132+0000 7f1762d9d640 1 --2- 172.21.15.16:0/1424221818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 0x7f1764161570 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f175c009870 tx=0x7f175c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb70b41d7ac28a32 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.132+0000 7f176159a640 1 -- 172.21.15.16:0/1424221818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c031640 con 0x7f17640d2260 2023-11-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.132+0000 7f176159a640 1 -- 172.21.15.16:0/1424221818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c0317a0 con 0x7f17640d2260 2023-11-24T06:31:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.132+0000 7f176159a640 1 -- 172.21.15.16:0/1424221818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c031b30 con 0x7f17640d2260 2023-11-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.133+0000 7f17689d5640 1 -- 172.21.15.16:0/1424221818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 msgr2=0x7f1764161570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.133+0000 7f17689d5640 1 --2- 172.21.15.16:0/1424221818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 0x7f1764161570 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f175c009870 tx=0x7f175c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.133+0000 7f17689d5640 1 -- 172.21.15.16:0/1424221818 shutdown_connections 2023-11-24T06:31:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.133+0000 7f17689d5640 1 --2- 172.21.15.16:0/1424221818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 0x7f1764161570 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.133+0000 7f17689d5640 1 --2- 172.21.15.16:0/1424221818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f17640d0560 0x7f1764161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.133+0000 7f17689d5640 1 -- 172.21.15.16:0/1424221818 >> 172.21.15.16:0/1424221818 conn(0x7f176409c560 msgr2=0x7f17640d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.133+0000 7f17689d5640 1 -- 172.21.15.16:0/1424221818 shutdown_connections 2023-11-24T06:31:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.133+0000 7f17689d5640 1 -- 172.21.15.16:0/1424221818 wait complete. 2023-11-24T06:31:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.135+0000 7f17689d5640 1 Processor -- start 2023-11-24T06:31:02.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.135+0000 7f17689d5640 1 -- start start 2023-11-24T06:31:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.136+0000 7f17689d5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17640c9870 0x7f17640cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.136+0000 7f17689d5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f17640d0560 0x7f17640b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.136+0000 7f17689d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 0x7f176414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.136+0000 7f17689d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640cb8b0 con 0x7f17640d2260 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.136+0000 7f17689d5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640cbb50 con 0x7f17640c9870 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.136+0000 7f17689d5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17640cbe30 con 0x7f17640d0560 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.136+0000 7f1762d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 0x7f176414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.136+0000 7f1762d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 0x7f176414e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40034/0 (socket says 172.21.15.16:40034) 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.136+0000 7f1762d9d640 1 -- 172.21.15.16:0/238441024 learned_addr learned my addr 172.21.15.16:0/238441024 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f176259c640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17640c9870 0x7f17640cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f1762d9d640 1 -- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f17640d0560 msgr2=0x7f17640b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f1762d9d640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f17640d0560 0x7f17640b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f1762d9d640 1 -- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17640c9870 msgr2=0x7f17640cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f1762d9d640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17640c9870 0x7f17640cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f1762d9d640 1 -- 172.21.15.16:0/238441024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0094d0 con 0x7f17640d2260 2023-11-24T06:31:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f1762d9d640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 0x7f176414e200 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f175c034040 tx=0x7f175c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f175affd640 1 -- 172.21.15.16:0/238441024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c03abe0 con 0x7f17640d2260 2023-11-24T06:31:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f175affd640 1 -- 172.21.15.16:0/238441024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c03ad40 con 0x7f17640d2260 2023-11-24T06:31:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.137+0000 7f175affd640 1 -- 172.21.15.16:0/238441024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c0445f0 con 0x7f17640d2260 2023-11-24T06:31:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.138+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17640cc110 con 0x7f17640d2260 2023-11-24T06:31:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.138+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17640cc6d0 con 0x7f17640d2260 2023-11-24T06:31:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.139+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17640cc8c0 con 0x7f17640d2260 2023-11-24T06:31:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.140+0000 7f175affd640 1 -- 172.21.15.16:0/238441024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f175c02f050 con 0x7f17640d2260 2023-11-24T06:31:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.141+0000 7f175affd640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f174406b3e0 0x7f174406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.141+0000 7f175affd640 1 -- 172.21.15.16:0/238441024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f175c0ac140 con 0x7f17640d2260 2023-11-24T06:31:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.142+0000 7f176259c640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f174406b3e0 0x7f174406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:02.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.143+0000 7f176259c640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f174406b3e0 0x7f174406d8a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1754005520 tx=0x7f1754009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.146+0000 7f175affd640 1 -- 172.21.15.16:0/238441024 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f175c0dee90 con 0x7f17640d2260 2023-11-24T06:31:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.359+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17640675d0 con 0x7f17640d2260 2023-11-24T06:31:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.360+0000 7f175affd640 1 -- 172.21.15.16:0/238441024 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7f175c035080 con 0x7f17640d2260 2023-11-24T06:31:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:31:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f174406b3e0 msgr2=0x7f174406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f174406b3e0 0x7f174406d8a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1754005520 tx=0x7f1754009040 comp rx=0 tx=0).stop 2023-11-24T06:31:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 msgr2=0x7f176414e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 0x7f176414e200 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f175c034040 tx=0x7f175c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 shutdown_connections 2023-11-24T06:31:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f174406b3e0 0x7f174406d8a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17640d2260 0x7f176414e200 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f17640d0560 0x7f17640b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 --2- 172.21.15.16:0/238441024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17640c9870 0x7f17640cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.363+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 >> 172.21.15.16:0/238441024 conn(0x7f176409c560 msgr2=0x7f17640ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.364+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 shutdown_connections 2023-11-24T06:31:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:02.364+0000 7f17689d5640 1 -- 172.21.15.16:0/238441024 wait complete. 2023-11-24T06:31:02.384 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi173.front.sepia.ceph.com... 2023-11-24T06:31:02.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:02.384 DEBUG:teuthology.orchestra.run.smithi173:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:31:02.486 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[97548]: fuse finished with error 0 and tester_r 0 2023-11-24T06:31:02.489 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:31:02.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:02.490 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:31:02.672 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:31:02.672 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:02.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:31:02.736 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:31:02.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:02.737 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:31:02.793 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:31:02.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:31:02.794 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:31:02.794 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:02.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:31:02.855 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:31:02.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:02.855 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:31:02.910 INFO:teuthology.orchestra.run.smithi173.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:31:02.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:31:02.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:03.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.150+0000 7fdf64dcb640 1 Processor -- start 2023-11-24T06:31:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.150+0000 7fdf64dcb640 1 -- start start 2023-11-24T06:31:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.151+0000 7fdf64dcb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdf6013cfa0 0x7fdf6015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.151+0000 7fdf64dcb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf600cd530 0x7fdf6013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.151+0000 7fdf64dcb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf600cdec0 con 0x7fdf600cd9b0 2023-11-24T06:31:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.151+0000 7fdf64dcb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf600ce1a0 con 0x7fdf6013cfa0 2023-11-24T06:31:03.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.151+0000 7fdf64dcb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600ce480 con 0x7fdf600cd530 2023-11-24T06:31:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.152+0000 7fdf5effd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf600cd530 0x7fdf6013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.152+0000 7fdf5effd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf600cd530 0x7fdf6013f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:42404/0 (socket says 172.21.15.16:42404) 2023-11-24T06:31:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.152+0000 7fdf5effd640 1 -- 172.21.15.16:0/1181337125 learned_addr learned my addr 172.21.15.16:0/1181337125 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.153+0000 7fdf5f7fe640 1 --2- 172.21.15.16:0/1181337125 >> v2:172.21.15.173:3300/0 conn(0x7fdf6013cfa0 0x7fdf6015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:03.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.153+0000 7fdf5effd640 1 -- 172.21.15.16:0/1181337125 >> v2:172.21.15.173:3300/0 conn(0x7fdf6013cfa0 msgr2=0x7fdf6015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.153+0000 7fdf5effd640 1 --2- 172.21.15.16:0/1181337125 >> v2:172.21.15.173:3300/0 conn(0x7fdf6013cfa0 0x7fdf6015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.153+0000 7fdf5effd640 1 -- 172.21.15.16:0/1181337125 >> v1:172.21.15.173:6789/0 conn(0x7fdf600cd9b0 legacy=0x7fdf6015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.153+0000 7fdf5effd640 1 -- 172.21.15.16:0/1181337125 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600ce760 con 0x7fdf600cd530 2023-11-24T06:31:03.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.155+0000 7fdf5effd640 1 --2- 172.21.15.16:0/1181337125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf600cd530 0x7fdf6013f5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009870 tx=0x7fdf5402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=507880646c89e3b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:03.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.156+0000 7fdf5e7fc640 1 -- 172.21.15.16:0/1181337125 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54031640 con 0x7fdf600cd530 2023-11-24T06:31:03.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.156+0000 7fdf5e7fc640 1 -- 172.21.15.16:0/1181337125 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf540317a0 con 0x7fdf600cd530 2023-11-24T06:31:03.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.156+0000 7fdf5e7fc640 1 -- 172.21.15.16:0/1181337125 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54031b30 con 0x7fdf600cd530 2023-11-24T06:31:03.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.157+0000 7fdf64dcb640 1 -- 172.21.15.16:0/1181337125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf600cd530 msgr2=0x7fdf6013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.157+0000 7fdf64dcb640 1 --2- 172.21.15.16:0/1181337125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf600cd530 0x7fdf6013f5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009870 tx=0x7fdf5402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.157+0000 7fdf64dcb640 1 -- 172.21.15.16:0/1181337125 shutdown_connections 2023-11-24T06:31:03.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.157+0000 7fdf64dcb640 1 --2- 172.21.15.16:0/1181337125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf600cd530 0x7fdf6013f5a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.157+0000 7fdf64dcb640 1 --2- 172.21.15.16:0/1181337125 >> v2:172.21.15.173:3300/0 conn(0x7fdf6013cfa0 0x7fdf6015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.157+0000 7fdf64dcb640 1 -- 172.21.15.16:0/1181337125 >> 172.21.15.16:0/1181337125 conn(0x7fdf6006a800 msgr2=0x7fdf6006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:03.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.157+0000 7fdf64dcb640 1 -- 172.21.15.16:0/1181337125 shutdown_connections 2023-11-24T06:31:03.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.157+0000 7fdf64dcb640 1 -- 172.21.15.16:0/1181337125 wait complete. 2023-11-24T06:31:03.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.159+0000 7fdf64dcb640 1 Processor -- start 2023-11-24T06:31:03.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.159+0000 7fdf64dcb640 1 -- start start 2023-11-24T06:31:03.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.160+0000 7fdf64dcb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf600cd530 0x7fdf6006d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.160+0000 7fdf64dcb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf600cd9b0 0x7fdf600c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.160+0000 7fdf64dcb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf6013cfa0 0x7fdf600cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.160+0000 7fdf64dcb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf60153890 con 0x7fdf600cd530 2023-11-24T06:31:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.160+0000 7fdf64dcb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf601535b0 con 0x7fdf600cd9b0 2023-11-24T06:31:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.160+0000 7fdf64dcb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600c67b0 con 0x7fdf6013cfa0 2023-11-24T06:31:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.161+0000 7fdf5f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf600cd530 0x7fdf6006d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.161+0000 7fdf5f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf600cd530 0x7fdf6006d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36330/0 (socket says 172.21.15.16:36330) 2023-11-24T06:31:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.161+0000 7fdf5f7fe640 1 -- 172.21.15.16:0/3103905437 learned_addr learned my addr 172.21.15.16:0/3103905437 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.161+0000 7fdf5f7fe640 1 -- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf6013cfa0 msgr2=0x7fdf600cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.161+0000 7fdf5f7fe640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf6013cfa0 0x7fdf600cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.161+0000 7fdf5f7fe640 1 -- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf600cd9b0 msgr2=0x7fdf600c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.161+0000 7fdf5f7fe640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf600cd9b0 0x7fdf600c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.161+0000 7fdf5f7fe640 1 -- 172.21.15.16:0/3103905437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540094d0 con 0x7fdf600cd530 2023-11-24T06:31:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.161+0000 7fdf5f7fe640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf600cd530 0x7fdf6006d2c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000e870 tx=0x7fdf5000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.162+0000 7fdf47fff640 1 -- 172.21.15.16:0/3103905437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf500094f0 con 0x7fdf600cd530 2023-11-24T06:31:03.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.162+0000 7fdf47fff640 1 -- 172.21.15.16:0/3103905437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5000c940 con 0x7fdf600cd530 2023-11-24T06:31:03.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.162+0000 7fdf47fff640 1 -- 172.21.15.16:0/3103905437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000ccf0 con 0x7fdf600cd530 2023-11-24T06:31:03.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.162+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf600c6a90 con 0x7fdf600cd530 2023-11-24T06:31:03.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.162+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6006d800 con 0x7fdf600cd530 2023-11-24T06:31:03.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.163+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf600c6610 con 0x7fdf600cd530 2023-11-24T06:31:03.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.164+0000 7fdf47fff640 1 -- 172.21.15.16:0/3103905437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fdf50013470 con 0x7fdf600cd530 2023-11-24T06:31:03.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.165+0000 7fdf47fff640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4c06b3e0 0x7fdf4c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.165+0000 7fdf47fff640 1 -- 172.21.15.16:0/3103905437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fdf5008a230 con 0x7fdf600cd530 2023-11-24T06:31:03.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.165+0000 7fdf5effd640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4c06b3e0 0x7fdf4c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:03.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.167+0000 7fdf5effd640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4c06b3e0 0x7fdf4c06d8a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdf54036000 tx=0x7fdf54030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:03.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.170+0000 7fdf47fff640 1 -- 172.21.15.16:0/3103905437 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf600c6610 con 0x7fdf600cd530 2023-11-24T06:31:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.401+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf600c6610 con 0x7fdf600cd530 2023-11-24T06:31:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.403+0000 7fdf47fff640 1 -- 172.21.15.16:0/3103905437 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fdf600c6610 con 0x7fdf600cd530 2023-11-24T06:31:03.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:03.405 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1164611025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1164611025},{"type":"v1","addr":"172.21.15.16:6834","nonce":1164611025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5650,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1157106761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1157106761},{"type":"v1","addr":"172.21.15.173:6836","nonce":1157106761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/151946729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":151946729},{"type":"v1","addr":"172.21.15.173:6837","nonce":151946729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:30:33.707795+0000","modified":"2023-11-24T06:30:36.770163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6693":{"gid":6693,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2360619406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2360619406},{"type":"v1","addr":"172.21.15.16:6835","nonce":2360619406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:31:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 45 2023-11-24T06:31:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4c06b3e0 msgr2=0x7fdf4c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4c06b3e0 0x7fdf4c06d8a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdf54036000 tx=0x7fdf54030000 comp rx=0 tx=0).stop 2023-11-24T06:31:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf600cd530 msgr2=0x7fdf6006d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf600cd530 0x7fdf6006d2c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000e870 tx=0x7fdf5000ed30 comp rx=0 tx=0).stop 2023-11-24T06:31:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 shutdown_connections 2023-11-24T06:31:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdf4c06b3e0 0x7fdf4c06d8a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf6013cfa0 0x7fdf600cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf600cd9b0 0x7fdf600c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 --2- 172.21.15.16:0/3103905437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf600cd530 0x7fdf6006d2c0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.405+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 >> 172.21.15.16:0/3103905437 conn(0x7fdf6006a800 msgr2=0x7fdf6013c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.406+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 shutdown_connections 2023-11-24T06:31:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.406+0000 7fdf64dcb640 1 -- 172.21.15.16:0/3103905437 wait complete. 2023-11-24T06:31:03.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.653+0000 7f155876f640 1 Processor -- start 2023-11-24T06:31:03.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.654+0000 7f155876f640 1 -- start start 2023-11-24T06:31:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.655+0000 7f155876f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f155013df80 0x7f155015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.655+0000 7f155876f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f155013e360 0x7f15500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.655+0000 7f155876f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f155015f9c0 0x7f15500cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.655+0000 7f155876f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155006c470 con 0x7f155013e360 2023-11-24T06:31:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.655+0000 7f155876f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155006c750 con 0x7f155015f9c0 2023-11-24T06:31:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.655+0000 7f155876f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155006ca30 con 0x7f155013df80 2023-11-24T06:31:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.656+0000 7f1555d0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f155013e360 0x7f15500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.656+0000 7f1555d0a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f155013e360 0x7f15500cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36340/0 (socket says 172.21.15.16:36340) 2023-11-24T06:31:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.656+0000 7f1555d0a640 1 -- 172.21.15.16:0/3132779564 learned_addr learned my addr 172.21.15.16:0/3132779564 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.656+0000 7f1555d0a640 1 -- 172.21.15.16:0/3132779564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f155013df80 msgr2=0x7f155015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.656+0000 7f1555d0a640 1 --2- 172.21.15.16:0/3132779564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f155013df80 0x7f155015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.656+0000 7f1555d0a640 1 -- 172.21.15.16:0/3132779564 >> v2:172.21.15.173:3300/0 conn(0x7f155015f9c0 msgr2=0x7f15500cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:03.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.656+0000 7f1555d0a640 1 --2- 172.21.15.16:0/3132779564 >> v2:172.21.15.173:3300/0 conn(0x7f155015f9c0 0x7f15500cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.656+0000 7f1555d0a640 1 -- 172.21.15.16:0/3132779564 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155006cd10 con 0x7f155013e360 2023-11-24T06:31:03.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.657+0000 7f1555d0a640 1 --2- 172.21.15.16:0/3132779564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013e360 0x7f15500cf290 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f155006e6d0 tx=0x7f154002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d7ce642d001e04 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:03.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.658+0000 7f1555509640 1 -- 172.21.15.16:0/3132779564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540031860 con 0x7f155013e360 2023-11-24T06:31:03.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.658+0000 7f1555509640 1 -- 172.21.15.16:0/3132779564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15400319c0 con 0x7f155013e360 2023-11-24T06:31:03.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.658+0000 7f1555509640 1 -- 172.21.15.16:0/3132779564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540031d50 con 0x7f155013e360 2023-11-24T06:31:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.658+0000 7f155876f640 1 -- 172.21.15.16:0/3132779564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013e360 msgr2=0x7f15500cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.658+0000 7f155876f640 1 --2- 172.21.15.16:0/3132779564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013e360 0x7f15500cf290 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f155006e6d0 tx=0x7f154002b900 comp rx=0 tx=0).stop 2023-11-24T06:31:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.658+0000 7f155876f640 1 -- 172.21.15.16:0/3132779564 shutdown_connections 2023-11-24T06:31:03.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.659+0000 7f155876f640 1 --2- 172.21.15.16:0/3132779564 >> v2:172.21.15.173:3300/0 conn(0x7f155015f9c0 0x7f15500cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.659+0000 7f155876f640 1 --2- 172.21.15.16:0/3132779564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013e360 0x7f15500cf290 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.659+0000 7f155876f640 1 --2- 172.21.15.16:0/3132779564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f155013df80 0x7f155015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.659+0000 7f155876f640 1 -- 172.21.15.16:0/3132779564 >> 172.21.15.16:0/3132779564 conn(0x7f155006a800 msgr2=0x7f155006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:03.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.659+0000 7f155876f640 1 -- 172.21.15.16:0/3132779564 shutdown_connections 2023-11-24T06:31:03.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.659+0000 7f155876f640 1 -- 172.21.15.16:0/3132779564 wait complete. 2023-11-24T06:31:03.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.661+0000 7f155876f640 1 Processor -- start 2023-11-24T06:31:03.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.661+0000 7f155876f640 1 -- start start 2023-11-24T06:31:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155876f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013df80 0x7f15501e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155876f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f155013e360 0x7f1550149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155876f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f155015f9c0 0x7f15501f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155876f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1550070490 con 0x7f155013df80 2023-11-24T06:31:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155876f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1550070730 con 0x7f155013e360 2023-11-24T06:31:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155876f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1550149c80 con 0x7f155015f9c0 2023-11-24T06:31:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155650b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013df80 0x7f15501e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155650b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013df80 0x7f15501e4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36354/0 (socket says 172.21.15.16:36354) 2023-11-24T06:31:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155650b640 1 -- 172.21.15.16:0/2156523101 learned_addr learned my addr 172.21.15.16:0/2156523101 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:03.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155650b640 1 -- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f155015f9c0 msgr2=0x7f15501f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155650b640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f155015f9c0 0x7f15501f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155650b640 1 -- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f155013e360 msgr2=0x7f1550149740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.662+0000 7f155650b640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f155013e360 0x7f1550149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.663+0000 7f155650b640 1 -- 172.21.15.16:0/2156523101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15400094d0 con 0x7f155013df80 2023-11-24T06:31:03.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.663+0000 7f155650b640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013df80 0x7f15501e4ae0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f154c00b410 tx=0x7f154c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.663+0000 7f1546ffd640 1 -- 172.21.15.16:0/2156523101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c011830 con 0x7f155013df80 2023-11-24T06:31:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.663+0000 7f1546ffd640 1 -- 172.21.15.16:0/2156523101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c0119b0 con 0x7f155013df80 2023-11-24T06:31:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.663+0000 7f1546ffd640 1 -- 172.21.15.16:0/2156523101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c011d60 con 0x7f155013df80 2023-11-24T06:31:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.663+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15501f6d20 con 0x7f155013df80 2023-11-24T06:31:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.664+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15501f72a0 con 0x7f155013df80 2023-11-24T06:31:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.665+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1550155970 con 0x7f155013df80 2023-11-24T06:31:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.666+0000 7f1546ffd640 1 -- 172.21.15.16:0/2156523101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f154c01a400 con 0x7f155013df80 2023-11-24T06:31:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.667+0000 7f1546ffd640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f153406b3e0 0x7f153406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.667+0000 7f1546ffd640 1 -- 172.21.15.16:0/2156523101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f154c08a2a0 con 0x7f155013df80 2023-11-24T06:31:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.667+0000 7f1555d0a640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f153406b3e0 0x7f153406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:03.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.669+0000 7f1555d0a640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f153406b3e0 0x7f153406d8a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1540036000 tx=0x7f1540030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.671+0000 7f1546ffd640 1 -- 172.21.15.16:0/2156523101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f154c054a10 con 0x7f155013df80 2023-11-24T06:31:03.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.865+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15500675d0 con 0x7f155013df80 2023-11-24T06:31:03.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.868+0000 7f1546ffd640 1 -- 172.21.15.16:0/2156523101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16824 (secure 0 0 0) 0x7f154c058030 con 0x7f155013df80 2023-11-24T06:31:03.871 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:03.872 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:59.119430+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:30:31.631710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:30:32.732622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:03.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f153406b3e0 msgr2=0x7f153406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f153406b3e0 0x7f153406d8a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1540036000 tx=0x7f1540030000 comp rx=0 tx=0).stop 2023-11-24T06:31:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013df80 msgr2=0x7f15501e4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013df80 0x7f15501e4ae0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f154c00b410 tx=0x7f154c00b8d0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 shutdown_connections 2023-11-24T06:31:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f153406b3e0 0x7f153406d8a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f155015f9c0 0x7f15501f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f155013e360 0x7f1550149740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 --2- 172.21.15.16:0/2156523101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155013df80 0x7f15501e4ae0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 >> 172.21.15.16:0/2156523101 conn(0x7f155006a800 msgr2=0x7f155013ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 shutdown_connections 2023-11-24T06:31:03.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:03.871+0000 7f155876f640 1 -- 172.21.15.16:0/2156523101 wait complete. 2023-11-24T06:31:03.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:30:31.631710+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:30:32.732622+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 74, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:31:03.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:31:04.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.117+0000 7f3119ebb640 1 Processor -- start 2023-11-24T06:31:04.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.117+0000 7f3119ebb640 1 -- start start 2023-11-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.118+0000 7f3119ebb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f311415d4b0 0x7f31140d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.119+0000 7f3119ebb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31140b2730 con 0x7f31140b41e0 2023-11-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.119+0000 7f3119ebb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31140d1ec0 con 0x7f31140cf9a0 2023-11-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.119+0000 7f3119ebb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31140b29d0 con 0x7f311415d4b0 2023-11-24T06:31:04.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.119+0000 7f3118eb9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f311415d4b0 0x7f31140d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.119+0000 7f31196ba640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f31140b41e0 0x7f31140ce500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48406/0 (socket says 172.21.15.16:48406) 2023-11-24T06:31:04.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.119+0000 7f31196ba640 1 -- 172.21.15.16:0/2475559100 learned_addr learned my addr 172.21.15.16:0/2475559100 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:04.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.119+0000 7f3118eb9640 1 -- 172.21.15.16:0/2475559100 >> v1:172.21.15.173:6789/0 conn(0x7f31140cf9a0 legacy=0x7f31140cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:04.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.119+0000 7f3118eb9640 1 -- 172.21.15.16:0/2475559100 >> v1:172.21.15.16:6789/0 conn(0x7f31140b41e0 legacy=0x7f31140ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.119+0000 7f3118eb9640 1 -- 172.21.15.16:0/2475559100 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31140b2cb0 con 0x7f311415d4b0 2023-11-24T06:31:04.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.121+0000 7f3118eb9640 1 --2- 172.21.15.16:0/2475559100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311415d4b0 0x7f31140d1550 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f31040097a0 tx=0x7f310402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967401f55a8b9756 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:04.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.121+0000 7f31137fe640 1 -- 172.21.15.16:0/2475559100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104031830 con 0x7f311415d4b0 2023-11-24T06:31:04.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.121+0000 7f31137fe640 1 -- 172.21.15.16:0/2475559100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3104031990 con 0x7f311415d4b0 2023-11-24T06:31:04.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.121+0000 7f31137fe640 1 -- 172.21.15.16:0/2475559100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104031d20 con 0x7f311415d4b0 2023-11-24T06:31:04.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.122+0000 7f3119ebb640 1 -- 172.21.15.16:0/2475559100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311415d4b0 msgr2=0x7f31140d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.122+0000 7f3119ebb640 1 --2- 172.21.15.16:0/2475559100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311415d4b0 0x7f31140d1550 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f31040097a0 tx=0x7f310402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.122+0000 7f3119ebb640 1 -- 172.21.15.16:0/2475559100 shutdown_connections 2023-11-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.122+0000 7f3119ebb640 1 --2- 172.21.15.16:0/2475559100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311415d4b0 0x7f31140d1550 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.122+0000 7f3119ebb640 1 -- 172.21.15.16:0/2475559100 >> 172.21.15.16:0/2475559100 conn(0x7f31140d3000 msgr2=0x7f31140d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.122+0000 7f3119ebb640 1 -- 172.21.15.16:0/2475559100 shutdown_connections 2023-11-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.122+0000 7f3119ebb640 1 -- 172.21.15.16:0/2475559100 wait complete. 2023-11-24T06:31:04.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.124+0000 7f3119ebb640 1 Processor -- start 2023-11-24T06:31:04.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.124+0000 7f3119ebb640 1 -- start start 2023-11-24T06:31:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.125+0000 7f3119ebb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31140b41e0 0x7f31140cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.125+0000 7f3119ebb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31140cf9a0 0x7f31140b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.125+0000 7f3119ebb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f311415d4b0 0x7f31140b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.125+0000 7f3119ebb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31140cafc0 con 0x7f31140b41e0 2023-11-24T06:31:04.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.125+0000 7f3119ebb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31140ca0d0 con 0x7f311415d4b0 2023-11-24T06:31:04.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.125+0000 7f3119ebb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31140ca3b0 con 0x7f31140cf9a0 2023-11-24T06:31:04.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.125+0000 7f3118eb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31140b41e0 0x7f31140cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.125+0000 7f3118eb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31140b41e0 0x7f31140cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36376/0 (socket says 172.21.15.16:36376) 2023-11-24T06:31:04.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.125+0000 7f3118eb9640 1 -- 172.21.15.16:0/1696237153 learned_addr learned my addr 172.21.15.16:0/1696237153 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:04.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f3118eb9640 1 -- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31140cf9a0 msgr2=0x7f31140b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f3113fff640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31140cf9a0 0x7f31140b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f3118eb9640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31140cf9a0 0x7f31140b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f3118eb9640 1 -- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f311415d4b0 msgr2=0x7f31140b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f3118eb9640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f311415d4b0 0x7f31140b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f3118eb9640 1 -- 172.21.15.16:0/1696237153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3104009400 con 0x7f31140b41e0 2023-11-24T06:31:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f3113fff640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31140cf9a0 0x7f31140b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f3118eb9640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31140b41e0 0x7f31140cb450 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3104000c00 tx=0x7f31040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f31117fa640 1 -- 172.21.15.16:0/1696237153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310403aa60 con 0x7f31140b41e0 2023-11-24T06:31:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f31117fa640 1 -- 172.21.15.16:0/1696237153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310403abc0 con 0x7f31140b41e0 2023-11-24T06:31:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f31117fa640 1 -- 172.21.15.16:0/1696237153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31040443f0 con 0x7f31140b41e0 2023-11-24T06:31:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.126+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311415da10 con 0x7f31140b41e0 2023-11-24T06:31:04.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.127+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31140ca970 con 0x7f31140b41e0 2023-11-24T06:31:04.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.128+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3114067450 con 0x7f31140b41e0 2023-11-24T06:31:04.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.129+0000 7f31117fa640 1 -- 172.21.15.16:0/1696237153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f310404d450 con 0x7f31140b41e0 2023-11-24T06:31:04.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.130+0000 7f31117fa640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f30f406b3e0 0x7f30f406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.130+0000 7f31117fa640 1 -- 172.21.15.16:0/1696237153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f31040ac470 con 0x7f31140b41e0 2023-11-24T06:31:04.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.130+0000 7f3113fff640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f30f406b3e0 0x7f30f406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.132+0000 7f3113fff640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f30f406b3e0 0x7f30f406d8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f31140b78a0 tx=0x7f30fc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:04.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.134+0000 7f31117fa640 1 -- 172.21.15.16:0/1696237153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3104076ad0 con 0x7f31140b41e0 2023-11-24T06:31:04.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.328+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f311406be90 con 0x7f31140b41e0 2023-11-24T06:31:04.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.329+0000 7f31117fa640 1 -- 172.21.15.16:0/1696237153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f310407a0f0 con 0x7f31140b41e0 2023-11-24T06:31:04.331 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:31:04.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f30f406b3e0 msgr2=0x7f30f406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f30f406b3e0 0x7f30f406d8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f31140b78a0 tx=0x7f30fc009040 comp rx=0 tx=0).stop 2023-11-24T06:31:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31140b41e0 msgr2=0x7f31140cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31140b41e0 0x7f31140cb450 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3104000c00 tx=0x7f31040046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 shutdown_connections 2023-11-24T06:31:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f30f406b3e0 0x7f30f406d8a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f311415d4b0 0x7f31140b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31140cf9a0 0x7f31140b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 --2- 172.21.15.16:0/1696237153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31140b41e0 0x7f31140cb450 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 >> 172.21.15.16:0/1696237153 conn(0x7f31140d3000 msgr2=0x7f31140ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.331+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 shutdown_connections 2023-11-24T06:31:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.332+0000 7f3119ebb640 1 -- 172.21.15.16:0/1696237153 wait complete. 2023-11-24T06:31:04.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:31:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.578+0000 7f361ffff640 1 Processor -- start 2023-11-24T06:31:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.578+0000 7f361ffff640 1 -- start start 2023-11-24T06:31:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.580+0000 7f361ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 0x7f3620070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.580+0000 7f361ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f362015bf10 0x7f36200cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.580+0000 7f361ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36200cc980 0x7f3620143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.580+0000 7f361ffff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362015f970 con 0x7f36200cc980 2023-11-24T06:31:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.580+0000 7f361ffff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362015f4e0 con 0x7f362015bf10 2023-11-24T06:31:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.580+0000 7f361ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200cce10 con 0x7f36200adf20 2023-11-24T06:31:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.580+0000 7f361f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36200cc980 0x7f3620143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.580+0000 7f361f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36200cc980 0x7f3620143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36386/0 (socket says 172.21.15.16:36386) 2023-11-24T06:31:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.580+0000 7f361f7fe640 1 -- 172.21.15.16:0/240083814 learned_addr learned my addr 172.21.15.16:0/240083814 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.581+0000 7f361effd640 1 --2- 172.21.15.16:0/240083814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 0x7f3620070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.581+0000 7f361f7fe640 1 -- 172.21.15.16:0/240083814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 msgr2=0x7f3620070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.581+0000 7f36167fc640 1 --2- 172.21.15.16:0/240083814 >> v2:172.21.15.173:3300/0 conn(0x7f362015bf10 0x7f36200cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.581+0000 7f361f7fe640 1 --2- 172.21.15.16:0/240083814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 0x7f3620070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.581+0000 7f361f7fe640 1 -- 172.21.15.16:0/240083814 >> v2:172.21.15.173:3300/0 conn(0x7f362015bf10 msgr2=0x7f36200cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.581+0000 7f361f7fe640 1 --2- 172.21.15.16:0/240083814 >> v2:172.21.15.173:3300/0 conn(0x7f362015bf10 0x7f36200cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.581+0000 7f361f7fe640 1 -- 172.21.15.16:0/240083814 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620071130 con 0x7f36200cc980 2023-11-24T06:31:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.581+0000 7f36167fc640 1 --2- 172.21.15.16:0/240083814 >> v2:172.21.15.173:3300/0 conn(0x7f362015bf10 0x7f36200cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.582+0000 7f361f7fe640 1 --2- 172.21.15.16:0/240083814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 0x7f3620143d30 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f36200b4180 tx=0x7f361802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fd863fe4ffdb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.582+0000 7f361e7fc640 1 -- 172.21.15.16:0/240083814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36180317e0 con 0x7f36200cc980 2023-11-24T06:31:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361e7fc640 1 -- 172.21.15.16:0/240083814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618031940 con 0x7f36200cc980 2023-11-24T06:31:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361e7fc640 1 -- 172.21.15.16:0/240083814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618031cd0 con 0x7f36200cc980 2023-11-24T06:31:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361ffff640 1 -- 172.21.15.16:0/240083814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 msgr2=0x7f3620143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361ffff640 1 --2- 172.21.15.16:0/240083814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 0x7f3620143d30 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f36200b4180 tx=0x7f361802b880 comp rx=0 tx=0).stop 2023-11-24T06:31:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361ffff640 1 -- 172.21.15.16:0/240083814 shutdown_connections 2023-11-24T06:31:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361ffff640 1 --2- 172.21.15.16:0/240083814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 0x7f3620143d30 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361ffff640 1 --2- 172.21.15.16:0/240083814 >> v2:172.21.15.173:3300/0 conn(0x7f362015bf10 0x7f36200cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361ffff640 1 --2- 172.21.15.16:0/240083814 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 0x7f3620070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361ffff640 1 -- 172.21.15.16:0/240083814 >> 172.21.15.16:0/240083814 conn(0x7f362006a7d0 msgr2=0x7f362006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361ffff640 1 -- 172.21.15.16:0/240083814 shutdown_connections 2023-11-24T06:31:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.583+0000 7f361ffff640 1 -- 172.21.15.16:0/240083814 wait complete. 2023-11-24T06:31:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.585+0000 7f361ffff640 1 Processor -- start 2023-11-24T06:31:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.585+0000 7f361ffff640 1 -- start start 2023-11-24T06:31:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.586+0000 7f361ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 0x7f362015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.586+0000 7f361ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 0x7f36200afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.586+0000 7f361ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f362015bf10 0x7f36200b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.586+0000 7f361ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362015af50 con 0x7f36200cc980 2023-11-24T06:31:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.586+0000 7f361ffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36201f0940 con 0x7f362015bf10 2023-11-24T06:31:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.586+0000 7f361ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36201f3ec0 con 0x7f36200adf20 2023-11-24T06:31:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f36167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 0x7f36200afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f36167fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 0x7f36200afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36394/0 (socket says 172.21.15.16:36394) 2023-11-24T06:31:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f36167fc640 1 -- 172.21.15.16:0/3883537871 learned_addr learned my addr 172.21.15.16:0/3883537871 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f361effd640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 0x7f362015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f36167fc640 1 -- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 msgr2=0x7f362015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f361f7fe640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f362015bf10 0x7f36200b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f36167fc640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 0x7f362015aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f36167fc640 1 -- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f362015bf10 msgr2=0x7f36200b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f36167fc640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f362015bf10 0x7f36200b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f36167fc640 1 -- 172.21.15.16:0/3883537871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180094d0 con 0x7f36200cc980 2023-11-24T06:31:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.587+0000 7f36167fc640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 0x7f36200afe20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f360800c7f0 tx=0x7f360800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.588+0000 7f3617fff640 1 -- 172.21.15.16:0/3883537871 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608007a30 con 0x7f36200cc980 2023-11-24T06:31:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.588+0000 7f3617fff640 1 -- 172.21.15.16:0/3883537871 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3608007b90 con 0x7f36200cc980 2023-11-24T06:31:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.588+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36201f41a0 con 0x7f36200cc980 2023-11-24T06:31:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.588+0000 7f3617fff640 1 -- 172.21.15.16:0/3883537871 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800ce40 con 0x7f36200cc980 2023-11-24T06:31:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.588+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36201f4760 con 0x7f36200cc980 2023-11-24T06:31:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.589+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3620067450 con 0x7f36200cc980 2023-11-24T06:31:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.590+0000 7f3617fff640 1 -- 172.21.15.16:0/3883537871 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f36080113f0 con 0x7f36200cc980 2023-11-24T06:31:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.591+0000 7f3617fff640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f360006b3e0 0x7f360006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.591+0000 7f3617fff640 1 -- 172.21.15.16:0/3883537871 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f360808a370 con 0x7f36200cc980 2023-11-24T06:31:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.592+0000 7f361effd640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f360006b3e0 0x7f360006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.593+0000 7f361effd640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f360006b3e0 0x7f360006d8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3620070760 tx=0x7f3610003a50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.596+0000 7f3617fff640 1 -- 172.21.15.16:0/3883537871 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3608054a10 con 0x7f36200cc980 2023-11-24T06:31:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.784+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36200cbea0 con 0x7f36200cc980 2023-11-24T06:31:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.785+0000 7f3617fff640 1 -- 172.21.15.16:0/3883537871 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3608058030 con 0x7f36200cc980 2023-11-24T06:31:04.788 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:31:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f360006b3e0 msgr2=0x7f360006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f360006b3e0 0x7f360006d8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3620070760 tx=0x7f3610003a50 comp rx=0 tx=0).stop 2023-11-24T06:31:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 msgr2=0x7f36200afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 0x7f36200afe20 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f360800c7f0 tx=0x7f360800cbf0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 shutdown_connections 2023-11-24T06:31:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f360006b3e0 0x7f360006d8a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f362015bf10 0x7f36200b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36200cc980 0x7f36200afe20 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 --2- 172.21.15.16:0/3883537871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36200adf20 0x7f362015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 >> 172.21.15.16:0/3883537871 conn(0x7f362006a7d0 msgr2=0x7f36200af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 shutdown_connections 2023-11-24T06:31:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:04.788+0000 7f361ffff640 1 -- 172.21.15.16:0/3883537871 wait complete. 2023-11-24T06:31:04.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:31:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.036+0000 7f1085ad1640 1 Processor -- start 2023-11-24T06:31:05.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.037+0000 7f1085ad1640 1 -- start start 2023-11-24T06:31:05.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.038+0000 7f1085ad1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f108015de80 0x7f10800ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.038+0000 7f1085ad1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108006c730 con 0x7f1080143810 2023-11-24T06:31:05.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.038+0000 7f1085ad1640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10800724d0 con 0x7f1080071e40 2023-11-24T06:31:05.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.038+0000 7f1085ad1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1080143bf0 con 0x7f108015de80 2023-11-24T06:31:05.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.039+0000 7f10852d0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f108015de80 0x7f10800ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.039+0000 7f1084acf640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1080143810 0x7f108015e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48408/0 (socket says 172.21.15.16:48408) 2023-11-24T06:31:05.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.039+0000 7f1084acf640 1 -- 172.21.15.16:0/792058297 learned_addr learned my addr 172.21.15.16:0/792058297 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:05.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.039+0000 7f107f7fe640 1 -- 172.21.15.16:0/792058297 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523214188 0 0) 0x7f108006c730 con 0x7f1080143810 2023-11-24T06:31:05.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.039+0000 7f10852d0640 1 -- 172.21.15.16:0/792058297 >> v1:172.21.15.173:6789/0 conn(0x7f1080071e40 legacy=0x7f108015d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.039+0000 7f10852d0640 1 -- 172.21.15.16:0/792058297 >> v1:172.21.15.16:6789/0 conn(0x7f1080143810 legacy=0x7f108015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.039+0000 7f10852d0640 1 -- 172.21.15.16:0/792058297 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108006c9f0 con 0x7f108015de80 2023-11-24T06:31:05.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.041+0000 7f10852d0640 1 --2- 172.21.15.16:0/792058297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108015de80 0x7f10800ae170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1078009d90 tx=0x7f107802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0be240e636c9a2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:05.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.041+0000 7f107f7fe640 1 -- 172.21.15.16:0/792058297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107802cbd0 con 0x7f108015de80 2023-11-24T06:31:05.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.041+0000 7f107f7fe640 1 -- 172.21.15.16:0/792058297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107802cd30 con 0x7f108015de80 2023-11-24T06:31:05.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.041+0000 7f107f7fe640 1 -- 172.21.15.16:0/792058297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107802cea0 con 0x7f108015de80 2023-11-24T06:31:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.042+0000 7f1085ad1640 1 -- 172.21.15.16:0/792058297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108015de80 msgr2=0x7f10800ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.042+0000 7f1085ad1640 1 --2- 172.21.15.16:0/792058297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108015de80 0x7f10800ae170 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1078009d90 tx=0x7f107802be20 comp rx=0 tx=0).stop 2023-11-24T06:31:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.042+0000 7f1085ad1640 1 -- 172.21.15.16:0/792058297 shutdown_connections 2023-11-24T06:31:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.042+0000 7f1085ad1640 1 --2- 172.21.15.16:0/792058297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108015de80 0x7f10800ae170 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.042+0000 7f1085ad1640 1 -- 172.21.15.16:0/792058297 >> 172.21.15.16:0/792058297 conn(0x7f108006a7d0 msgr2=0x7f108006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.042+0000 7f1085ad1640 1 -- 172.21.15.16:0/792058297 shutdown_connections 2023-11-24T06:31:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.042+0000 7f1085ad1640 1 -- 172.21.15.16:0/792058297 wait complete. 2023-11-24T06:31:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.044+0000 7f1085ad1640 1 Processor -- start 2023-11-24T06:31:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.044+0000 7f1085ad1640 1 -- start start 2023-11-24T06:31:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.045+0000 7f1085ad1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1080071e40 0x7f108015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.045+0000 7f1085ad1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1080143810 0x7f10801459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.045+0000 7f1085ad1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f108015de80 0x7f10800b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.045+0000 7f1085ad1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10800722f0 con 0x7f1080071e40 2023-11-24T06:31:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.045+0000 7f1085ad1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1080145520 con 0x7f1080143810 2023-11-24T06:31:05.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.045+0000 7f1085ad1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1080144bf0 con 0x7f108015de80 2023-11-24T06:31:05.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.045+0000 7f1084acf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1080071e40 0x7f108015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.045+0000 7f1084acf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1080071e40 0x7f108015cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36422/0 (socket says 172.21.15.16:36422) 2023-11-24T06:31:05.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.045+0000 7f1084acf640 1 -- 172.21.15.16:0/934409554 learned_addr learned my addr 172.21.15.16:0/934409554 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:05.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f10852d0640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f108015de80 0x7f10800b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f107ffff640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1080143810 0x7f10801459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f1084acf640 1 -- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f108015de80 msgr2=0x7f10800b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f1084acf640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f108015de80 0x7f10800b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f1084acf640 1 -- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1080143810 msgr2=0x7f10801459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f1084acf640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1080143810 0x7f10801459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f1084acf640 1 -- 172.21.15.16:0/934409554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1078009a70 con 0x7f1080071e40 2023-11-24T06:31:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f1084acf640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1080071e40 0x7f108015cbe0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f107400c4a0 tx=0x7f107400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f107d7fa640 1 -- 172.21.15.16:0/934409554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740074e0 con 0x7f1080071e40 2023-11-24T06:31:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f107d7fa640 1 -- 172.21.15.16:0/934409554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074007640 con 0x7f1080071e40 2023-11-24T06:31:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.046+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1080144ed0 con 0x7f1080071e40 2023-11-24T06:31:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.047+0000 7f107d7fa640 1 -- 172.21.15.16:0/934409554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074004a00 con 0x7f1080071e40 2023-11-24T06:31:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.047+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1080145ef0 con 0x7f1080071e40 2023-11-24T06:31:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.048+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1080067450 con 0x7f1080071e40 2023-11-24T06:31:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.049+0000 7f107d7fa640 1 -- 172.21.15.16:0/934409554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1074004b60 con 0x7f1080071e40 2023-11-24T06:31:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.050+0000 7f107d7fa640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f106406b3e0 0x7f106406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.050+0000 7f107d7fa640 1 -- 172.21.15.16:0/934409554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f1074089680 con 0x7f1080071e40 2023-11-24T06:31:05.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.050+0000 7f107ffff640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f106406b3e0 0x7f106406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.051+0000 7f107ffff640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f106406b3e0 0x7f106406d8a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f108006ddf0 tx=0x7f106c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:05.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.054+0000 7f107d7fa640 1 -- 172.21.15.16:0/934409554 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1074053df0 con 0x7f1080071e40 2023-11-24T06:31:05.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.248+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10800cb120 con 0x7f1080071e40 2023-11-24T06:31:05.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.249+0000 7f107d7fa640 1 -- 172.21.15.16:0/934409554 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1074057410 con 0x7f1080071e40 2023-11-24T06:31:05.252 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:31:05.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f106406b3e0 msgr2=0x7f106406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f106406b3e0 0x7f106406d8a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f108006ddf0 tx=0x7f106c009040 comp rx=0 tx=0).stop 2023-11-24T06:31:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1080071e40 msgr2=0x7f108015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1080071e40 0x7f108015cbe0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f107400c4a0 tx=0x7f107400c960 comp rx=0 tx=0).stop 2023-11-24T06:31:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 shutdown_connections 2023-11-24T06:31:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f106406b3e0 0x7f106406d8a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f108015de80 0x7f10800b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1080143810 0x7f10801459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 --2- 172.21.15.16:0/934409554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1080071e40 0x7f108015cbe0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.251+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 >> 172.21.15.16:0/934409554 conn(0x7f108006a7d0 msgr2=0x7f1080160370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.252+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 shutdown_connections 2023-11-24T06:31:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.252+0000 7f1085ad1640 1 -- 172.21.15.16:0/934409554 wait complete. 2023-11-24T06:31:05.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.490+0000 7f6e18c13640 1 Processor -- start 2023-11-24T06:31:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.490+0000 7f6e18c13640 1 -- start start 2023-11-24T06:31:05.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.492+0000 7f6e18c13640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e141607a0 0x7f6e140b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.492+0000 7f6e18c13640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6e140b3eb0 0x7f6e141610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.492+0000 7f6e18c13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e14160b80 0x7f6e14161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.492+0000 7f6e18c13640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e140b2b90 con 0x7f6e14160b80 2023-11-24T06:31:05.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.492+0000 7f6e18c13640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e140b2e30 con 0x7f6e140b3eb0 2023-11-24T06:31:05.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.492+0000 7f6e18c13640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140b3110 con 0x7f6e141607a0 2023-11-24T06:31:05.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.492+0000 7f6e13fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e14160b80 0x7f6e14161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.492+0000 7f6e13fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e14160b80 0x7f6e14161600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36428/0 (socket says 172.21.15.16:36428) 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.492+0000 7f6e13fff640 1 -- 172.21.15.16:0/3219464766 learned_addr learned my addr 172.21.15.16:0/3219464766 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.493+0000 7f6e0bfff640 1 --2- 172.21.15.16:0/3219464766 >> v2:172.21.15.173:3300/0 conn(0x7f6e140b3eb0 0x7f6e141610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.493+0000 7f6e137fe640 1 --2- 172.21.15.16:0/3219464766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e141607a0 0x7f6e140b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.493+0000 7f6e13fff640 1 -- 172.21.15.16:0/3219464766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e141607a0 msgr2=0x7f6e140b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.493+0000 7f6e13fff640 1 --2- 172.21.15.16:0/3219464766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e141607a0 0x7f6e140b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.493+0000 7f6e13fff640 1 -- 172.21.15.16:0/3219464766 >> v2:172.21.15.173:3300/0 conn(0x7f6e140b3eb0 msgr2=0x7f6e141610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.493+0000 7f6e13fff640 1 --2- 172.21.15.16:0/3219464766 >> v2:172.21.15.173:3300/0 conn(0x7f6e140b3eb0 0x7f6e141610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.493+0000 7f6e13fff640 1 -- 172.21.15.16:0/3219464766 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e140b33f0 con 0x7f6e14160b80 2023-11-24T06:31:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.493+0000 7f6e0bfff640 1 --2- 172.21.15.16:0/3219464766 >> v2:172.21.15.173:3300/0 conn(0x7f6e140b3eb0 0x7f6e141610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.493+0000 7f6e137fe640 1 --2- 172.21.15.16:0/3219464766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e141607a0 0x7f6e140b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:05.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.494+0000 7f6e13fff640 1 --2- 172.21.15.16:0/3219464766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 0x7f6e14161600 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6e1408bbe0 tx=0x7f6e0c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd4053933f702e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:05.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.494+0000 7f6e12ffd640 1 -- 172.21.15.16:0/3219464766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c02ca80 con 0x7f6e14160b80 2023-11-24T06:31:05.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.494+0000 7f6e12ffd640 1 -- 172.21.15.16:0/3219464766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c02cbe0 con 0x7f6e14160b80 2023-11-24T06:31:05.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.494+0000 7f6e12ffd640 1 -- 172.21.15.16:0/3219464766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c0343f0 con 0x7f6e14160b80 2023-11-24T06:31:05.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.495+0000 7f6e18c13640 1 -- 172.21.15.16:0/3219464766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 msgr2=0x7f6e14161600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.495+0000 7f6e18c13640 1 --2- 172.21.15.16:0/3219464766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 0x7f6e14161600 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6e1408bbe0 tx=0x7f6e0c02bee0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.495+0000 7f6e18c13640 1 -- 172.21.15.16:0/3219464766 shutdown_connections 2023-11-24T06:31:05.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.495+0000 7f6e18c13640 1 --2- 172.21.15.16:0/3219464766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 0x7f6e14161600 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.495+0000 7f6e18c13640 1 --2- 172.21.15.16:0/3219464766 >> v2:172.21.15.173:3300/0 conn(0x7f6e140b3eb0 0x7f6e141610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.495+0000 7f6e18c13640 1 --2- 172.21.15.16:0/3219464766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e141607a0 0x7f6e140b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.495+0000 7f6e18c13640 1 -- 172.21.15.16:0/3219464766 >> 172.21.15.16:0/3219464766 conn(0x7f6e1409c590 msgr2=0x7f6e140d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.495+0000 7f6e18c13640 1 -- 172.21.15.16:0/3219464766 shutdown_connections 2023-11-24T06:31:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.495+0000 7f6e18c13640 1 -- 172.21.15.16:0/3219464766 wait complete. 2023-11-24T06:31:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.497+0000 7f6e18c13640 1 Processor -- start 2023-11-24T06:31:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.497+0000 7f6e18c13640 1 -- start start 2023-11-24T06:31:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.498+0000 7f6e18c13640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e140b3eb0 0x7f6e14149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.498+0000 7f6e18c13640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e141607a0 0x7f6e140d3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.498+0000 7f6e18c13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 0x7f6e140d3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.498+0000 7f6e18c13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1414d540 con 0x7f6e14160b80 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.498+0000 7f6e18c13640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e141f64b0 con 0x7f6e141607a0 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.498+0000 7f6e18c13640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e141f6790 con 0x7f6e140b3eb0 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.498+0000 7f6e13fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 0x7f6e140d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.498+0000 7f6e13fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 0x7f6e140d3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36438/0 (socket says 172.21.15.16:36438) 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.498+0000 7f6e13fff640 1 -- 172.21.15.16:0/2659532125 learned_addr learned my addr 172.21.15.16:0/2659532125 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.499+0000 7f6e137fe640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e140b3eb0 0x7f6e14149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.499+0000 7f6e0bfff640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e141607a0 0x7f6e140d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.499+0000 7f6e13fff640 1 -- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e140b3eb0 msgr2=0x7f6e14149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.499+0000 7f6e13fff640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e140b3eb0 0x7f6e14149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.499+0000 7f6e13fff640 1 -- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e141607a0 msgr2=0x7f6e140d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.499+0000 7f6e13fff640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e141607a0 0x7f6e140d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.499+0000 7f6e13fff640 1 -- 172.21.15.16:0/2659532125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c009b30 con 0x7f6e14160b80 2023-11-24T06:31:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.499+0000 7f6e13fff640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 0x7f6e140d3750 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c036040 tx=0x7f6e0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.500+0000 7f6e10ff9640 1 -- 172.21.15.16:0/2659532125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c03e450 con 0x7f6e14160b80 2023-11-24T06:31:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.500+0000 7f6e10ff9640 1 -- 172.21.15.16:0/2659532125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c03e5b0 con 0x7f6e14160b80 2023-11-24T06:31:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.500+0000 7f6e10ff9640 1 -- 172.21.15.16:0/2659532125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c03e960 con 0x7f6e14160b80 2023-11-24T06:31:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.500+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e141f6a70 con 0x7f6e14160b80 2023-11-24T06:31:05.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.500+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1414a2b0 con 0x7f6e14160b80 2023-11-24T06:31:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.501+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e141f6f30 con 0x7f6e14160b80 2023-11-24T06:31:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.502+0000 7f6e10ff9640 1 -- 172.21.15.16:0/2659532125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6e0c02cd50 con 0x7f6e14160b80 2023-11-24T06:31:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.503+0000 7f6e10ff9640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6df406b430 0x7f6df406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.503+0000 7f6e10ff9640 1 -- 172.21.15.16:0/2659532125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f6e0c0accd0 con 0x7f6e14160b80 2023-11-24T06:31:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.504+0000 7f6e137fe640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6df406b430 0x7f6df406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.506+0000 7f6e137fe640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6df406b430 0x7f6df406d8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400a910 tx=0x7f6e04008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.508+0000 7f6e10ff9640 1 -- 172.21.15.16:0/2659532125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e0c0774c0 con 0x7f6e14160b80 2023-11-24T06:31:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.748+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e14067450 con 0x7f6e14160b80 2023-11-24T06:31:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.749+0000 7f6e10ff9640 1 -- 172.21.15.16:0/2659532125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f6e0c043020 con 0x7f6e14160b80 2023-11-24T06:31:05.752 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:05.752 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1164611025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1164611025},{"type":"v1","addr":"172.21.15.16:6834","nonce":1164611025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5650,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1157106761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1157106761},{"type":"v1","addr":"172.21.15.173:6836","nonce":1157106761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/151946729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":151946729},{"type":"v1","addr":"172.21.15.173:6837","nonce":151946729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:30:33.707795+0000","modified":"2023-11-24T06:30:36.770163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6693":{"gid":6693,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2360619406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2360619406},{"type":"v1","addr":"172.21.15.16:6835","nonce":2360619406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:31:05.753 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 45 2023-11-24T06:31:05.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6df406b430 msgr2=0x7f6df406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6df406b430 0x7f6df406d8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400a910 tx=0x7f6e04008040 comp rx=0 tx=0).stop 2023-11-24T06:31:05.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 msgr2=0x7f6e140d3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 0x7f6e140d3750 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c036040 tx=0x7f6e0c004a40 comp rx=0 tx=0).stop 2023-11-24T06:31:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 shutdown_connections 2023-11-24T06:31:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6df406b430 0x7f6df406d8f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e14160b80 0x7f6e140d3750 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6e141607a0 0x7f6e140d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 --2- 172.21.15.16:0/2659532125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6e140b3eb0 0x7f6e14149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 >> 172.21.15.16:0/2659532125 conn(0x7f6e1409c590 msgr2=0x7f6e1415fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 shutdown_connections 2023-11-24T06:31:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.752+0000 7f6e18c13640 1 -- 172.21.15.16:0/2659532125 wait complete. 2023-11-24T06:31:05.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.992+0000 7f33c9639640 1 Processor -- start 2023-11-24T06:31:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.992+0000 7f33c9639640 1 -- start start 2023-11-24T06:31:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.994+0000 7f33c9639640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33c4160920 0x7f33c40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.994+0000 7f33c9639640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f33c40cfea0 0x7f33c4161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.994+0000 7f33c9639640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c40d0a10 con 0x7f33c40cf9f0 2023-11-24T06:31:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.994+0000 7f33c9639640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c40d1f50 con 0x7f33c4160920 2023-11-24T06:31:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.994+0000 7f33c9639640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c40cf080 con 0x7f33c40cfea0 2023-11-24T06:31:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.994+0000 7f33c3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33c4160920 0x7f33c40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.994+0000 7f33c3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33c4160920 0x7f33c40b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36448/0 (socket says 172.21.15.16:36448) 2023-11-24T06:31:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.994+0000 7f33c3fff640 1 -- 172.21.15.16:0/3966991675 learned_addr learned my addr 172.21.15.16:0/3966991675 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.995+0000 7f33c8e38640 1 --2- 172.21.15.16:0/3966991675 >> v2:172.21.15.173:3300/0 conn(0x7f33c40cfea0 0x7f33c4161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.995+0000 7f33c3fff640 1 -- 172.21.15.16:0/3966991675 >> v2:172.21.15.173:3300/0 conn(0x7f33c40cfea0 msgr2=0x7f33c4161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.995+0000 7f33c3fff640 1 --2- 172.21.15.16:0/3966991675 >> v2:172.21.15.173:3300/0 conn(0x7f33c40cfea0 0x7f33c4161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.995+0000 7f33c3fff640 1 -- 172.21.15.16:0/3966991675 >> v1:172.21.15.173:6789/0 conn(0x7f33c40cf9f0 legacy=0x7f33c4161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.995+0000 7f33c3fff640 1 -- 172.21.15.16:0/3966991675 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c40d0310 con 0x7f33c4160920 2023-11-24T06:31:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.996+0000 7f33c3fff640 1 --2- 172.21.15.16:0/3966991675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c4160920 0x7f33c40b3f30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f33c408c2c0 tx=0x7f33b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e453b55d5604c039 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.996+0000 7f33c2ffd640 1 -- 172.21.15.16:0/3966991675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b402ca80 con 0x7f33c4160920 2023-11-24T06:31:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.996+0000 7f33c2ffd640 1 -- 172.21.15.16:0/3966991675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b402cbe0 con 0x7f33c4160920 2023-11-24T06:31:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.996+0000 7f33c2ffd640 1 -- 172.21.15.16:0/3966991675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b40343f0 con 0x7f33c4160920 2023-11-24T06:31:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.997+0000 7f33c9639640 1 -- 172.21.15.16:0/3966991675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c4160920 msgr2=0x7f33c40b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.997+0000 7f33c9639640 1 --2- 172.21.15.16:0/3966991675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c4160920 0x7f33c40b3f30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f33c408c2c0 tx=0x7f33b402bee0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.997+0000 7f33c9639640 1 -- 172.21.15.16:0/3966991675 shutdown_connections 2023-11-24T06:31:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.997+0000 7f33c9639640 1 --2- 172.21.15.16:0/3966991675 >> v2:172.21.15.173:3300/0 conn(0x7f33c40cfea0 0x7f33c4161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.997+0000 7f33c9639640 1 --2- 172.21.15.16:0/3966991675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c4160920 0x7f33c40b3f30 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.997+0000 7f33c9639640 1 -- 172.21.15.16:0/3966991675 >> 172.21.15.16:0/3966991675 conn(0x7f33c40d3090 msgr2=0x7f33c40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:06.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.997+0000 7f33c9639640 1 -- 172.21.15.16:0/3966991675 shutdown_connections 2023-11-24T06:31:06.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.997+0000 7f33c9639640 1 -- 172.21.15.16:0/3966991675 wait complete. 2023-11-24T06:31:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.999+0000 7f33c9639640 1 Processor -- start 2023-11-24T06:31:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:05.999+0000 7f33c9639640 1 -- start start 2023-11-24T06:31:06.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.000+0000 7f33c9639640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c40cf9f0 0x7f33c4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.000+0000 7f33c9639640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33c40cfea0 0x7f33c41510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.000+0000 7f33c9639640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33c4160920 0x7f33c4151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.000+0000 7f33c9639640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4151b60 con 0x7f33c40cf9f0 2023-11-24T06:31:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.000+0000 7f33c9639640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40ca160 con 0x7f33c4160920 2023-11-24T06:31:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.000+0000 7f33c9639640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4152240 con 0x7f33c40cfea0 2023-11-24T06:31:06.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c40cf9f0 0x7f33c4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c40cf9f0 0x7f33c4158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36456/0 (socket says 172.21.15.16:36456) 2023-11-24T06:31:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c3fff640 1 -- 172.21.15.16:0/1001959118 learned_addr learned my addr 172.21.15.16:0/1001959118 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c37fe640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33c40cfea0 0x7f33c41510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c8e38640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33c4160920 0x7f33c4151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c3fff640 1 -- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33c40cfea0 msgr2=0x7f33c41510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c3fff640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33c40cfea0 0x7f33c41510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c3fff640 1 -- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33c4160920 msgr2=0x7f33c4151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c3fff640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33c4160920 0x7f33c4151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.001+0000 7f33c3fff640 1 -- 172.21.15.16:0/1001959118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4009b30 con 0x7f33c40cf9f0 2023-11-24T06:31:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.002+0000 7f33c3fff640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c40cf9f0 0x7f33c4158e80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f33b4000f80 tx=0x7f33b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.002+0000 7f33c0ff9640 1 -- 172.21.15.16:0/1001959118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4009510 con 0x7f33c40cf9f0 2023-11-24T06:31:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.002+0000 7f33c0ff9640 1 -- 172.21.15.16:0/1001959118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b403e450 con 0x7f33c40cf9f0 2023-11-24T06:31:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.002+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c40b8b50 con 0x7f33c40cf9f0 2023-11-24T06:31:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.002+0000 7f33c0ff9640 1 -- 172.21.15.16:0/1001959118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b403e800 con 0x7f33c40cf9f0 2023-11-24T06:31:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.002+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c40b8360 con 0x7f33c40cf9f0 2023-11-24T06:31:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.003+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c40b5ac0 con 0x7f33c40cf9f0 2023-11-24T06:31:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.004+0000 7f33c0ff9640 1 -- 172.21.15.16:0/1001959118 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f33b402cd50 con 0x7f33c40cf9f0 2023-11-24T06:31:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.005+0000 7f33c0ff9640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f33b006b3e0 0x7f33b006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.005+0000 7f33c0ff9640 1 -- 172.21.15.16:0/1001959118 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f33b40acc60 con 0x7f33c40cf9f0 2023-11-24T06:31:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.006+0000 7f33c37fe640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f33b006b3e0 0x7f33b006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.007+0000 7f33c37fe640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f33b006b3e0 0x7f33b006d8a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006fd0 tx=0x7f33b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:06.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.010+0000 7f33c0ff9640 1 -- 172.21.15.16:0/1001959118 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33b40773d0 con 0x7f33c40cf9f0 2023-11-24T06:31:06.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.207+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33c40b8990 con 0x7f33c40cf9f0 2023-11-24T06:31:06.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.210+0000 7f33c0ff9640 1 -- 172.21.15.16:0/1001959118 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16824 (secure 0 0 0) 0x7f33b407a9f0 con 0x7f33c40cf9f0 2023-11-24T06:31:06.213 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:06.213 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:59.119430+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:30:31.631710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:30:32.732622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f33b006b3e0 msgr2=0x7f33b006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f33b006b3e0 0x7f33b006d8a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006fd0 tx=0x7f33b8008040 comp rx=0 tx=0).stop 2023-11-24T06:31:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c40cf9f0 msgr2=0x7f33c4158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c40cf9f0 0x7f33c4158e80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f33b4000f80 tx=0x7f33b4004a40 comp rx=0 tx=0).stop 2023-11-24T06:31:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 shutdown_connections 2023-11-24T06:31:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f33b006b3e0 0x7f33b006d8a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33c4160920 0x7f33c4151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33c40cfea0 0x7f33c41510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 --2- 172.21.15.16:0/1001959118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33c40cf9f0 0x7f33c4158e80 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 >> 172.21.15.16:0/1001959118 conn(0x7f33c40d3090 msgr2=0x7f33c40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.212+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 shutdown_connections 2023-11-24T06:31:06.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.213+0000 7f33c9639640 1 -- 172.21.15.16:0/1001959118 wait complete. 2023-11-24T06:31:06.234 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:31:06.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.447+0000 7f244d0d1640 1 Processor -- start 2023-11-24T06:31:06.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.448+0000 7f244d0d1640 1 -- start start 2023-11-24T06:31:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.449+0000 7f244d0d1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24480cc990 0x7f244806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.449+0000 7f244d0d1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2448070e50 con 0x7f24480cc500 2023-11-24T06:31:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.449+0000 7f244d0d1640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2448144020 con 0x7f2448143c40 2023-11-24T06:31:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.450+0000 7f244d0d1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24480723c0 con 0x7f24480cc990 2023-11-24T06:31:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.450+0000 7f243ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24480cc500 0x7f244806d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48410/0 (socket says 172.21.15.16:48410) 2023-11-24T06:31:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.450+0000 7f243ffff640 1 -- 172.21.15.16:0/719570965 learned_addr learned my addr 172.21.15.16:0/719570965 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.450+0000 7f244c8d0640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24480cc990 0x7f244806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.450+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360556858 0 0) 0x7f2448070e50 con 0x7f24480cc500 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.450+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2434003520 con 0x7f24480cc500 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4009947476 0 0) 0x7f2434003520 con 0x7f24480cc500 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24480cc990 msgr2=0x7f244806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24480cc990 0x7f244806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 >> v1:172.21.15.173:6789/0 conn(0x7f2448143c40 legacy=0x7f2448071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448071e00 con 0x7f24480cc500 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f2430001690 con 0x7f24480cc500 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 >> v1:172.21.15.16:6789/0 conn(0x7f24480cc500 legacy=0x7f244806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24340039a0 0x7f2434003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24340047c0 0x7f2434004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24480cc990 0x7f24340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480723c0 con 0x7f24340047c0 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243400bae0 con 0x7f24340039a0 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243400bca0 con 0x7f24480cc990 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.451+0000 7f243ffff640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24340047c0 0x7f2434004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.452+0000 7f243ffff640 1 -- 172.21.15.16:0/719570965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24480cc990 msgr2=0x7f24340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.452+0000 7f243ffff640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24480cc990 0x7f24340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.452+0000 7f243ffff640 1 -- 172.21.15.16:0/719570965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24340039a0 msgr2=0x7f2434003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.452+0000 7f243ffff640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24340039a0 0x7f2434003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.452+0000 7f243ffff640 1 -- 172.21.15.16:0/719570965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434003520 con 0x7f24340047c0 2023-11-24T06:31:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.453+0000 7f243ffff640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24340047c0 0x7f2434004c40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f24480cbd00 tx=0x7f24340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc67bb40cad1d65c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.453+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430005ad0 con 0x7f24340047c0 2023-11-24T06:31:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.453+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430005c30 con 0x7f24340047c0 2023-11-24T06:31:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.453+0000 7f24477fe640 1 -- 172.21.15.16:0/719570965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243002cab0 con 0x7f24340047c0 2023-11-24T06:31:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.454+0000 7f244d0d1640 1 -- 172.21.15.16:0/719570965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24340047c0 msgr2=0x7f2434004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.454+0000 7f244d0d1640 1 --2- 172.21.15.16:0/719570965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24340047c0 0x7f2434004c40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f24480cbd00 tx=0x7f24340057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.454+0000 7f244c8d0640 1 -- 172.21.15.16:0/719570965 reap_dead start 2023-11-24T06:31:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.454+0000 7f244d0d1640 1 -- 172.21.15.16:0/719570965 shutdown_connections 2023-11-24T06:31:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.454+0000 7f244d0d1640 1 -- 172.21.15.16:0/719570965 >> 172.21.15.16:0/719570965 conn(0x7f244806a800 msgr2=0x7f244806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.454+0000 7f244d0d1640 1 -- 172.21.15.16:0/719570965 shutdown_connections 2023-11-24T06:31:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.454+0000 7f244d0d1640 1 -- 172.21.15.16:0/719570965 wait complete. 2023-11-24T06:31:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.456+0000 7f244d0d1640 1 Processor -- start 2023-11-24T06:31:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.456+0000 7f244d0d1640 1 -- start start 2023-11-24T06:31:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f244d0d1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24340047c0 0x7f2448158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f244d0d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24481f3980 0x7f24480abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f244d0d1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24481f3e00 0x7f24480ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f244d0d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24481f28e0 con 0x7f24481f3980 2023-11-24T06:31:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f244d0d1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24481f2bc0 con 0x7f24481f3e00 2023-11-24T06:31:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f244d0d1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24481f2ea0 con 0x7f24340047c0 2023-11-24T06:31:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f243ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24481f3980 0x7f24480abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f243ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24481f3980 0x7f24480abb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36474/0 (socket says 172.21.15.16:36474) 2023-11-24T06:31:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f243ffff640 1 -- 172.21.15.16:0/729726409 learned_addr learned my addr 172.21.15.16:0/729726409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:06.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.457+0000 7f2447fff640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24340047c0 0x7f2448158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f244c8d0640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24481f3e00 0x7f24480ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f243ffff640 1 -- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24340047c0 msgr2=0x7f2448158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f243ffff640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24340047c0 0x7f2448158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f243ffff640 1 -- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24481f3e00 msgr2=0x7f24480ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f243ffff640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24481f3e00 0x7f24480ac0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f243ffff640 1 -- 172.21.15.16:0/729726409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24481f3180 con 0x7f24481f3980 2023-11-24T06:31:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f2447fff640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24340047c0 0x7f2448158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:06.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f243ffff640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24481f3980 0x7f24480abb90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2430001950 tx=0x7f2430001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f24457fa640 1 -- 172.21.15.16:0/729726409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430044070 con 0x7f24481f3980 2023-11-24T06:31:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f24457fa640 1 -- 172.21.15.16:0/729726409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243002caa0 con 0x7f24481f3980 2023-11-24T06:31:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f24457fa640 1 -- 172.21.15.16:0/729726409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243002cc50 con 0x7f24481f3980 2023-11-24T06:31:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24481f3460 con 0x7f24481f3980 2023-11-24T06:31:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.458+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24481587c0 con 0x7f24481f3980 2023-11-24T06:31:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.459+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2448067450 con 0x7f24481f3980 2023-11-24T06:31:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.460+0000 7f24457fa640 1 -- 172.21.15.16:0/729726409 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2430032070 con 0x7f24481f3980 2023-11-24T06:31:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.461+0000 7f24457fa640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f242806b3e0 0x7f242806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.461+0000 7f24457fa640 1 -- 172.21.15.16:0/729726409 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f24300ab6f0 con 0x7f24481f3980 2023-11-24T06:31:06.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.462+0000 7f2447fff640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f242806b3e0 0x7f242806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.463+0000 7f2447fff640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f242806b3e0 0x7f242806d8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2438001710 tx=0x7f2438009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:06.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.465+0000 7f24457fa640 1 -- 172.21.15.16:0/729726409 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2430075fb0 con 0x7f24481f3980 2023-11-24T06:31:06.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.694+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2448067450 con 0x7f24481f3980 2023-11-24T06:31:06.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.696+0000 7f24457fa640 1 -- 172.21.15.16:0/729726409 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f243002cdb0 con 0x7f24481f3980 2023-11-24T06:31:06.699 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:06.700 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1164611025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1164611025},{"type":"v1","addr":"172.21.15.16:6834","nonce":1164611025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5650,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1157106761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1157106761},{"type":"v1","addr":"172.21.15.173:6836","nonce":1157106761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/151946729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":151946729},{"type":"v1","addr":"172.21.15.173:6837","nonce":151946729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:30:33.707795+0000","modified":"2023-11-24T06:30:36.770163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6693":{"gid":6693,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2360619406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2360619406},{"type":"v1","addr":"172.21.15.16:6835","nonce":2360619406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:31:06.700 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 45 2023-11-24T06:31:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f242806b3e0 msgr2=0x7f242806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f242806b3e0 0x7f242806d8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2438001710 tx=0x7f2438009040 comp rx=0 tx=0).stop 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24481f3980 msgr2=0x7f24480abb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24481f3980 0x7f24480abb90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2430001950 tx=0x7f2430001a20 comp rx=0 tx=0).stop 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 shutdown_connections 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f242806b3e0 0x7f242806d8a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24481f3e00 0x7f24480ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24481f3980 0x7f24480abb90 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 --2- 172.21.15.16:0/729726409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24340047c0 0x7f2448158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.698+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 >> 172.21.15.16:0/729726409 conn(0x7f244806a800 msgr2=0x7f24480b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.699+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 shutdown_connections 2023-11-24T06:31:06.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.699+0000 7f244d0d1640 1 -- 172.21.15.16:0/729726409 wait complete. 2023-11-24T06:31:06.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.935+0000 7feaa8e52640 1 Processor -- start 2023-11-24T06:31:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.935+0000 7feaa8e52640 1 -- start start 2023-11-24T06:31:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.937+0000 7feaa8e52640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7feaa413ff20 0x7feaa415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.937+0000 7feaa8e52640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa406aaf0 0x7feaa4141c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.937+0000 7feaa8e52640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa40b5440 con 0x7feaa4140300 2023-11-24T06:31:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.937+0000 7feaa8e52640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa4161c80 con 0x7feaa413ff20 2023-11-24T06:31:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.937+0000 7feaa8e52640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa415e6f0 con 0x7feaa406aaf0 2023-11-24T06:31:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.938+0000 7fea9bfff640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7feaa4140300 0x7feaa406a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:34696/0 (socket says 172.21.15.16:34696) 2023-11-24T06:31:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.938+0000 7feaa3fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa406aaf0 0x7feaa4141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.938+0000 7fea9bfff640 1 -- 172.21.15.16:0/276168642 learned_addr learned my addr 172.21.15.16:0/276168642 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.938+0000 7feaa37fe640 1 --2- 172.21.15.16:0/276168642 >> v2:172.21.15.173:3300/0 conn(0x7feaa413ff20 0x7feaa415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.939+0000 7feaa2ffd640 1 -- 172.21.15.16:0/276168642 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013678219 0 0) 0x7feaa40b5440 con 0x7feaa4140300 2023-11-24T06:31:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.939+0000 7feaa2ffd640 1 -- 172.21.15.16:0/276168642 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea90003520 con 0x7feaa4140300 2023-11-24T06:31:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.939+0000 7feaa3fff640 1 -- 172.21.15.16:0/276168642 >> v2:172.21.15.173:3300/0 conn(0x7feaa413ff20 msgr2=0x7feaa415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.939+0000 7feaa3fff640 1 --2- 172.21.15.16:0/276168642 >> v2:172.21.15.173:3300/0 conn(0x7feaa413ff20 0x7feaa415e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.939+0000 7feaa3fff640 1 -- 172.21.15.16:0/276168642 >> v1:172.21.15.173:6789/0 conn(0x7feaa4140300 legacy=0x7feaa406a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.939+0000 7feaa3fff640 1 -- 172.21.15.16:0/276168642 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa40b4830 con 0x7feaa406aaf0 2023-11-24T06:31:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.940+0000 7feaa3fff640 1 --2- 172.21.15.16:0/276168642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa406aaf0 0x7feaa4141c50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009dc0 tx=0x7fea9c02bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b67230572d4480 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.941+0000 7feaa2ffd640 1 -- 172.21.15.16:0/276168642 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c02cba0 con 0x7feaa406aaf0 2023-11-24T06:31:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.941+0000 7feaa2ffd640 1 -- 172.21.15.16:0/276168642 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9c02cd00 con 0x7feaa406aaf0 2023-11-24T06:31:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.941+0000 7feaa2ffd640 1 -- 172.21.15.16:0/276168642 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9c034660 con 0x7feaa406aaf0 2023-11-24T06:31:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.942+0000 7feaa8e52640 1 -- 172.21.15.16:0/276168642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa406aaf0 msgr2=0x7feaa4141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.942+0000 7feaa8e52640 1 --2- 172.21.15.16:0/276168642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa406aaf0 0x7feaa4141c50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009dc0 tx=0x7fea9c02bdf0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.942+0000 7feaa8e52640 1 -- 172.21.15.16:0/276168642 shutdown_connections 2023-11-24T06:31:06.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.942+0000 7feaa8e52640 1 --2- 172.21.15.16:0/276168642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa406aaf0 0x7feaa4141c50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.942+0000 7feaa8e52640 1 --2- 172.21.15.16:0/276168642 >> v2:172.21.15.173:3300/0 conn(0x7feaa413ff20 0x7feaa415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.942+0000 7feaa8e52640 1 -- 172.21.15.16:0/276168642 >> 172.21.15.16:0/276168642 conn(0x7feaa40b1cd0 msgr2=0x7feaa40b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.942+0000 7feaa8e52640 1 -- 172.21.15.16:0/276168642 shutdown_connections 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.942+0000 7feaa8e52640 1 -- 172.21.15.16:0/276168642 wait complete. 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.944+0000 7feaa8e52640 1 Processor -- start 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.944+0000 7feaa8e52640 1 -- start start 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7feaa8e52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa406aaf0 0x7feaa4146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7feaa8e52640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa413ff20 0x7feaa4149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7feaa8e52640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa4140300 0x7feaa40af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7feaa8e52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa40af750 con 0x7feaa406aaf0 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7feaa8e52640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa4157bb0 con 0x7feaa413ff20 2023-11-24T06:31:06.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7feaa8e52640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa4159b80 con 0x7feaa4140300 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7feaa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa406aaf0 0x7feaa4146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7feaa37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa406aaf0 0x7feaa4146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36482/0 (socket says 172.21.15.16:36482) 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7feaa37fe640 1 -- 172.21.15.16:0/2067961185 learned_addr learned my addr 172.21.15.16:0/2067961185 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.945+0000 7fea9bfff640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa413ff20 0x7feaa4149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa3fff640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa4140300 0x7feaa40af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa37fe640 1 -- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa4140300 msgr2=0x7feaa40af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa37fe640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa4140300 0x7feaa40af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa37fe640 1 -- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa413ff20 msgr2=0x7feaa4149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa37fe640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa413ff20 0x7feaa4149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa37fe640 1 -- 172.21.15.16:0/2067961185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c009a70 con 0x7feaa406aaf0 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa37fe640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa406aaf0 0x7feaa4146e90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fea9400e9e0 tx=0x7fea9400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa0ff9640 1 -- 172.21.15.16:0/2067961185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94009670 con 0x7feaa406aaf0 2023-11-24T06:31:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa0ff9640 1 -- 172.21.15.16:0/2067961185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9400c9d0 con 0x7feaa406aaf0 2023-11-24T06:31:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa0ff9640 1 -- 172.21.15.16:0/2067961185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9400cd80 con 0x7feaa406aaf0 2023-11-24T06:31:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.946+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa4159850 con 0x7feaa406aaf0 2023-11-24T06:31:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.947+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa4157e00 con 0x7feaa406aaf0 2023-11-24T06:31:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.948+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa4067450 con 0x7feaa406aaf0 2023-11-24T06:31:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.949+0000 7feaa0ff9640 1 -- 172.21.15.16:0/2067961185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fea94013470 con 0x7feaa406aaf0 2023-11-24T06:31:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.950+0000 7feaa0ff9640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea8406b410 0x7fea8406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.950+0000 7feaa0ff9640 1 -- 172.21.15.16:0/2067961185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fea9408a260 con 0x7feaa406aaf0 2023-11-24T06:31:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.950+0000 7fea9bfff640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea8406b410 0x7fea8406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.951+0000 7fea9bfff640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea8406b410 0x7fea8406d8d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feaa41f6550 tx=0x7fea8c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:06.954+0000 7feaa0ff9640 1 -- 172.21.15.16:0/2067961185 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea94054a60 con 0x7feaa406aaf0 2023-11-24T06:31:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.183+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feaa413ebe0 con 0x7feaa406aaf0 2023-11-24T06:31:07.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.186+0000 7feaa0ff9640 1 -- 172.21.15.16:0/2067961185 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fea94058080 con 0x7feaa406aaf0 2023-11-24T06:31:07.190 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:07.190 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1164611025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1164611025},{"type":"v1","addr":"172.21.15.16:6834","nonce":1164611025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5650,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1157106761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1157106761},{"type":"v1","addr":"172.21.15.173:6836","nonce":1157106761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/151946729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":151946729},{"type":"v1","addr":"172.21.15.173:6837","nonce":151946729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:30:33.707795+0000","modified":"2023-11-24T06:30:36.770163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6693},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6693":{"gid":6693,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2360619406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2360619406},{"type":"v1","addr":"172.21.15.16:6835","nonce":2360619406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:31:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 45 2023-11-24T06:31:07.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea8406b410 msgr2=0x7fea8406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea8406b410 0x7fea8406d8d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feaa41f6550 tx=0x7fea8c009040 comp rx=0 tx=0).stop 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa406aaf0 msgr2=0x7feaa4146e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa406aaf0 0x7feaa4146e90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fea9400e9e0 tx=0x7fea9400eea0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 shutdown_connections 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fea8406b410 0x7fea8406d8d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feaa4140300 0x7feaa40af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feaa413ff20 0x7feaa4149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 --2- 172.21.15.16:0/2067961185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa406aaf0 0x7feaa4146e90 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 >> 172.21.15.16:0/2067961185 conn(0x7feaa40b1cd0 msgr2=0x7feaa406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 shutdown_connections 2023-11-24T06:31:07.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.189+0000 7feaa8e52640 1 -- 172.21.15.16:0/2067961185 wait complete. 2023-11-24T06:31:07.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.417+0000 7f12d99db640 1 Processor -- start 2023-11-24T06:31:07.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.417+0000 7f12d99db640 1 -- start start 2023-11-24T06:31:07.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.419+0000 7f12d99db640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f12d413fcb0 0x7f12d4164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.419+0000 7f12d99db640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d40cdc40 con 0x7f12d413cfa0 2023-11-24T06:31:07.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.419+0000 7f12d99db640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d40cdec0 con 0x7f12d415afb0 2023-11-24T06:31:07.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.419+0000 7f12d99db640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d40ce1a0 con 0x7f12d413fcb0 2023-11-24T06:31:07.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.419+0000 7f12d89d9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f12d413cfa0 0x7f12d40cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48412/0 (socket says 172.21.15.16:48412) 2023-11-24T06:31:07.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.419+0000 7f12d89d9640 1 -- 172.21.15.16:0/2535521093 learned_addr learned my addr 172.21.15.16:0/2535521093 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:07.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.420+0000 7f12d91da640 1 --2- 172.21.15.16:0/2535521093 >> v2:172.21.15.173:3300/0 conn(0x7f12d413fcb0 0x7f12d4164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.420+0000 7f12d37fe640 1 -- 172.21.15.16:0/2535521093 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430483188 0 0) 0x7f12d40cdc40 con 0x7f12d413cfa0 2023-11-24T06:31:07.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.420+0000 7f12d37fe640 1 -- 172.21.15.16:0/2535521093 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c4003520 con 0x7f12d413cfa0 2023-11-24T06:31:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.420+0000 7f12d37fe640 1 -- 172.21.15.16:0/2535521093 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687113256 0 0) 0x7f12d40cdec0 con 0x7f12d415afb0 2023-11-24T06:31:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.420+0000 7f12d37fe640 1 -- 172.21.15.16:0/2535521093 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d40cdc40 con 0x7f12d415afb0 2023-11-24T06:31:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.420+0000 7f12d37fe640 1 -- 172.21.15.16:0/2535521093 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 813344130 0 0) 0x7f12c4003520 con 0x7f12d413cfa0 2023-11-24T06:31:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.420+0000 7f12d91da640 1 -- 172.21.15.16:0/2535521093 >> v1:172.21.15.173:6789/0 conn(0x7f12d415afb0 legacy=0x7f12d413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.420+0000 7f12d91da640 1 -- 172.21.15.16:0/2535521093 >> v1:172.21.15.16:6789/0 conn(0x7f12d413cfa0 legacy=0x7f12d40cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.421+0000 7f12d91da640 1 -- 172.21.15.16:0/2535521093 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40ce480 con 0x7f12d413fcb0 2023-11-24T06:31:07.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.423+0000 7f12d91da640 1 --2- 172.21.15.16:0/2535521093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12d413fcb0 0x7f12d4164d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009d90 tx=0x7f12cc02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6b7e0d309365251 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.423+0000 7f12d37fe640 1 -- 172.21.15.16:0/2535521093 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80015f0 con 0x7f12d413fcb0 2023-11-24T06:31:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.423+0000 7f12d37fe640 1 -- 172.21.15.16:0/2535521093 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc02c8b0 con 0x7f12d413fcb0 2023-11-24T06:31:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.423+0000 7f12d37fe640 1 -- 172.21.15.16:0/2535521093 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc02cc40 con 0x7f12d413fcb0 2023-11-24T06:31:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.424+0000 7f12d99db640 1 -- 172.21.15.16:0/2535521093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12d413fcb0 msgr2=0x7f12d4164d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.424+0000 7f12d99db640 1 --2- 172.21.15.16:0/2535521093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12d413fcb0 0x7f12d4164d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009d90 tx=0x7f12cc02bd80 comp rx=0 tx=0).stop 2023-11-24T06:31:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.424+0000 7f12d99db640 1 -- 172.21.15.16:0/2535521093 shutdown_connections 2023-11-24T06:31:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.424+0000 7f12d99db640 1 --2- 172.21.15.16:0/2535521093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12d413fcb0 0x7f12d4164d80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.424+0000 7f12d99db640 1 -- 172.21.15.16:0/2535521093 >> 172.21.15.16:0/2535521093 conn(0x7f12d406a800 msgr2=0x7f12d406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.424+0000 7f12d99db640 1 -- 172.21.15.16:0/2535521093 shutdown_connections 2023-11-24T06:31:07.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.424+0000 7f12d99db640 1 -- 172.21.15.16:0/2535521093 wait complete. 2023-11-24T06:31:07.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.426+0000 7f12d99db640 1 Processor -- start 2023-11-24T06:31:07.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.426+0000 7f12d99db640 1 -- start start 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.427+0000 7f12d99db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12d413cfa0 0x7f12d41568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.427+0000 7f12d99db640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12d413fcb0 0x7f12d40a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.427+0000 7f12d99db640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12d415afb0 0x7f12d4139a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.427+0000 7f12d99db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40a9410 con 0x7f12d413cfa0 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.427+0000 7f12d99db640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d4139f40 con 0x7f12d415afb0 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.427+0000 7f12d99db640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40a9620 con 0x7f12d413fcb0 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d89d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12d413cfa0 0x7f12d41568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d89d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12d413cfa0 0x7f12d41568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36498/0 (socket says 172.21.15.16:36498) 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d89d9640 1 -- 172.21.15.16:0/2451893409 learned_addr learned my addr 172.21.15.16:0/2451893409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:07.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d3fff640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12d413fcb0 0x7f12d40a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d91da640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12d415afb0 0x7f12d4139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d89d9640 1 -- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12d413fcb0 msgr2=0x7f12d40a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d89d9640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12d413fcb0 0x7f12d40a8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d89d9640 1 -- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12d415afb0 msgr2=0x7f12d4139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d89d9640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12d415afb0 0x7f12d4139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.428+0000 7f12d89d9640 1 -- 172.21.15.16:0/2451893409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc009a70 con 0x7f12d413cfa0 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.429+0000 7f12d89d9640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12d413cfa0 0x7f12d41568b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f12c800c780 tx=0x7f12c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.429+0000 7f12d17fa640 1 -- 172.21.15.16:0/2451893409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8005ad0 con 0x7f12d413cfa0 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.429+0000 7f12d17fa640 1 -- 172.21.15.16:0/2451893409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c8005c30 con 0x7f12d413cfa0 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.429+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d40a9900 con 0x7f12d413cfa0 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.429+0000 7f12d17fa640 1 -- 172.21.15.16:0/2451893409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8007070 con 0x7f12d413cfa0 2023-11-24T06:31:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.429+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d40a9ec0 con 0x7f12d413cfa0 2023-11-24T06:31:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.430+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d4067450 con 0x7f12d413cfa0 2023-11-24T06:31:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.431+0000 7f12d17fa640 1 -- 172.21.15.16:0/2451893409 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f12c80071d0 con 0x7f12d413cfa0 2023-11-24T06:31:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.432+0000 7f12d17fa640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12b806b3e0 0x7f12b806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.432+0000 7f12d17fa640 1 -- 172.21.15.16:0/2451893409 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f12c8089e90 con 0x7f12d413cfa0 2023-11-24T06:31:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.433+0000 7f12d3fff640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12b806b3e0 0x7f12b806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.434+0000 7f12d3fff640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12b806b3e0 0x7f12b806d8a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f12d4153650 tx=0x7f12c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.437+0000 7f12d17fa640 1 -- 172.21.15.16:0/2451893409 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12c8054600 con 0x7f12d413cfa0 2023-11-24T06:31:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.630+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12d40cbff0 con 0x7f12d413cfa0 2023-11-24T06:31:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.633+0000 7f12d17fa640 1 -- 172.21.15.16:0/2451893409 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16824 (secure 0 0 0) 0x7f12c8057c20 con 0x7f12d413cfa0 2023-11-24T06:31:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:07.637 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:30:59.119430+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:30:31.631710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:30:32.732622+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":74,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.20","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12b806b3e0 msgr2=0x7f12b806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12b806b3e0 0x7f12b806d8a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f12d4153650 tx=0x7f12c0009040 comp rx=0 tx=0).stop 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12d413cfa0 msgr2=0x7f12d41568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12d413cfa0 0x7f12d41568b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f12c800c780 tx=0x7f12c800cc40 comp rx=0 tx=0).stop 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 shutdown_connections 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f12b806b3e0 0x7f12b806d8a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12d415afb0 0x7f12d4139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12d413fcb0 0x7f12d40a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 --2- 172.21.15.16:0/2451893409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12d413cfa0 0x7f12d41568b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 >> 172.21.15.16:0/2451893409 conn(0x7f12d406a800 msgr2=0x7f12d40b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 shutdown_connections 2023-11-24T06:31:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.636+0000 7f12d99db640 1 -- 172.21.15.16:0/2451893409 wait complete. 2023-11-24T06:31:07.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:31:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.885+0000 7fb68dd06640 1 Processor -- start 2023-11-24T06:31:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.885+0000 7fb68dd06640 1 -- start start 2023-11-24T06:31:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb68dd06640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb688162430 0x7fb6880b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb68dd06640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb688162810 0x7fb688071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb68dd06640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb6880b3650 0x7fb688071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb68dd06640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb688071cc0 con 0x7fb688162810 2023-11-24T06:31:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb68dd06640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6880702e0 con 0x7fb6880b3650 2023-11-24T06:31:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb68dd06640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880705c0 con 0x7fb688162430 2023-11-24T06:31:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb67ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb688162810 0x7fb688071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb67ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb688162810 0x7fb688071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36500/0 (socket says 172.21.15.16:36500) 2023-11-24T06:31:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb67ffff640 1 -- 172.21.15.16:0/1222098611 learned_addr learned my addr 172.21.15.16:0/1222098611 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb68cd04640 1 --2- 172.21.15.16:0/1222098611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb688162430 0x7fb6880b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.887+0000 7fb68d505640 1 --2- 172.21.15.16:0/1222098611 >> v2:172.21.15.173:3300/0 conn(0x7fb6880b3650 0x7fb688071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.888+0000 7fb67ffff640 1 -- 172.21.15.16:0/1222098611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb688162430 msgr2=0x7fb6880b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.888+0000 7fb67ffff640 1 --2- 172.21.15.16:0/1222098611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb688162430 0x7fb6880b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.888+0000 7fb67ffff640 1 -- 172.21.15.16:0/1222098611 >> v2:172.21.15.173:3300/0 conn(0x7fb6880b3650 msgr2=0x7fb688071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.888+0000 7fb67ffff640 1 --2- 172.21.15.16:0/1222098611 >> v2:172.21.15.173:3300/0 conn(0x7fb6880b3650 0x7fb688071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.888+0000 7fb67ffff640 1 -- 172.21.15.16:0/1222098611 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6880708a0 con 0x7fb688162810 2023-11-24T06:31:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.889+0000 7fb67ffff640 1 --2- 172.21.15.16:0/1222098611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162810 0x7fb688071240 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb6880b2bc0 tx=0x7fb67802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa16613bbe3da202 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:07.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.889+0000 7fb67f7fe640 1 -- 172.21.15.16:0/1222098611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678031870 con 0x7fb688162810 2023-11-24T06:31:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.889+0000 7fb67f7fe640 1 -- 172.21.15.16:0/1222098611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6780319d0 con 0x7fb688162810 2023-11-24T06:31:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.889+0000 7fb67f7fe640 1 -- 172.21.15.16:0/1222098611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678031d60 con 0x7fb688162810 2023-11-24T06:31:07.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.890+0000 7fb68dd06640 1 -- 172.21.15.16:0/1222098611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162810 msgr2=0x7fb688071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.890+0000 7fb68dd06640 1 --2- 172.21.15.16:0/1222098611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162810 0x7fb688071240 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb6880b2bc0 tx=0x7fb67802b910 comp rx=0 tx=0).stop 2023-11-24T06:31:07.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.890+0000 7fb68dd06640 1 -- 172.21.15.16:0/1222098611 shutdown_connections 2023-11-24T06:31:07.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.890+0000 7fb68dd06640 1 --2- 172.21.15.16:0/1222098611 >> v2:172.21.15.173:3300/0 conn(0x7fb6880b3650 0x7fb688071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.890+0000 7fb68dd06640 1 --2- 172.21.15.16:0/1222098611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162810 0x7fb688071240 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.890+0000 7fb68dd06640 1 --2- 172.21.15.16:0/1222098611 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb688162430 0x7fb6880b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.890+0000 7fb68dd06640 1 -- 172.21.15.16:0/1222098611 >> 172.21.15.16:0/1222098611 conn(0x7fb68806a800 msgr2=0x7fb68806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:07.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.890+0000 7fb68dd06640 1 -- 172.21.15.16:0/1222098611 shutdown_connections 2023-11-24T06:31:07.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.890+0000 7fb68dd06640 1 -- 172.21.15.16:0/1222098611 wait complete. 2023-11-24T06:31:07.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.891+0000 7fb68dd06640 1 Processor -- start 2023-11-24T06:31:07.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.892+0000 7fb68dd06640 1 -- start start 2023-11-24T06:31:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb68dd06640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6880b3650 0x7fb68813aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb68dd06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162430 0x7fb6881ec260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb68dd06640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880aea70 0x7fb6881eccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb68dd06640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68813b1e0 con 0x7fb688162430 2023-11-24T06:31:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb68dd06640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6880af210 con 0x7fb6880aea70 2023-11-24T06:31:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb68dd06640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6881ec7a0 con 0x7fb6880b3650 2023-11-24T06:31:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb67ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162430 0x7fb6881ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb67ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162430 0x7fb6881ec260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36504/0 (socket says 172.21.15.16:36504) 2023-11-24T06:31:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb67ffff640 1 -- 172.21.15.16:0/3069871519 learned_addr learned my addr 172.21.15.16:0/3069871519 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb67ffff640 1 -- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6880b3650 msgr2=0x7fb68813aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb67ffff640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6880b3650 0x7fb68813aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb67ffff640 1 -- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880aea70 msgr2=0x7fb6881eccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb67ffff640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880aea70 0x7fb6881eccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.893+0000 7fb67ffff640 1 -- 172.21.15.16:0/3069871519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780094d0 con 0x7fb688162430 2023-11-24T06:31:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.894+0000 7fb67ffff640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162430 0x7fb6881ec260 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb67802b950 tx=0x7fb6780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:07.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.894+0000 7fb67d7fa640 1 -- 172.21.15.16:0/3069871519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67803acb0 con 0x7fb688162430 2023-11-24T06:31:07.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.894+0000 7fb67d7fa640 1 -- 172.21.15.16:0/3069871519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67803ae10 con 0x7fb688162430 2023-11-24T06:31:07.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.894+0000 7fb67d7fa640 1 -- 172.21.15.16:0/3069871519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678044630 con 0x7fb688162430 2023-11-24T06:31:07.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.894+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6880aef30 con 0x7fb688162430 2023-11-24T06:31:07.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.894+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb688154d60 con 0x7fb688162430 2023-11-24T06:31:07.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.895+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6881548e0 con 0x7fb688162430 2023-11-24T06:31:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.896+0000 7fb67d7fa640 1 -- 172.21.15.16:0/3069871519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb67804d450 con 0x7fb688162430 2023-11-24T06:31:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.897+0000 7fb67d7fa640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb67406b3e0 0x7fb67406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.897+0000 7fb67d7fa640 1 -- 172.21.15.16:0/3069871519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fb67802d040 con 0x7fb688162430 2023-11-24T06:31:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.898+0000 7fb68cd04640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb67406b3e0 0x7fb67406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.899+0000 7fb68cd04640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb67406b3e0 0x7fb67406d8a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb684009540 tx=0x7fb684009450 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:07.902+0000 7fb67d7fa640 1 -- 172.21.15.16:0/3069871519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6881548e0 con 0x7fb688162430 2023-11-24T06:31:08.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.126+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb6881548e0 con 0x7fb688162430 2023-11-24T06:31:08.130 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-24T06:31:08.127+0000 7f082ae01640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:31:08.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.131+0000 7fb67d7fa640 1 -- 172.21.15.16:0/3069871519 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb6881548e0 con 0x7fb688162430 2023-11-24T06:31:08.134 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:31:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb67406b3e0 msgr2=0x7fb67406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb67406b3e0 0x7fb67406d8a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb684009540 tx=0x7fb684009450 comp rx=0 tx=0).stop 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162430 msgr2=0x7fb6881ec260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162430 0x7fb6881ec260 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb67802b950 tx=0x7fb6780046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 shutdown_connections 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb67406b3e0 0x7fb67406d8a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6880aea70 0x7fb6881eccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb688162430 0x7fb6881ec260 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 --2- 172.21.15.16:0/3069871519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6880b3650 0x7fb68813aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 >> 172.21.15.16:0/3069871519 conn(0x7fb68806a800 msgr2=0x7fb6880b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 shutdown_connections 2023-11-24T06:31:08.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.135+0000 7fb68dd06640 1 -- 172.21.15.16:0/3069871519 wait complete. 2023-11-24T06:31:08.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:31:08.222 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:31:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.356+0000 7f4b1aaf8640 1 Processor -- start 2023-11-24T06:31:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.357+0000 7f4b1aaf8640 1 -- start start 2023-11-24T06:31:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.358+0000 7f4b1aaf8640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4b1406c730 0x7f4b140b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.358+0000 7f4b1aaf8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b14070160 0x7f4b14071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.358+0000 7f4b1aaf8640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b140708d0 con 0x7f4b1406cb10 2023-11-24T06:31:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.358+0000 7f4b1aaf8640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b14070bb0 con 0x7f4b1406c730 2023-11-24T06:31:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.358+0000 7f4b1aaf8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b14070e90 con 0x7f4b14070160 2023-11-24T06:31:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.359+0000 7f4b1a2f7640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f4b1406cb10 0x7f4b1406cfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:34712/0 (socket says 172.21.15.16:34712) 2023-11-24T06:31:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.359+0000 7f4b1a2f7640 1 -- 172.21.15.16:0/2368307658 learned_addr learned my addr 172.21.15.16:0/2368307658 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.359+0000 7f4b192f5640 1 --2- 172.21.15.16:0/2368307658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b14070160 0x7f4b14071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.359+0000 7f4b19af6640 1 --2- 172.21.15.16:0/2368307658 >> v2:172.21.15.173:3300/0 conn(0x7f4b1406c730 0x7f4b140b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.360+0000 7f4b18af4640 1 -- 172.21.15.16:0/2368307658 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145282587 0 0) 0x7f4b140708d0 con 0x7f4b1406cb10 2023-11-24T06:31:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.360+0000 7f4b18af4640 1 -- 172.21.15.16:0/2368307658 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b00003520 con 0x7f4b1406cb10 2023-11-24T06:31:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.360+0000 7f4b192f5640 1 -- 172.21.15.16:0/2368307658 >> v2:172.21.15.173:3300/0 conn(0x7f4b1406c730 msgr2=0x7f4b140b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.360+0000 7f4b192f5640 1 --2- 172.21.15.16:0/2368307658 >> v2:172.21.15.173:3300/0 conn(0x7f4b1406c730 0x7f4b140b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.360+0000 7f4b192f5640 1 -- 172.21.15.16:0/2368307658 >> v1:172.21.15.173:6789/0 conn(0x7f4b1406cb10 legacy=0x7f4b1406cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.360+0000 7f4b192f5640 1 -- 172.21.15.16:0/2368307658 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b14071170 con 0x7f4b14070160 2023-11-24T06:31:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.361+0000 7f4b192f5640 1 --2- 172.21.15.16:0/2368307658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b14070160 0x7f4b14071970 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4b040097a0 tx=0x7f4b0402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c1b7937eefa79bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.362+0000 7f4b18af4640 1 -- 172.21.15.16:0/2368307658 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04031780 con 0x7f4b14070160 2023-11-24T06:31:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.362+0000 7f4b18af4640 1 -- 172.21.15.16:0/2368307658 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b040318e0 con 0x7f4b14070160 2023-11-24T06:31:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.362+0000 7f4b18af4640 1 -- 172.21.15.16:0/2368307658 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04031c70 con 0x7f4b14070160 2023-11-24T06:31:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.363+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/2368307658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b14070160 msgr2=0x7f4b14071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.363+0000 7f4b1aaf8640 1 --2- 172.21.15.16:0/2368307658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b14070160 0x7f4b14071970 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4b040097a0 tx=0x7f4b0402b820 comp rx=0 tx=0).stop 2023-11-24T06:31:08.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.363+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/2368307658 shutdown_connections 2023-11-24T06:31:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.363+0000 7f4b1aaf8640 1 --2- 172.21.15.16:0/2368307658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b14070160 0x7f4b14071970 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.363+0000 7f4b1aaf8640 1 --2- 172.21.15.16:0/2368307658 >> v2:172.21.15.173:3300/0 conn(0x7f4b1406c730 0x7f4b140b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.363+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/2368307658 >> 172.21.15.16:0/2368307658 conn(0x7f4b1406a7d0 msgr2=0x7f4b1406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.363+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/2368307658 shutdown_connections 2023-11-24T06:31:08.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.363+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/2368307658 wait complete. 2023-11-24T06:31:08.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.365+0000 7f4b1aaf8640 1 Processor -- start 2023-11-24T06:31:08.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.366+0000 7f4b1aaf8640 1 -- start start 2023-11-24T06:31:08.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.367+0000 7f4b1aaf8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b1406c730 0x7f4b140aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:08.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.367+0000 7f4b1aaf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b1406cb10 0x7f4b140adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:08.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.367+0000 7f4b1aaf8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b14070160 0x7f4b14149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:08.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.367+0000 7f4b1aaf8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b141f8750 con 0x7f4b1406cb10 2023-11-24T06:31:08.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.367+0000 7f4b1aaf8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b141f8a30 con 0x7f4b14070160 2023-11-24T06:31:08.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.367+0000 7f4b1aaf8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b141f8d10 con 0x7f4b1406c730 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.367+0000 7f4b192f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b1406cb10 0x7f4b140adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.367+0000 7f4b192f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b1406cb10 0x7f4b140adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36516/0 (socket says 172.21.15.16:36516) 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.367+0000 7f4b192f5640 1 -- 172.21.15.16:0/862386593 learned_addr learned my addr 172.21.15.16:0/862386593 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.368+0000 7f4b19af6640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b1406c730 0x7f4b140aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.368+0000 7f4b1a2f7640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b14070160 0x7f4b14149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.368+0000 7f4b192f5640 1 -- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b1406c730 msgr2=0x7f4b140aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.368+0000 7f4b192f5640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b1406c730 0x7f4b140aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.368+0000 7f4b192f5640 1 -- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b14070160 msgr2=0x7f4b14149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.368+0000 7f4b192f5640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b14070160 0x7f4b14149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.368+0000 7f4b192f5640 1 -- 172.21.15.16:0/862386593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b04009400 con 0x7f4b1406cb10 2023-11-24T06:31:08.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.368+0000 7f4b1a2f7640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b14070160 0x7f4b14149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.368+0000 7f4b192f5640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b1406cb10 0x7f4b140adda0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f4b040098c0 tx=0x7f4b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.369+0000 7f4b0a7fc640 1 -- 172.21.15.16:0/862386593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04032820 con 0x7f4b1406cb10 2023-11-24T06:31:08.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.369+0000 7f4b0a7fc640 1 -- 172.21.15.16:0/862386593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b040329a0 con 0x7f4b1406cb10 2023-11-24T06:31:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.369+0000 7f4b0a7fc640 1 -- 172.21.15.16:0/862386593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04032d50 con 0x7f4b1406cb10 2023-11-24T06:31:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.369+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b141f8ff0 con 0x7f4b1406cb10 2023-11-24T06:31:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.369+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b140ab3a0 con 0x7f4b1406cb10 2023-11-24T06:31:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.370+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b141f9510 con 0x7f4b1406cb10 2023-11-24T06:31:08.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.371+0000 7f4b0a7fc640 1 -- 172.21.15.16:0/862386593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4b0404d3f0 con 0x7f4b1406cb10 2023-11-24T06:31:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.373+0000 7f4b0a7fc640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4af806b3e0 0x7f4af806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:08.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.373+0000 7f4b0a7fc640 1 -- 172.21.15.16:0/862386593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f4b040abe70 con 0x7f4b1406cb10 2023-11-24T06:31:08.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.374+0000 7f4b19af6640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4af806b3e0 0x7f4af806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.375+0000 7f4b19af6640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4af806b3e0 0x7f4af806d8a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009ad0 tx=0x7f4b10008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:08.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.378+0000 7f4b0a7fc640 1 -- 172.21.15.16:0/862386593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b141f9510 con 0x7f4b1406cb10 2023-11-24T06:31:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:08.608+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4b141f9510 con 0x7f4b1406cb10 2023-11-24T06:31:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.196+0000 7f4b0a7fc640 1 -- 172.21.15.16:0/862386593 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4b141f9510 con 0x7f4b1406cb10 2023-11-24T06:31:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.199+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4af806b3e0 msgr2=0x7f4af806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.199+0000 7f4b1aaf8640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4af806b3e0 0x7f4af806d8a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009ad0 tx=0x7f4b10008040 comp rx=0 tx=0).stop 2023-11-24T06:31:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.199+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b1406cb10 msgr2=0x7f4b140adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:09.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.199+0000 7f4b1aaf8640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b1406cb10 0x7f4b140adda0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f4b040098c0 tx=0x7f4b040046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.200+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 shutdown_connections 2023-11-24T06:31:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.200+0000 7f4b1aaf8640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4af806b3e0 0x7f4af806d8a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.200+0000 7f4b1aaf8640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b14070160 0x7f4b14149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.200+0000 7f4b1aaf8640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b1406cb10 0x7f4b140adda0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.200+0000 7f4b1aaf8640 1 --2- 172.21.15.16:0/862386593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b1406c730 0x7f4b140aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.200+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 >> 172.21.15.16:0/862386593 conn(0x7f4b1406a7d0 msgr2=0x7f4b140cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.200+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 shutdown_connections 2023-11-24T06:31:09.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.200+0000 7f4b1aaf8640 1 -- 172.21.15.16:0/862386593 wait complete. 2023-11-24T06:31:09.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:31:09.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.453+0000 7fc0d8947640 1 Processor -- start 2023-11-24T06:31:09.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.453+0000 7fc0d8947640 1 -- start start 2023-11-24T06:31:09.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.455+0000 7fc0d8947640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0d406c470 0x7fc0d40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:09.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.455+0000 7fc0d8947640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406dab0 0x7fc0d40ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:09.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.455+0000 7fc0d8947640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d406ca10 con 0x7fc0d41447f0 2023-11-24T06:31:09.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.455+0000 7fc0d8947640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d406d690 con 0x7fc0d406c470 2023-11-24T06:31:09.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.455+0000 7fc0d8947640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40cebb0 con 0x7fc0d406dab0 2023-11-24T06:31:09.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.455+0000 7fc0caffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0d406c470 0x7fc0d40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:09.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.456+0000 7fc0caffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc0d406c470 0x7fc0d40cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36532/0 (socket says 172.21.15.16:36532) 2023-11-24T06:31:09.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.456+0000 7fc0caffd640 1 -- 172.21.15.16:0/3910416836 learned_addr learned my addr 172.21.15.16:0/3910416836 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:09.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.456+0000 7fc0d3fff640 1 --2- 172.21.15.16:0/3910416836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406dab0 0x7fc0d40ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.456+0000 7fc0caffd640 1 -- 172.21.15.16:0/3910416836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406dab0 msgr2=0x7fc0d40ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.456+0000 7fc0caffd640 1 --2- 172.21.15.16:0/3910416836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406dab0 0x7fc0d40ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.456+0000 7fc0caffd640 1 -- 172.21.15.16:0/3910416836 >> v1:172.21.15.173:6789/0 conn(0x7fc0d41447f0 legacy=0x7fc0d415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.456+0000 7fc0caffd640 1 -- 172.21.15.16:0/3910416836 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d406ccf0 con 0x7fc0d406c470 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.457+0000 7fc0caffd640 1 --2- 172.21.15.16:0/3910416836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406c470 0x7fc0d40cc5e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009870 tx=0x7fc0bc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2349d3effacbbadf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d2ffd640 1 -- 172.21.15.16:0/3910416836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0317c0 con 0x7fc0d406c470 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d2ffd640 1 -- 172.21.15.16:0/3910416836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc031920 con 0x7fc0d406c470 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d2ffd640 1 -- 172.21.15.16:0/3910416836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc031cb0 con 0x7fc0d406c470 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d8947640 1 -- 172.21.15.16:0/3910416836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406c470 msgr2=0x7fc0d40cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d8947640 1 --2- 172.21.15.16:0/3910416836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406c470 0x7fc0d40cc5e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009870 tx=0x7fc0bc02b860 comp rx=0 tx=0).stop 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d8947640 1 -- 172.21.15.16:0/3910416836 shutdown_connections 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d8947640 1 --2- 172.21.15.16:0/3910416836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406dab0 0x7fc0d40ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d8947640 1 --2- 172.21.15.16:0/3910416836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406c470 0x7fc0d40cc5e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d8947640 1 -- 172.21.15.16:0/3910416836 >> 172.21.15.16:0/3910416836 conn(0x7fc0d406a3e0 msgr2=0x7fc0d406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:09.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d8947640 1 -- 172.21.15.16:0/3910416836 shutdown_connections 2023-11-24T06:31:09.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.458+0000 7fc0d8947640 1 -- 172.21.15.16:0/3910416836 wait complete. 2023-11-24T06:31:09.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.460+0000 7fc0d8947640 1 Processor -- start 2023-11-24T06:31:09.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.460+0000 7fc0d8947640 1 -- start start 2023-11-24T06:31:09.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.461+0000 7fc0d8947640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406c470 0x7fc0d40c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:09.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.461+0000 7fc0d8947640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406dab0 0x7fc0d40c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:09.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.461+0000 7fc0d8947640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d41447f0 0x7fc0d40c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:09.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.461+0000 7fc0d8947640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40ca020 con 0x7fc0d406dab0 2023-11-24T06:31:09.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.461+0000 7fc0d8947640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40c6a90 con 0x7fc0d41447f0 2023-11-24T06:31:09.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.461+0000 7fc0d8947640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d40c6d70 con 0x7fc0d406c470 2023-11-24T06:31:09.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406dab0 0x7fc0d40c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:09.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0caffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406dab0 0x7fc0d40c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36542/0 (socket says 172.21.15.16:36542) 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0caffd640 1 -- 172.21.15.16:0/462564854 learned_addr learned my addr 172.21.15.16:0/462564854 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0d37fe640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406c470 0x7fc0d40c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0caffd640 1 -- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406c470 msgr2=0x7fc0d40c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0d3fff640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d41447f0 0x7fc0d40c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0caffd640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406c470 0x7fc0d40c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0caffd640 1 -- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d41447f0 msgr2=0x7fc0d40c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0caffd640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d41447f0 0x7fc0d40c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0caffd640 1 -- 172.21.15.16:0/462564854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0094d0 con 0x7fc0d406dab0 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.462+0000 7fc0d37fe640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406c470 0x7fc0d40c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.463+0000 7fc0caffd640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406dab0 0x7fc0d40c7cd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc02b8a0 tx=0x7fc0bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.463+0000 7fc0d0ff9640 1 -- 172.21.15.16:0/462564854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc032860 con 0x7fc0d406dab0 2023-11-24T06:31:09.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.463+0000 7fc0d0ff9640 1 -- 172.21.15.16:0/462564854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc032a00 con 0x7fc0d406dab0 2023-11-24T06:31:09.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.463+0000 7fc0d0ff9640 1 -- 172.21.15.16:0/462564854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc032db0 con 0x7fc0d406dab0 2023-11-24T06:31:09.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.463+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d40c7050 con 0x7fc0d406dab0 2023-11-24T06:31:09.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.463+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d41f2ba0 con 0x7fc0d406dab0 2023-11-24T06:31:09.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.464+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d41f2d90 con 0x7fc0d406dab0 2023-11-24T06:31:09.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.465+0000 7fc0d0ff9640 1 -- 172.21.15.16:0/462564854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fc0bc04e450 con 0x7fc0d406dab0 2023-11-24T06:31:09.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.466+0000 7fc0d0ff9640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc0b406b410 0x7fc0b406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:09.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.466+0000 7fc0d0ff9640 1 -- 172.21.15.16:0/462564854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fc0bc0ac370 con 0x7fc0d406dab0 2023-11-24T06:31:09.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.467+0000 7fc0d37fe640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc0b406b410 0x7fc0b406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:09.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.469+0000 7fc0d37fe640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc0b406b410 0x7fc0b406d8d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40b1b80 tx=0x7fc0c4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.471+0000 7fc0d0ff9640 1 -- 172.21.15.16:0/462564854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0bc076b60 con 0x7fc0d406dab0 2023-11-24T06:31:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:09.662+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0d4067450 con 0x7fc0d406dab0 2023-11-24T06:31:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.209+0000 7fc0d0ff9640 1 -- 172.21.15.16:0/462564854 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc0bc07a180 con 0x7fc0d406dab0 2023-11-24T06:31:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:31:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.212+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc0b406b410 msgr2=0x7fc0b406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.212+0000 7fc0d8947640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc0b406b410 0x7fc0b406d8d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40b1b80 tx=0x7fc0c4009040 comp rx=0 tx=0).stop 2023-11-24T06:31:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.212+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406dab0 msgr2=0x7fc0d40c7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:10.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.212+0000 7fc0d8947640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406dab0 0x7fc0d40c7cd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc02b8a0 tx=0x7fc0bc0046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.214+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 shutdown_connections 2023-11-24T06:31:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.214+0000 7fc0d8947640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc0b406b410 0x7fc0b406d8d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.214+0000 7fc0d8947640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0d41447f0 0x7fc0d40c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.214+0000 7fc0d8947640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d406dab0 0x7fc0d40c7cd0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.214+0000 7fc0d8947640 1 --2- 172.21.15.16:0/462564854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0d406c470 0x7fc0d40c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.214+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 >> 172.21.15.16:0/462564854 conn(0x7fc0d406a3e0 msgr2=0x7fc0d415cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.215+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 shutdown_connections 2023-11-24T06:31:10.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.216+0000 7fc0d8947640 1 -- 172.21.15.16:0/462564854 wait complete. 2023-11-24T06:31:10.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:31:10.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.448+0000 7fad0d767640 1 Processor -- start 2023-11-24T06:31:10.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.448+0000 7fad0d767640 1 -- start start 2023-11-24T06:31:10.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.450+0000 7fad0d767640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080cc990 0x7fad0806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:10.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.450+0000 7fad0d767640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad08070e50 con 0x7fad08143c40 2023-11-24T06:31:10.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.450+0000 7fad0d767640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad08144020 con 0x7fad080cc500 2023-11-24T06:31:10.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.450+0000 7fad0d767640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad080723c0 con 0x7fad080cc990 2023-11-24T06:31:10.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.450+0000 7fad07fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fad08143c40 0x7fad08071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48422/0 (socket says 172.21.15.16:48422) 2023-11-24T06:31:10.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.450+0000 7fad07fff640 1 -- 172.21.15.16:0/2068835447 learned_addr learned my addr 172.21.15.16:0/2068835447 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:10.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad0cf66640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080cc990 0x7fad0806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230891717 0 0) 0x7fad08070e50 con 0x7fad08143c40 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf0003520 con 0x7fad08143c40 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1697639096 0 0) 0x7facf0003520 con 0x7fad08143c40 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080cc990 msgr2=0x7fad0806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080cc990 0x7fad0806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 >> v1:172.21.15.173:6789/0 conn(0x7fad080cc500 legacy=0x7fad0806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad08071e00 con 0x7fad08143c40 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7facf8001690 con 0x7fad08143c40 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 >> v1:172.21.15.16:6789/0 conn(0x7fad08143c40 legacy=0x7fad08071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080cc990 0x7facf00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facf0004020 0x7facf00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7facf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad080723c0 con 0x7facf0004d00 2023-11-24T06:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf000bc60 con 0x7facf0004020 2023-11-24T06:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.451+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf000be20 con 0x7fad080cc990 2023-11-24T06:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.452+0000 7fad0cf66640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7facf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.452+0000 7fad07fff640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080cc990 0x7facf00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.452+0000 7fad077fe640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facf0004020 0x7facf00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.452+0000 7fad0cf66640 1 -- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080cc990 msgr2=0x7facf00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.452+0000 7fad0cf66640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080cc990 0x7facf00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.452+0000 7fad0cf66640 1 -- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facf0004020 msgr2=0x7facf00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.452+0000 7fad0cf66640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7facf0004020 0x7facf00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.452+0000 7fad0cf66640 1 -- 172.21.15.16:0/2068835447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf0003520 con 0x7facf0004d00 2023-11-24T06:31:10.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.453+0000 7fad0cf66640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7facf0005180 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fad00007360 tx=0x7facf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0ec607d79d1bc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:10.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0002ccd0 con 0x7facf0004d00 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0002ce30 con 0x7facf0004d00 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad06ffd640 1 -- 172.21.15.16:0/2068835447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00034660 con 0x7facf0004d00 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad0d767640 1 -- 172.21.15.16:0/2068835447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 msgr2=0x7facf0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad0d767640 1 --2- 172.21.15.16:0/2068835447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7facf0005180 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fad00007360 tx=0x7facf00057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad0cf66640 1 -- 172.21.15.16:0/2068835447 reap_dead start 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad0d767640 1 -- 172.21.15.16:0/2068835447 shutdown_connections 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad0d767640 1 -- 172.21.15.16:0/2068835447 >> 172.21.15.16:0/2068835447 conn(0x7fad0806a800 msgr2=0x7fad0806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad0d767640 1 -- 172.21.15.16:0/2068835447 shutdown_connections 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.454+0000 7fad0d767640 1 -- 172.21.15.16:0/2068835447 wait complete. 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.456+0000 7fad0d767640 1 Processor -- start 2023-11-24T06:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.456+0000 7fad0d767640 1 -- start start 2023-11-24T06:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.457+0000 7fad0d767640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7fad08159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.457+0000 7fad0d767640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad080aa780 0x7fad080ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.457+0000 7fad0d767640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080ad600 0x7fad080a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.457+0000 7fad0d767640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad080ada80 con 0x7facf0004d00 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.457+0000 7fad0d767640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08130a80 con 0x7fad080aa780 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.457+0000 7fad0d767640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad080aabe0 con 0x7fad080ad600 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.457+0000 7fad07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7fad08159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad07fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7fad08159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36556/0 (socket says 172.21.15.16:36556) 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad07fff640 1 -- 172.21.15.16:0/1488261488 learned_addr learned my addr 172.21.15.16:0/1488261488 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad0cf66640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080ad600 0x7fad080a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad077fe640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad080aa780 0x7fad080ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad07fff640 1 -- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080ad600 msgr2=0x7fad080a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad07fff640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080ad600 0x7fad080a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad07fff640 1 -- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad080aa780 msgr2=0x7fad080ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad07fff640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad080aa780 0x7fad080ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad07fff640 1 -- 172.21.15.16:0/1488261488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0812fe70 con 0x7facf0004d00 2023-11-24T06:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad077fe640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad080aa780 0x7fad080ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.458+0000 7fad07fff640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7fad08159c50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7facf800e7f0 tx=0x7facf800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.459+0000 7fad04ff9640 1 -- 172.21.15.16:0/1488261488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf800c8d0 con 0x7facf0004d00 2023-11-24T06:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.459+0000 7fad04ff9640 1 -- 172.21.15.16:0/1488261488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf800ca30 con 0x7facf0004d00 2023-11-24T06:31:10.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.459+0000 7fad04ff9640 1 -- 172.21.15.16:0/1488261488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf800cde0 con 0x7facf0004d00 2023-11-24T06:31:10.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.459+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad08130150 con 0x7facf0004d00 2023-11-24T06:31:10.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.459+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad08130710 con 0x7facf0004d00 2023-11-24T06:31:10.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.460+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad081308d0 con 0x7facf0004d00 2023-11-24T06:31:10.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.461+0000 7fad04ff9640 1 -- 172.21.15.16:0/1488261488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7facf801d460 con 0x7facf0004d00 2023-11-24T06:31:10.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.462+0000 7fad04ff9640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7facf406b410 0x7facf406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:10.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.462+0000 7fad04ff9640 1 -- 172.21.15.16:0/1488261488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5472+0+0 (secure 0 0 0) 0x7facf808a020 con 0x7facf0004d00 2023-11-24T06:31:10.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.463+0000 7fad077fe640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7facf406b410 0x7facf406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:10.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.464+0000 7fad077fe640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7facf406b410 0x7facf406d8d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fad0815f1c0 tx=0x7facfc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.467+0000 7fad04ff9640 1 -- 172.21.15.16:0/1488261488 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7facf8054880 con 0x7facf0004d00 2023-11-24T06:31:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:10.669+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad08067450 con 0x7facf0004d00 2023-11-24T06:31:11.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.221+0000 7fad04ff9640 1 -- 172.21.15.16:0/1488261488 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 169+0+0 (secure 0 0 0) 0x7facf8057ea0 con 0x7facf0004d00 2023-11-24T06:31:11.224 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-24T06:31:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.224+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7facf406b410 msgr2=0x7facf406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.224+0000 7fad0d767640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7facf406b410 0x7facf406d8d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fad0815f1c0 tx=0x7facfc009040 comp rx=0 tx=0).stop 2023-11-24T06:31:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.224+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 msgr2=0x7fad08159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.224+0000 7fad0d767640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7fad08159c50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7facf800e7f0 tx=0x7facf800ecb0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.228+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 shutdown_connections 2023-11-24T06:31:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.228+0000 7fad0d767640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7facf406b410 0x7facf406d8d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.228+0000 7fad0d767640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad080ad600 0x7fad080a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.228+0000 7fad0d767640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad080aa780 0x7fad080ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.228+0000 7fad0d767640 1 --2- 172.21.15.16:0/1488261488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facf0004d00 0x7fad08159c50 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.228+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 >> 172.21.15.16:0/1488261488 conn(0x7fad0806a800 msgr2=0x7fad0806d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.228+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 shutdown_connections 2023-11-24T06:31:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.228+0000 7fad0d767640 1 -- 172.21.15.16:0/1488261488 wait complete. 2023-11-24T06:31:11.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:31:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.467+0000 7f16a96ff640 1 Processor -- start 2023-11-24T06:31:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.467+0000 7f16a96ff640 1 -- start start 2023-11-24T06:31:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.469+0000 7f16a96ff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f16a40ccb50 0x7f16a4070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.469+0000 7f16a96ff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a415bf10 0x7f16a4143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.469+0000 7f16a96ff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a415f4e0 con 0x7f16a40adf20 2023-11-24T06:31:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.469+0000 7f16a96ff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a415f970 con 0x7f16a40ccb50 2023-11-24T06:31:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.469+0000 7f16a96ff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a415c3c0 con 0x7f16a415bf10 2023-11-24T06:31:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.470+0000 7f16a3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f16a40adf20 0x7f16a40cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48426/0 (socket says 172.21.15.16:48426) 2023-11-24T06:31:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.470+0000 7f16a3fff640 1 -- 172.21.15.16:0/1086313780 learned_addr learned my addr 172.21.15.16:0/1086313780 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:11.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.470+0000 7f169b7fe640 1 --2- 172.21.15.16:0/1086313780 >> v2:172.21.15.173:3300/0 conn(0x7f16a40ccb50 0x7f16a4070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.470+0000 7f16a8efe640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a415bf10 0x7f16a4143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.470+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036991277 0 0) 0x7f16a415f4e0 con 0x7f16a40adf20 2023-11-24T06:31:11.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.470+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1690003520 con 0x7f16a40adf20 2023-11-24T06:31:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 117746603 0 0) 0x7f1690003520 con 0x7f16a40adf20 2023-11-24T06:31:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a415bf10 msgr2=0x7f16a4143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a415bf10 0x7f16a4143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 >> v2:172.21.15.173:3300/0 conn(0x7f16a40ccb50 msgr2=0x7f16a4070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 --2- 172.21.15.16:0/1086313780 >> v2:172.21.15.173:3300/0 conn(0x7f16a40ccb50 0x7f16a4070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4071130 con 0x7f16a40adf20 2023-11-24T06:31:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f1694001670 con 0x7f16a40adf20 2023-11-24T06:31:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 >> v1:172.21.15.16:6789/0 conn(0x7f16a40adf20 legacy=0x7f16a40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1690003970 0x7f1690003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16900047c0 0x7f1690004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a415bf10 0x7f1690005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a415f970 con 0x7f1690003970 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a415c3c0 con 0x7f16900047c0 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169000b980 con 0x7f16a415bf10 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.471+0000 7f16a3fff640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1690003970 0x7f1690003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.472+0000 7f169b7fe640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16900047c0 0x7f1690004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.472+0000 7f16a3fff640 1 -- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a415bf10 msgr2=0x7f1690005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.472+0000 7f16a8efe640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a415bf10 0x7f1690005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.472+0000 7f16a3fff640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a415bf10 0x7f1690005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.472+0000 7f16a3fff640 1 -- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16900047c0 msgr2=0x7f1690004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.472+0000 7f16a3fff640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16900047c0 0x7f1690004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.472+0000 7f16a3fff640 1 -- 172.21.15.16:0/1086313780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690003520 con 0x7f1690003970 2023-11-24T06:31:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.473+0000 7f16a3fff640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1690003970 0x7f1690003d90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f169400c270 tx=0x7f16900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc99074ed326ea2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.473+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694005ad0 con 0x7f1690003970 2023-11-24T06:31:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.473+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694005c50 con 0x7f1690003970 2023-11-24T06:31:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.473+0000 7f16a37fe640 1 -- 172.21.15.16:0/1086313780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169402c8b0 con 0x7f1690003970 2023-11-24T06:31:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.474+0000 7f16a96ff640 1 -- 172.21.15.16:0/1086313780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1690003970 msgr2=0x7f1690003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.474+0000 7f16a96ff640 1 --2- 172.21.15.16:0/1086313780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1690003970 0x7f1690003d90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f169400c270 tx=0x7f16900057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.474+0000 7f16a8efe640 1 -- 172.21.15.16:0/1086313780 reap_dead start 2023-11-24T06:31:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.474+0000 7f16a96ff640 1 -- 172.21.15.16:0/1086313780 shutdown_connections 2023-11-24T06:31:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.474+0000 7f16a96ff640 1 -- 172.21.15.16:0/1086313780 >> 172.21.15.16:0/1086313780 conn(0x7f16a406a7d0 msgr2=0x7f16a406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:11.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.474+0000 7f16a96ff640 1 -- 172.21.15.16:0/1086313780 shutdown_connections 2023-11-24T06:31:11.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.474+0000 7f16a96ff640 1 -- 172.21.15.16:0/1086313780 wait complete. 2023-11-24T06:31:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.476+0000 7f16a96ff640 1 Processor -- start 2023-11-24T06:31:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.476+0000 7f16a96ff640 1 -- start start 2023-11-24T06:31:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.477+0000 7f16a96ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1690003970 0x7f16a4139410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.477+0000 7f16a96ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a41efdf0 0x7f16a413eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.477+0000 7f16a96ff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a41f0270 0x7f16a414be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.477+0000 7f16a96ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a413f990 con 0x7f16a41efdf0 2023-11-24T06:31:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.477+0000 7f16a96ff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a413fce0 con 0x7f1690003970 2023-11-24T06:31:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.477+0000 7f16a96ff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a413ffc0 con 0x7f16a41f0270 2023-11-24T06:31:11.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.477+0000 7f169b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a41efdf0 0x7f16a413eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.477+0000 7f169b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a41efdf0 0x7f16a413eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36568/0 (socket says 172.21.15.16:36568) 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.477+0000 7f169b7fe640 1 -- 172.21.15.16:0/3071214932 learned_addr learned my addr 172.21.15.16:0/3071214932 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f16a3fff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1690003970 0x7f16a4139410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f169b7fe640 1 -- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a41f0270 msgr2=0x7f16a414be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f169b7fe640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a41f0270 0x7f16a414be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f169b7fe640 1 -- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1690003970 msgr2=0x7f16a4139410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f169b7fe640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1690003970 0x7f16a4139410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f169b7fe640 1 -- 172.21.15.16:0/3071214932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a41402a0 con 0x7f16a41efdf0 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f16a3fff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1690003970 0x7f16a4139410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f169b7fe640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a41efdf0 0x7f16a413eed0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f168c011590 tx=0x7f168c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f16a17fa640 1 -- 172.21.15.16:0/3071214932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c004220 con 0x7f16a41efdf0 2023-11-24T06:31:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f16a17fa640 1 -- 172.21.15.16:0/3071214932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c004380 con 0x7f16a41efdf0 2023-11-24T06:31:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a4140580 con 0x7f16a41efdf0 2023-11-24T06:31:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f16a17fa640 1 -- 172.21.15.16:0/3071214932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c015930 con 0x7f16a41efdf0 2023-11-24T06:31:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.478+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a413f410 con 0x7f16a41efdf0 2023-11-24T06:31:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.480+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a413f6f0 con 0x7f16a41efdf0 2023-11-24T06:31:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.480+0000 7f16a17fa640 1 -- 172.21.15.16:0/3071214932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f168c015a90 con 0x7f16a41efdf0 2023-11-24T06:31:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.481+0000 7f16a17fa640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f168406b410 0x7f168406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.481+0000 7f16a17fa640 1 -- 172.21.15.16:0/3071214932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f168c089db0 con 0x7f16a41efdf0 2023-11-24T06:31:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.482+0000 7f16a3fff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f168406b410 0x7f168406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.483+0000 7f16a3fff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f168406b410 0x7f168406d8d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1694034000 tx=0x7f1694007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:11.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.486+0000 7f16a17fa640 1 -- 172.21.15.16:0/3071214932 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f168c054890 con 0x7f16a41efdf0 2023-11-24T06:31:11.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.681+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f16a4067450 con 0x7f16a41efdf0 2023-11-24T06:31:11.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.689+0000 7f16a17fa640 1 -- 172.21.15.16:0/3071214932 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f168c057eb0 con 0x7f16a41efdf0 2023-11-24T06:31:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f168406b410 msgr2=0x7f168406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f168406b410 0x7f168406d8d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1694034000 tx=0x7f1694007000 comp rx=0 tx=0).stop 2023-11-24T06:31:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a41efdf0 msgr2=0x7f16a413eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a41efdf0 0x7f16a413eed0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f168c011590 tx=0x7f168c011a50 comp rx=0 tx=0).stop 2023-11-24T06:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 shutdown_connections 2023-11-24T06:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f168406b410 0x7f168406d8d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16a41f0270 0x7f16a414be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a41efdf0 0x7f16a413eed0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 --2- 172.21.15.16:0/3071214932 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1690003970 0x7f16a4139410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 >> 172.21.15.16:0/3071214932 conn(0x7f16a406a7d0 msgr2=0x7f16a4141f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 shutdown_connections 2023-11-24T06:31:11.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.692+0000 7f16a96ff640 1 -- 172.21.15.16:0/3071214932 wait complete. 2023-11-24T06:31:11.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T06:31:11.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.921+0000 7f1f10a15640 1 Processor -- start 2023-11-24T06:31:11.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.921+0000 7f1f10a15640 1 -- start start 2023-11-24T06:31:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.923+0000 7f1f10a15640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1f0c13df80 0x7f1f0c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.923+0000 7f1f10a15640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c06c470 con 0x7f1f0c0cf9a0 2023-11-24T06:31:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.923+0000 7f1f10a15640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c06c750 con 0x7f1f0c13e360 2023-11-24T06:31:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.923+0000 7f1f10a15640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c06ca30 con 0x7f1f0c13df80 2023-11-24T06:31:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.923+0000 7f1f0ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1f0c0cf9a0 0x7f1f0c0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48432/0 (socket says 172.21.15.16:48432) 2023-11-24T06:31:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.923+0000 7f1f0ad9d640 1 -- 172.21.15.16:0/3495321553 learned_addr learned my addr 172.21.15.16:0/3495321553 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.924+0000 7f1f0a59c640 1 --2- 172.21.15.16:0/3495321553 >> v2:172.21.15.173:3300/0 conn(0x7f1f0c13df80 0x7f1f0c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.924+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877690685 0 0) 0x7f1f0c06c470 con 0x7f1f0c0cf9a0 2023-11-24T06:31:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.924+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef8003520 con 0x7f1f0c0cf9a0 2023-11-24T06:31:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.924+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1847161805 0 0) 0x7f1ef8003520 con 0x7f1f0c0cf9a0 2023-11-24T06:31:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.924+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 >> v2:172.21.15.173:3300/0 conn(0x7f1f0c13df80 msgr2=0x7f1f0c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.924+0000 7f1f09d9b640 1 --2- 172.21.15.16:0/3495321553 >> v2:172.21.15.173:3300/0 conn(0x7f1f0c13df80 0x7f1f0c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.924+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 >> v1:172.21.15.173:6789/0 conn(0x7f1f0c13e360 legacy=0x7f1f0c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.924+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c06cd10 con 0x7f1f0c0cf9a0 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.924+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f1f040019f0 con 0x7f1f0c0cf9a0 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 >> v1:172.21.15.16:6789/0 conn(0x7f1f0c0cf9a0 legacy=0x7f1f0c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f09d9b640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1ef8003970 0x7f1ef8003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f09d9b640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef8004370 0x7f1ef8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f09d9b640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8005040 0x7f1ef800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c06ca30 con 0x7f1ef8004370 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef800bfa0 con 0x7f1ef8005040 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef800c160 con 0x7f1ef8003970 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f03fff640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef8004370 0x7f1ef8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f0a59c640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1ef8003970 0x7f1ef8003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f03fff640 1 -- 172.21.15.16:0/3495321553 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1ef8003970 msgr2=0x7f1ef8003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f03fff640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1ef8003970 0x7f1ef8003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f03fff640 1 -- 172.21.15.16:0/3495321553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8005040 msgr2=0x7f1ef800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.925+0000 7f1f0ad9d640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8005040 0x7f1ef800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.926+0000 7f1f03fff640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8005040 0x7f1ef800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.926+0000 7f1f03fff640 1 -- 172.21.15.16:0/3495321553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8003520 con 0x7f1ef8004370 2023-11-24T06:31:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.926+0000 7f1f0ad9d640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8005040 0x7f1ef800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.926+0000 7f1f0ad9d640 1 -- 172.21.15.16:0/3495321553 reap_dead start 2023-11-24T06:31:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.927+0000 7f1f03fff640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef8004370 0x7f1ef8004840 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40018e0 tx=0x7f1ef80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde7470dbe398408 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.927+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4033630 con 0x7f1ef8004370 2023-11-24T06:31:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.927+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4033790 con 0x7f1ef8004370 2023-11-24T06:31:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.927+0000 7f1f09d9b640 1 -- 172.21.15.16:0/3495321553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4033b60 con 0x7f1ef8004370 2023-11-24T06:31:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.928+0000 7f1f10a15640 1 -- 172.21.15.16:0/3495321553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef8004370 msgr2=0x7f1ef8004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.928+0000 7f1f10a15640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef8004370 0x7f1ef8004840 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40018e0 tx=0x7f1ef80057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.928+0000 7f1f10a15640 1 -- 172.21.15.16:0/3495321553 shutdown_connections 2023-11-24T06:31:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.928+0000 7f1f10a15640 1 --2- 172.21.15.16:0/3495321553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef8004370 0x7f1ef8004840 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.928+0000 7f1f10a15640 1 -- 172.21.15.16:0/3495321553 >> 172.21.15.16:0/3495321553 conn(0x7f1f0c06a800 msgr2=0x7f1f0c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.928+0000 7f1f10a15640 1 -- 172.21.15.16:0/3495321553 shutdown_connections 2023-11-24T06:31:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.928+0000 7f1f10a15640 1 -- 172.21.15.16:0/3495321553 wait complete. 2023-11-24T06:31:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.931+0000 7f1f10a15640 1 Processor -- start 2023-11-24T06:31:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.931+0000 7f1f10a15640 1 -- start start 2023-11-24T06:31:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.932+0000 7f1f10a15640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8004370 0x7f1f0c148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.932+0000 7f1f10a15640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f0c0c4160 0x7f1f0c1f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.932+0000 7f1f10a15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c15d8a0 0x7f1f0c15dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.932+0000 7f1f10a15640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c149940 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.932+0000 7f1f10a15640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c15e260 con 0x7f1ef8004370 2023-11-24T06:31:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.932+0000 7f1f10a15640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c148cc0 con 0x7f1f0c0c4160 2023-11-24T06:31:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.932+0000 7f1f0ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c15d8a0 0x7f1f0c15dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.932+0000 7f1f0ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c15d8a0 0x7f1f0c15dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36586/0 (socket says 172.21.15.16:36586) 2023-11-24T06:31:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.933+0000 7f1f0ad9d640 1 -- 172.21.15.16:0/1495028679 learned_addr learned my addr 172.21.15.16:0/1495028679 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.933+0000 7f1f0a59c640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8004370 0x7f1f0c148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.933+0000 7f1f03fff640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f0c0c4160 0x7f1f0c1f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.933+0000 7f1f0ad9d640 1 -- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f0c0c4160 msgr2=0x7f1f0c1f2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.933+0000 7f1f0ad9d640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f0c0c4160 0x7f1f0c1f2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.933+0000 7f1f0ad9d640 1 -- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8004370 msgr2=0x7f1f0c148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.933+0000 7f1f0ad9d640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8004370 0x7f1f0c148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.933+0000 7f1f0ad9d640 1 -- 172.21.15.16:0/1495028679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c148fa0 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.933+0000 7f1f0ad9d640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c15d8a0 0x7f1f0c15dd20 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0cfea0 tx=0x7f1f0400b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.934+0000 7f1f02ffd640 1 -- 172.21.15.16:0/1495028679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0400b810 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.934+0000 7f1f02ffd640 1 -- 172.21.15.16:0/1495028679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0400b970 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.934+0000 7f1f02ffd640 1 -- 172.21.15.16:0/1495028679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0400bd20 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.934+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c149280 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.934+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c0c45e0 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.935+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c149a80 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.936+0000 7f1f02ffd640 1 -- 172.21.15.16:0/1495028679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1f04007070 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.937+0000 7f1f02ffd640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1eec06b410 0x7f1eec06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.937+0000 7f1f02ffd640 1 -- 172.21.15.16:0/1495028679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f1f0408a0d0 con 0x7f1f0c15d8a0 2023-11-24T06:31:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.938+0000 7f1f0a59c640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1eec06b410 0x7f1eec06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.939+0000 7f1f0a59c640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1eec06b410 0x7f1eec06d8d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009a90 tx=0x7f1efc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:11.941+0000 7f1f02ffd640 1 -- 172.21.15.16:0/1495028679 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f0c149a80 con 0x7f1f0c15d8a0 2023-11-24T06:31:12.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:12.222+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7f1f0c149a80 con 0x7f1f0c15d8a0 2023-11-24T06:31:13.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.226+0000 7f1f02ffd640 1 -- 172.21.15.16:0/1495028679 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v260) v1 ==== 166+0+0 (secure 0 0 0) 0x7f1f0c149a80 con 0x7f1f0c15d8a0 2023-11-24T06:31:13.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1eec06b410 msgr2=0x7f1eec06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1eec06b410 0x7f1eec06d8d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009a90 tx=0x7f1efc008040 comp rx=0 tx=0).stop 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c15d8a0 msgr2=0x7f1f0c15dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c15d8a0 0x7f1f0c15dd20 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0cfea0 tx=0x7f1f0400b0d0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 shutdown_connections 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f1eec06b410 0x7f1eec06d8d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0c15d8a0 0x7f1f0c15dd20 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1f0c0c4160 0x7f1f0c1f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 --2- 172.21.15.16:0/1495028679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ef8004370 0x7f1f0c148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 >> 172.21.15.16:0/1495028679 conn(0x7f1f0c06a800 msgr2=0x7f1f0c0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.230+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 shutdown_connections 2023-11-24T06:31:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.231+0000 7f1f10a15640 1 -- 172.21.15.16:0/1495028679 wait complete. 2023-11-24T06:31:13.255 INFO:tasks.cephfs_test_runner:tests the scenario where 'fs subvolume snapshot info' command ... ok 2023-11-24T06:31:13.255 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:31:13.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.474+0000 7f9a36813640 1 Processor -- start 2023-11-24T06:31:13.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.474+0000 7f9a36813640 1 -- start start 2023-11-24T06:31:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.476+0000 7f9a36813640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a30143c40 0x7f9a30071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.476+0000 7f9a36813640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a300ccc10 0x7f9a3006d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.476+0000 7f9a36813640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a30070e50 con 0x7f9a300717d0 2023-11-24T06:31:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.476+0000 7f9a36813640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a300723c0 con 0x7f9a30143c40 2023-11-24T06:31:13.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.476+0000 7f9a36813640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a30144020 con 0x7f9a300ccc10 2023-11-24T06:31:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.476+0000 7f9a35811640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a30143c40 0x7f9a30071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.476+0000 7f9a35010640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9a300717d0 0x7f9a300cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45674/0 (socket says 172.21.15.16:45674) 2023-11-24T06:31:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.476+0000 7f9a35010640 1 -- 172.21.15.16:0/3662769964 learned_addr learned my addr 172.21.15.16:0/3662769964 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.476+0000 7f9a36012640 1 --2- 172.21.15.16:0/3662769964 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a300ccc10 0x7f9a3006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.477+0000 7f9a35811640 1 -- 172.21.15.16:0/3662769964 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a300ccc10 msgr2=0x7f9a3006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.477+0000 7f9a35811640 1 --2- 172.21.15.16:0/3662769964 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a300ccc10 0x7f9a3006d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.477+0000 7f9a35811640 1 -- 172.21.15.16:0/3662769964 >> v1:172.21.15.16:6789/0 conn(0x7f9a300717d0 legacy=0x7f9a300cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.477+0000 7f9a35811640 1 -- 172.21.15.16:0/3662769964 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30071e00 con 0x7f9a30143c40 2023-11-24T06:31:13.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.477+0000 7f9a36012640 1 --2- 172.21.15.16:0/3662769964 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a300ccc10 0x7f9a3006d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.478+0000 7f9a35811640 1 --2- 172.21.15.16:0/3662769964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30143c40 0x7f9a30071290 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0097a0 tx=0x7f9a2c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69dcad6286b8014 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.478+0000 7f9a3480f640 1 -- 172.21.15.16:0/3662769964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c031780 con 0x7f9a30143c40 2023-11-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.478+0000 7f9a3480f640 1 -- 172.21.15.16:0/3662769964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c0318e0 con 0x7f9a30143c40 2023-11-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.478+0000 7f9a3480f640 1 -- 172.21.15.16:0/3662769964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c031c70 con 0x7f9a30143c40 2023-11-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.479+0000 7f9a36813640 1 -- 172.21.15.16:0/3662769964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30143c40 msgr2=0x7f9a30071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.479+0000 7f9a36813640 1 --2- 172.21.15.16:0/3662769964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30143c40 0x7f9a30071290 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c0097a0 tx=0x7f9a2c02b820 comp rx=0 tx=0).stop 2023-11-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.479+0000 7f9a36813640 1 -- 172.21.15.16:0/3662769964 shutdown_connections 2023-11-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.479+0000 7f9a36813640 1 --2- 172.21.15.16:0/3662769964 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a300ccc10 0x7f9a3006d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.479+0000 7f9a36813640 1 --2- 172.21.15.16:0/3662769964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30143c40 0x7f9a30071290 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.479+0000 7f9a36813640 1 -- 172.21.15.16:0/3662769964 >> 172.21.15.16:0/3662769964 conn(0x7f9a3006a800 msgr2=0x7f9a3006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.479+0000 7f9a36813640 1 -- 172.21.15.16:0/3662769964 shutdown_connections 2023-11-24T06:31:13.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.479+0000 7f9a36813640 1 -- 172.21.15.16:0/3662769964 wait complete. 2023-11-24T06:31:13.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.481+0000 7f9a36813640 1 Processor -- start 2023-11-24T06:31:13.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.481+0000 7f9a36813640 1 -- start start 2023-11-24T06:31:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.482+0000 7f9a36813640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a300717d0 0x7f9a300c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.482+0000 7f9a36813640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a300ccc10 0x7f9a30137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.482+0000 7f9a36813640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a30143c40 0x7f9a3015a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.482+0000 7f9a36813640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a301388b0 con 0x7f9a300717d0 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.482+0000 7f9a36813640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a30138b90 con 0x7f9a300ccc10 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.482+0000 7f9a36813640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a30138e70 con 0x7f9a30143c40 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.482+0000 7f9a35811640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a300717d0 0x7f9a300c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.482+0000 7f9a35811640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a300717d0 0x7f9a300c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33062/0 (socket says 172.21.15.16:33062) 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.482+0000 7f9a35811640 1 -- 172.21.15.16:0/3291357833 learned_addr learned my addr 172.21.15.16:0/3291357833 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.483+0000 7f9a36012640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a30143c40 0x7f9a3015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.483+0000 7f9a35010640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a300ccc10 0x7f9a30137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.483+0000 7f9a35811640 1 -- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a30143c40 msgr2=0x7f9a3015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.483+0000 7f9a35811640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a30143c40 0x7f9a3015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.483+0000 7f9a35811640 1 -- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a300ccc10 msgr2=0x7f9a30137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.483+0000 7f9a35811640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a300ccc10 0x7f9a30137ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.483+0000 7f9a35811640 1 -- 172.21.15.16:0/3291357833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c009400 con 0x7f9a300717d0 2023-11-24T06:31:13.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.483+0000 7f9a35811640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a300717d0 0x7f9a300c7d00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c034040 tx=0x7f9a2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:13.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.484+0000 7f9a267fc640 1 -- 172.21.15.16:0/3291357833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c0329e0 con 0x7f9a300717d0 2023-11-24T06:31:13.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.484+0000 7f9a267fc640 1 -- 172.21.15.16:0/3291357833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c032b60 con 0x7f9a300717d0 2023-11-24T06:31:13.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.484+0000 7f9a267fc640 1 -- 172.21.15.16:0/3291357833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c045400 con 0x7f9a300717d0 2023-11-24T06:31:13.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.484+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a30139150 con 0x7f9a300717d0 2023-11-24T06:31:13.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.484+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a300c8240 con 0x7f9a300717d0 2023-11-24T06:31:13.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.485+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a30139670 con 0x7f9a300717d0 2023-11-24T06:31:13.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.486+0000 7f9a267fc640 1 -- 172.21.15.16:0/3291357833 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f9a2c04e440 con 0x7f9a300717d0 2023-11-24T06:31:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.488+0000 7f9a267fc640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9a1406b3e0 0x7f9a1406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.488+0000 7f9a267fc640 1 -- 172.21.15.16:0/3291357833 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f9a2c0ac1b0 con 0x7f9a300717d0 2023-11-24T06:31:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.488+0000 7f9a35010640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9a1406b3e0 0x7f9a1406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.489+0000 7f9a35010640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9a1406b3e0 0x7f9a1406d8a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9a300c76d0 tx=0x7f9a20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.492+0000 7f9a267fc640 1 -- 172.21.15.16:0/3291357833 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a30139670 con 0x7f9a300717d0 2023-11-24T06:31:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.688+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a30139670 con 0x7f9a300717d0 2023-11-24T06:31:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.690+0000 7f9a267fc640 1 -- 172.21.15.16:0/3291357833 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+13031 (secure 0 0 0) 0x7f9a30139670 con 0x7f9a300717d0 2023-11-24T06:31:13.694 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:13.694 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:11.210875+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2360619406":"2023-11-25T06:31:08.128246+0000","172.21.15.16:6833/2360619406":"2023-11-25T06:31:08.128246+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9a1406b3e0 msgr2=0x7f9a1406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9a1406b3e0 0x7f9a1406d8a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9a300c76d0 tx=0x7f9a20009040 comp rx=0 tx=0).stop 2023-11-24T06:31:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a300717d0 msgr2=0x7f9a300c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a300717d0 0x7f9a300c7d00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c034040 tx=0x7f9a2c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 shutdown_connections 2023-11-24T06:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9a1406b3e0 0x7f9a1406d8a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a30143c40 0x7f9a3015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a300ccc10 0x7f9a30137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 --2- 172.21.15.16:0/3291357833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a300717d0 0x7f9a300c7d00 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 >> 172.21.15.16:0/3291357833 conn(0x7f9a3006a800 msgr2=0x7f9a3013f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 shutdown_connections 2023-11-24T06:31:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.693+0000 7f9a36813640 1 -- 172.21.15.16:0/3291357833 wait complete. 2023-11-24T06:31:13.716 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:31:13.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:31:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.948+0000 7f21a8be9640 1 Processor -- start 2023-11-24T06:31:13.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.949+0000 7f21a8be9640 1 -- start start 2023-11-24T06:31:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.950+0000 7f21a8be9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415ae40 0x7f21a4161610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.951+0000 7f21a8be9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f21a4071f80 0x7f21a415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.951+0000 7f21a8be9640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a415e690 con 0x7f21a415dc30 2023-11-24T06:31:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.951+0000 7f21a8be9640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a415e0d0 con 0x7f21a4071f80 2023-11-24T06:31:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.951+0000 7f21a8be9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a415e3b0 con 0x7f21a415ae40 2023-11-24T06:31:13.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.951+0000 7f21a37fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415ae40 0x7f21a4161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.951+0000 7f21a37fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415ae40 0x7f21a4161610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:57022/0 (socket says 172.21.15.16:57022) 2023-11-24T06:31:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.951+0000 7f21a37fe640 1 -- 172.21.15.16:0/3795697877 learned_addr learned my addr 172.21.15.16:0/3795697877 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.951+0000 7f219bfff640 1 --2- 172.21.15.16:0/3795697877 >> v2:172.21.15.173:3300/0 conn(0x7f21a4071f80 0x7f21a415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.952+0000 7f21a37fe640 1 -- 172.21.15.16:0/3795697877 >> v2:172.21.15.173:3300/0 conn(0x7f21a4071f80 msgr2=0x7f21a415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.952+0000 7f21a37fe640 1 --2- 172.21.15.16:0/3795697877 >> v2:172.21.15.173:3300/0 conn(0x7f21a4071f80 0x7f21a415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.952+0000 7f21a37fe640 1 -- 172.21.15.16:0/3795697877 >> v1:172.21.15.173:6789/0 conn(0x7f21a415dc30 legacy=0x7f21a406a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.952+0000 7f21a37fe640 1 -- 172.21.15.16:0/3795697877 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4160e10 con 0x7f21a415ae40 2023-11-24T06:31:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.954+0000 7f21a37fe640 1 --2- 172.21.15.16:0/3795697877 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415ae40 0x7f21a4161610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2194009870 tx=0x7f219402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e6cb4a24c005ab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.954+0000 7f21a2ffd640 1 -- 172.21.15.16:0/3795697877 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194031640 con 0x7f21a415ae40 2023-11-24T06:31:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.954+0000 7f21a2ffd640 1 -- 172.21.15.16:0/3795697877 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21940317a0 con 0x7f21a415ae40 2023-11-24T06:31:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.954+0000 7f21a2ffd640 1 -- 172.21.15.16:0/3795697877 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194031b30 con 0x7f21a415ae40 2023-11-24T06:31:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.955+0000 7f21a8be9640 1 -- 172.21.15.16:0/3795697877 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415ae40 msgr2=0x7f21a4161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.955+0000 7f21a8be9640 1 --2- 172.21.15.16:0/3795697877 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415ae40 0x7f21a4161610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2194009870 tx=0x7f219402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.955+0000 7f21a8be9640 1 -- 172.21.15.16:0/3795697877 shutdown_connections 2023-11-24T06:31:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.955+0000 7f21a8be9640 1 --2- 172.21.15.16:0/3795697877 >> v2:172.21.15.173:3300/0 conn(0x7f21a4071f80 0x7f21a415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.955+0000 7f21a8be9640 1 --2- 172.21.15.16:0/3795697877 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415ae40 0x7f21a4161610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.955+0000 7f21a8be9640 1 -- 172.21.15.16:0/3795697877 >> 172.21.15.16:0/3795697877 conn(0x7f21a406bb20 msgr2=0x7f21a40cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.955+0000 7f21a8be9640 1 -- 172.21.15.16:0/3795697877 shutdown_connections 2023-11-24T06:31:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.955+0000 7f21a8be9640 1 -- 172.21.15.16:0/3795697877 wait complete. 2023-11-24T06:31:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.957+0000 7f21a8be9640 1 Processor -- start 2023-11-24T06:31:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.957+0000 7f21a8be9640 1 -- start start 2023-11-24T06:31:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.958+0000 7f21a8be9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a4071f80 0x7f21a415bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.958+0000 7f21a8be9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415ae40 0x7f21a41e8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.958+0000 7f21a8be9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415dc30 0x7f21a415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.958+0000 7f21a8be9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a41e9320 con 0x7f21a415ae40 2023-11-24T06:31:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.958+0000 7f21a8be9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a415c2a0 con 0x7f21a4071f80 2023-11-24T06:31:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.958+0000 7f21a8be9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a415c580 con 0x7f21a415dc30 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.958+0000 7f219bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415ae40 0x7f21a41e8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.958+0000 7f219bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415ae40 0x7f21a41e8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33070/0 (socket says 172.21.15.16:33070) 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.958+0000 7f219bfff640 1 -- 172.21.15.16:0/2707431477 learned_addr learned my addr 172.21.15.16:0/2707431477 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f21a37fe640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a4071f80 0x7f21a415bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f21a3fff640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415dc30 0x7f21a415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f219bfff640 1 -- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415dc30 msgr2=0x7f21a415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f219bfff640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415dc30 0x7f21a415f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f219bfff640 1 -- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a4071f80 msgr2=0x7f21a415bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f219bfff640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a4071f80 0x7f21a415bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f219bfff640 1 -- 172.21.15.16:0/2707431477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940094d0 con 0x7f21a415ae40 2023-11-24T06:31:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f219bfff640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415ae40 0x7f21a41e8de0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f218c00e9b0 tx=0x7f218c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f21a0ff9640 1 -- 172.21.15.16:0/2707431477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c009670 con 0x7f21a415ae40 2023-11-24T06:31:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f21a0ff9640 1 -- 172.21.15.16:0/2707431477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c00c9a0 con 0x7f21a415ae40 2023-11-24T06:31:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f21a0ff9640 1 -- 172.21.15.16:0/2707431477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c00cd50 con 0x7f21a415ae40 2023-11-24T06:31:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.959+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a415c860 con 0x7f21a415ae40 2023-11-24T06:31:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.960+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a4150cd0 con 0x7f21a415ae40 2023-11-24T06:31:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.961+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a4067b60 con 0x7f21a415ae40 2023-11-24T06:31:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.962+0000 7f21a0ff9640 1 -- 172.21.15.16:0/2707431477 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f218c013470 con 0x7f21a415ae40 2023-11-24T06:31:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.963+0000 7f21a0ff9640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218406b3e0 0x7f218406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.963+0000 7f21a0ff9640 1 -- 172.21.15.16:0/2707431477 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f218c089f40 con 0x7f21a415ae40 2023-11-24T06:31:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.963+0000 7f21a37fe640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218406b3e0 0x7f218406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.965+0000 7f21a37fe640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218406b3e0 0x7f218406d8a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2194036000 tx=0x7f219403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:13.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:13.968+0000 7f21a0ff9640 1 -- 172.21.15.16:0/2707431477 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f218c054af0 con 0x7f21a415ae40 2023-11-24T06:31:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.159+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21a4071730 con 0x7f21a415ae40 2023-11-24T06:31:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.160+0000 7f21a0ff9640 1 -- 172.21.15.16:0/2707431477 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f218c058110 con 0x7f21a415ae40 2023-11-24T06:31:14.164 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218406b3e0 msgr2=0x7f218406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218406b3e0 0x7f218406d8a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2194036000 tx=0x7f219403c040 comp rx=0 tx=0).stop 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415ae40 msgr2=0x7f21a41e8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415ae40 0x7f21a41e8de0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f218c00e9b0 tx=0x7f218c00ee70 comp rx=0 tx=0).stop 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 shutdown_connections 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f218406b3e0 0x7f218406d8a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f21a415dc30 0x7f21a415f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a415ae40 0x7f21a41e8de0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 --2- 172.21.15.16:0/2707431477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f21a4071f80 0x7f21a415bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 >> 172.21.15.16:0/2707431477 conn(0x7f21a406bb20 msgr2=0x7f21a41561a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 shutdown_connections 2023-11-24T06:31:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.163+0000 7f21a8be9640 1 -- 172.21.15.16:0/2707431477 wait complete. 2023-11-24T06:31:14.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T06:31:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.409+0000 7f184e839640 1 Processor -- start 2023-11-24T06:31:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.410+0000 7f184e839640 1 -- start start 2023-11-24T06:31:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.411+0000 7f184e839640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18480b34e0 0x7f18480b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.411+0000 7f184e839640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f18480b4140 0x7f18480d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.411+0000 7f184e839640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480d1b90 0x7f18480cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.411+0000 7f184e839640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18480d2020 con 0x7f18480b34e0 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.411+0000 7f184e839640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18480d0420 con 0x7f18480b4140 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.411+0000 7f184e839640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18480d0e70 con 0x7f18480d1b90 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d837640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18480b34e0 0x7f18480b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d837640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18480b34e0 0x7f18480b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33072/0 (socket says 172.21.15.16:33072) 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d837640 1 -- 172.21.15.16:0/3915742130 learned_addr learned my addr 172.21.15.16:0/3915742130 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d036640 1 --2- 172.21.15.16:0/3915742130 >> v2:172.21.15.173:3300/0 conn(0x7f18480b4140 0x7f18480d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184e038640 1 --2- 172.21.15.16:0/3915742130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480d1b90 0x7f18480cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d837640 1 -- 172.21.15.16:0/3915742130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480d1b90 msgr2=0x7f18480cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d837640 1 --2- 172.21.15.16:0/3915742130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480d1b90 0x7f18480cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d837640 1 -- 172.21.15.16:0/3915742130 >> v2:172.21.15.173:3300/0 conn(0x7f18480b4140 msgr2=0x7f18480d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d837640 1 --2- 172.21.15.16:0/3915742130 >> v2:172.21.15.173:3300/0 conn(0x7f18480b4140 0x7f18480d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d837640 1 -- 172.21.15.16:0/3915742130 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18480d3d50 con 0x7f18480b34e0 2023-11-24T06:31:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184d036640 1 --2- 172.21.15.16:0/3915742130 >> v2:172.21.15.173:3300/0 conn(0x7f18480b4140 0x7f18480d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.412+0000 7f184e038640 1 --2- 172.21.15.16:0/3915742130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480d1b90 0x7f18480cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.413+0000 7f184d837640 1 --2- 172.21.15.16:0/3915742130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b34e0 0x7f18480b3c00 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f18480a0fa0 tx=0x7f184402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38612b4e056390 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184c835640 1 -- 172.21.15.16:0/3915742130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844031860 con 0x7f18480b34e0 2023-11-24T06:31:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184c835640 1 -- 172.21.15.16:0/3915742130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18440319c0 con 0x7f18480b34e0 2023-11-24T06:31:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184c835640 1 -- 172.21.15.16:0/3915742130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844031d50 con 0x7f18480b34e0 2023-11-24T06:31:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184e839640 1 -- 172.21.15.16:0/3915742130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b34e0 msgr2=0x7f18480b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184e839640 1 --2- 172.21.15.16:0/3915742130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b34e0 0x7f18480b3c00 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f18480a0fa0 tx=0x7f184402b900 comp rx=0 tx=0).stop 2023-11-24T06:31:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184e839640 1 -- 172.21.15.16:0/3915742130 shutdown_connections 2023-11-24T06:31:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184e839640 1 --2- 172.21.15.16:0/3915742130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480d1b90 0x7f18480cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184e839640 1 --2- 172.21.15.16:0/3915742130 >> v2:172.21.15.173:3300/0 conn(0x7f18480b4140 0x7f18480d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184e839640 1 --2- 172.21.15.16:0/3915742130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b34e0 0x7f18480b3c00 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.414+0000 7f184e839640 1 -- 172.21.15.16:0/3915742130 >> 172.21.15.16:0/3915742130 conn(0x7f18480d3090 msgr2=0x7f18480d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.415+0000 7f184e839640 1 -- 172.21.15.16:0/3915742130 shutdown_connections 2023-11-24T06:31:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.415+0000 7f184e839640 1 -- 172.21.15.16:0/3915742130 wait complete. 2023-11-24T06:31:14.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.417+0000 7f184e839640 1 Processor -- start 2023-11-24T06:31:14.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.417+0000 7f184e839640 1 -- start start 2023-11-24T06:31:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.418+0000 7f184e839640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480b34e0 0x7f18480b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.418+0000 7f184e839640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b4140 0x7f18480caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.418+0000 7f184e839640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18480d1b90 0x7f184815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.418+0000 7f184e839640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18480cb410 con 0x7f18480b4140 2023-11-24T06:31:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.418+0000 7f184e839640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18480cb6e0 con 0x7f18480d1b90 2023-11-24T06:31:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.418+0000 7f184e839640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18480b1080 con 0x7f18480b34e0 2023-11-24T06:31:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184d036640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b4140 0x7f18480caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184d036640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b4140 0x7f18480caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33084/0 (socket says 172.21.15.16:33084) 2023-11-24T06:31:14.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184d036640 1 -- 172.21.15.16:0/3086440494 learned_addr learned my addr 172.21.15.16:0/3086440494 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184e038640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18480d1b90 0x7f184815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184d837640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480b34e0 0x7f18480b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184d036640 1 -- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480b34e0 msgr2=0x7f18480b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184d036640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480b34e0 0x7f18480b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184d036640 1 -- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18480d1b90 msgr2=0x7f184815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184d036640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18480d1b90 0x7f184815b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.419+0000 7f184d036640 1 -- 172.21.15.16:0/3086440494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440094d0 con 0x7f18480b4140 2023-11-24T06:31:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.420+0000 7f184d036640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b4140 0x7f18480caed0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f183800c7c0 tx=0x7f183800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.420+0000 7f183e7fc640 1 -- 172.21.15.16:0/3086440494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838007a30 con 0x7f18480b4140 2023-11-24T06:31:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.420+0000 7f183e7fc640 1 -- 172.21.15.16:0/3086440494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1838007b90 con 0x7f18480b4140 2023-11-24T06:31:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.420+0000 7f183e7fc640 1 -- 172.21.15.16:0/3086440494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183800ce70 con 0x7f18480b4140 2023-11-24T06:31:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.420+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18480b1360 con 0x7f18480b4140 2023-11-24T06:31:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.420+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18480cb9c0 con 0x7f18480b4140 2023-11-24T06:31:14.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.421+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18480d1350 con 0x7f18480b4140 2023-11-24T06:31:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.422+0000 7f183e7fc640 1 -- 172.21.15.16:0/3086440494 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f18380113f0 con 0x7f18480b4140 2023-11-24T06:31:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.423+0000 7f183e7fc640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f182c06b410 0x7f182c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.423+0000 7f183e7fc640 1 -- 172.21.15.16:0/3086440494 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f183808a040 con 0x7f18480b4140 2023-11-24T06:31:14.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.424+0000 7f184d837640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f182c06b410 0x7f182c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:14.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.425+0000 7f184d837640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f182c06b410 0x7f182c06d8d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1844036000 tx=0x7f184403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.428+0000 7f183e7fc640 1 -- 172.21.15.16:0/3086440494 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18480d1350 con 0x7f18480b4140 2023-11-24T06:31:14.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:14.695+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7f18480b1880 con 0x7f18480b4140 2023-11-24T06:31:15.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.238+0000 7f183e7fc640 1 -- 172.21.15.16:0/3086440494 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v262) v1 ==== 167+0+0 (secure 0 0 0) 0x7f18480b1880 con 0x7f18480b4140 2023-11-24T06:31:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f182c06b410 msgr2=0x7f182c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:15.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f182c06b410 0x7f182c06d8d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1844036000 tx=0x7f184403c040 comp rx=0 tx=0).stop 2023-11-24T06:31:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b4140 msgr2=0x7f18480caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b4140 0x7f18480caed0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f183800c7c0 tx=0x7f183800cc80 comp rx=0 tx=0).stop 2023-11-24T06:31:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 shutdown_connections 2023-11-24T06:31:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f182c06b410 0x7f182c06d8d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18480d1b90 0x7f184815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18480b4140 0x7f18480caed0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 --2- 172.21.15.16:0/3086440494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18480b34e0 0x7f18480b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 >> 172.21.15.16:0/3086440494 conn(0x7f18480d3090 msgr2=0x7f18480c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 shutdown_connections 2023-11-24T06:31:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.242+0000 7f184e839640 1 -- 172.21.15.16:0/3086440494 wait complete. 2023-11-24T06:31:15.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:31:15.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.483+0000 7fe0a359e640 1 Processor -- start 2023-11-24T06:31:15.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.484+0000 7fe0a359e640 1 -- start start 2023-11-24T06:31:15.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.485+0000 7fe0a359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0a41447f0 0x7fe0a415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:15.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.485+0000 7fe0a359e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe0a406c470 0x7fe0a40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:15.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.485+0000 7fe0a359e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a415c5c0 con 0x7fe0a406dab0 2023-11-24T06:31:15.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.485+0000 7fe0a359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a406d690 con 0x7fe0a41447f0 2023-11-24T06:31:15.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.485+0000 7fe0a359e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a40cebb0 con 0x7fe0a406c470 2023-11-24T06:31:15.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.486+0000 7fe0a259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0a41447f0 0x7fe0a415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:15.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.486+0000 7fe0a259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0a41447f0 0x7fe0a415c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33098/0 (socket says 172.21.15.16:33098) 2023-11-24T06:31:15.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.486+0000 7fe0a259c640 1 -- 172.21.15.16:0/251766282 learned_addr learned my addr 172.21.15.16:0/251766282 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:15.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.486+0000 7fe0a259c640 1 -- 172.21.15.16:0/251766282 >> v2:172.21.15.173:3300/0 conn(0x7fe0a406c470 msgr2=0x7fe0a40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:15.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.486+0000 7fe0a259c640 1 --2- 172.21.15.16:0/251766282 >> v2:172.21.15.173:3300/0 conn(0x7fe0a406c470 0x7fe0a40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.486+0000 7fe0a259c640 1 -- 172.21.15.16:0/251766282 >> v1:172.21.15.173:6789/0 conn(0x7fe0a406dab0 legacy=0x7fe0a415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:15.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.486+0000 7fe0a259c640 1 -- 172.21.15.16:0/251766282 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a406ca10 con 0x7fe0a41447f0 2023-11-24T06:31:15.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.487+0000 7fe0a259c640 1 --2- 172.21.15.16:0/251766282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a41447f0 0x7fe0a415c080 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40cecf0 tx=0x7fe09402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e009d750cc0a26b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.488+0000 7fe0a1d9b640 1 -- 172.21.15.16:0/251766282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940317e0 con 0x7fe0a41447f0 2023-11-24T06:31:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.488+0000 7fe0a1d9b640 1 -- 172.21.15.16:0/251766282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe094031940 con 0x7fe0a41447f0 2023-11-24T06:31:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.488+0000 7fe0a1d9b640 1 -- 172.21.15.16:0/251766282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094031cd0 con 0x7fe0a41447f0 2023-11-24T06:31:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.488+0000 7fe0a359e640 1 -- 172.21.15.16:0/251766282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a41447f0 msgr2=0x7fe0a415c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.488+0000 7fe0a359e640 1 --2- 172.21.15.16:0/251766282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a41447f0 0x7fe0a415c080 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40cecf0 tx=0x7fe09402b880 comp rx=0 tx=0).stop 2023-11-24T06:31:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.488+0000 7fe0a359e640 1 -- 172.21.15.16:0/251766282 shutdown_connections 2023-11-24T06:31:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.488+0000 7fe0a359e640 1 --2- 172.21.15.16:0/251766282 >> v2:172.21.15.173:3300/0 conn(0x7fe0a406c470 0x7fe0a40cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.488+0000 7fe0a359e640 1 --2- 172.21.15.16:0/251766282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a41447f0 0x7fe0a415c080 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.488+0000 7fe0a359e640 1 -- 172.21.15.16:0/251766282 >> 172.21.15.16:0/251766282 conn(0x7fe0a406a3e0 msgr2=0x7fe0a406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:15.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.489+0000 7fe0a359e640 1 -- 172.21.15.16:0/251766282 shutdown_connections 2023-11-24T06:31:15.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.489+0000 7fe0a359e640 1 -- 172.21.15.16:0/251766282 wait complete. 2023-11-24T06:31:15.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.490+0000 7fe0a359e640 1 Processor -- start 2023-11-24T06:31:15.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.490+0000 7fe0a359e640 1 -- start start 2023-11-24T06:31:15.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.491+0000 7fe0a359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0a406c470 0x7fe0a414b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:15.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.491+0000 7fe0a359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a406dab0 0x7fe0a40adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:15.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.491+0000 7fe0a359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0a41447f0 0x7fe0a40abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:15.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.491+0000 7fe0a359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a412bca0 con 0x7fe0a406dab0 2023-11-24T06:31:15.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.491+0000 7fe0a359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a412bf80 con 0x7fe0a41447f0 2023-11-24T06:31:15.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.491+0000 7fe0a359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a412c260 con 0x7fe0a406c470 2023-11-24T06:31:15.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a406dab0 0x7fe0a40adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:15.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe09bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a406dab0 0x7fe0a40adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33112/0 (socket says 172.21.15.16:33112) 2023-11-24T06:31:15.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe09bfff640 1 -- 172.21.15.16:0/3925616646 learned_addr learned my addr 172.21.15.16:0/3925616646 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:15.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe0a259c640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0a406c470 0x7fe0a414b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:15.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe09bfff640 1 -- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0a406c470 msgr2=0x7fe0a414b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:15.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe0a2d9d640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0a41447f0 0x7fe0a40abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:15.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe09bfff640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0a406c470 0x7fe0a414b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe09bfff640 1 -- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0a41447f0 msgr2=0x7fe0a40abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:15.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe09bfff640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0a41447f0 0x7fe0a40abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe09bfff640 1 -- 172.21.15.16:0/3925616646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0940094d0 con 0x7fe0a406dab0 2023-11-24T06:31:15.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.492+0000 7fe0a259c640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0a406c470 0x7fe0a414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:15.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.493+0000 7fe09bfff640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a406dab0 0x7fe0a40adee0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00b4a0 tx=0x7fe08c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:15.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.493+0000 7fe09affd640 1 -- 172.21.15.16:0/3925616646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c010610 con 0x7fe0a406dab0 2023-11-24T06:31:15.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.493+0000 7fe09affd640 1 -- 172.21.15.16:0/3925616646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08c010770 con 0x7fe0a406dab0 2023-11-24T06:31:15.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.493+0000 7fe09affd640 1 -- 172.21.15.16:0/3925616646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c010b20 con 0x7fe0a406dab0 2023-11-24T06:31:15.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.493+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a412c540 con 0x7fe0a406dab0 2023-11-24T06:31:15.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.493+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a415ae50 con 0x7fe0a406dab0 2023-11-24T06:31:15.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.494+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a4067450 con 0x7fe0a406dab0 2023-11-24T06:31:15.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.495+0000 7fe09affd640 1 -- 172.21.15.16:0/3925616646 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe08c010c80 con 0x7fe0a406dab0 2023-11-24T06:31:15.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.496+0000 7fe09affd640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe08406b3e0 0x7fe08406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:15.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.496+0000 7fe09affd640 1 -- 172.21.15.16:0/3925616646 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fe08c089910 con 0x7fe0a406dab0 2023-11-24T06:31:15.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.497+0000 7fe0a259c640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe08406b3e0 0x7fe08406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:15.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.498+0000 7fe0a259c640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe08406b3e0 0x7fe08406d8a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe094036000 tx=0x7fe09403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:15.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.500+0000 7fe09affd640 1 -- 172.21.15.16:0/3925616646 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe08c054480 con 0x7fe0a406dab0 2023-11-24T06:31:15.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.699+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe0a4143dd0 con 0x7fe0a406dab0 2023-11-24T06:31:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.705+0000 7fe09affd640 1 -- 172.21.15.16:0/3925616646 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe08c057aa0 con 0x7fe0a406dab0 2023-11-24T06:31:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe08406b3e0 msgr2=0x7fe08406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe08406b3e0 0x7fe08406d8a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe094036000 tx=0x7fe09403d040 comp rx=0 tx=0).stop 2023-11-24T06:31:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a406dab0 msgr2=0x7fe0a40adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a406dab0 0x7fe0a40adee0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00b4a0 tx=0x7fe08c00b960 comp rx=0 tx=0).stop 2023-11-24T06:31:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 shutdown_connections 2023-11-24T06:31:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe08406b3e0 0x7fe08406d8a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0a41447f0 0x7fe0a40abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a406dab0 0x7fe0a40adee0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 --2- 172.21.15.16:0/3925616646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0a406c470 0x7fe0a414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 >> 172.21.15.16:0/3925616646 conn(0x7fe0a406a3e0 msgr2=0x7fe0a40b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 shutdown_connections 2023-11-24T06:31:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:15.708+0000 7fe0a359e640 1 -- 172.21.15.16:0/3925616646 wait complete. 2023-11-24T06:31:15.733 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:31:15.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:31:15.848 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:31:15.974 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:31:15.975 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:31:15.975 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:16.038 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:31:16.038 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:31:16.038 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:16.098 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:31:16.098 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:31:16.098 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:16.161 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:31:16.161 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:31:16.161 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:16.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:31:16.221 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:31:16.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.454+0000 7f523c0c2640 1 Processor -- start 2023-11-24T06:31:16.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.454+0000 7f523c0c2640 1 -- start start 2023-11-24T06:31:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.455+0000 7f523c0c2640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f523413df80 0x7f523415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.455+0000 7f523c0c2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f523413e360 0x7f52340cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.455+0000 7f523c0c2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523406c470 con 0x7f523415f9c0 2023-11-24T06:31:16.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.455+0000 7f523c0c2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523406c750 con 0x7f523413e360 2023-11-24T06:31:16.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.455+0000 7f523c0c2640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523406ca30 con 0x7f523413df80 2023-11-24T06:31:16.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.456+0000 7f523965d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f523413e360 0x7f52340cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:16.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.456+0000 7f523965d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f523413e360 0x7f52340cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33116/0 (socket says 172.21.15.16:33116) 2023-11-24T06:31:16.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.456+0000 7f523965d640 1 -- 172.21.15.16:0/4005392450 learned_addr learned my addr 172.21.15.16:0/4005392450 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:16.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.456+0000 7f5239e5e640 1 --2- 172.21.15.16:0/4005392450 >> v2:172.21.15.173:3300/0 conn(0x7f523413df80 0x7f523415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.456+0000 7f523965d640 1 -- 172.21.15.16:0/4005392450 >> v2:172.21.15.173:3300/0 conn(0x7f523413df80 msgr2=0x7f523415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.456+0000 7f523965d640 1 --2- 172.21.15.16:0/4005392450 >> v2:172.21.15.173:3300/0 conn(0x7f523413df80 0x7f523415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.456+0000 7f523965d640 1 -- 172.21.15.16:0/4005392450 >> v1:172.21.15.16:6789/0 conn(0x7f523415f9c0 legacy=0x7f52341408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.456+0000 7f523965d640 1 -- 172.21.15.16:0/4005392450 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523406cd10 con 0x7f523413e360 2023-11-24T06:31:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.456+0000 7f5238e5c640 1 -- 172.21.15.16:0/4005392450 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528254978 0 0) 0x7f523406c470 con 0x7f523415f9c0 2023-11-24T06:31:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.458+0000 7f523965d640 1 --2- 172.21.15.16:0/4005392450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413e360 0x7f52340cf290 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5234072770 tx=0x7f522002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7472b14d59e3a898 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.458+0000 7f5238e5c640 1 -- 172.21.15.16:0/4005392450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220030860 con 0x7f523413e360 2023-11-24T06:31:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.458+0000 7f5238e5c640 1 -- 172.21.15.16:0/4005392450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52200309c0 con 0x7f523413e360 2023-11-24T06:31:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.458+0000 7f5238e5c640 1 -- 172.21.15.16:0/4005392450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220030d50 con 0x7f523413e360 2023-11-24T06:31:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.459+0000 7f523c0c2640 1 -- 172.21.15.16:0/4005392450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413e360 msgr2=0x7f52340cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.459+0000 7f523c0c2640 1 --2- 172.21.15.16:0/4005392450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413e360 0x7f52340cf290 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5234072770 tx=0x7f522002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.459+0000 7f523c0c2640 1 -- 172.21.15.16:0/4005392450 shutdown_connections 2023-11-24T06:31:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.459+0000 7f523c0c2640 1 --2- 172.21.15.16:0/4005392450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413e360 0x7f52340cf290 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.459+0000 7f523c0c2640 1 --2- 172.21.15.16:0/4005392450 >> v2:172.21.15.173:3300/0 conn(0x7f523413df80 0x7f523415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.459+0000 7f523c0c2640 1 -- 172.21.15.16:0/4005392450 >> 172.21.15.16:0/4005392450 conn(0x7f523406a800 msgr2=0x7f523406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.459+0000 7f523c0c2640 1 -- 172.21.15.16:0/4005392450 shutdown_connections 2023-11-24T06:31:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.459+0000 7f523c0c2640 1 -- 172.21.15.16:0/4005392450 wait complete. 2023-11-24T06:31:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.461+0000 7f523c0c2640 1 Processor -- start 2023-11-24T06:31:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.461+0000 7f523c0c2640 1 -- start start 2023-11-24T06:31:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.462+0000 7f523c0c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413df80 0x7f523415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.462+0000 7f523c0c2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523413e360 0x7f523415dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.462+0000 7f523c0c2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f523415f9c0 0x7f52341517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.462+0000 7f523c0c2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234151020 con 0x7f523413df80 2023-11-24T06:31:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.462+0000 7f523c0c2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523414d090 con 0x7f523415f9c0 2023-11-24T06:31:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.462+0000 7f523c0c2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523414d350 con 0x7f523413e360 2023-11-24T06:31:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.462+0000 7f5239e5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413df80 0x7f523415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.462+0000 7f5239e5e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413df80 0x7f523415d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33118/0 (socket says 172.21.15.16:33118) 2023-11-24T06:31:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.462+0000 7f5239e5e640 1 -- 172.21.15.16:0/3505852215 learned_addr learned my addr 172.21.15.16:0/3505852215 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f5239e5e640 1 -- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523413e360 msgr2=0x7f523415dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f523965d640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523413e360 0x7f523415dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f523a65f640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f523415f9c0 0x7f52341517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f5239e5e640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523413e360 0x7f523415dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f5239e5e640 1 -- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f523415f9c0 msgr2=0x7f52341517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f5239e5e640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f523415f9c0 0x7f52341517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f5239e5e640 1 -- 172.21.15.16:0/3505852215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52200094d0 con 0x7f523413df80 2023-11-24T06:31:16.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f523965d640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523413e360 0x7f523415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:16.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f5239e5e640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413df80 0x7f523415d8a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5228007e40 tx=0x7f522800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:16.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f52327fc640 1 -- 172.21.15.16:0/3505852215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228014620 con 0x7f523413df80 2023-11-24T06:31:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f52327fc640 1 -- 172.21.15.16:0/3505852215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522800ee90 con 0x7f523413df80 2023-11-24T06:31:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523414d630 con 0x7f523413df80 2023-11-24T06:31:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.463+0000 7f52327fc640 1 -- 172.21.15.16:0/3505852215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280149c0 con 0x7f523413df80 2023-11-24T06:31:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.464+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52341f2d80 con 0x7f523413df80 2023-11-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.465+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52340675d0 con 0x7f523413df80 2023-11-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.466+0000 7f52327fc640 1 -- 172.21.15.16:0/3505852215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5228014b20 con 0x7f523413df80 2023-11-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.467+0000 7f52327fc640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f521806b3e0 0x7f521806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.467+0000 7f52327fc640 1 -- 172.21.15.16:0/3505852215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f522808a6d0 con 0x7f523413df80 2023-11-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.468+0000 7f523965d640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f521806b3e0 0x7f521806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.469+0000 7f523965d640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f521806b3e0 0x7f521806d8a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5220036000 tx=0x7f522003b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.472+0000 7f52327fc640 1 -- 172.21.15.16:0/3505852215 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5228055240 con 0x7f523413df80 2023-11-24T06:31:16.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.707+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52340722b0 con 0x7f523413df80 2023-11-24T06:31:16.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.710+0000 7f52327fc640 1 -- 172.21.15.16:0/3505852215 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f5228058860 con 0x7f523413df80 2023-11-24T06:31:16.714 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:16.714 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5228,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/1164611025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1164611025},{"type":"v1","addr":"172.21.15.16:6834","nonce":1164611025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5650,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/1157106761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1157106761},{"type":"v1","addr":"172.21.15.173:6836","nonce":1157106761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":5653,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/151946729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":151946729},{"type":"v1","addr":"172.21.15.173:6837","nonce":151946729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3095186010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3095186010},{"type":"v1","addr":"172.21.15.16:6835","nonce":3095186010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[]} 2023-11-24T06:31:16.715 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 47 2023-11-24T06:31:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.712+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f521806b3e0 msgr2=0x7f521806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.712+0000 7f523c0c2640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f521806b3e0 0x7f521806d8a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5220036000 tx=0x7f522003b040 comp rx=0 tx=0).stop 2023-11-24T06:31:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.712+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413df80 msgr2=0x7f523415d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:16.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.712+0000 7f523c0c2640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413df80 0x7f523415d8a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5228007e40 tx=0x7f522800e4b0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.713+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 shutdown_connections 2023-11-24T06:31:16.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.713+0000 7f523c0c2640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f521806b3e0 0x7f521806d8a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.713+0000 7f523c0c2640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f523415f9c0 0x7f52341517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.713+0000 7f523c0c2640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523413e360 0x7f523415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.713+0000 7f523c0c2640 1 --2- 172.21.15.16:0/3505852215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f523413df80 0x7f523415d8a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.713+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 >> 172.21.15.16:0/3505852215 conn(0x7f523406a800 msgr2=0x7f52340b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.713+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 shutdown_connections 2023-11-24T06:31:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:16.713+0000 7f523c0c2640 1 -- 172.21.15.16:0/3505852215 wait complete. 2023-11-24T06:31:16.737 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:31:16.737 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:31:16.738 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:31:16.738 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:31:16.738 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:31:16.739 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:31:16.739 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:31:16.739 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:31:16.739 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:31:16.739 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:31:16.740 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:31:16.740 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:31:16.740 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:31:16.740 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:31:16.740 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:31:16.740 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:31:16.756 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:31:16.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:31:16.758 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:31:16.759 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:31:16.763 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:31:16.763 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:31:16.765 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:31:16.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:31:16.811 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:31:16.811 DEBUG:teuthology.parallel:result is None 2023-11-24T06:31:16.813 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:31:16.814 DEBUG:teuthology.parallel:result is None 2023-11-24T06:31:16.819 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:31:16.819 DEBUG:teuthology.parallel:result is None 2023-11-24T06:31:16.821 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:31:16.821 DEBUG:teuthology.parallel:result is None 2023-11-24T06:31:16.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:31:16.978 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-11-24T06:31:16.979 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:31:16.979 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2023-11-24T06:31:16.979 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2023-11-24T06:31:17.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.013+0000 7f6a1f7a9640 1 Processor -- start 2023-11-24T06:31:17.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.014+0000 7f6a1f7a9640 1 -- start start 2023-11-24T06:31:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.015+0000 7f6a1f7a9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a1806b690 0x7f6a1815dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:17.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.015+0000 7f6a1f7a9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1815e270 0x7f6a18070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.015+0000 7f6a1f7a9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a18154490 con 0x7f6a181540b0 2023-11-24T06:31:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.015+0000 7f6a1f7a9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1815d550 con 0x7f6a1806b690 2023-11-24T06:31:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.015+0000 7f6a1f7a9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1815a7b0 con 0x7f6a1815e270 2023-11-24T06:31:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.015+0000 7f6a1cd44640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a1806b690 0x7f6a1815dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.015+0000 7f6a1d545640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a181540b0 0x7f6a18159f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45714/0 (socket says 172.21.15.16:45714) 2023-11-24T06:31:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.015+0000 7f6a1d545640 1 -- 172.21.15.16:0/1398455886 learned_addr learned my addr 172.21.15.16:0/1398455886 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:17.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.016+0000 7f6a1dd46640 1 --2- 172.21.15.16:0/1398455886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1815e270 0x7f6a18070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.016+0000 7f6a1cd44640 1 -- 172.21.15.16:0/1398455886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1815e270 msgr2=0x7f6a18070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.016+0000 7f6a1cd44640 1 --2- 172.21.15.16:0/1398455886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1815e270 0x7f6a18070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.016+0000 7f6a1cd44640 1 -- 172.21.15.16:0/1398455886 >> v1:172.21.15.16:6789/0 conn(0x7f6a181540b0 legacy=0x7f6a18159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.016+0000 7f6a1cd44640 1 -- 172.21.15.16:0/1398455886 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1815aa90 con 0x7f6a1806b690 2023-11-24T06:31:17.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.016+0000 7f6a1dd46640 1 --2- 172.21.15.16:0/1398455886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1815e270 0x7f6a18070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:17.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.017+0000 7f6a1cd44640 1 --2- 172.21.15.16:0/1398455886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1806b690 0x7f6a1815dd30 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6a080097a0 tx=0x7f6a0802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aca7bb8fbd81dab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:17.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.017+0000 7f6a0ffff640 1 -- 172.21.15.16:0/1398455886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0802d040 con 0x7f6a1806b690 2023-11-24T06:31:17.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.017+0000 7f6a0ffff640 1 -- 172.21.15.16:0/1398455886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a08031810 con 0x7f6a1806b690 2023-11-24T06:31:17.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.017+0000 7f6a0ffff640 1 -- 172.21.15.16:0/1398455886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08031ba0 con 0x7f6a1806b690 2023-11-24T06:31:17.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.018+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/1398455886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1806b690 msgr2=0x7f6a1815dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:17.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.018+0000 7f6a1f7a9640 1 --2- 172.21.15.16:0/1398455886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1806b690 0x7f6a1815dd30 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6a080097a0 tx=0x7f6a0802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:31:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.018+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/1398455886 shutdown_connections 2023-11-24T06:31:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.018+0000 7f6a1f7a9640 1 --2- 172.21.15.16:0/1398455886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1815e270 0x7f6a18070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.018+0000 7f6a1f7a9640 1 --2- 172.21.15.16:0/1398455886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a1806b690 0x7f6a1815dd30 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.018+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/1398455886 >> 172.21.15.16:0/1398455886 conn(0x7f6a180cf700 msgr2=0x7f6a180cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.018+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/1398455886 shutdown_connections 2023-11-24T06:31:17.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.018+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/1398455886 wait complete. 2023-11-24T06:31:17.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.020+0000 7f6a1f7a9640 1 Processor -- start 2023-11-24T06:31:17.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.020+0000 7f6a1f7a9640 1 -- start start 2023-11-24T06:31:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1f7a9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1806b690 0x7f6a1814c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1f7a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a181540b0 0x7f6a180ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1f7a9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a180ce950 0x7f6a1814f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1f7a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1814cba0 con 0x7f6a181540b0 2023-11-24T06:31:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1f7a9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a180cedd0 con 0x7f6a180ce950 2023-11-24T06:31:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1f7a9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a18158210 con 0x7f6a1806b690 2023-11-24T06:31:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1cd44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a181540b0 0x7f6a180ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:17.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1d545640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1806b690 0x7f6a1814c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1cd44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a181540b0 0x7f6a180ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33172/0 (socket says 172.21.15.16:33172) 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1cd44640 1 -- 172.21.15.16:0/2158032334 learned_addr learned my addr 172.21.15.16:0/2158032334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.021+0000 7f6a1dd46640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a180ce950 0x7f6a1814f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a1cd44640 1 -- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1806b690 msgr2=0x7f6a1814c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a1cd44640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1806b690 0x7f6a1814c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a1cd44640 1 -- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a180ce950 msgr2=0x7f6a1814f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a1cd44640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a180ce950 0x7f6a1814f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a1cd44640 1 -- 172.21.15.16:0/2158032334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a08009400 con 0x7f6a181540b0 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a1d545640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1806b690 0x7f6a1814c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a1dd46640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a180ce950 0x7f6a1814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:31:17.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a1cd44640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a181540b0 0x7f6a180ce410 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6a080098c0 tx=0x7f6a080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a0dffb640 1 -- 172.21.15.16:0/2158032334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0803ad60 con 0x7f6a181540b0 2023-11-24T06:31:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a0dffb640 1 -- 172.21.15.16:0/2158032334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a08044400 con 0x7f6a181540b0 2023-11-24T06:31:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a0dffb640 1 -- 172.21.15.16:0/2158032334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a080447b0 con 0x7f6a181540b0 2023-11-24T06:31:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.022+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a18150680 con 0x7f6a181540b0 2023-11-24T06:31:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.023+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a181584b0 con 0x7f6a181540b0 2023-11-24T06:31:17.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.024+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a180675d0 con 0x7f6a181540b0 2023-11-24T06:31:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.025+0000 7f6a0dffb640 1 -- 172.21.15.16:0/2158032334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6a0804d440 con 0x7f6a181540b0 2023-11-24T06:31:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.026+0000 7f6a0dffb640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b410 0x7f69fc06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:17.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.026+0000 7f6a0dffb640 1 -- 172.21.15.16:0/2158032334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f6a080ac220 con 0x7f6a181540b0 2023-11-24T06:31:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.027+0000 7f6a1d545640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b410 0x7f69fc06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:17.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.027+0000 7f6a1d545640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b410 0x7f69fc06d8d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6a14005880 tx=0x7f6a14009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:17.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.031+0000 7f6a0dffb640 1 -- 172.21.15.16:0/2158032334 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a08076da0 con 0x7f6a181540b0 2023-11-24T06:31:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:17.242+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6a180cf400 con 0x7f6a181540b0 2023-11-24T06:31:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.052+0000 7f6a0dffb640 1 -- 172.21.15.16:0/2158032334 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v82) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6a0807a3c0 con 0x7f6a181540b0 2023-11-24T06:31:18.055 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-11-24T06:31:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b410 msgr2=0x7f69fc06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b410 0x7f69fc06d8d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6a14005880 tx=0x7f6a14009040 comp rx=0 tx=0).stop 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a181540b0 msgr2=0x7f6a180ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a181540b0 0x7f6a180ce410 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6a080098c0 tx=0x7f6a080046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 shutdown_connections 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f69fc06b410 0x7f69fc06d8d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a180ce950 0x7f6a1814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a181540b0 0x7f6a180ce410 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 --2- 172.21.15.16:0/2158032334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a1806b690 0x7f6a1814c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 >> 172.21.15.16:0/2158032334 conn(0x7f6a180cf700 msgr2=0x7f6a180cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 shutdown_connections 2023-11-24T06:31:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.056+0000 7f6a1f7a9640 1 -- 172.21.15.16:0/2158032334 wait complete. 2023-11-24T06:31:18.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:31:18.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.303+0000 7f3c9581a640 1 Processor -- start 2023-11-24T06:31:18.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.303+0000 7f3c9581a640 1 -- start start 2023-11-24T06:31:18.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.305+0000 7f3c9581a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3c90071ce0 0x7f3c900ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.305+0000 7f3c9581a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c900b3490 con 0x7f3c90146850 2023-11-24T06:31:18.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.305+0000 7f3c9581a640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c900b3770 con 0x7f3c900adb00 2023-11-24T06:31:18.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.305+0000 7f3c9581a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c900b3a50 con 0x7f3c90071ce0 2023-11-24T06:31:18.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.305+0000 7f3c94818640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3c90146850 0x7f3c900715d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45718/0 (socket says 172.21.15.16:45718) 2023-11-24T06:31:18.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.305+0000 7f3c94818640 1 -- 172.21.15.16:0/3163503550 learned_addr learned my addr 172.21.15.16:0/3163503550 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.306+0000 7f3c8ffff640 1 --2- 172.21.15.16:0/3163503550 >> v2:172.21.15.173:3300/0 conn(0x7f3c90071ce0 0x7f3c900ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.306+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474625158 0 0) 0x7f3c900b3490 con 0x7f3c90146850 2023-11-24T06:31:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.306+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c80003520 con 0x7f3c90146850 2023-11-24T06:31:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.306+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2427263396 0 0) 0x7f3c80003520 con 0x7f3c90146850 2023-11-24T06:31:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.306+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 >> v2:172.21.15.173:3300/0 conn(0x7f3c90071ce0 msgr2=0x7f3c900ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.306+0000 7f3c8f7fe640 1 --2- 172.21.15.16:0/3163503550 >> v2:172.21.15.173:3300/0 conn(0x7f3c90071ce0 0x7f3c900ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.306+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 >> v1:172.21.15.173:6789/0 conn(0x7f3c900adb00 legacy=0x7f3c9006c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900b3d30 con 0x7f3c90146850 2023-11-24T06:31:18.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f3c84001670 con 0x7f3c90146850 2023-11-24T06:31:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 >> v1:172.21.15.16:6789/0 conn(0x7f3c90146850 legacy=0x7f3c900715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8f7fe640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c80003970 0x7f3c80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8f7fe640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c800047c0 0x7f3c80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8f7fe640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c80005180 0x7f3c8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900b3a50 con 0x7f3c800047c0 2023-11-24T06:31:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8000c020 con 0x7f3c80003970 2023-11-24T06:31:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8000c1e0 con 0x7f3c80005180 2023-11-24T06:31:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8ffff640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c800047c0 0x7f3c80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c94818640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c80003970 0x7f3c80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8ffff640 1 -- 172.21.15.16:0/3163503550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c80005180 msgr2=0x7f3c8000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8ffff640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c80005180 0x7f3c8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.307+0000 7f3c8ffff640 1 -- 172.21.15.16:0/3163503550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c80003970 msgr2=0x7f3c80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.308+0000 7f3c8ffff640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c80003970 0x7f3c80003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.308+0000 7f3c8ffff640 1 -- 172.21.15.16:0/3163503550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c80003520 con 0x7f3c800047c0 2023-11-24T06:31:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.308+0000 7f3c95019640 1 -- 172.21.15.16:0/3163503550 reap_dead start 2023-11-24T06:31:18.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.308+0000 7f3c94818640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c80003970 0x7f3c80003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:18.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.309+0000 7f3c8ffff640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c800047c0 0x7f3c80004c40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009a90 tx=0x7f3c800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a7f0b0bcb9f54f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:18.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.309+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c02c680 con 0x7f3c800047c0 2023-11-24T06:31:18.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.309+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c02c7e0 con 0x7f3c800047c0 2023-11-24T06:31:18.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.309+0000 7f3c8f7fe640 1 -- 172.21.15.16:0/3163503550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c02cb70 con 0x7f3c800047c0 2023-11-24T06:31:18.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.310+0000 7f3c9581a640 1 -- 172.21.15.16:0/3163503550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c800047c0 msgr2=0x7f3c80004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.310+0000 7f3c9581a640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c800047c0 0x7f3c80004c40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009a90 tx=0x7f3c800057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.310+0000 7f3c9581a640 1 -- 172.21.15.16:0/3163503550 shutdown_connections 2023-11-24T06:31:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.310+0000 7f3c9581a640 1 --2- 172.21.15.16:0/3163503550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c800047c0 0x7f3c80004c40 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.310+0000 7f3c9581a640 1 -- 172.21.15.16:0/3163503550 >> 172.21.15.16:0/3163503550 conn(0x7f3c9006a3e0 msgr2=0x7f3c9006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.310+0000 7f3c9581a640 1 -- 172.21.15.16:0/3163503550 shutdown_connections 2023-11-24T06:31:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.310+0000 7f3c9581a640 1 -- 172.21.15.16:0/3163503550 wait complete. 2023-11-24T06:31:18.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.312+0000 7f3c9581a640 1 Processor -- start 2023-11-24T06:31:18.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.312+0000 7f3c9581a640 1 -- start start 2023-11-24T06:31:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.313+0000 7f3c9581a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c800047c0 0x7f3c90138f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.313+0000 7f3c9581a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c900c3de0 0x7f3c901386b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.313+0000 7f3c9581a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9015a5b0 0x7f3c9015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.313+0000 7f3c9581a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c90138bf0 con 0x7f3c9015a5b0 2023-11-24T06:31:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.313+0000 7f3c9581a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900c4270 con 0x7f3c800047c0 2023-11-24T06:31:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.313+0000 7f3c9581a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900c4550 con 0x7f3c900c3de0 2023-11-24T06:31:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.313+0000 7f3c95019640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9015a5b0 0x7f3c9015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.313+0000 7f3c95019640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9015a5b0 0x7f3c9015bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33188/0 (socket says 172.21.15.16:33188) 2023-11-24T06:31:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.313+0000 7f3c95019640 1 -- 172.21.15.16:0/1876136541 learned_addr learned my addr 172.21.15.16:0/1876136541 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c94818640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c800047c0 0x7f3c90138f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c95019640 1 -- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c900c3de0 msgr2=0x7f3c901386b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c95019640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c900c3de0 0x7f3c901386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c95019640 1 -- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c800047c0 msgr2=0x7f3c90138f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c95019640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c800047c0 0x7f3c90138f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c95019640 1 -- 172.21.15.16:0/1876136541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900c4830 con 0x7f3c9015a5b0 2023-11-24T06:31:18.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c95019640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9015a5b0 0x7f3c9015bcf0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3c900ad470 tx=0x7f3c8800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c8d7fa640 1 -- 172.21.15.16:0/1876136541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88005d40 con 0x7f3c9015a5b0 2023-11-24T06:31:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c8d7fa640 1 -- 172.21.15.16:0/1876136541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8800ced0 con 0x7f3c9015a5b0 2023-11-24T06:31:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c8d7fa640 1 -- 172.21.15.16:0/1876136541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8800a850 con 0x7f3c9015a5b0 2023-11-24T06:31:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c900c4b10 con 0x7f3c9015a5b0 2023-11-24T06:31:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.314+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9015aa60 con 0x7f3c9015a5b0 2023-11-24T06:31:18.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.316+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c900c5030 con 0x7f3c9015a5b0 2023-11-24T06:31:18.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.316+0000 7f3c8d7fa640 1 -- 172.21.15.16:0/1876136541 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f3c8800a9b0 con 0x7f3c9015a5b0 2023-11-24T06:31:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.317+0000 7f3c8d7fa640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3c7406b410 0x7f3c7406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.318+0000 7f3c8d7fa640 1 -- 172.21.15.16:0/1876136541 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f3c88088ca0 con 0x7f3c9015a5b0 2023-11-24T06:31:18.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.318+0000 7f3c94818640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3c7406b410 0x7f3c7406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.319+0000 7f3c94818640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3c7406b410 0x7f3c7406d8d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3c90050140 tx=0x7f3c84000ef0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:18.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.322+0000 7f3c8d7fa640 1 -- 172.21.15.16:0/1876136541 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c900c5030 con 0x7f3c9015a5b0 2023-11-24T06:31:18.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.581+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3c900c5030 con 0x7f3c9015a5b0 2023-11-24T06:31:18.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.582+0000 7f3c8d7fa640 1 -- 172.21.15.16:0/1876136541 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3c900c5030 con 0x7f3c9015a5b0 2023-11-24T06:31:18.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:18.585 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:31:18.585 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBl2seAMxAAL8jbBlU0yStjuIivg2ehGQ==", 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.586 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBlbyW+GxAAy15inYIiNwnGVHj7Ghs5hg==", 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-11-24T06:31:18.587 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl79LhCRAALZnwoYyKju4aeWNS+jF31A==", 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBl/NBDLBAAR4qYgD5THE/yNeZT9XgK5g==", 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:31:18.588 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:31:18.589 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:31:18.589 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.589 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.589 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-11-24T06:31:18.589 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD4QWBlarihIxAAczpFVQAODZogIKH0TTdMhw==", 2023-11-24T06:31:18.589 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD8QWBlITYFAhAAMlJyCTojY7WZihUegTcBcQ==", 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:31:18.590 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD/QWBlA/N0DRAAThU0hfAL0+MggufKZC4K8g==", 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.591 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQACQmBlM7E3JxAAdmfhQHbjK/JGHgeD0urDZw==", 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-11-24T06:31:18.592 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAKQmBlrM97ARAApYaqyASXJqB5ITNLeBec/w==", 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQANQmBl3s5dHBAA3spWOZQiqazGS6isl8ft3Q==", 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:31:18.593 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAQQmBlKVmeNhAAuV5j9qZ2NNvEePQx77EM6Q==", 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQAUQmBlwADrBRAAvS3M6+H3X+OQD1h6xwhnBQ==", 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA==", 2023-11-24T06:31:18.595 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBlKhfoFxAA8A+MObA/cjawPN7aclqS9Q==", 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:31:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBliwu9CxAAPcBaB9CWK2QqRD1k2x+4ig==", 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.597 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD2QWBl8HWRIhAAqzUXni0Uqgbmi/imvnNIZg==", 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-11-24T06:31:18.598 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDzQWBlo6bBIRAADWG8Peciy2ws4wHagrfFZw==", 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlz+KlJhAAXtiyVVA72YrbDKpNSBUnwA==", 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.599 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD0QWBlN4GDGhAAF9YIRqu011Xj6oWUv9LYlw==", 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.600 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:18.601 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-11-24T06:31:18.601 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3c7406b410 msgr2=0x7f3c7406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3c7406b410 0x7f3c7406d8d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3c90050140 tx=0x7f3c84000ef0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9015a5b0 msgr2=0x7f3c9015bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9015a5b0 0x7f3c9015bcf0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3c900ad470 tx=0x7f3c8800cb90 comp rx=0 tx=0).stop 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 shutdown_connections 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f3c7406b410 0x7f3c7406d8d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9015a5b0 0x7f3c9015bcf0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c900c3de0 0x7f3c901386b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 --2- 172.21.15.16:0/1876136541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c800047c0 0x7f3c90138f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 >> 172.21.15.16:0/1876136541 conn(0x7f3c9006a3e0 msgr2=0x7f3c900ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:18.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.584+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 shutdown_connections 2023-11-24T06:31:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.585+0000 7f3c9581a640 1 -- 172.21.15.16:0/1876136541 wait complete. 2023-11-24T06:31:18.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.832+0000 7f48f1d73640 1 Processor -- start 2023-11-24T06:31:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.832+0000 7f48f1d73640 1 -- start start 2023-11-24T06:31:18.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.834+0000 7f48f1d73640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48ec143c40 0x7f48ec071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.834+0000 7f48f1d73640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 0x7f48ec0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.834+0000 7f48f1d73640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ec070e50 con 0x7f48ec0cca40 2023-11-24T06:31:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.834+0000 7f48f1d73640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ec0723c0 con 0x7f48ec143c40 2023-11-24T06:31:18.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.834+0000 7f48f1d73640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0cce70 con 0x7f48ec0717d0 2023-11-24T06:31:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.835+0000 7f48f1572640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f48ec0cca40 0x7f48ec15eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45730/0 (socket says 172.21.15.16:45730) 2023-11-24T06:31:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.835+0000 7f48f1572640 1 -- 172.21.15.16:0/3419349394 learned_addr learned my addr 172.21.15.16:0/3419349394 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:18.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.835+0000 7f48f0d71640 1 --2- 172.21.15.16:0/3419349394 >> v2:172.21.15.16:3300/0 conn(0x7f48ec143c40 0x7f48ec071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.835+0000 7f48ebfff640 1 --2- 172.21.15.16:0/3419349394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 0x7f48ec0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.835+0000 7f48eb7fe640 1 -- 172.21.15.16:0/3419349394 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540392585 0 0) 0x7f48ec070e50 con 0x7f48ec0cca40 2023-11-24T06:31:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.835+0000 7f48f0d71640 1 -- 172.21.15.16:0/3419349394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 msgr2=0x7f48ec0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.835+0000 7f48f0d71640 1 --2- 172.21.15.16:0/3419349394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 0x7f48ec0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.835+0000 7f48f0d71640 1 -- 172.21.15.16:0/3419349394 >> v1:172.21.15.16:6789/0 conn(0x7f48ec0cca40 legacy=0x7f48ec15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.835+0000 7f48f0d71640 1 -- 172.21.15.16:0/3419349394 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec06e170 con 0x7f48ec143c40 2023-11-24T06:31:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.837+0000 7f48f0d71640 1 --2- 172.21.15.16:0/3419349394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 0x7f48ec071290 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f48e00097a0 tx=0x7f48e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0f5918c30aaa55 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48eb7fe640 1 -- 172.21.15.16:0/3419349394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e002e070 con 0x7f48ec143c40 2023-11-24T06:31:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48eb7fe640 1 -- 172.21.15.16:0/3419349394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e00315a0 con 0x7f48ec143c40 2023-11-24T06:31:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48eb7fe640 1 -- 172.21.15.16:0/3419349394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0031930 con 0x7f48ec143c40 2023-11-24T06:31:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48f1d73640 1 -- 172.21.15.16:0/3419349394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 msgr2=0x7f48ec071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48f1d73640 1 --2- 172.21.15.16:0/3419349394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 0x7f48ec071290 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f48e00097a0 tx=0x7f48e002b840 comp rx=0 tx=0).stop 2023-11-24T06:31:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48f1d73640 1 -- 172.21.15.16:0/3419349394 shutdown_connections 2023-11-24T06:31:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48f1d73640 1 --2- 172.21.15.16:0/3419349394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 0x7f48ec0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48f1d73640 1 --2- 172.21.15.16:0/3419349394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 0x7f48ec071290 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48f1d73640 1 -- 172.21.15.16:0/3419349394 >> 172.21.15.16:0/3419349394 conn(0x7f48ec06a800 msgr2=0x7f48ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48f1d73640 1 -- 172.21.15.16:0/3419349394 shutdown_connections 2023-11-24T06:31:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.838+0000 7f48f1d73640 1 -- 172.21.15.16:0/3419349394 wait complete. 2023-11-24T06:31:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.840+0000 7f48f1d73640 1 Processor -- start 2023-11-24T06:31:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.840+0000 7f48f1d73640 1 -- start start 2023-11-24T06:31:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.841+0000 7f48f1d73640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 0x7f48ec0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.841+0000 7f48f1d73640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48ec0cca40 0x7f48ec142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.841+0000 7f48f1d73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 0x7f48ec1427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.841+0000 7f48f1d73640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0c55e0 con 0x7f48ec143c40 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.841+0000 7f48f1d73640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0cb020 con 0x7f48ec0cca40 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.841+0000 7f48f1d73640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ec0cb300 con 0x7f48ec0717d0 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f1572640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 0x7f48ec1427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f1572640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 0x7f48ec1427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33216/0 (socket says 172.21.15.16:33216) 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f1572640 1 -- 172.21.15.16:0/2660953584 learned_addr learned my addr 172.21.15.16:0/2660953584 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48ebfff640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48ec0cca40 0x7f48ec142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f0d71640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 0x7f48ec0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f1572640 1 -- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 msgr2=0x7f48ec0c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f1572640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 0x7f48ec0c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f1572640 1 -- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48ec0cca40 msgr2=0x7f48ec142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f1572640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48ec0cca40 0x7f48ec142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f1572640 1 -- 172.21.15.16:0/2660953584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e0009400 con 0x7f48ec143c40 2023-11-24T06:31:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.842+0000 7f48f0d71640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 0x7f48ec0c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.843+0000 7f48f1572640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 0x7f48ec1427c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c7a0 tx=0x7f48e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.843+0000 7f48e97fa640 1 -- 172.21.15.16:0/2660953584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e400a640 con 0x7f48ec143c40 2023-11-24T06:31:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.843+0000 7f48e97fa640 1 -- 172.21.15.16:0/2660953584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e400a7a0 con 0x7f48ec143c40 2023-11-24T06:31:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.843+0000 7f48e97fa640 1 -- 172.21.15.16:0/2660953584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e400abd0 con 0x7f48ec143c40 2023-11-24T06:31:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.843+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ec0c5830 con 0x7f48ec143c40 2023-11-24T06:31:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.843+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ec0c8b00 con 0x7f48ec143c40 2023-11-24T06:31:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.844+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ec0c5d50 con 0x7f48ec143c40 2023-11-24T06:31:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.845+0000 7f48e97fa640 1 -- 172.21.15.16:0/2660953584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f48e400ad30 con 0x7f48ec143c40 2023-11-24T06:31:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.846+0000 7f48e97fa640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d006b3e0 0x7f48d006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:18.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.846+0000 7f48e97fa640 1 -- 172.21.15.16:0/2660953584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f48e4089320 con 0x7f48ec143c40 2023-11-24T06:31:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.847+0000 7f48f0d71640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d006b3e0 0x7f48d006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.848+0000 7f48f0d71640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d006b3e0 0x7f48d006d8a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f48e0036000 tx=0x7f48e0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:18.851+0000 7f48e97fa640 1 -- 172.21.15.16:0/2660953584 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48e4053e50 con 0x7f48ec143c40 2023-11-24T06:31:19.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.047+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48ec0cb6f0 con 0x7f48ec143c40 2023-11-24T06:31:19.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.048+0000 7f48e97fa640 1 -- 172.21.15.16:0/2660953584 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+12902 (secure 0 0 0) 0x7f48e4018090 con 0x7f48ec143c40 2023-11-24T06:31:19.052 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:19.052 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":82,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:18.049586+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:19.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.051+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d006b3e0 msgr2=0x7f48d006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.051+0000 7f48f1d73640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d006b3e0 0x7f48d006d8a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f48e0036000 tx=0x7f48e0008030 comp rx=0 tx=0).stop 2023-11-24T06:31:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.051+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 msgr2=0x7f48ec1427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.051+0000 7f48f1d73640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 0x7f48ec1427c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c7a0 tx=0x7f48e400cc60 comp rx=0 tx=0).stop 2023-11-24T06:31:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.052+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 shutdown_connections 2023-11-24T06:31:19.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.052+0000 7f48f1d73640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f48d006b3e0 0x7f48d006d8a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.052+0000 7f48f1d73640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48ec143c40 0x7f48ec1427c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.052+0000 7f48f1d73640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48ec0cca40 0x7f48ec142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.052+0000 7f48f1d73640 1 --2- 172.21.15.16:0/2660953584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48ec0717d0 0x7f48ec0c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.052+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 >> 172.21.15.16:0/2660953584 conn(0x7f48ec06a800 msgr2=0x7f48ec06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.052+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 shutdown_connections 2023-11-24T06:31:19.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.052+0000 7f48f1d73640 1 -- 172.21.15.16:0/2660953584 wait complete. 2023-11-24T06:31:19.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:31:19.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:31:19.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.303+0000 7f91e5127640 1 Processor -- start 2023-11-24T06:31:19.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.303+0000 7f91e5127640 1 -- start start 2023-11-24T06:31:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.305+0000 7f91e5127640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e015d4b0 0x7f91e00d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.305+0000 7f91e5127640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f91e00cf9a0 0x7f91e00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.305+0000 7f91e5127640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e00d0320 con 0x7f91e00cddf0 2023-11-24T06:31:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.305+0000 7f91e5127640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e00b2730 con 0x7f91e00cf9a0 2023-11-24T06:31:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.305+0000 7f91e5127640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00d1ec0 con 0x7f91e015d4b0 2023-11-24T06:31:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.306+0000 7f91dffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e015d4b0 0x7f91e00d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.306+0000 7f91e4926640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f91e00cddf0 0x7f91e00ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:46456/0 (socket says 172.21.15.16:46456) 2023-11-24T06:31:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.306+0000 7f91e4926640 1 -- 172.21.15.16:0/1005192019 learned_addr learned my addr 172.21.15.16:0/1005192019 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:19.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.306+0000 7f91df7fe640 1 --2- 172.21.15.16:0/1005192019 >> v2:172.21.15.173:3300/0 conn(0x7f91e00cf9a0 0x7f91e00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.307+0000 7f91df7fe640 1 -- 172.21.15.16:0/1005192019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e015d4b0 msgr2=0x7f91e00d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.307+0000 7f91df7fe640 1 --2- 172.21.15.16:0/1005192019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e015d4b0 0x7f91e00d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.307+0000 7f91df7fe640 1 -- 172.21.15.16:0/1005192019 >> v1:172.21.15.173:6789/0 conn(0x7f91e00cddf0 legacy=0x7f91e00ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.307+0000 7f91df7fe640 1 -- 172.21.15.16:0/1005192019 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00b29d0 con 0x7f91e00cf9a0 2023-11-24T06:31:19.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.307+0000 7f91dffff640 1 --2- 172.21.15.16:0/1005192019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e015d4b0 0x7f91e00d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:31:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.308+0000 7f91df7fe640 1 --2- 172.21.15.16:0/1005192019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91e00cf9a0 0x7f91e00cfde0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f91d40097d0 tx=0x7f91d402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab2558a99561ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.308+0000 7f91deffd640 1 -- 172.21.15.16:0/1005192019 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4031800 con 0x7f91e00cf9a0 2023-11-24T06:31:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91deffd640 1 -- 172.21.15.16:0/1005192019 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d4031960 con 0x7f91e00cf9a0 2023-11-24T06:31:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91deffd640 1 -- 172.21.15.16:0/1005192019 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4031cf0 con 0x7f91e00cf9a0 2023-11-24T06:31:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91e5127640 1 -- 172.21.15.16:0/1005192019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91e00cf9a0 msgr2=0x7f91e00cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91e5127640 1 --2- 172.21.15.16:0/1005192019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91e00cf9a0 0x7f91e00cfde0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f91d40097d0 tx=0x7f91d402b8a0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91e5127640 1 -- 172.21.15.16:0/1005192019 shutdown_connections 2023-11-24T06:31:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91e5127640 1 --2- 172.21.15.16:0/1005192019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91e00cf9a0 0x7f91e00cfde0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91e5127640 1 --2- 172.21.15.16:0/1005192019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e015d4b0 0x7f91e00d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91e5127640 1 -- 172.21.15.16:0/1005192019 >> 172.21.15.16:0/1005192019 conn(0x7f91e00d3000 msgr2=0x7f91e00d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91e5127640 1 -- 172.21.15.16:0/1005192019 shutdown_connections 2023-11-24T06:31:19.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.309+0000 7f91e5127640 1 -- 172.21.15.16:0/1005192019 wait complete. 2023-11-24T06:31:19.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.311+0000 7f91e5127640 1 Processor -- start 2023-11-24T06:31:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.311+0000 7f91e5127640 1 -- start start 2023-11-24T06:31:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.312+0000 7f91e5127640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e00cddf0 0x7f91e015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.312+0000 7f91e5127640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e00cf9a0 0x7f91e00cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.312+0000 7f91e5127640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91e015d4b0 0x7f91e00b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.312+0000 7f91e5127640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01608d0 con 0x7f91e00cddf0 2023-11-24T06:31:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.312+0000 7f91e5127640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e015c390 con 0x7f91e015d4b0 2023-11-24T06:31:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.312+0000 7f91e5127640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e015fc50 con 0x7f91e00cf9a0 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e00cddf0 0x7f91e015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e00cddf0 0x7f91e015be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33220/0 (socket says 172.21.15.16:33220) 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91dffff640 1 -- 172.21.15.16:0/2373864146 learned_addr learned my addr 172.21.15.16:0/2373864146 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91df7fe640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e00cf9a0 0x7f91e00cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91e4926640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91e015d4b0 0x7f91e00b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91dffff640 1 -- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e00cf9a0 msgr2=0x7f91e00cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91dffff640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e00cf9a0 0x7f91e00cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91dffff640 1 -- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91e015d4b0 msgr2=0x7f91e00b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91dffff640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91e015d4b0 0x7f91e00b7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.313+0000 7f91dffff640 1 -- 172.21.15.16:0/2373864146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d0009400 con 0x7f91e00cddf0 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.314+0000 7f91dffff640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e00cddf0 0x7f91e015be50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f91d000e960 tx=0x7f91d000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.314+0000 7f91dcff9640 1 -- 172.21.15.16:0/2373864146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000ca00 con 0x7f91e00cddf0 2023-11-24T06:31:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.314+0000 7f91dcff9640 1 -- 172.21.15.16:0/2373864146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d000cb60 con 0x7f91e00cddf0 2023-11-24T06:31:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.314+0000 7f91dcff9640 1 -- 172.21.15.16:0/2373864146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0013470 con 0x7f91e00cddf0 2023-11-24T06:31:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.314+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d4009400 con 0x7f91e00cddf0 2023-11-24T06:31:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.314+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e015da10 con 0x7f91e00cddf0 2023-11-24T06:31:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.315+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e0067450 con 0x7f91e00cddf0 2023-11-24T06:31:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.316+0000 7f91dcff9640 1 -- 172.21.15.16:0/2373864146 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f91d00135d0 con 0x7f91e00cddf0 2023-11-24T06:31:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.317+0000 7f91dcff9640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f91cc06b3e0 0x7f91cc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.317+0000 7f91dcff9640 1 -- 172.21.15.16:0/2373864146 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f91d0089fa0 con 0x7f91e00cddf0 2023-11-24T06:31:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.318+0000 7f91df7fe640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f91cc06b3e0 0x7f91cc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.319+0000 7f91df7fe640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f91cc06b3e0 0x7f91cc06d8a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f91d4036000 tx=0x7f91d403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:19.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.321+0000 7f91dcff9640 1 -- 172.21.15.16:0/2373864146 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91d0054b60 con 0x7f91e00cddf0 2023-11-24T06:31:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.513+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91e00cd3c0 con 0x7f91e00cddf0 2023-11-24T06:31:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.514+0000 7f91dcff9640 1 -- 172.21.15.16:0/2373864146 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91d0058180 con 0x7f91e00cddf0 2023-11-24T06:31:19.517 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:31:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f91cc06b3e0 msgr2=0x7f91cc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f91cc06b3e0 0x7f91cc06d8a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f91d4036000 tx=0x7f91d403c040 comp rx=0 tx=0).stop 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e00cddf0 msgr2=0x7f91e015be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e00cddf0 0x7f91e015be50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f91d000e960 tx=0x7f91d000ee20 comp rx=0 tx=0).stop 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 shutdown_connections 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f91cc06b3e0 0x7f91cc06d8a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91e015d4b0 0x7f91e00b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91e00cf9a0 0x7f91e00cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 --2- 172.21.15.16:0/2373864146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e00cddf0 0x7f91e015be50 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 >> 172.21.15.16:0/2373864146 conn(0x7f91e00d3000 msgr2=0x7f91e00d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 shutdown_connections 2023-11-24T06:31:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.516+0000 7f91e5127640 1 -- 172.21.15.16:0/2373864146 wait complete. 2023-11-24T06:31:19.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:31:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.765+0000 7fd43b023640 1 Processor -- start 2023-11-24T06:31:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.765+0000 7fd43b023640 1 -- start start 2023-11-24T06:31:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.767+0000 7fd43b023640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd43413e360 0x7fd4340cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.767+0000 7fd43b023640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43415c5f0 con 0x7fd43413df80 2023-11-24T06:31:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.767+0000 7fd43b023640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43406c470 con 0x7fd43415f9c0 2023-11-24T06:31:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.767+0000 7fd43b023640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43406c750 con 0x7fd43413e360 2023-11-24T06:31:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.768+0000 7fd438dbf640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd43413df80 0x7fd43415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45732/0 (socket says 172.21.15.16:45732) 2023-11-24T06:31:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.768+0000 7fd433fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd43413e360 0x7fd4340cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.768+0000 7fd438dbf640 1 -- 172.21.15.16:0/3501654248 learned_addr learned my addr 172.21.15.16:0/3501654248 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.768+0000 7fd433fff640 1 -- 172.21.15.16:0/3501654248 >> v1:172.21.15.173:6789/0 conn(0x7fd43415f9c0 legacy=0x7fd4340cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.768+0000 7fd4337fe640 1 -- 172.21.15.16:0/3501654248 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967857402 0 0) 0x7fd43415c5f0 con 0x7fd43413df80 2023-11-24T06:31:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.768+0000 7fd433fff640 1 -- 172.21.15.16:0/3501654248 >> v1:172.21.15.16:6789/0 conn(0x7fd43413df80 legacy=0x7fd43415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.768+0000 7fd433fff640 1 -- 172.21.15.16:0/3501654248 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43406ca30 con 0x7fd43413e360 2023-11-24T06:31:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.770+0000 7fd433fff640 1 --2- 172.21.15.16:0/3501654248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 0x7fd4340cf290 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd4280097a0 tx=0x7fd42802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febe6b2413e27d7e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.770+0000 7fd4337fe640 1 -- 172.21.15.16:0/3501654248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428030610 con 0x7fd43413e360 2023-11-24T06:31:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.770+0000 7fd4337fe640 1 -- 172.21.15.16:0/3501654248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd428030770 con 0x7fd43413e360 2023-11-24T06:31:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.770+0000 7fd4337fe640 1 -- 172.21.15.16:0/3501654248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428030b00 con 0x7fd43413e360 2023-11-24T06:31:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.771+0000 7fd43b023640 1 -- 172.21.15.16:0/3501654248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 msgr2=0x7fd4340cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.771+0000 7fd43b023640 1 --2- 172.21.15.16:0/3501654248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 0x7fd4340cf290 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd4280097a0 tx=0x7fd42802b960 comp rx=0 tx=0).stop 2023-11-24T06:31:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.771+0000 7fd43b023640 1 -- 172.21.15.16:0/3501654248 shutdown_connections 2023-11-24T06:31:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.771+0000 7fd43b023640 1 --2- 172.21.15.16:0/3501654248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 0x7fd4340cf290 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.771+0000 7fd43b023640 1 -- 172.21.15.16:0/3501654248 >> 172.21.15.16:0/3501654248 conn(0x7fd43406a800 msgr2=0x7fd43406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.771+0000 7fd43b023640 1 -- 172.21.15.16:0/3501654248 shutdown_connections 2023-11-24T06:31:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.771+0000 7fd43b023640 1 -- 172.21.15.16:0/3501654248 wait complete. 2023-11-24T06:31:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.773+0000 7fd43b023640 1 Processor -- start 2023-11-24T06:31:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.773+0000 7fd43b023640 1 -- start start 2023-11-24T06:31:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.774+0000 7fd43b023640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd43413df80 0x7fd43413b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd43b023640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 0x7fd434154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd43b023640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd43415f9c0 0x7fd434150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd43b023640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340afb30 con 0x7fd43413e360 2023-11-24T06:31:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd43b023640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43414a440 con 0x7fd43415f9c0 2023-11-24T06:31:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd43b023640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340aefb0 con 0x7fd43413df80 2023-11-24T06:31:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd433fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 0x7fd434154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd433fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 0x7fd434154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33246/0 (socket says 172.21.15.16:33246) 2023-11-24T06:31:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd433fff640 1 -- 172.21.15.16:0/3740948421 learned_addr learned my addr 172.21.15.16:0/3740948421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd438dbf640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd43413df80 0x7fd43413b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd433fff640 1 -- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd43413df80 msgr2=0x7fd43413b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd433fff640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd43413df80 0x7fd43413b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd433fff640 1 -- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd43415f9c0 msgr2=0x7fd434150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd433fff640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd43415f9c0 0x7fd434150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.775+0000 7fd433fff640 1 -- 172.21.15.16:0/3740948421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd428009400 con 0x7fd43413e360 2023-11-24T06:31:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.776+0000 7fd433fff640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 0x7fd434154520 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd428000c00 tx=0x7fd4280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.776+0000 7fd4317fa640 1 -- 172.21.15.16:0/3740948421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428039c90 con 0x7fd43413e360 2023-11-24T06:31:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.776+0000 7fd4317fa640 1 -- 172.21.15.16:0/3740948421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42803f040 con 0x7fd43413e360 2023-11-24T06:31:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.776+0000 7fd4317fa640 1 -- 172.21.15.16:0/3740948421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428004030 con 0x7fd43413e360 2023-11-24T06:31:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.776+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4340af290 con 0x7fd43413e360 2023-11-24T06:31:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.776+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4340af850 con 0x7fd43413e360 2023-11-24T06:31:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.778+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4340675d0 con 0x7fd43413e360 2023-11-24T06:31:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.778+0000 7fd4317fa640 1 -- 172.21.15.16:0/3740948421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd428004190 con 0x7fd43413e360 2023-11-24T06:31:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.779+0000 7fd4317fa640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd42006b3e0 0x7fd42006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.779+0000 7fd4317fa640 1 -- 172.21.15.16:0/3740948421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fd4280abbc0 con 0x7fd43413e360 2023-11-24T06:31:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.780+0000 7fd438dbf640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd42006b3e0 0x7fd42006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.781+0000 7fd438dbf640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd42006b3e0 0x7fd42006d8a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd424000f60 tx=0x7fd424009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.784+0000 7fd4317fa640 1 -- 172.21.15.16:0/3740948421 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd428076670 con 0x7fd43413e360 2023-11-24T06:31:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.984+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd43406df00 con 0x7fd43413e360 2023-11-24T06:31:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.985+0000 7fd4317fa640 1 -- 172.21.15.16:0/3740948421 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd428048060 con 0x7fd43413e360 2023-11-24T06:31:19.988 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:19.989 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:31:19.989 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:19.989 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-11-24T06:31:19.989 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-11-24T06:31:19.989 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:19.989 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd42006b3e0 msgr2=0x7fd42006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd42006b3e0 0x7fd42006d8a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd424000f60 tx=0x7fd424009040 comp rx=0 tx=0).stop 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 msgr2=0x7fd434154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 0x7fd434154520 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fd428000c00 tx=0x7fd4280046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 shutdown_connections 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd42006b3e0 0x7fd42006d8a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd43415f9c0 0x7fd434150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd43413e360 0x7fd434154520 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 --2- 172.21.15.16:0/3740948421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd43413df80 0x7fd43413b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 >> 172.21.15.16:0/3740948421 conn(0x7fd43406a800 msgr2=0x7fd43415cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 shutdown_connections 2023-11-24T06:31:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:19.988+0000 7fd43b023640 1 -- 172.21.15.16:0/3740948421 wait complete. 2023-11-24T06:31:20.012 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:31:20.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:31:20.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.244+0000 7ff31ef89640 1 Processor -- start 2023-11-24T06:31:20.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.245+0000 7ff31ef89640 1 -- start start 2023-11-24T06:31:20.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.246+0000 7ff31ef89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff31813df80 0x7ff31815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.246+0000 7ff31ef89640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff31813e360 0x7ff3180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.246+0000 7ff31ef89640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31806c470 con 0x7ff31815f9c0 2023-11-24T06:31:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.246+0000 7ff31ef89640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31806c750 con 0x7ff31813e360 2023-11-24T06:31:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.246+0000 7ff31ef89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31806ca30 con 0x7ff31813df80 2023-11-24T06:31:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.247+0000 7ff31d526640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff31815f9c0 0x7ff3181408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45746/0 (socket says 172.21.15.16:45746) 2023-11-24T06:31:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.247+0000 7ff31d526640 1 -- 172.21.15.16:0/1052817110 learned_addr learned my addr 172.21.15.16:0/1052817110 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.247+0000 7ff30ffff640 1 --2- 172.21.15.16:0/1052817110 >> v2:172.21.15.173:3300/0 conn(0x7ff31813e360 0x7ff3180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.247+0000 7ff31cd25640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff31813df80 0x7ff31815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.247+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103271847 0 0) 0x7ff31806c470 con 0x7ff31815f9c0 2023-11-24T06:31:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.247+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff300003520 con 0x7ff31815f9c0 2023-11-24T06:31:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.247+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2658966656 0 0) 0x7ff300003520 con 0x7ff31815f9c0 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff31813df80 msgr2=0x7ff31815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff31813df80 0x7ff31815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 >> v2:172.21.15.173:3300/0 conn(0x7ff31813e360 msgr2=0x7ff3180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 --2- 172.21.15.16:0/1052817110 >> v2:172.21.15.173:3300/0 conn(0x7ff31813e360 0x7ff3180cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31806cd10 con 0x7ff31815f9c0 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7ff310001670 con 0x7ff31815f9c0 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 >> v1:172.21.15.16:6789/0 conn(0x7ff31815f9c0 legacy=0x7ff3181408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff300003970 0x7ff300003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3000047c0 0x7ff300004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff31813df80 0x7ff300005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31806c750 con 0x7ff3000047c0 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31806ca30 con 0x7ff300003970 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30000b980 con 0x7ff31813df80 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30ffff640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3000047c0 0x7ff300004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff31d526640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff31813df80 0x7ff300005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.248+0000 7ff30ffff640 1 -- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff31813df80 msgr2=0x7ff300005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.249+0000 7ff31cd25640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff300003970 0x7ff300003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.249+0000 7ff30ffff640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff31813df80 0x7ff300005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.249+0000 7ff30ffff640 1 -- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff300003970 msgr2=0x7ff300003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.249+0000 7ff30ffff640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff300003970 0x7ff300003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.249+0000 7ff30ffff640 1 -- 172.21.15.16:0/1052817110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300003520 con 0x7ff3000047c0 2023-11-24T06:31:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.249+0000 7ff31d526640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff31813df80 0x7ff300005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.250+0000 7ff30ffff640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3000047c0 0x7ff300004c40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ff308009ac0 tx=0x7ff3000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5333774b62eec29 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.250+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30802c6d0 con 0x7ff3000047c0 2023-11-24T06:31:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.250+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30802c830 con 0x7ff3000047c0 2023-11-24T06:31:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.250+0000 7ff30f7fe640 1 -- 172.21.15.16:0/1052817110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30802cbc0 con 0x7ff3000047c0 2023-11-24T06:31:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.251+0000 7ff31ef89640 1 -- 172.21.15.16:0/1052817110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3000047c0 msgr2=0x7ff300004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.251+0000 7ff31ef89640 1 --2- 172.21.15.16:0/1052817110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3000047c0 0x7ff300004c40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7ff308009ac0 tx=0x7ff3000057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.251+0000 7ff31d526640 1 -- 172.21.15.16:0/1052817110 reap_dead start 2023-11-24T06:31:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.251+0000 7ff31ef89640 1 -- 172.21.15.16:0/1052817110 shutdown_connections 2023-11-24T06:31:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.251+0000 7ff31ef89640 1 -- 172.21.15.16:0/1052817110 >> 172.21.15.16:0/1052817110 conn(0x7ff31806a800 msgr2=0x7ff31806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.251+0000 7ff31ef89640 1 -- 172.21.15.16:0/1052817110 shutdown_connections 2023-11-24T06:31:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.251+0000 7ff31ef89640 1 -- 172.21.15.16:0/1052817110 wait complete. 2023-11-24T06:31:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.253+0000 7ff31ef89640 1 Processor -- start 2023-11-24T06:31:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.253+0000 7ff31ef89640 1 -- start start 2023-11-24T06:31:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.254+0000 7ff31ef89640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3000047c0 0x7ff3181f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.254+0000 7ff31ef89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31813dac0 0x7ff31814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.254+0000 7ff31ef89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff318154480 0x7ff318154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.254+0000 7ff31ef89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff318154e50 con 0x7ff31813dac0 2023-11-24T06:31:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.254+0000 7ff31ef89640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31814feb0 con 0x7ff3000047c0 2023-11-24T06:31:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.254+0000 7ff31ef89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3181f3270 con 0x7ff318154480 2023-11-24T06:31:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.254+0000 7ff30ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31813dac0 0x7ff31814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.254+0000 7ff30ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31813dac0 0x7ff31814d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33250/0 (socket says 172.21.15.16:33250) 2023-11-24T06:31:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.254+0000 7ff30ffff640 1 -- 172.21.15.16:0/1795225751 learned_addr learned my addr 172.21.15.16:0/1795225751 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff31cd25640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3000047c0 0x7ff3181f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff30ffff640 1 -- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff318154480 msgr2=0x7ff318154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff30ffff640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff318154480 0x7ff318154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff30ffff640 1 -- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3000047c0 msgr2=0x7ff3181f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff30ffff640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3000047c0 0x7ff3181f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff30ffff640 1 -- 172.21.15.16:0/1795225751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31814f2a0 con 0x7ff31813dac0 2023-11-24T06:31:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff31cd25640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3000047c0 0x7ff3181f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff30ffff640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31813dac0 0x7ff31814d4e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff30802c2d0 tx=0x7ff308004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff30d7fa640 1 -- 172.21.15.16:0/1795225751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308003810 con 0x7ff31813dac0 2023-11-24T06:31:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff30d7fa640 1 -- 172.21.15.16:0/1795225751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff308003970 con 0x7ff31813dac0 2023-11-24T06:31:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.255+0000 7ff30d7fa640 1 -- 172.21.15.16:0/1795225751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308003ce0 con 0x7ff31813dac0 2023-11-24T06:31:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.256+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31814f580 con 0x7ff31813dac0 2023-11-24T06:31:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.256+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31814fb40 con 0x7ff31813dac0 2023-11-24T06:31:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.257+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31814fcd0 con 0x7ff31813dac0 2023-11-24T06:31:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.258+0000 7ff30d7fa640 1 -- 172.21.15.16:0/1795225751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff308040660 con 0x7ff31813dac0 2023-11-24T06:31:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.259+0000 7ff30d7fa640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff30406b430 0x7ff30406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.259+0000 7ff30d7fa640 1 -- 172.21.15.16:0/1795225751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4885+0+0 (secure 0 0 0) 0x7ff3080abda0 con 0x7ff31813dac0 2023-11-24T06:31:20.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.260+0000 7ff31cd25640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff30406b430 0x7ff30406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.261+0000 7ff31cd25640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff30406b430 0x7ff30406d8f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3140073e0 tx=0x7ff31400d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:20.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.264+0000 7ff30d7fa640 1 -- 172.21.15.16:0/1795225751 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff31814fcd0 con 0x7ff31813dac0 2023-11-24T06:31:20.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.464+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff31814fcd0 con 0x7ff31813dac0 2023-11-24T06:31:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.741+0000 7ff30d7fa640 1 -- 172.21.15.16:0/1795225751 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff30807a4a0 con 0x7ff31813dac0 2023-11-24T06:31:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-11-24T06:31:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff30406b430 msgr2=0x7ff30406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff30406b430 0x7ff30406d8f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3140073e0 tx=0x7ff31400d040 comp rx=0 tx=0).stop 2023-11-24T06:31:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31813dac0 msgr2=0x7ff31814d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31813dac0 0x7ff31814d4e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff30802c2d0 tx=0x7ff308004730 comp rx=0 tx=0).stop 2023-11-24T06:31:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 shutdown_connections 2023-11-24T06:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff30406b430 0x7ff30406d8f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff318154480 0x7ff318154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31813dac0 0x7ff31814d4e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 --2- 172.21.15.16:0/1795225751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3000047c0 0x7ff3181f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.745+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 >> 172.21.15.16:0/1795225751 conn(0x7ff31806a800 msgr2=0x7ff3180cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.746+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 shutdown_connections 2023-11-24T06:31:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.746+0000 7ff31ef89640 1 -- 172.21.15.16:0/1795225751 wait complete. 2023-11-24T06:31:20.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:31:20.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.983+0000 7f83ccda5640 1 Processor -- start 2023-11-24T06:31:20.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.983+0000 7f83ccda5640 1 -- start start 2023-11-24T06:31:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.985+0000 7f83ccda5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c8157dc0 0x7f83c8070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.985+0000 7f83ccda5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c815ba40 con 0x7f83c815e450 2023-11-24T06:31:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.985+0000 7f83ccda5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c815eb10 con 0x7f83c8071230 2023-11-24T06:31:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.985+0000 7f83ccda5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81581a0 con 0x7f83c8157dc0 2023-11-24T06:31:20.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.985+0000 7f83c6d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f83c815e450 0x7f83c815bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45758/0 (socket says 172.21.15.16:45758) 2023-11-24T06:31:20.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.985+0000 7f83c6d9d640 1 -- 172.21.15.16:0/2182113802 learned_addr learned my addr 172.21.15.16:0/2182113802 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:20.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c659c640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c8157dc0 0x7f83c8070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079588158 0 0) 0x7f83c815ba40 con 0x7f83c815e450 2023-11-24T06:31:20.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83b4003520 con 0x7f83c815e450 2023-11-24T06:31:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574393867 0 0) 0x7f83b4003520 con 0x7f83c815e450 2023-11-24T06:31:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c8157dc0 msgr2=0x7f83c8070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c8157dc0 0x7f83c8070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 >> v1:172.21.15.173:6789/0 conn(0x7f83c8071230 legacy=0x7f83c815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8071860 con 0x7f83c815e450 2023-11-24T06:31:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f83c0001670 con 0x7f83c815e450 2023-11-24T06:31:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 >> v1:172.21.15.16:6789/0 conn(0x7f83c815e450 legacy=0x7f83c815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83b40039a0 0x7f83b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c8157dc0 0x7f83b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81581a0 con 0x7f83b4004d00 2023-11-24T06:31:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b400ba30 con 0x7f83b40039a0 2023-11-24T06:31:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.986+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b400de00 con 0x7f83c8157dc0 2023-11-24T06:31:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.987+0000 7f83c6d9d640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.987+0000 7f83c659c640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83b40039a0 0x7f83b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.987+0000 7f83bdd9b640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c8157dc0 0x7f83b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.987+0000 7f83c6d9d640 1 -- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c8157dc0 msgr2=0x7f83b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:20.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.987+0000 7f83c6d9d640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c8157dc0 0x7f83b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.987+0000 7f83c6d9d640 1 -- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83b40039a0 msgr2=0x7f83b4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:20.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.987+0000 7f83c6d9d640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83b40039a0 0x7f83b4003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.987+0000 7f83c6d9d640 1 -- 172.21.15.16:0/2182113802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4003520 con 0x7f83b4004d00 2023-11-24T06:31:20.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.987+0000 7f83bdd9b640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c8157dc0 0x7f83b40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.988+0000 7f83c6d9d640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83b4005180 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c210 tx=0x7f83b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf87bcd9131039a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0005ad0 con 0x7f83b4004d00 2023-11-24T06:31:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c0005c30 con 0x7f83b4004d00 2023-11-24T06:31:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83c5d9b640 1 -- 172.21.15.16:0/2182113802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c002c890 con 0x7f83b4004d00 2023-11-24T06:31:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83ccda5640 1 -- 172.21.15.16:0/2182113802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 msgr2=0x7f83b4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83ccda5640 1 --2- 172.21.15.16:0/2182113802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83b4005180 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c210 tx=0x7f83b40057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83c6d9d640 1 -- 172.21.15.16:0/2182113802 reap_dead start 2023-11-24T06:31:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83ccda5640 1 -- 172.21.15.16:0/2182113802 shutdown_connections 2023-11-24T06:31:20.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83ccda5640 1 -- 172.21.15.16:0/2182113802 >> 172.21.15.16:0/2182113802 conn(0x7f83c80b0be0 msgr2=0x7f83c80b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:20.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83ccda5640 1 -- 172.21.15.16:0/2182113802 shutdown_connections 2023-11-24T06:31:20.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.989+0000 7f83ccda5640 1 -- 172.21.15.16:0/2182113802 wait complete. 2023-11-24T06:31:20.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.991+0000 7f83ccda5640 1 Processor -- start 2023-11-24T06:31:20.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.991+0000 7f83ccda5640 1 -- start start 2023-11-24T06:31:20.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.992+0000 7f83ccda5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83c8146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.992+0000 7f83ccda5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c80cee10 0x7f83c80cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.992+0000 7f83ccda5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c80ac520 0x7f83c80ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:20.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.992+0000 7f83ccda5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80a9ad0 con 0x7f83b4004d00 2023-11-24T06:31:20.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.992+0000 7f83ccda5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80a9db0 con 0x7f83c80ac520 2023-11-24T06:31:20.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.992+0000 7f83ccda5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80aa090 con 0x7f83c80cee10 2023-11-24T06:31:20.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83c8146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83c8146dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33280/0 (socket says 172.21.15.16:33280) 2023-11-24T06:31:20.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c659c640 1 -- 172.21.15.16:0/198339680 learned_addr learned my addr 172.21.15.16:0/198339680 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c659c640 1 -- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c80cee10 msgr2=0x7f83c80cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c659c640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c80cee10 0x7f83c80cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c659c640 1 -- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c80ac520 msgr2=0x7f83c80ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c6d9d640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c80ac520 0x7f83c80ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c659c640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c80ac520 0x7f83c80ac9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c659c640 1 -- 172.21.15.16:0/198339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80cd480 con 0x7f83b4004d00 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c6d9d640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c80ac520 0x7f83c80ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.993+0000 7f83c659c640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83c8146dd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f83b800f350 tx=0x7f83b800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.994+0000 7f83bf7fe640 1 -- 172.21.15.16:0/198339680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8017600 con 0x7f83b4004d00 2023-11-24T06:31:20.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.994+0000 7f83bf7fe640 1 -- 172.21.15.16:0/198339680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b8011040 con 0x7f83b4004d00 2023-11-24T06:31:21.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.994+0000 7f83bf7fe640 1 -- 172.21.15.16:0/198339680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b80179c0 con 0x7f83b4004d00 2023-11-24T06:31:21.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.994+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c8155a60 con 0x7f83b4004d00 2023-11-24T06:31:21.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.994+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c80cd9a0 con 0x7f83b4004d00 2023-11-24T06:31:21.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.995+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c8067670 con 0x7f83b4004d00 2023-11-24T06:31:21.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.997+0000 7f83bf7fe640 1 -- 172.21.15.16:0/198339680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f83b8017b20 con 0x7f83b4004d00 2023-11-24T06:31:21.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.998+0000 7f83bf7fe640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f83a806b410 0x7f83a806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:21.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.999+0000 7f83bf7fe640 1 -- 172.21.15.16:0/198339680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f83b808aef0 con 0x7f83b4004d00 2023-11-24T06:31:21.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:20.999+0000 7f83bdd9b640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f83a806b410 0x7f83a806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:21.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.001+0000 7f83bdd9b640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f83a806b410 0x7f83a806d8d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f83c80aead0 tx=0x7f83b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:21.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.003+0000 7f83bf7fe640 1 -- 172.21.15.16:0/198339680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83b80558e0 con 0x7f83b4004d00 2023-11-24T06:31:21.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.204+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f83c80cd610 con 0x7f83b4004d00 2023-11-24T06:31:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.748+0000 7f83bf7fe640 1 -- 172.21.15.16:0/198339680 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f83b801c070 con 0x7f83b4004d00 2023-11-24T06:31:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-11-24T06:31:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f83a806b410 msgr2=0x7f83a806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f83a806b410 0x7f83a806d8d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f83c80aead0 tx=0x7f83b0009040 comp rx=0 tx=0).stop 2023-11-24T06:31:21.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 msgr2=0x7f83c8146dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:21.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83c8146dd0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f83b800f350 tx=0x7f83b800f810 comp rx=0 tx=0).stop 2023-11-24T06:31:21.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 shutdown_connections 2023-11-24T06:31:21.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f83a806b410 0x7f83a806d8d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:21.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c80ac520 0x7f83c80ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:21.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83c80cee10 0x7f83c80cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:21.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 --2- 172.21.15.16:0/198339680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b4004d00 0x7f83c8146dd0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:21.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.753+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 >> 172.21.15.16:0/198339680 conn(0x7f83c80b0be0 msgr2=0x7f83c80b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:21.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.754+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 shutdown_connections 2023-11-24T06:31:21.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.755+0000 7f83ccda5640 1 -- 172.21.15.16:0/198339680 wait complete. 2023-11-24T06:31:21.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:31:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.982+0000 7ff49dbe3640 1 Processor -- start 2023-11-24T06:31:21.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.982+0000 7ff49dbe3640 1 -- start start 2023-11-24T06:31:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.983+0000 7ff49dbe3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff49813df80 0x7ff49815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.984+0000 7ff49dbe3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49806c470 con 0x7ff4980cf9a0 2023-11-24T06:31:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.984+0000 7ff49dbe3640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49806c750 con 0x7ff49813e360 2023-11-24T06:31:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.984+0000 7ff49dbe3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49806ca30 con 0x7ff49813df80 2023-11-24T06:31:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.984+0000 7ff4977fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff49813df80 0x7ff49815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.984+0000 7ff497fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff4980cf9a0 0x7ff4980cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45760/0 (socket says 172.21.15.16:45760) 2023-11-24T06:31:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.984+0000 7ff497fff640 1 -- 172.21.15.16:0/2774624484 learned_addr learned my addr 172.21.15.16:0/2774624484 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.985+0000 7ff4977fe640 1 -- 172.21.15.16:0/2774624484 >> v1:172.21.15.173:6789/0 conn(0x7ff49813e360 legacy=0x7ff4980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.985+0000 7ff4977fe640 1 -- 172.21.15.16:0/2774624484 >> v1:172.21.15.16:6789/0 conn(0x7ff4980cf9a0 legacy=0x7ff4980cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.985+0000 7ff4977fe640 1 -- 172.21.15.16:0/2774624484 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49806cd10 con 0x7ff49813df80 2023-11-24T06:31:21.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.985+0000 7ff4967fc640 1 -- 172.21.15.16:0/2774624484 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932715785 0 0) 0x7ff49806c470 con 0x7ff4980cf9a0 2023-11-24T06:31:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.986+0000 7ff4977fe640 1 --2- 172.21.15.16:0/2774624484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813df80 0x7ff49815bee0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff488009d90 tx=0x7ff48802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e457fc3d0789b5c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.986+0000 7ff4967fc640 1 -- 172.21.15.16:0/2774624484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48802cd00 con 0x7ff49813df80 2023-11-24T06:31:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.986+0000 7ff4967fc640 1 -- 172.21.15.16:0/2774624484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48802ce60 con 0x7ff49813df80 2023-11-24T06:31:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.986+0000 7ff4967fc640 1 -- 172.21.15.16:0/2774624484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488034660 con 0x7ff49813df80 2023-11-24T06:31:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.987+0000 7ff49dbe3640 1 -- 172.21.15.16:0/2774624484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813df80 msgr2=0x7ff49815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.987+0000 7ff49dbe3640 1 --2- 172.21.15.16:0/2774624484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813df80 0x7ff49815bee0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff488009d90 tx=0x7ff48802bf50 comp rx=0 tx=0).stop 2023-11-24T06:31:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.987+0000 7ff49dbe3640 1 -- 172.21.15.16:0/2774624484 shutdown_connections 2023-11-24T06:31:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.987+0000 7ff49dbe3640 1 --2- 172.21.15.16:0/2774624484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813df80 0x7ff49815bee0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:21.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.987+0000 7ff49dbe3640 1 -- 172.21.15.16:0/2774624484 >> 172.21.15.16:0/2774624484 conn(0x7ff49806a800 msgr2=0x7ff49806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.987+0000 7ff49dbe3640 1 -- 172.21.15.16:0/2774624484 shutdown_connections 2023-11-24T06:31:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.987+0000 7ff49dbe3640 1 -- 172.21.15.16:0/2774624484 wait complete. 2023-11-24T06:31:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.989+0000 7ff49dbe3640 1 Processor -- start 2023-11-24T06:31:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.989+0000 7ff49dbe3640 1 -- start start 2023-11-24T06:31:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.990+0000 7ff49dbe3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4980cf9a0 0x7ff4981f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.990+0000 7ff49dbe3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff49813df80 0x7ff49813d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.990+0000 7ff49dbe3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813e360 0x7ff4980715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.990+0000 7ff49dbe3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980702c0 con 0x7ff49813e360 2023-11-24T06:31:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.990+0000 7ff49dbe3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981f3c40 con 0x7ff49813df80 2023-11-24T06:31:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.990+0000 7ff49dbe3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49813dd30 con 0x7ff4980cf9a0 2023-11-24T06:31:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813e360 0x7ff4980715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff497fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813e360 0x7ff4980715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33308/0 (socket says 172.21.15.16:33308) 2023-11-24T06:31:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff497fff640 1 -- 172.21.15.16:0/1962393237 learned_addr learned my addr 172.21.15.16:0/1962393237 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff4977fe640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4980cf9a0 0x7ff4981f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff496ffd640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff49813df80 0x7ff49813d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff497fff640 1 -- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4980cf9a0 msgr2=0x7ff4981f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff497fff640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4980cf9a0 0x7ff4981f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff497fff640 1 -- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff49813df80 msgr2=0x7ff49813d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff497fff640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff49813df80 0x7ff49813d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff497fff640 1 -- 172.21.15.16:0/1962393237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488009a70 con 0x7ff49813e360 2023-11-24T06:31:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.991+0000 7ff497fff640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813e360 0x7ff4980715b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c3e0 tx=0x7ff49000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.992+0000 7ff48ffff640 1 -- 172.21.15.16:0/1962393237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900074e0 con 0x7ff49813e360 2023-11-24T06:31:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.992+0000 7ff48ffff640 1 -- 172.21.15.16:0/1962393237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff490007640 con 0x7ff49813e360 2023-11-24T06:31:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.992+0000 7ff48ffff640 1 -- 172.21.15.16:0/1962393237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490004a20 con 0x7ff49813e360 2023-11-24T06:31:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.992+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff498070510 con 0x7ff49813e360 2023-11-24T06:31:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.992+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff498070ad0 con 0x7ff49813e360 2023-11-24T06:31:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.993+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4980675d0 con 0x7ff49813e360 2023-11-24T06:31:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.995+0000 7ff48ffff640 1 -- 172.21.15.16:0/1962393237 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff490004b80 con 0x7ff49813e360 2023-11-24T06:31:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.995+0000 7ff48ffff640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff47806b3e0 0x7ff47806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.996+0000 7ff48ffff640 1 -- 172.21.15.16:0/1962393237 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5637+0+0 (secure 0 0 0) 0x7ff49008a420 con 0x7ff49813e360 2023-11-24T06:31:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.997+0000 7ff4977fe640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff47806b3e0 0x7ff47806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.998+0000 7ff4977fe640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff47806b3e0 0x7ff47806d8a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff488036000 tx=0x7ff488030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:21.999+0000 7ff48ffff640 1 -- 172.21.15.16:0/1962393237 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff490054ce0 con 0x7ff49813e360 2023-11-24T06:31:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:22.234+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff49815e760 con 0x7ff49813e360 2023-11-24T06:31:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.047+0000 7ff48ffff640 1 -- 172.21.15.16:0/1962393237 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff490058300 con 0x7ff49813e360 2023-11-24T06:31:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 12 and data pool 13 2023-11-24T06:31:23.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff47806b3e0 msgr2=0x7ff47806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff47806b3e0 0x7ff47806d8a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff488036000 tx=0x7ff488030000 comp rx=0 tx=0).stop 2023-11-24T06:31:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813e360 msgr2=0x7ff4980715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813e360 0x7ff4980715b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c3e0 tx=0x7ff49000c8a0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 shutdown_connections 2023-11-24T06:31:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff47806b3e0 0x7ff47806d8a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff49813e360 0x7ff4980715b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff49813df80 0x7ff49813d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 --2- 172.21.15.16:0/1962393237 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4980cf9a0 0x7ff4981f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.051+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 >> 172.21.15.16:0/1962393237 conn(0x7ff49806a800 msgr2=0x7ff4980cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:23.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.052+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 shutdown_connections 2023-11-24T06:31:23.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.052+0000 7ff49dbe3640 1 -- 172.21.15.16:0/1962393237 wait complete. 2023-11-24T06:31:23.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.271+0000 7f7e12d83640 1 Processor -- start 2023-11-24T06:31:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.272+0000 7f7e12d83640 1 -- start start 2023-11-24T06:31:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.273+0000 7f7e12d83640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e0c0b3eb0 0x7f7e0c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.273+0000 7f7e12d83640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e0c160b80 0x7f7e0c1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.273+0000 7f7e12d83640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c0b2b90 con 0x7f7e0c1607a0 2023-11-24T06:31:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.273+0000 7f7e12d83640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0c0d21d0 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.273+0000 7f7e12d83640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b2e30 con 0x7f7e0c160b80 2023-11-24T06:31:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e11d81640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7e0c1607a0 0x7f7e0c1610c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55654/0 (socket says 172.21.15.16:55654) 2023-11-24T06:31:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e11d81640 1 -- 172.21.15.16:0/766095062 learned_addr learned my addr 172.21.15.16:0/766095062 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:23.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e11580640 1 --2- 172.21.15.16:0/766095062 >> v2:172.21.15.16:3300/0 conn(0x7f7e0c0b3eb0 0x7f7e0c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e12582640 1 --2- 172.21.15.16:0/766095062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e0c160b80 0x7f7e0c1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e10d7f640 1 -- 172.21.15.16:0/766095062 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432982701 0 0) 0x7f7e0c0b2b90 con 0x7f7e0c1607a0 2023-11-24T06:31:23.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e10d7f640 1 -- 172.21.15.16:0/766095062 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dfc003520 con 0x7f7e0c1607a0 2023-11-24T06:31:23.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e11580640 1 -- 172.21.15.16:0/766095062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e0c160b80 msgr2=0x7f7e0c1617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:23.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e11580640 1 --2- 172.21.15.16:0/766095062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e0c160b80 0x7f7e0c1617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e11580640 1 -- 172.21.15.16:0/766095062 >> v1:172.21.15.16:6789/0 conn(0x7f7e0c1607a0 legacy=0x7f7e0c1610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.274+0000 7f7e11580640 1 -- 172.21.15.16:0/766095062 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c0b3110 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.275+0000 7f7e11580640 1 --2- 172.21.15.16:0/766095062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 0x7f7e0c0b4330 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7df80097a0 tx=0x7f7df802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479830f6930d9aca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:23.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e10d7f640 1 -- 172.21.15.16:0/766095062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df802d040 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e10d7f640 1 -- 172.21.15.16:0/766095062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df8031810 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e10d7f640 1 -- 172.21.15.16:0/766095062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8031ba0 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e12d83640 1 -- 172.21.15.16:0/766095062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 msgr2=0x7f7e0c0b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e12d83640 1 --2- 172.21.15.16:0/766095062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 0x7f7e0c0b4330 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f7df80097a0 tx=0x7f7df802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e12d83640 1 -- 172.21.15.16:0/766095062 shutdown_connections 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e12d83640 1 --2- 172.21.15.16:0/766095062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e0c160b80 0x7f7e0c1617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e12d83640 1 --2- 172.21.15.16:0/766095062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 0x7f7e0c0b4330 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e12d83640 1 -- 172.21.15.16:0/766095062 >> 172.21.15.16:0/766095062 conn(0x7f7e0c09c590 msgr2=0x7f7e0c0d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.276+0000 7f7e12d83640 1 -- 172.21.15.16:0/766095062 shutdown_connections 2023-11-24T06:31:23.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.277+0000 7f7e12d83640 1 -- 172.21.15.16:0/766095062 wait complete. 2023-11-24T06:31:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.278+0000 7f7e12d83640 1 Processor -- start 2023-11-24T06:31:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.279+0000 7f7e12d83640 1 -- start start 2023-11-24T06:31:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.279+0000 7f7e12d83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 0x7f7e0c0b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.279+0000 7f7e12d83640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e0c1607a0 0x7f7e0c15b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e12d83640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c160b80 0x7f7e0c0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e12d83640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b6680 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e12d83640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b6960 con 0x7f7e0c160b80 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e12d83640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b6c40 con 0x7f7e0c1607a0 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e11d81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 0x7f7e0c0b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e11d81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 0x7f7e0c0b7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53818/0 (socket says 172.21.15.16:53818) 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e11d81640 1 -- 172.21.15.16:0/3529009954 learned_addr learned my addr 172.21.15.16:0/3529009954 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e11d81640 1 -- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e0c1607a0 msgr2=0x7f7e0c15b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e11d81640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e0c1607a0 0x7f7e0c15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e11d81640 1 -- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c160b80 msgr2=0x7f7e0c0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e11d81640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c160b80 0x7f7e0c0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e11d81640 1 -- 172.21.15.16:0/3529009954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df8009400 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.280+0000 7f7e11d81640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 0x7f7e0c0b7720 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000cbc0 tx=0x7f7e0000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.281+0000 7f7e0a7fc640 1 -- 172.21.15.16:0/3529009954 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000ad40 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.281+0000 7f7e0a7fc640 1 -- 172.21.15.16:0/3529009954 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e00012400 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.281+0000 7f7e0a7fc640 1 -- 172.21.15.16:0/3529009954 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000aea0 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.281+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c0b6f20 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.281+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c0b6370 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.282+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c0b7440 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.283+0000 7f7e0a7fc640 1 -- 172.21.15.16:0/3529009954 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f7e00007070 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.284+0000 7f7e0a7fc640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7df006b3e0 0x7f7df006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.284+0000 7f7e0a7fc640 1 -- 172.21.15.16:0/3529009954 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7e0008a050 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.285+0000 7f7e11580640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7df006b3e0 0x7f7df006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.286+0000 7f7e11580640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7df006b3e0 0x7f7df006d8a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7df8036000 tx=0x7f7df8030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:23.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.289+0000 7f7e0a7fc640 1 -- 172.21.15.16:0/3529009954 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e0c0b7440 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.484+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e0c0b7440 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.487+0000 7f7e0a7fc640 1 -- 172.21.15.16:0/3529009954 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f7e00017020 con 0x7f7e0c0b3eb0 2023-11-24T06:31:23.491 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:23.491 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:22.236299+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:31:20.522140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:31:21.257255+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.489+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7df006b3e0 msgr2=0x7f7df006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.489+0000 7f7e12d83640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7df006b3e0 0x7f7df006d8a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7df8036000 tx=0x7f7df8030000 comp rx=0 tx=0).stop 2023-11-24T06:31:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 msgr2=0x7f7e0c0b7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 0x7f7e0c0b7720 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000cbc0 tx=0x7f7e0000a430 comp rx=0 tx=0).stop 2023-11-24T06:31:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 shutdown_connections 2023-11-24T06:31:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7df006b3e0 0x7f7df006d8a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e0c160b80 0x7f7e0c0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e0c1607a0 0x7f7e0c15b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 --2- 172.21.15.16:0/3529009954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0b3eb0 0x7f7e0c0b7720 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 >> 172.21.15.16:0/3529009954 conn(0x7f7e0c09c590 msgr2=0x7f7e0c161030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 shutdown_connections 2023-11-24T06:31:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.490+0000 7f7e12d83640 1 -- 172.21.15.16:0/3529009954 wait complete. 2023-11-24T06:31:23.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.748+0000 7fdb668b4640 1 Processor -- start 2023-11-24T06:31:23.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.748+0000 7fdb668b4640 1 -- start start 2023-11-24T06:31:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.749+0000 7fdb668b4640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdb60161740 0x7fdb60168380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.749+0000 7fdb668b4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb600cf260 con 0x7fdb600c9870 2023-11-24T06:31:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.749+0000 7fdb668b4640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb600cf4c0 con 0x7fdb600d0560 2023-11-24T06:31:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.749+0000 7fdb668b4640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb600d2ef0 con 0x7fdb60161740 2023-11-24T06:31:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.750+0000 7fdb5ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdb600c9870 0x7fdb600ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55666/0 (socket says 172.21.15.16:55666) 2023-11-24T06:31:23.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.750+0000 7fdb5ffff640 1 -- 172.21.15.16:0/2132942097 learned_addr learned my addr 172.21.15.16:0/2132942097 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.750+0000 7fdb64e51640 1 --2- 172.21.15.16:0/2132942097 >> v2:172.21.15.173:3300/0 conn(0x7fdb60161740 0x7fdb60168380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.750+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857666093 0 0) 0x7fdb600cf260 con 0x7fdb600c9870 2023-11-24T06:31:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb4c003520 con 0x7fdb600c9870 2023-11-24T06:31:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3001405539 0 0) 0x7fdb4c003520 con 0x7fdb600c9870 2023-11-24T06:31:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 >> v2:172.21.15.173:3300/0 conn(0x7fdb60161740 msgr2=0x7fdb60168380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 --2- 172.21.15.16:0/2132942097 >> v2:172.21.15.173:3300/0 conn(0x7fdb60161740 0x7fdb60168380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 >> v1:172.21.15.173:6789/0 conn(0x7fdb600d0560 legacy=0x7fdb60161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600cf7a0 con 0x7fdb600c9870 2023-11-24T06:31:23.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fdb500019f0 con 0x7fdb600c9870 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 >> v1:172.21.15.16:6789/0 conn(0x7fdb600c9870 legacy=0x7fdb600ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb4c003970 0x7fdb4c003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb4c005180 0x7fdb4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600d2ef0 con 0x7fdb4c0047c0 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c00bfd0 con 0x7fdb4c005180 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.751+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c00c190 con 0x7fdb4c003970 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.752+0000 7fdb5f7fe640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.752+0000 7fdb64e51640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb4c005180 0x7fdb4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.752+0000 7fdb5ffff640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb4c003970 0x7fdb4c003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.752+0000 7fdb5f7fe640 1 -- 172.21.15.16:0/2132942097 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb4c003970 msgr2=0x7fdb4c003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.752+0000 7fdb5f7fe640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb4c003970 0x7fdb4c003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.752+0000 7fdb5f7fe640 1 -- 172.21.15.16:0/2132942097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb4c005180 msgr2=0x7fdb4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.752+0000 7fdb5f7fe640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb4c005180 0x7fdb4c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.752+0000 7fdb5f7fe640 1 -- 172.21.15.16:0/2132942097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c003520 con 0x7fdb4c0047c0 2023-11-24T06:31:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.752+0000 7fdb64e51640 1 -- 172.21.15.16:0/2132942097 reap_dead start 2023-11-24T06:31:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.753+0000 7fdb5f7fe640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fdb48001830 tx=0x7fdb4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ca6d4a1872d98e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:23.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480052d0 con 0x7fdb4c0047c0 2023-11-24T06:31:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4802e050 con 0x7fdb4c0047c0 2023-11-24T06:31:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb5effd640 1 -- 172.21.15.16:0/2132942097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480035f0 con 0x7fdb4c0047c0 2023-11-24T06:31:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb668b4640 1 -- 172.21.15.16:0/2132942097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 msgr2=0x7fdb4c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb668b4640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fdb48001830 tx=0x7fdb4c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb668b4640 1 -- 172.21.15.16:0/2132942097 shutdown_connections 2023-11-24T06:31:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb668b4640 1 --2- 172.21.15.16:0/2132942097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb668b4640 1 -- 172.21.15.16:0/2132942097 >> 172.21.15.16:0/2132942097 conn(0x7fdb6009c560 msgr2=0x7fdb600d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb668b4640 1 -- 172.21.15.16:0/2132942097 shutdown_connections 2023-11-24T06:31:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.754+0000 7fdb668b4640 1 -- 172.21.15.16:0/2132942097 wait complete. 2023-11-24T06:31:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.756+0000 7fdb668b4640 1 Processor -- start 2023-11-24T06:31:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.756+0000 7fdb668b4640 1 -- start start 2023-11-24T06:31:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.757+0000 7fdb668b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb601586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.757+0000 7fdb668b4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6015ec60 0x7fdb6015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.757+0000 7fdb668b4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb601f31a0 0x7fdb601f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.757+0000 7fdb668b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6015f620 con 0x7fdb4c0047c0 2023-11-24T06:31:23.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.757+0000 7fdb668b4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601f3ef0 con 0x7fdb601f31a0 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.757+0000 7fdb668b4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600b7a50 con 0x7fdb6015ec60 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.758+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb601586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.758+0000 7fdb5ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb601586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53840/0 (socket says 172.21.15.16:53840) 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.758+0000 7fdb5ffff640 1 -- 172.21.15.16:0/247768660 learned_addr learned my addr 172.21.15.16:0/247768660 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.758+0000 7fdb5ffff640 1 -- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6015ec60 msgr2=0x7fdb6015f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.758+0000 7fdb5ffff640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6015ec60 0x7fdb6015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.758+0000 7fdb5ffff640 1 -- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb601f31a0 msgr2=0x7fdb601f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.758+0000 7fdb5ffff640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb601f31a0 0x7fdb601f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.758+0000 7fdb5ffff640 1 -- 172.21.15.16:0/247768660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600b7120 con 0x7fdb4c0047c0 2023-11-24T06:31:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.758+0000 7fdb5ffff640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb601586d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdb500079e0 tx=0x7fdb5000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:23.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.759+0000 7fdb5cff9640 1 -- 172.21.15.16:0/247768660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50007ad0 con 0x7fdb4c0047c0 2023-11-24T06:31:23.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.759+0000 7fdb5cff9640 1 -- 172.21.15.16:0/247768660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5000edd0 con 0x7fdb4c0047c0 2023-11-24T06:31:23.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.759+0000 7fdb5cff9640 1 -- 172.21.15.16:0/247768660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50013660 con 0x7fdb4c0047c0 2023-11-24T06:31:23.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.759+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb600b7400 con 0x7fdb4c0047c0 2023-11-24T06:31:23.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.759+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb60158c10 con 0x7fdb4c0047c0 2023-11-24T06:31:23.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.760+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb600675d0 con 0x7fdb4c0047c0 2023-11-24T06:31:23.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.761+0000 7fdb5cff9640 1 -- 172.21.15.16:0/247768660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fdb500137c0 con 0x7fdb4c0047c0 2023-11-24T06:31:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.762+0000 7fdb5cff9640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb4006b410 0x7fdb4006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.762+0000 7fdb5cff9640 1 -- 172.21.15.16:0/247768660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdb5008a680 con 0x7fdb4c0047c0 2023-11-24T06:31:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.763+0000 7fdb5f7fe640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb4006b410 0x7fdb4006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.764+0000 7fdb5f7fe640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb4006b410 0x7fdb4006d8d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdb48033000 tx=0x7fdb48009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.766+0000 7fdb5cff9640 1 -- 172.21.15.16:0/247768660 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb50054ef0 con 0x7fdb4c0047c0 2023-11-24T06:31:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.965+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb600d0a20 con 0x7fdb4c0047c0 2023-11-24T06:31:23.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.967+0000 7fdb5cff9640 1 -- 172.21.15.16:0/247768660 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16604 (secure 0 0 0) 0x7fdb50058510 con 0x7fdb4c0047c0 2023-11-24T06:31:23.971 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:23.769353+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:31:20.522140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:31:21.257255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.970+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb4006b410 msgr2=0x7fdb4006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.970+0000 7fdb668b4640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb4006b410 0x7fdb4006d8d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdb48033000 tx=0x7fdb48009030 comp rx=0 tx=0).stop 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 msgr2=0x7fdb601586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb601586d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fdb500079e0 tx=0x7fdb5000e530 comp rx=0 tx=0).stop 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 shutdown_connections 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fdb4006b410 0x7fdb4006d8d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb601f31a0 0x7fdb601f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb6015ec60 0x7fdb6015f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 --2- 172.21.15.16:0/247768660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb4c0047c0 0x7fdb601586d0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 >> 172.21.15.16:0/247768660 conn(0x7fdb6009c560 msgr2=0x7fdb600ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 shutdown_connections 2023-11-24T06:31:23.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:23.971+0000 7fdb668b4640 1 -- 172.21.15.16:0/247768660 wait complete. 2023-11-24T06:31:23.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:31:23.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:31:24.044 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:31:24.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.241+0000 7f4d5a8bb640 1 Processor -- start 2023-11-24T06:31:24.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.241+0000 7f4d5a8bb640 1 -- start start 2023-11-24T06:31:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.243+0000 7f4d5a8bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d540ccb50 0x7f4d54070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.243+0000 7f4d5a8bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5415f4e0 con 0x7f4d540adf20 2023-11-24T06:31:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.243+0000 7f4d5a8bb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d54071130 con 0x7f4d5415bf10 2023-11-24T06:31:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.243+0000 7f4d5a8bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5415f970 con 0x7f4d540ccb50 2023-11-24T06:31:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.244+0000 7f4d598b9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4d540adf20 0x7f4d540cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55672/0 (socket says 172.21.15.16:55672) 2023-11-24T06:31:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.244+0000 7f4d598b9640 1 -- 172.21.15.16:0/1792369143 learned_addr learned my addr 172.21.15.16:0/1792369143 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.244+0000 7f4d590b8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d540ccb50 0x7f4d54070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.244+0000 7f4d590b8640 1 -- 172.21.15.16:0/1792369143 >> v1:172.21.15.173:6789/0 conn(0x7f4d5415bf10 legacy=0x7f4d54143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.244+0000 7f4d590b8640 1 -- 172.21.15.16:0/1792369143 >> v1:172.21.15.16:6789/0 conn(0x7f4d540adf20 legacy=0x7f4d540cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.244+0000 7f4d590b8640 1 -- 172.21.15.16:0/1792369143 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5415ebb0 con 0x7f4d540ccb50 2023-11-24T06:31:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.245+0000 7f4d590b8640 1 --2- 172.21.15.16:0/1792369143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540ccb50 0x7f4d54070a30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4d440097a0 tx=0x7f4d4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2e618d3f560a3a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:24.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.246+0000 7f4d588b7640 1 -- 172.21.15.16:0/1792369143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4402e070 con 0x7f4d540ccb50 2023-11-24T06:31:24.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.246+0000 7f4d588b7640 1 -- 172.21.15.16:0/1792369143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d440315a0 con 0x7f4d540ccb50 2023-11-24T06:31:24.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.246+0000 7f4d588b7640 1 -- 172.21.15.16:0/1792369143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44031930 con 0x7f4d540ccb50 2023-11-24T06:31:24.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.246+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/1792369143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540ccb50 msgr2=0x7f4d54070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:24.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.246+0000 7f4d5a8bb640 1 --2- 172.21.15.16:0/1792369143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540ccb50 0x7f4d54070a30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f4d440097a0 tx=0x7f4d4402b840 comp rx=0 tx=0).stop 2023-11-24T06:31:24.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.246+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/1792369143 shutdown_connections 2023-11-24T06:31:24.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.246+0000 7f4d5a8bb640 1 --2- 172.21.15.16:0/1792369143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540ccb50 0x7f4d54070a30 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:24.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.246+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/1792369143 >> 172.21.15.16:0/1792369143 conn(0x7f4d5406a7d0 msgr2=0x7f4d5406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:24.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.247+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/1792369143 shutdown_connections 2023-11-24T06:31:24.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.247+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/1792369143 wait complete. 2023-11-24T06:31:24.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.248+0000 7f4d5a8bb640 1 Processor -- start 2023-11-24T06:31:24.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.248+0000 7f4d5a8bb640 1 -- start start 2023-11-24T06:31:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.249+0000 7f4d5a8bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540adf20 0x7f4d5415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.249+0000 7f4d5a8bb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d540ccb50 0x7f4d5413a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.249+0000 7f4d5a8bb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d5415bf10 0x7f4d5413a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:24.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.249+0000 7f4d5a8bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540acb10 con 0x7f4d540adf20 2023-11-24T06:31:24.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.249+0000 7f4d5a8bb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540acdf0 con 0x7f4d540ccb50 2023-11-24T06:31:24.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.249+0000 7f4d5a8bb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540ad0d0 con 0x7f4d5415bf10 2023-11-24T06:31:24.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.250+0000 7f4d598b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540adf20 0x7f4d5415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:24.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.250+0000 7f4d598b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540adf20 0x7f4d5415a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53850/0 (socket says 172.21.15.16:53850) 2023-11-24T06:31:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.250+0000 7f4d598b9640 1 -- 172.21.15.16:0/500143279 learned_addr learned my addr 172.21.15.16:0/500143279 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.250+0000 7f4d598b9640 1 -- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d5415bf10 msgr2=0x7f4d5413a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.250+0000 7f4d598b9640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d5415bf10 0x7f4d5413a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.250+0000 7f4d598b9640 1 -- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d540ccb50 msgr2=0x7f4d5413a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.250+0000 7f4d598b9640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d540ccb50 0x7f4d5413a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:24.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.250+0000 7f4d598b9640 1 -- 172.21.15.16:0/500143279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d44009400 con 0x7f4d540adf20 2023-11-24T06:31:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.250+0000 7f4d598b9640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540adf20 0x7f4d5415a7e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c480 tx=0x7f4d5000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.251+0000 7f4d4a7fc640 1 -- 172.21.15.16:0/500143279 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50005ce0 con 0x7f4d540adf20 2023-11-24T06:31:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.251+0000 7f4d4a7fc640 1 -- 172.21.15.16:0/500143279 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d50005e40 con 0x7f4d540adf20 2023-11-24T06:31:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.251+0000 7f4d4a7fc640 1 -- 172.21.15.16:0/500143279 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d500072f0 con 0x7f4d540adf20 2023-11-24T06:31:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.251+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d540ad3b0 con 0x7f4d540adf20 2023-11-24T06:31:24.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.251+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d540abd90 con 0x7f4d540adf20 2023-11-24T06:31:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.252+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d540704b0 con 0x7f4d540adf20 2023-11-24T06:31:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.253+0000 7f4d4a7fc640 1 -- 172.21.15.16:0/500143279 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4d50007510 con 0x7f4d540adf20 2023-11-24T06:31:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.254+0000 7f4d4a7fc640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3806b3e0 0x7f4d3806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.254+0000 7f4d4a7fc640 1 -- 172.21.15.16:0/500143279 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4d500894f0 con 0x7f4d540adf20 2023-11-24T06:31:24.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.255+0000 7f4d590b8640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3806b3e0 0x7f4d3806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:24.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.256+0000 7f4d590b8640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3806b3e0 0x7f4d3806d8a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4d44036000 tx=0x7f4d44030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:24.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.259+0000 7f4d4a7fc640 1 -- 172.21.15.16:0/500143279 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d50053e50 con 0x7f4d540adf20 2023-11-24T06:31:24.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:24.494+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4d54067450 con 0x7f4d540adf20 2023-11-24T06:31:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.072+0000 7f4d4a7fc640 1 -- 172.21.15.16:0/500143279 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4d50057470 con 0x7f4d540adf20 2023-11-24T06:31:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.075+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3806b3e0 msgr2=0x7f4d3806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3806b3e0 0x7f4d3806d8a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4d44036000 tx=0x7f4d44030000 comp rx=0 tx=0).stop 2023-11-24T06:31:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540adf20 msgr2=0x7f4d5415a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540adf20 0x7f4d5415a7e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c480 tx=0x7f4d5000c940 comp rx=0 tx=0).stop 2023-11-24T06:31:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 shutdown_connections 2023-11-24T06:31:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f4d3806b3e0 0x7f4d3806d8a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d5415bf10 0x7f4d5413a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d540ccb50 0x7f4d5413a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 --2- 172.21.15.16:0/500143279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d540adf20 0x7f4d5415a7e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 >> 172.21.15.16:0/500143279 conn(0x7f4d5406a7d0 msgr2=0x7f4d5415b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 shutdown_connections 2023-11-24T06:31:25.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.076+0000 7f4d5a8bb640 1 -- 172.21.15.16:0/500143279 wait complete. 2023-11-24T06:31:25.099 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:31:25.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:31:25.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.328+0000 7f55c2221640 1 Processor -- start 2023-11-24T06:31:25.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.328+0000 7f55c2221640 1 -- start start 2023-11-24T06:31:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.330+0000 7f55c2221640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc1505b0 0x7f55bc0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.330+0000 7f55c2221640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0b1e30 con 0x7f55bc15a690 2023-11-24T06:31:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.330+0000 7f55c2221640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0b20a0 con 0x7f55bc06d6d0 2023-11-24T06:31:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.330+0000 7f55c2221640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0b2380 con 0x7f55bc1505b0 2023-11-24T06:31:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.331+0000 7f55c0a1e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f55bc15a690 0x7f55bc06cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55682/0 (socket says 172.21.15.16:55682) 2023-11-24T06:31:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.331+0000 7f55c0a1e640 1 -- 172.21.15.16:0/3528048741 learned_addr learned my addr 172.21.15.16:0/3528048741 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.331+0000 7f55c121f640 1 --2- 172.21.15.16:0/3528048741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc1505b0 0x7f55bc0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.332+0000 7f55b3fff640 1 -- 172.21.15.16:0/3528048741 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430520138 0 0) 0x7f55bc0b1e30 con 0x7f55bc15a690 2023-11-24T06:31:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.332+0000 7f55b3fff640 1 -- 172.21.15.16:0/3528048741 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a8003520 con 0x7f55bc15a690 2023-11-24T06:31:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.332+0000 7f55c121f640 1 -- 172.21.15.16:0/3528048741 >> v1:172.21.15.173:6789/0 conn(0x7f55bc06d6d0 legacy=0x7f55bc0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.332+0000 7f55b3fff640 1 -- 172.21.15.16:0/3528048741 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908994069 0 0) 0x7f55bc0b20a0 con 0x7f55bc06d6d0 2023-11-24T06:31:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.332+0000 7f55c121f640 1 -- 172.21.15.16:0/3528048741 >> v1:172.21.15.16:6789/0 conn(0x7f55bc15a690 legacy=0x7f55bc06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:25.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.332+0000 7f55c121f640 1 -- 172.21.15.16:0/3528048741 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0b2660 con 0x7f55bc1505b0 2023-11-24T06:31:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.333+0000 7f55c121f640 1 --2- 172.21.15.16:0/3528048741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc1505b0 0x7f55bc0b3b20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55b80097d0 tx=0x7f55b802b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9804d7a46bc1ea87 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.334+0000 7f55b3fff640 1 -- 172.21.15.16:0/3528048741 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0015f0 con 0x7f55bc1505b0 2023-11-24T06:31:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.334+0000 7f55b3fff640 1 -- 172.21.15.16:0/3528048741 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b802e050 con 0x7f55bc1505b0 2023-11-24T06:31:25.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.334+0000 7f55b3fff640 1 -- 172.21.15.16:0/3528048741 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8030710 con 0x7f55bc1505b0 2023-11-24T06:31:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.335+0000 7f55c2221640 1 -- 172.21.15.16:0/3528048741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc1505b0 msgr2=0x7f55bc0b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:25.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.335+0000 7f55c2221640 1 --2- 172.21.15.16:0/3528048741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc1505b0 0x7f55bc0b3b20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55b80097d0 tx=0x7f55b802b730 comp rx=0 tx=0).stop 2023-11-24T06:31:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.335+0000 7f55c2221640 1 -- 172.21.15.16:0/3528048741 shutdown_connections 2023-11-24T06:31:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.335+0000 7f55c2221640 1 --2- 172.21.15.16:0/3528048741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc1505b0 0x7f55bc0b3b20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.335+0000 7f55c2221640 1 -- 172.21.15.16:0/3528048741 >> 172.21.15.16:0/3528048741 conn(0x7f55bc06a7d0 msgr2=0x7f55bc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.335+0000 7f55c2221640 1 -- 172.21.15.16:0/3528048741 shutdown_connections 2023-11-24T06:31:25.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.335+0000 7f55c2221640 1 -- 172.21.15.16:0/3528048741 wait complete. 2023-11-24T06:31:25.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.337+0000 7f55c2221640 1 Processor -- start 2023-11-24T06:31:25.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.338+0000 7f55c2221640 1 -- start start 2023-11-24T06:31:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.339+0000 7f55c2221640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55bc06d6d0 0x7f55bc070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.339+0000 7f55c2221640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55bc1505b0 0x7f55bc1495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.339+0000 7f55c2221640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc15a690 0x7f55bc147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.339+0000 7f55c2221640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0706a0 con 0x7f55bc06d6d0 2023-11-24T06:31:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.339+0000 7f55c2221640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc149b30 con 0x7f55bc1505b0 2023-11-24T06:31:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.339+0000 7f55c2221640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc070890 con 0x7f55bc15a690 2023-11-24T06:31:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.339+0000 7f55c121f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55bc06d6d0 0x7f55bc070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c121f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55bc06d6d0 0x7f55bc070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53864/0 (socket says 172.21.15.16:53864) 2023-11-24T06:31:25.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c121f640 1 -- 172.21.15.16:0/706536090 learned_addr learned my addr 172.21.15.16:0/706536090 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c0a1e640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55bc1505b0 0x7f55bc1495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:25.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c1a20640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc15a690 0x7f55bc147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c121f640 1 -- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc15a690 msgr2=0x7f55bc147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c121f640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc15a690 0x7f55bc147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c121f640 1 -- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55bc1505b0 msgr2=0x7f55bc1495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c121f640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55bc1505b0 0x7f55bc1495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c121f640 1 -- 172.21.15.16:0/706536090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8009400 con 0x7f55bc06d6d0 2023-11-24T06:31:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.340+0000 7f55c1a20640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc15a690 0x7f55bc147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:31:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.341+0000 7f55c121f640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55bc06d6d0 0x7f55bc070160 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f55b802b770 tx=0x7f55b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:25.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.341+0000 7f55b1ffb640 1 -- 172.21.15.16:0/706536090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b803d890 con 0x7f55bc06d6d0 2023-11-24T06:31:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.341+0000 7f55b1ffb640 1 -- 172.21.15.16:0/706536090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b803d9f0 con 0x7f55bc06d6d0 2023-11-24T06:31:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.341+0000 7f55b1ffb640 1 -- 172.21.15.16:0/706536090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b803dda0 con 0x7f55bc06d6d0 2023-11-24T06:31:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.341+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc070b70 con 0x7f55bc06d6d0 2023-11-24T06:31:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.341+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc071130 con 0x7f55bc06d6d0 2023-11-24T06:31:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.342+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc067450 con 0x7f55bc06d6d0 2023-11-24T06:31:25.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.343+0000 7f55b1ffb640 1 -- 172.21.15.16:0/706536090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f55b8039040 con 0x7f55bc06d6d0 2023-11-24T06:31:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.344+0000 7f55b1ffb640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55a006b3e0 0x7f55a006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:25.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.344+0000 7f55b1ffb640 1 -- 172.21.15.16:0/706536090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f55b80ac1a0 con 0x7f55bc06d6d0 2023-11-24T06:31:25.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.345+0000 7f55c0a1e640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55a006b3e0 0x7f55a006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:25.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.346+0000 7f55c0a1e640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55a006b3e0 0x7f55a006d8a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0b28c0 tx=0x7f55ac001870 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:25.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.349+0000 7f55b1ffb640 1 -- 172.21.15.16:0/706536090 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55b8076900 con 0x7f55bc06d6d0 2023-11-24T06:31:25.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:25.581+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f55bc0b3660 con 0x7f55bc06d6d0 2023-11-24T06:31:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.075+0000 7f55b1ffb640 1 -- 172.21.15.16:0/706536090 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f55b8079f20 con 0x7f55bc06d6d0 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55a006b3e0 msgr2=0x7f55a006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55a006b3e0 0x7f55a006d8a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0b28c0 tx=0x7f55ac001870 comp rx=0 tx=0).stop 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55bc06d6d0 msgr2=0x7f55bc070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55bc06d6d0 0x7f55bc070160 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f55b802b770 tx=0x7f55b80046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 shutdown_connections 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55a006b3e0 0x7f55a006d8a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55bc15a690 0x7f55bc147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55bc1505b0 0x7f55bc1495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 --2- 172.21.15.16:0/706536090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55bc06d6d0 0x7f55bc070160 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.079+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 >> 172.21.15.16:0/706536090 conn(0x7f55bc06a7d0 msgr2=0x7f55bc06abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.080+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 shutdown_connections 2023-11-24T06:31:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.080+0000 7f55c2221640 1 -- 172.21.15.16:0/706536090 wait complete. 2023-11-24T06:31:26.107 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:31:26.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.330+0000 7f67e9706640 1 Processor -- start 2023-11-24T06:31:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.331+0000 7f67e9706640 1 -- start start 2023-11-24T06:31:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.332+0000 7f67e9706640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67e40cc990 0x7f67e406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.332+0000 7f67e9706640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e4070e50 con 0x7f67e40cc500 2023-11-24T06:31:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.332+0000 7f67e9706640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e4144020 con 0x7f67e4143c40 2023-11-24T06:31:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.332+0000 7f67e9706640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e40723c0 con 0x7f67e40cc990 2023-11-24T06:31:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.333+0000 7f67e8f05640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67e40cc990 0x7f67e406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.333+0000 7f67e8f05640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67e40cc990 0x7f67e406d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53876/0 (socket says 172.21.15.16:53876) 2023-11-24T06:31:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.333+0000 7f67e8f05640 1 -- 172.21.15.16:0/3662233707 learned_addr learned my addr 172.21.15.16:0/3662233707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.333+0000 7f67e8f05640 1 -- 172.21.15.16:0/3662233707 >> v1:172.21.15.173:6789/0 conn(0x7f67e4143c40 legacy=0x7f67e4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.333+0000 7f67e8f05640 1 -- 172.21.15.16:0/3662233707 >> v1:172.21.15.16:6789/0 conn(0x7f67e40cc500 legacy=0x7f67e406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.333+0000 7f67e8f05640 1 -- 172.21.15.16:0/3662233707 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e4071e00 con 0x7f67e40cc990 2023-11-24T06:31:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.334+0000 7f67e8f05640 1 --2- 172.21.15.16:0/3662233707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc990 0x7f67e406d870 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009870 tx=0x7f67dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ff8639538d96e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:26.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.335+0000 7f67e2ffd640 1 -- 172.21.15.16:0/3662233707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc031870 con 0x7f67e40cc990 2023-11-24T06:31:26.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.335+0000 7f67e2ffd640 1 -- 172.21.15.16:0/3662233707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc0319d0 con 0x7f67e40cc990 2023-11-24T06:31:26.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.335+0000 7f67e2ffd640 1 -- 172.21.15.16:0/3662233707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc031d60 con 0x7f67e40cc990 2023-11-24T06:31:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.335+0000 7f67e9706640 1 -- 172.21.15.16:0/3662233707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc990 msgr2=0x7f67e406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.335+0000 7f67e9706640 1 --2- 172.21.15.16:0/3662233707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc990 0x7f67e406d870 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009870 tx=0x7f67dc02b910 comp rx=0 tx=0).stop 2023-11-24T06:31:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.335+0000 7f67e9706640 1 -- 172.21.15.16:0/3662233707 shutdown_connections 2023-11-24T06:31:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.335+0000 7f67e9706640 1 --2- 172.21.15.16:0/3662233707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc990 0x7f67e406d870 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.335+0000 7f67e9706640 1 -- 172.21.15.16:0/3662233707 >> 172.21.15.16:0/3662233707 conn(0x7f67e406a800 msgr2=0x7f67e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.335+0000 7f67e9706640 1 -- 172.21.15.16:0/3662233707 shutdown_connections 2023-11-24T06:31:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.336+0000 7f67e9706640 1 -- 172.21.15.16:0/3662233707 wait complete. 2023-11-24T06:31:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.338+0000 7f67e9706640 1 Processor -- start 2023-11-24T06:31:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.338+0000 7f67e9706640 1 -- start start 2023-11-24T06:31:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e9706640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc500 0x7f67e40c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e9706640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e40cc990 0x7f67e4133350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e9706640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67e4143c40 0x7f67e40c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e9706640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40c8c40 con 0x7f67e40cc500 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e9706640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4133890 con 0x7f67e40cc990 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e9706640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4133c20 con 0x7f67e4143c40 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc500 0x7f67e40c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc500 0x7f67e40c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53882/0 (socket says 172.21.15.16:53882) 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e3fff640 1 -- 172.21.15.16:0/4003832823 learned_addr learned my addr 172.21.15.16:0/4003832823 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e3fff640 1 -- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67e4143c40 msgr2=0x7f67e40c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e3fff640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67e4143c40 0x7f67e40c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e3fff640 1 -- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e40cc990 msgr2=0x7f67e4133350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e3fff640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e40cc990 0x7f67e4133350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.340+0000 7f67e3fff640 1 -- 172.21.15.16:0/4003832823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc0094d0 con 0x7f67e40cc500 2023-11-24T06:31:26.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.341+0000 7f67e3fff640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc500 0x7f67e40c8700 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f67d400ba50 tx=0x7f67d400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:26.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.341+0000 7f67e0ff9640 1 -- 172.21.15.16:0/4003832823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400cbe0 con 0x7f67e40cc500 2023-11-24T06:31:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.341+0000 7f67e0ff9640 1 -- 172.21.15.16:0/4003832823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d400cd80 con 0x7f67e40cc500 2023-11-24T06:31:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.341+0000 7f67e0ff9640 1 -- 172.21.15.16:0/4003832823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4014680 con 0x7f67e40cc500 2023-11-24T06:31:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.341+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e4133f00 con 0x7f67e40cc500 2023-11-24T06:31:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.341+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e41344c0 con 0x7f67e40cc500 2023-11-24T06:31:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.342+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e4067450 con 0x7f67e40cc500 2023-11-24T06:31:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.343+0000 7f67e0ff9640 1 -- 172.21.15.16:0/4003832823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f67d40147e0 con 0x7f67e40cc500 2023-11-24T06:31:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.344+0000 7f67e0ff9640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f67c406b3e0 0x7f67c406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.344+0000 7f67e0ff9640 1 -- 172.21.15.16:0/4003832823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f67d4089f60 con 0x7f67e40cc500 2023-11-24T06:31:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.345+0000 7f67e37fe640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f67c406b3e0 0x7f67c406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:26.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.346+0000 7f67e37fe640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f67c406b3e0 0x7f67c406d8a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f67cc000f60 tx=0x7f67cc0079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.348+0000 7f67e0ff9640 1 -- 172.21.15.16:0/4003832823 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67d40547f0 con 0x7f67e40cc500 2023-11-24T06:31:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.577+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67e40ceb30 con 0x7f67e40cc500 2023-11-24T06:31:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.578+0000 7f67e0ff9640 1 -- 172.21.15.16:0/4003832823 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f67d4019090 con 0x7f67e40cc500 2023-11-24T06:31:26.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:26.582 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3235524397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":3235524397},{"type":"v1","addr":"172.21.15.173:6837","nonce":3235524397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/593524744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":593524744},{"type":"v1","addr":"172.21.15.173:6836","nonce":593524744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7173,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/825050165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":825050165},{"type":"v1","addr":"172.21.15.16:6834","nonce":825050165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:31:22.237194+0000","modified":"2023-11-24T06:31:25.583915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2773872585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2773872585},{"type":"v1","addr":"172.21.15.16:6835","nonce":2773872585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:31:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 53 2023-11-24T06:31:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f67c406b3e0 msgr2=0x7f67c406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f67c406b3e0 0x7f67c406d8a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f67cc000f60 tx=0x7f67cc0079b0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc500 msgr2=0x7f67e40c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc500 0x7f67e40c8700 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f67d400ba50 tx=0x7f67d400bf10 comp rx=0 tx=0).stop 2023-11-24T06:31:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 shutdown_connections 2023-11-24T06:31:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f67c406b3e0 0x7f67c406d8a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67e4143c40 0x7f67e40c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e40cc990 0x7f67e4133350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 --2- 172.21.15.16:0/4003832823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e40cc500 0x7f67e40c8700 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 >> 172.21.15.16:0/4003832823 conn(0x7f67e406a800 msgr2=0x7f67e40acee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 shutdown_connections 2023-11-24T06:31:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.581+0000 7f67e9706640 1 -- 172.21.15.16:0/4003832823 wait complete. 2023-11-24T06:31:26.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.823+0000 7f1120947640 1 Processor -- start 2023-11-24T06:31:26.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.823+0000 7f1120947640 1 -- start start 2023-11-24T06:31:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.825+0000 7f1120947640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f111c162810 0x7f111c071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.825+0000 7f1120947640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c071780 0x7f111c06c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.825+0000 7f1120947640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c071c10 con 0x7f111c162430 2023-11-24T06:31:26.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.825+0000 7f1120947640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c0702e0 con 0x7f111c162810 2023-11-24T06:31:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.825+0000 7f1120947640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0705c0 con 0x7f111c071780 2023-11-24T06:31:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.826+0000 7f111b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f111c162430 0x7f111c0b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55694/0 (socket says 172.21.15.16:55694) 2023-11-24T06:31:26.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.826+0000 7f111b7fe640 1 -- 172.21.15.16:0/1168657980 learned_addr learned my addr 172.21.15.16:0/1168657980 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:26.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.826+0000 7f1112ffd640 1 --2- 172.21.15.16:0/1168657980 >> v2:172.21.15.16:3300/0 conn(0x7f111c162810 0x7f111c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:26.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.826+0000 7f111bfff640 1 --2- 172.21.15.16:0/1168657980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c071780 0x7f111c06c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:26.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.826+0000 7f1112ffd640 1 -- 172.21.15.16:0/1168657980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c071780 msgr2=0x7f111c06c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:26.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.826+0000 7f1112ffd640 1 --2- 172.21.15.16:0/1168657980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c071780 0x7f111c06c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.826+0000 7f1112ffd640 1 -- 172.21.15.16:0/1168657980 >> v1:172.21.15.16:6789/0 conn(0x7f111c162430 legacy=0x7f111c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.826+0000 7f1112ffd640 1 -- 172.21.15.16:0/1168657980 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0708a0 con 0x7f111c162810 2023-11-24T06:31:26.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.826+0000 7f111affd640 1 -- 172.21.15.16:0/1168657980 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188700784 0 0) 0x7f111c071c10 con 0x7f111c162430 2023-11-24T06:31:26.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.827+0000 7f1112ffd640 1 --2- 172.21.15.16:0/1168657980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c162810 0x7f111c071240 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f11040097a0 tx=0x7f110402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502c2d5a47fbc47f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:26.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.828+0000 7f111affd640 1 -- 172.21.15.16:0/1168657980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104031830 con 0x7f111c162810 2023-11-24T06:31:26.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.828+0000 7f111affd640 1 -- 172.21.15.16:0/1168657980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1104031990 con 0x7f111c162810 2023-11-24T06:31:26.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.828+0000 7f111affd640 1 -- 172.21.15.16:0/1168657980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104031d20 con 0x7f111c162810 2023-11-24T06:31:26.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.828+0000 7f1120947640 1 -- 172.21.15.16:0/1168657980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c162810 msgr2=0x7f111c071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.828+0000 7f1120947640 1 --2- 172.21.15.16:0/1168657980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c162810 0x7f111c071240 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f11040097a0 tx=0x7f110402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.829+0000 7f1120947640 1 -- 172.21.15.16:0/1168657980 shutdown_connections 2023-11-24T06:31:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.829+0000 7f1120947640 1 --2- 172.21.15.16:0/1168657980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c071780 0x7f111c06c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.829+0000 7f1120947640 1 --2- 172.21.15.16:0/1168657980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c162810 0x7f111c071240 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.829+0000 7f1120947640 1 -- 172.21.15.16:0/1168657980 >> 172.21.15.16:0/1168657980 conn(0x7f111c06a800 msgr2=0x7f111c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:26.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.829+0000 7f1120947640 1 -- 172.21.15.16:0/1168657980 shutdown_connections 2023-11-24T06:31:26.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.829+0000 7f1120947640 1 -- 172.21.15.16:0/1168657980 wait complete. 2023-11-24T06:31:26.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.831+0000 7f1120947640 1 Processor -- start 2023-11-24T06:31:26.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.832+0000 7f1120947640 1 -- start start 2023-11-24T06:31:26.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.833+0000 7f1120947640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f111c071780 0x7f111c162e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.833+0000 7f1120947640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c162430 0x7f111c1f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.833+0000 7f1120947640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c1f6d40 0x7f111c152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.833+0000 7f1120947640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1592c0 con 0x7f111c1f6d40 2023-11-24T06:31:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.833+0000 7f1120947640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c153950 con 0x7f111c071780 2023-11-24T06:31:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.833+0000 7f1120947640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1f71d0 con 0x7f111c162430 2023-11-24T06:31:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.833+0000 7f111bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c1f6d40 0x7f111c152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.833+0000 7f111bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c1f6d40 0x7f111c152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53896/0 (socket says 172.21.15.16:53896) 2023-11-24T06:31:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.833+0000 7f111bfff640 1 -- 172.21.15.16:0/3775472482 learned_addr learned my addr 172.21.15.16:0/3775472482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.834+0000 7f111b7fe640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f111c071780 0x7f111c162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.834+0000 7f1112ffd640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c162430 0x7f111c1f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.834+0000 7f111bfff640 1 -- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c162430 msgr2=0x7f111c1f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.834+0000 7f111bfff640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c162430 0x7f111c1f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.834+0000 7f111bfff640 1 -- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f111c071780 msgr2=0x7f111c162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.834+0000 7f111bfff640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f111c071780 0x7f111c162e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.834+0000 7f111bfff640 1 -- 172.21.15.16:0/3775472482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1104009400 con 0x7f111c1f6d40 2023-11-24T06:31:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.834+0000 7f111bfff640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c1f6d40 0x7f111c152940 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1114007e40 tx=0x7f111400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.834+0000 7f1118ff9640 1 -- 172.21.15.16:0/3775472482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114014620 con 0x7f111c1f6d40 2023-11-24T06:31:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.835+0000 7f1118ff9640 1 -- 172.21.15.16:0/3775472482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111400ee90 con 0x7f111c1f6d40 2023-11-24T06:31:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.835+0000 7f1118ff9640 1 -- 172.21.15.16:0/3775472482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140149e0 con 0x7f111c1f6d40 2023-11-24T06:31:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.835+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c153670 con 0x7f111c1f6d40 2023-11-24T06:31:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.835+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c158f50 con 0x7f111c1f6d40 2023-11-24T06:31:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.836+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c159470 con 0x7f111c1f6d40 2023-11-24T06:31:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.837+0000 7f1118ff9640 1 -- 172.21.15.16:0/3775472482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1114014b40 con 0x7f111c1f6d40 2023-11-24T06:31:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.838+0000 7f1118ff9640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f10fc06b3e0 0x7f10fc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.838+0000 7f1118ff9640 1 -- 172.21.15.16:0/3775472482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f111408aa70 con 0x7f111c1f6d40 2023-11-24T06:31:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.838+0000 7f111b7fe640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f10fc06b3e0 0x7f10fc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.840+0000 7f111b7fe640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f10fc06b3e0 0x7f10fc06d8a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f110c000f60 tx=0x7f110c007b50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:26.842+0000 7f1118ff9640 1 -- 172.21.15.16:0/3775472482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f111c159470 con 0x7f111c1f6d40 2023-11-24T06:31:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.030+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f111c159470 con 0x7f111c1f6d40 2023-11-24T06:31:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.033+0000 7f1118ff9640 1 -- 172.21.15.16:0/3775472482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16604 (secure 0 0 0) 0x7f111c159470 con 0x7f111c1f6d40 2023-11-24T06:31:27.036 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:27.037 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:23.769353+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:31:20.522140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:31:21.257255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f10fc06b3e0 msgr2=0x7f10fc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f10fc06b3e0 0x7f10fc06d8a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f110c000f60 tx=0x7f110c007b50 comp rx=0 tx=0).stop 2023-11-24T06:31:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c1f6d40 msgr2=0x7f111c152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c1f6d40 0x7f111c152940 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1114007e40 tx=0x7f111400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 shutdown_connections 2023-11-24T06:31:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f10fc06b3e0 0x7f10fc06d8a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c1f6d40 0x7f111c152940 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f111c162430 0x7f111c1f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 --2- 172.21.15.16:0/3775472482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f111c071780 0x7f111c162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.036+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 >> 172.21.15.16:0/3775472482 conn(0x7f111c06a800 msgr2=0x7f111c0b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.037+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 shutdown_connections 2023-11-24T06:31:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.037+0000 7f1120947640 1 -- 172.21.15.16:0/3775472482 wait complete. 2023-11-24T06:31:27.060 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:31:27.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:31:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.271+0000 7f6009621640 1 Processor -- start 2023-11-24T06:31:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.271+0000 7f6009621640 1 -- start start 2023-11-24T06:31:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.273+0000 7f6009621640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6004160920 0x7f60040b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.273+0000 7f6009621640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f60040cf9f0 0x7f60040cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.273+0000 7f6009621640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60040d0a10 con 0x7f6004161010 2023-11-24T06:31:27.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.273+0000 7f6009621640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60040d03e0 con 0x7f6004160920 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.273+0000 7f6009621640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60040d1f50 con 0x7f60040cf9f0 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.273+0000 7f6003fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6004160920 0x7f60040b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.273+0000 7f6008e20640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6004161010 0x7f6004161480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55702/0 (socket says 172.21.15.16:55702) 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.273+0000 7f6008e20640 1 -- 172.21.15.16:0/2592781046 learned_addr learned my addr 172.21.15.16:0/2592781046 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.274+0000 7f5ffb7fe640 1 --2- 172.21.15.16:0/2592781046 >> v2:172.21.15.173:3300/0 conn(0x7f60040cf9f0 0x7f60040cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.274+0000 7f6003fff640 1 -- 172.21.15.16:0/2592781046 >> v2:172.21.15.173:3300/0 conn(0x7f60040cf9f0 msgr2=0x7f60040cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.274+0000 7f6003fff640 1 --2- 172.21.15.16:0/2592781046 >> v2:172.21.15.173:3300/0 conn(0x7f60040cf9f0 0x7f60040cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.274+0000 7f60037fe640 1 -- 172.21.15.16:0/2592781046 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141914659 0 0) 0x7f60040d0a10 con 0x7f6004161010 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.274+0000 7f6003fff640 1 -- 172.21.15.16:0/2592781046 >> v1:172.21.15.16:6789/0 conn(0x7f6004161010 legacy=0x7f6004161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.274+0000 7f6003fff640 1 -- 172.21.15.16:0/2592781046 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040cf080 con 0x7f6004160920 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.274+0000 7f5ffb7fe640 1 --2- 172.21.15.16:0/2592781046 >> v2:172.21.15.173:3300/0 conn(0x7f60040cf9f0 0x7f60040cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:31:27.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.275+0000 7f6003fff640 1 --2- 172.21.15.16:0/2592781046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 0x7f60040b3f30 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f600408c1f0 tx=0x7f5ff402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef78288cb57f65e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.275+0000 7f60037fe640 1 -- 172.21.15.16:0/2592781046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4031830 con 0x7f6004160920 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.275+0000 7f60037fe640 1 -- 172.21.15.16:0/2592781046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff4031990 con 0x7f6004160920 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.275+0000 7f60037fe640 1 -- 172.21.15.16:0/2592781046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4031d20 con 0x7f6004160920 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.276+0000 7f6009621640 1 -- 172.21.15.16:0/2592781046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 msgr2=0x7f60040b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.276+0000 7f6009621640 1 --2- 172.21.15.16:0/2592781046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 0x7f60040b3f30 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f600408c1f0 tx=0x7f5ff402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.276+0000 7f6009621640 1 -- 172.21.15.16:0/2592781046 shutdown_connections 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.276+0000 7f6009621640 1 --2- 172.21.15.16:0/2592781046 >> v2:172.21.15.173:3300/0 conn(0x7f60040cf9f0 0x7f60040cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.276+0000 7f6009621640 1 --2- 172.21.15.16:0/2592781046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 0x7f60040b3f30 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.276+0000 7f6009621640 1 -- 172.21.15.16:0/2592781046 >> 172.21.15.16:0/2592781046 conn(0x7f60040d3090 msgr2=0x7f60040d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.276+0000 7f6009621640 1 -- 172.21.15.16:0/2592781046 shutdown_connections 2023-11-24T06:31:27.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.276+0000 7f6009621640 1 -- 172.21.15.16:0/2592781046 wait complete. 2023-11-24T06:31:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.278+0000 7f6009621640 1 Processor -- start 2023-11-24T06:31:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.278+0000 7f6009621640 1 -- start start 2023-11-24T06:31:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.279+0000 7f6009621640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60040cf9f0 0x7f600414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.279+0000 7f6009621640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 0x7f600415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.279+0000 7f6009621640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6004161010 0x7f6004148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.279+0000 7f6009621640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040b8cf0 con 0x7f6004160920 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.279+0000 7f6009621640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040b6130 con 0x7f60040cf9f0 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.279+0000 7f6009621640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60040b8730 con 0x7f6004161010 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.279+0000 7f5ffb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 0x7f600415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.279+0000 7f5ffb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 0x7f600415dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53918/0 (socket says 172.21.15.16:53918) 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.279+0000 7f5ffb7fe640 1 -- 172.21.15.16:0/761858505 learned_addr learned my addr 172.21.15.16:0/761858505 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.280+0000 7f6003fff640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60040cf9f0 0x7f600414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.280+0000 7f6008e20640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6004161010 0x7f6004148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.280+0000 7f5ffb7fe640 1 -- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6004161010 msgr2=0x7f6004148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.280+0000 7f5ffb7fe640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6004161010 0x7f6004148cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.280+0000 7f5ffb7fe640 1 -- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60040cf9f0 msgr2=0x7f600414f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.280+0000 7f5ffb7fe640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60040cf9f0 0x7f600414f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.280+0000 7f5ffb7fe640 1 -- 172.21.15.16:0/761858505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff4009400 con 0x7f6004160920 2023-11-24T06:31:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.280+0000 7f5ffb7fe640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 0x7f600415dc90 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5fec007ef0 tx=0x7f5fec00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.281+0000 7f60017fa640 1 -- 172.21.15.16:0/761858505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec003b90 con 0x7f6004160920 2023-11-24T06:31:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.281+0000 7f60017fa640 1 -- 172.21.15.16:0/761858505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec00f040 con 0x7f6004160920 2023-11-24T06:31:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.281+0000 7f60017fa640 1 -- 172.21.15.16:0/761858505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec00de80 con 0x7f6004160920 2023-11-24T06:31:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.281+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60040b8a10 con 0x7f6004160920 2023-11-24T06:31:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.281+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600414be10 con 0x7f6004160920 2023-11-24T06:31:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.282+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6004066640 con 0x7f6004160920 2023-11-24T06:31:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.283+0000 7f60017fa640 1 -- 172.21.15.16:0/761858505 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5fec004030 con 0x7f6004160920 2023-11-24T06:31:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.284+0000 7f60017fa640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fe406b3e0 0x7f5fe406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.284+0000 7f60017fa640 1 -- 172.21.15.16:0/761858505 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5fec08a3d0 con 0x7f6004160920 2023-11-24T06:31:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.285+0000 7f6003fff640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fe406b3e0 0x7f5fe406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.286+0000 7f6003fff640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fe406b3e0 0x7f5fe406d8a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4036000 tx=0x7f5ff403c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.288+0000 7f60017fa640 1 -- 172.21.15.16:0/761858505 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5fec054be0 con 0x7f6004160920 2023-11-24T06:31:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.479+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60040ceac0 con 0x7f5fe406b3e0 2023-11-24T06:31:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.505+0000 7f60017fa640 1 -- 172.21.15.16:0/761858505 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286885 (secure 0 0 0) 0x7f5ff4045410 con 0x7f5fe406b3e0 2023-11-24T06:31:27.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:27.514 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:31:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fe406b3e0 msgr2=0x7f5fe406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fe406b3e0 0x7f5fe406d8a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4036000 tx=0x7f5ff403c040 comp rx=0 tx=0).stop 2023-11-24T06:31:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 msgr2=0x7f600415dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 0x7f600415dc90 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5fec007ef0 tx=0x7f5fec00d4a0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 shutdown_connections 2023-11-24T06:31:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5fe406b3e0 0x7f5fe406d8a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6004161010 0x7f6004148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6004160920 0x7f600415dc90 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 --2- 172.21.15.16:0/761858505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60040cf9f0 0x7f600414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 >> 172.21.15.16:0/761858505 conn(0x7f60040d3090 msgr2=0x7f60040c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.512+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 shutdown_connections 2023-11-24T06:31:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.513+0000 7f6009621640 1 -- 172.21.15.16:0/761858505 wait complete. 2023-11-24T06:31:27.537 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2023-11-24T06:31:26.195783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258892,"kb_used_data":9396,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715828,"statfs":{"total":773094113280,"available":772829007872,"internally_reserved":0,"allocated":9621504,"data_stored":4763268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.448834"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.813796+0000","last_change":"2023-11-24T06:31:22.809145+0000","last_active":"2023-11-24T06:31:22.813796+0000","last_peered":"2023-11-24T06:31:22.813796+0000","last_clean":"2023-11-24T06:31:22.813796+0000","last_became_active":"2023-11-24T06:31:22.808851+0000","last_became_peered":"2023-11-24T06:31:22.808851+0000","last_unstale":"2023-11-24T06:31:22.813796+0000","last_undegraded":"2023-11-24T06:31:22.813796+0000","last_fullsized":"2023-11-24T06:31:22.813796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:47.718070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816048+0000","last_change":"2023-11-24T06:31:22.812457+0000","last_active":"2023-11-24T06:31:22.816048+0000","last_peered":"2023-11-24T06:31:22.816048+0000","last_clean":"2023-11-24T06:31:22.816048+0000","last_became_active":"2023-11-24T06:31:22.812147+0000","last_became_peered":"2023-11-24T06:31:22.812147+0000","last_unstale":"2023-11-24T06:31:22.816048+0000","last_undegraded":"2023-11-24T06:31:22.816048+0000","last_fullsized":"2023-11-24T06:31:22.816048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:07.045761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.781784+0000","last_change":"2023-11-24T06:31:22.805000+0000","last_active":"2023-11-24T06:31:23.781784+0000","last_peered":"2023-11-24T06:31:23.781784+0000","last_clean":"2023-11-24T06:31:23.781784+0000","last_became_active":"2023-11-24T06:31:22.804696+0000","last_became_peered":"2023-11-24T06:31:22.804696+0000","last_unstale":"2023-11-24T06:31:23.781784+0000","last_undegraded":"2023-11-24T06:31:23.781784+0000","last_fullsized":"2023-11-24T06:31:23.781784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:00.114881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775146+0000","last_change":"2023-11-24T06:31:22.814662+0000","last_active":"2023-11-24T06:31:23.775146+0000","last_peered":"2023-11-24T06:31:23.775146+0000","last_clean":"2023-11-24T06:31:23.775146+0000","last_became_active":"2023-11-24T06:31:22.814120+0000","last_became_peered":"2023-11-24T06:31:22.814120+0000","last_unstale":"2023-11-24T06:31:23.775146+0000","last_undegraded":"2023-11-24T06:31:23.775146+0000","last_fullsized":"2023-11-24T06:31:23.775146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:35.282625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.814911+0000","last_change":"2023-11-24T06:31:22.808935+0000","last_active":"2023-11-24T06:31:22.814911+0000","last_peered":"2023-11-24T06:31:22.814911+0000","last_clean":"2023-11-24T06:31:22.814911+0000","last_became_active":"2023-11-24T06:31:22.808636+0000","last_became_peered":"2023-11-24T06:31:22.808636+0000","last_unstale":"2023-11-24T06:31:22.814911+0000","last_undegraded":"2023-11-24T06:31:22.814911+0000","last_fullsized":"2023-11-24T06:31:22.814911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:02.080378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816661+0000","last_change":"2023-11-24T06:31:22.814342+0000","last_active":"2023-11-24T06:31:22.816661+0000","last_peered":"2023-11-24T06:31:22.816661+0000","last_clean":"2023-11-24T06:31:22.816661+0000","last_became_active":"2023-11-24T06:31:22.812307+0000","last_became_peered":"2023-11-24T06:31:22.812307+0000","last_unstale":"2023-11-24T06:31:22.816661+0000","last_undegraded":"2023-11-24T06:31:22.816661+0000","last_fullsized":"2023-11-24T06:31:22.816661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:53.964055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774458+0000","last_change":"2023-11-24T06:31:22.800036+0000","last_active":"2023-11-24T06:31:23.774458+0000","last_peered":"2023-11-24T06:31:23.774458+0000","last_clean":"2023-11-24T06:31:23.774458+0000","last_became_active":"2023-11-24T06:31:22.799699+0000","last_became_peered":"2023-11-24T06:31:22.799699+0000","last_unstale":"2023-11-24T06:31:23.774458+0000","last_undegraded":"2023-11-24T06:31:23.774458+0000","last_fullsized":"2023-11-24T06:31:23.774458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:49.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.810703+0000","last_change":"2023-11-24T06:31:22.802557+0000","last_active":"2023-11-24T06:31:22.810703+0000","last_peered":"2023-11-24T06:31:22.810703+0000","last_clean":"2023-11-24T06:31:22.810703+0000","last_became_active":"2023-11-24T06:31:22.801079+0000","last_became_peered":"2023-11-24T06:31:22.801079+0000","last_unstale":"2023-11-24T06:31:22.810703+0000","last_undegraded":"2023-11-24T06:31:22.810703+0000","last_fullsized":"2023-11-24T06:31:22.810703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:24.195691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816628+0000","last_change":"2023-11-24T06:31:22.814013+0000","last_active":"2023-11-24T06:31:22.816628+0000","last_peered":"2023-11-24T06:31:22.816628+0000","last_clean":"2023-11-24T06:31:22.816628+0000","last_became_active":"2023-11-24T06:31:22.813304+0000","last_became_peered":"2023-11-24T06:31:22.813304+0000","last_unstale":"2023-11-24T06:31:22.816628+0000","last_undegraded":"2023-11-24T06:31:22.816628+0000","last_fullsized":"2023-11-24T06:31:22.816628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:47.718070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.807987+0000","last_change":"2023-11-24T06:31:22.802610+0000","last_active":"2023-11-24T06:31:22.807987+0000","last_peered":"2023-11-24T06:31:22.807987+0000","last_clean":"2023-11-24T06:31:22.807987+0000","last_became_active":"2023-11-24T06:31:22.802319+0000","last_became_peered":"2023-11-24T06:31:22.802319+0000","last_unstale":"2023-11-24T06:31:22.807987+0000","last_undegraded":"2023-11-24T06:31:22.807987+0000","last_fullsized":"2023-11-24T06:31:22.807987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:50.981025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.798395+0000","last_change":"2023-11-24T06:31:22.793464+0000","last_active":"2023-11-24T06:31:22.798395+0000","last_peered":"2023-11-24T06:31:22.798395+0000","last_clean":"2023-11-24T06:31:22.798395+0000","last_became_active":"2023-11-24T06:31:22.792951+0000","last_became_peered":"2023-11-24T06:31:22.792951+0000","last_unstale":"2023-11-24T06:31:22.798395+0000","last_undegraded":"2023-11-24T06:31:22.798395+0000","last_fullsized":"2023-11-24T06:31:22.798395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:06.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.808974+0000","last_change":"2023-11-24T06:31:22.804463+0000","last_active":"2023-11-24T06:31:22.808974+0000","last_peered":"2023-11-24T06:31:22.808974+0000","last_clean":"2023-11-24T06:31:22.808974+0000","last_became_active":"2023-11-24T06:31:22.804161+0000","last_became_peered":"2023-11-24T06:31:22.804161+0000","last_unstale":"2023-11-24T06:31:22.808974+0000","last_undegraded":"2023-11-24T06:31:22.808974+0000","last_fullsized":"2023-11-24T06:31:22.808974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:34.125229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782051+0000","last_change":"2023-11-24T06:31:22.802795+0000","last_active":"2023-11-24T06:31:23.782051+0000","last_peered":"2023-11-24T06:31:23.782051+0000","last_clean":"2023-11-24T06:31:23.782051+0000","last_became_active":"2023-11-24T06:31:22.802522+0000","last_became_peered":"2023-11-24T06:31:22.802522+0000","last_unstale":"2023-11-24T06:31:23.782051+0000","last_undegraded":"2023-11-24T06:31:23.782051+0000","last_fullsized":"2023-11-24T06:31:23.782051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:18.381790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775098+0000","last_change":"2023-11-24T06:31:22.807462+0000","last_active":"2023-11-24T06:31:23.775098+0000","last_peered":"2023-11-24T06:31:23.775098+0000","last_clean":"2023-11-24T06:31:23.775098+0000","last_became_active":"2023-11-24T06:31:22.806195+0000","last_became_peered":"2023-11-24T06:31:22.806195+0000","last_unstale":"2023-11-24T06:31:23.775098+0000","last_undegraded":"2023-11-24T06:31:23.775098+0000","last_fullsized":"2023-11-24T06:31:23.775098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:32.005494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.777956+0000","last_change":"2023-11-24T06:31:22.793391+0000","last_active":"2023-11-24T06:31:23.777956+0000","last_peered":"2023-11-24T06:31:23.777956+0000","last_clean":"2023-11-24T06:31:23.777956+0000","last_became_active":"2023-11-24T06:31:22.792878+0000","last_became_peered":"2023-11-24T06:31:22.792878+0000","last_unstale":"2023-11-24T06:31:23.777956+0000","last_undegraded":"2023-11-24T06:31:23.777956+0000","last_fullsized":"2023-11-24T06:31:23.777956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:01.118116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.813160+0000","last_change":"2023-11-24T06:31:22.809923+0000","last_active":"2023-11-24T06:31:22.813160+0000","last_peered":"2023-11-24T06:31:22.813160+0000","last_clean":"2023-11-24T06:31:22.813160+0000","last_became_active":"2023-11-24T06:31:22.809351+0000","last_became_peered":"2023-11-24T06:31:22.809351+0000","last_unstale":"2023-11-24T06:31:22.813160+0000","last_undegraded":"2023-11-24T06:31:22.813160+0000","last_fullsized":"2023-11-24T06:31:22.813160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:30.966186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818630+0000","last_change":"2023-11-24T06:31:22.812999+0000","last_active":"2023-11-24T06:31:22.818630+0000","last_peered":"2023-11-24T06:31:22.818630+0000","last_clean":"2023-11-24T06:31:22.818630+0000","last_became_active":"2023-11-24T06:31:22.812697+0000","last_became_peered":"2023-11-24T06:31:22.812697+0000","last_unstale":"2023-11-24T06:31:22.818630+0000","last_undegraded":"2023-11-24T06:31:22.818630+0000","last_fullsized":"2023-11-24T06:31:22.818630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:53.964055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774884+0000","last_change":"2023-11-24T06:31:22.812199+0000","last_active":"2023-11-24T06:31:23.774884+0000","last_peered":"2023-11-24T06:31:23.774884+0000","last_clean":"2023-11-24T06:31:23.774884+0000","last_became_active":"2023-11-24T06:31:22.809074+0000","last_became_peered":"2023-11-24T06:31:22.809074+0000","last_unstale":"2023-11-24T06:31:23.774884+0000","last_undegraded":"2023-11-24T06:31:23.774884+0000","last_fullsized":"2023-11-24T06:31:23.774884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:07.045761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.812036+0000","last_change":"2023-11-24T06:31:22.810468+0000","last_active":"2023-11-24T06:31:22.812036+0000","last_peered":"2023-11-24T06:31:22.812036+0000","last_clean":"2023-11-24T06:31:22.812036+0000","last_became_active":"2023-11-24T06:31:22.810113+0000","last_became_peered":"2023-11-24T06:31:22.810113+0000","last_unstale":"2023-11-24T06:31:22.812036+0000","last_undegraded":"2023-11-24T06:31:22.812036+0000","last_fullsized":"2023-11-24T06:31:22.812036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:05.625519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775147+0000","last_change":"2023-11-24T06:31:22.814399+0000","last_active":"2023-11-24T06:31:23.775147+0000","last_peered":"2023-11-24T06:31:23.775147+0000","last_clean":"2023-11-24T06:31:23.775147+0000","last_became_active":"2023-11-24T06:31:22.813681+0000","last_became_peered":"2023-11-24T06:31:22.813681+0000","last_unstale":"2023-11-24T06:31:23.775147+0000","last_undegraded":"2023-11-24T06:31:23.775147+0000","last_fullsized":"2023-11-24T06:31:23.775147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:23.564974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774858+0000","last_change":"2023-11-24T06:31:22.815603+0000","last_active":"2023-11-24T06:31:23.774858+0000","last_peered":"2023-11-24T06:31:23.774858+0000","last_clean":"2023-11-24T06:31:23.774858+0000","last_became_active":"2023-11-24T06:31:22.815264+0000","last_became_peered":"2023-11-24T06:31:22.815264+0000","last_unstale":"2023-11-24T06:31:23.774858+0000","last_undegraded":"2023-11-24T06:31:23.774858+0000","last_fullsized":"2023-11-24T06:31:23.774858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:07.380352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.820290+0000","last_change":"2023-11-24T06:31:22.819388+0000","last_active":"2023-11-24T06:31:22.820290+0000","last_peered":"2023-11-24T06:31:22.820290+0000","last_clean":"2023-11-24T06:31:22.820290+0000","last_became_active":"2023-11-24T06:31:22.818851+0000","last_became_peered":"2023-11-24T06:31:22.818851+0000","last_unstale":"2023-11-24T06:31:22.820290+0000","last_undegraded":"2023-11-24T06:31:22.820290+0000","last_fullsized":"2023-11-24T06:31:22.820290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:51:30.186773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818905+0000","last_change":"2023-11-24T06:31:22.815168+0000","last_active":"2023-11-24T06:31:22.818905+0000","last_peered":"2023-11-24T06:31:22.818905+0000","last_clean":"2023-11-24T06:31:22.818905+0000","last_became_active":"2023-11-24T06:31:22.813528+0000","last_became_peered":"2023-11-24T06:31:22.813528+0000","last_unstale":"2023-11-24T06:31:22.818905+0000","last_undegraded":"2023-11-24T06:31:22.818905+0000","last_fullsized":"2023-11-24T06:31:22.818905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:50.076620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774531+0000","last_change":"2023-11-24T06:31:22.813482+0000","last_active":"2023-11-24T06:31:23.774531+0000","last_peered":"2023-11-24T06:31:23.774531+0000","last_clean":"2023-11-24T06:31:23.774531+0000","last_became_active":"2023-11-24T06:31:22.813023+0000","last_became_peered":"2023-11-24T06:31:22.813023+0000","last_unstale":"2023-11-24T06:31:23.774531+0000","last_undegraded":"2023-11-24T06:31:23.774531+0000","last_fullsized":"2023-11-24T06:31:23.774531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:04.248794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.808334+0000","last_change":"2023-11-24T06:31:22.803746+0000","last_active":"2023-11-24T06:31:22.808334+0000","last_peered":"2023-11-24T06:31:22.808334+0000","last_clean":"2023-11-24T06:31:22.808334+0000","last_became_active":"2023-11-24T06:31:22.803366+0000","last_became_peered":"2023-11-24T06:31:22.803366+0000","last_unstale":"2023-11-24T06:31:22.808334+0000","last_undegraded":"2023-11-24T06:31:22.808334+0000","last_fullsized":"2023-11-24T06:31:22.808334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:49.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782053+0000","last_change":"2023-11-24T06:31:22.800309+0000","last_active":"2023-11-24T06:31:23.782053+0000","last_peered":"2023-11-24T06:31:23.782053+0000","last_clean":"2023-11-24T06:31:23.782053+0000","last_became_active":"2023-11-24T06:31:22.799795+0000","last_became_peered":"2023-11-24T06:31:22.799795+0000","last_unstale":"2023-11-24T06:31:23.782053+0000","last_undegraded":"2023-11-24T06:31:23.782053+0000","last_fullsized":"2023-11-24T06:31:23.782053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:01.118116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.811060+0000","last_change":"2023-11-24T06:31:22.806492+0000","last_active":"2023-11-24T06:31:22.811060+0000","last_peered":"2023-11-24T06:31:22.811060+0000","last_clean":"2023-11-24T06:31:22.811060+0000","last_became_active":"2023-11-24T06:31:22.806138+0000","last_became_peered":"2023-11-24T06:31:22.806138+0000","last_unstale":"2023-11-24T06:31:22.811060+0000","last_undegraded":"2023-11-24T06:31:22.811060+0000","last_fullsized":"2023-11-24T06:31:22.811060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:50.076620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.817219+0000","last_change":"2023-11-24T06:31:22.810597+0000","last_active":"2023-11-24T06:31:22.817219+0000","last_peered":"2023-11-24T06:31:22.817219+0000","last_clean":"2023-11-24T06:31:22.817219+0000","last_became_active":"2023-11-24T06:31:22.810245+0000","last_became_peered":"2023-11-24T06:31:22.810245+0000","last_unstale":"2023-11-24T06:31:22.817219+0000","last_undegraded":"2023-11-24T06:31:22.817219+0000","last_fullsized":"2023-11-24T06:31:22.817219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:24.195691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.806003+0000","last_change":"2023-11-24T06:31:22.801112+0000","last_active":"2023-11-24T06:31:22.806003+0000","last_peered":"2023-11-24T06:31:22.806003+0000","last_clean":"2023-11-24T06:31:22.806003+0000","last_became_active":"2023-11-24T06:31:22.800423+0000","last_became_peered":"2023-11-24T06:31:22.800423+0000","last_unstale":"2023-11-24T06:31:22.806003+0000","last_undegraded":"2023-11-24T06:31:22.806003+0000","last_fullsized":"2023-11-24T06:31:22.806003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:23.564974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.819151+0000","last_change":"2023-11-24T06:31:22.815866+0000","last_active":"2023-11-24T06:31:22.819151+0000","last_peered":"2023-11-24T06:31:22.819151+0000","last_clean":"2023-11-24T06:31:22.819151+0000","last_became_active":"2023-11-24T06:31:22.814823+0000","last_became_peered":"2023-11-24T06:31:22.814823+0000","last_unstale":"2023-11-24T06:31:22.819151+0000","last_undegraded":"2023-11-24T06:31:22.819151+0000","last_fullsized":"2023-11-24T06:31:22.819151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:59:46.639287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818986+0000","last_change":"2023-11-24T06:31:22.816143+0000","last_active":"2023-11-24T06:31:22.818986+0000","last_peered":"2023-11-24T06:31:22.818986+0000","last_clean":"2023-11-24T06:31:22.818986+0000","last_became_active":"2023-11-24T06:31:22.815843+0000","last_became_peered":"2023-11-24T06:31:22.815843+0000","last_unstale":"2023-11-24T06:31:22.818986+0000","last_undegraded":"2023-11-24T06:31:22.818986+0000","last_fullsized":"2023-11-24T06:31:22.818986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:14.637441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818664+0000","last_change":"2023-11-24T06:31:22.812999+0000","last_active":"2023-11-24T06:31:22.818664+0000","last_peered":"2023-11-24T06:31:22.818664+0000","last_clean":"2023-11-24T06:31:22.818664+0000","last_became_active":"2023-11-24T06:31:22.812637+0000","last_became_peered":"2023-11-24T06:31:22.812637+0000","last_unstale":"2023-11-24T06:31:22.818664+0000","last_undegraded":"2023-11-24T06:31:22.818664+0000","last_fullsized":"2023-11-24T06:31:22.818664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:02.080378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818712+0000","last_change":"2023-11-24T06:31:22.814717+0000","last_active":"2023-11-24T06:31:22.818712+0000","last_peered":"2023-11-24T06:31:22.818712+0000","last_clean":"2023-11-24T06:31:22.818712+0000","last_became_active":"2023-11-24T06:31:22.813612+0000","last_became_peered":"2023-11-24T06:31:22.813612+0000","last_unstale":"2023-11-24T06:31:22.818712+0000","last_undegraded":"2023-11-24T06:31:22.818712+0000","last_fullsized":"2023-11-24T06:31:22.818712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:34.125229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"85'4","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.074524+0000","last_change":"2023-11-24T06:31:21.781696+0000","last_active":"2023-11-24T06:31:23.074524+0000","last_peered":"2023-11-24T06:31:23.074524+0000","last_clean":"2023-11-24T06:31:23.074524+0000","last_became_active":"2023-11-24T06:31:21.781341+0000","last_became_peered":"2023-11-24T06:31:21.781341+0000","last_unstale":"2023-11-24T06:31:23.074524+0000","last_undegraded":"2023-11-24T06:31:23.074524+0000","last_fullsized":"2023-11-24T06:31:23.074524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:46:00.242159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818718+0000","last_change":"2023-11-24T06:31:22.813808+0000","last_active":"2023-11-24T06:31:22.818718+0000","last_peered":"2023-11-24T06:31:22.818718+0000","last_clean":"2023-11-24T06:31:22.818718+0000","last_became_active":"2023-11-24T06:31:22.813452+0000","last_became_peered":"2023-11-24T06:31:22.813452+0000","last_unstale":"2023-11-24T06:31:22.818718+0000","last_undegraded":"2023-11-24T06:31:22.818718+0000","last_fullsized":"2023-11-24T06:31:22.818718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:15.145918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.067118+0000","last_change":"2023-11-24T06:31:21.784554+0000","last_active":"2023-11-24T06:31:23.067118+0000","last_peered":"2023-11-24T06:31:23.067118+0000","last_clean":"2023-11-24T06:31:23.067118+0000","last_became_active":"2023-11-24T06:31:21.783583+0000","last_became_peered":"2023-11-24T06:31:21.783583+0000","last_unstale":"2023-11-24T06:31:23.067118+0000","last_undegraded":"2023-11-24T06:31:23.067118+0000","last_fullsized":"2023-11-24T06:31:23.067118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:34.292815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.817795+0000","last_change":"2023-11-24T06:31:22.812711+0000","last_active":"2023-11-24T06:31:22.817795+0000","last_peered":"2023-11-24T06:31:22.817795+0000","last_clean":"2023-11-24T06:31:22.817795+0000","last_became_active":"2023-11-24T06:31:22.812410+0000","last_became_peered":"2023-11-24T06:31:22.812410+0000","last_unstale":"2023-11-24T06:31:22.817795+0000","last_undegraded":"2023-11-24T06:31:22.817795+0000","last_fullsized":"2023-11-24T06:31:22.817795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:27.745579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.065429+0000","last_change":"2023-11-24T06:31:21.778008+0000","last_active":"2023-11-24T06:31:23.065429+0000","last_peered":"2023-11-24T06:31:23.065429+0000","last_clean":"2023-11-24T06:31:23.065429+0000","last_became_active":"2023-11-24T06:31:21.777701+0000","last_became_peered":"2023-11-24T06:31:21.777701+0000","last_unstale":"2023-11-24T06:31:23.065429+0000","last_undegraded":"2023-11-24T06:31:23.065429+0000","last_fullsized":"2023-11-24T06:31:23.065429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:44.977659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782496+0000","last_change":"2023-11-24T06:31:22.804893+0000","last_active":"2023-11-24T06:31:23.782496+0000","last_peered":"2023-11-24T06:31:23.782496+0000","last_clean":"2023-11-24T06:31:23.782496+0000","last_became_active":"2023-11-24T06:31:22.804250+0000","last_became_peered":"2023-11-24T06:31:22.804250+0000","last_unstale":"2023-11-24T06:31:23.782496+0000","last_undegraded":"2023-11-24T06:31:23.782496+0000","last_fullsized":"2023-11-24T06:31:23.782496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:40.633471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.066805+0000","last_change":"2023-11-24T06:31:21.783413+0000","last_active":"2023-11-24T06:31:23.066805+0000","last_peered":"2023-11-24T06:31:23.066805+0000","last_clean":"2023-11-24T06:31:23.066805+0000","last_became_active":"2023-11-24T06:31:21.781374+0000","last_became_peered":"2023-11-24T06:31:21.781374+0000","last_unstale":"2023-11-24T06:31:23.066805+0000","last_undegraded":"2023-11-24T06:31:23.066805+0000","last_fullsized":"2023-11-24T06:31:23.066805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:41.406583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.809744+0000","last_change":"2023-11-24T06:31:22.806921+0000","last_active":"2023-11-24T06:31:22.809744+0000","last_peered":"2023-11-24T06:31:22.809744+0000","last_clean":"2023-11-24T06:31:22.809744+0000","last_became_active":"2023-11-24T06:31:22.805416+0000","last_became_peered":"2023-11-24T06:31:22.805416+0000","last_unstale":"2023-11-24T06:31:22.809744+0000","last_undegraded":"2023-11-24T06:31:22.809744+0000","last_fullsized":"2023-11-24T06:31:22.809744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:32.005494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.067178+0000","last_change":"2023-11-24T06:31:21.788045+0000","last_active":"2023-11-24T06:31:23.067178+0000","last_peered":"2023-11-24T06:31:23.067178+0000","last_clean":"2023-11-24T06:31:23.067178+0000","last_became_active":"2023-11-24T06:31:21.787739+0000","last_became_peered":"2023-11-24T06:31:21.787739+0000","last_unstale":"2023-11-24T06:31:23.067178+0000","last_undegraded":"2023-11-24T06:31:23.067178+0000","last_fullsized":"2023-11-24T06:31:23.067178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:14.130062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.819001+0000","last_change":"2023-11-24T06:31:22.816709+0000","last_active":"2023-11-24T06:31:22.819001+0000","last_peered":"2023-11-24T06:31:22.819001+0000","last_clean":"2023-11-24T06:31:22.819001+0000","last_became_active":"2023-11-24T06:31:22.814530+0000","last_became_peered":"2023-11-24T06:31:22.814530+0000","last_unstale":"2023-11-24T06:31:22.819001+0000","last_undegraded":"2023-11-24T06:31:22.819001+0000","last_fullsized":"2023-11-24T06:31:22.819001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:27:32.639784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.067000+0000","last_change":"2023-11-24T06:31:21.787938+0000","last_active":"2023-11-24T06:31:23.067000+0000","last_peered":"2023-11-24T06:31:23.067000+0000","last_clean":"2023-11-24T06:31:23.067000+0000","last_became_active":"2023-11-24T06:31:21.787631+0000","last_became_peered":"2023-11-24T06:31:21.787631+0000","last_unstale":"2023-11-24T06:31:23.067000+0000","last_undegraded":"2023-11-24T06:31:23.067000+0000","last_fullsized":"2023-11-24T06:31:23.067000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:49.965169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.781174+0000","last_change":"2023-11-24T06:31:22.802217+0000","last_active":"2023-11-24T06:31:23.781174+0000","last_peered":"2023-11-24T06:31:23.781174+0000","last_clean":"2023-11-24T06:31:23.781174+0000","last_became_active":"2023-11-24T06:31:22.801806+0000","last_became_peered":"2023-11-24T06:31:22.801806+0000","last_unstale":"2023-11-24T06:31:23.781174+0000","last_undegraded":"2023-11-24T06:31:23.781174+0000","last_fullsized":"2023-11-24T06:31:23.781174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:55:27.893485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.808104+0000","last_change":"2023-11-24T06:31:21.786736+0000","last_active":"2023-11-24T06:31:22.808104+0000","last_peered":"2023-11-24T06:31:22.808104+0000","last_clean":"2023-11-24T06:31:22.808104+0000","last_became_active":"2023-11-24T06:31:21.786457+0000","last_became_peered":"2023-11-24T06:31:21.786457+0000","last_unstale":"2023-11-24T06:31:22.808104+0000","last_undegraded":"2023-11-24T06:31:22.808104+0000","last_fullsized":"2023-11-24T06:31:22.808104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:53.789132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.778075+0000","last_change":"2023-11-24T06:31:22.793185+0000","last_active":"2023-11-24T06:31:23.778075+0000","last_peered":"2023-11-24T06:31:23.778075+0000","last_clean":"2023-11-24T06:31:23.778075+0000","last_became_active":"2023-11-24T06:31:22.792441+0000","last_became_peered":"2023-11-24T06:31:22.792441+0000","last_unstale":"2023-11-24T06:31:23.778075+0000","last_undegraded":"2023-11-24T06:31:23.778075+0000","last_fullsized":"2023-11-24T06:31:23.778075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:10:17.639377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.776285+0000","last_change":"2023-11-24T06:31:21.785502+0000","last_active":"2023-11-24T06:31:22.776285+0000","last_peered":"2023-11-24T06:31:22.776285+0000","last_clean":"2023-11-24T06:31:22.776285+0000","last_became_active":"2023-11-24T06:31:21.785199+0000","last_became_peered":"2023-11-24T06:31:21.785199+0000","last_unstale":"2023-11-24T06:31:22.776285+0000","last_undegraded":"2023-11-24T06:31:22.776285+0000","last_fullsized":"2023-11-24T06:31:22.776285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:14.041150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.778375+0000","last_change":"2023-11-24T06:31:22.799283+0000","last_active":"2023-11-24T06:31:23.778375+0000","last_peered":"2023-11-24T06:31:23.778375+0000","last_clean":"2023-11-24T06:31:23.778375+0000","last_became_active":"2023-11-24T06:31:22.798927+0000","last_became_peered":"2023-11-24T06:31:22.798927+0000","last_unstale":"2023-11-24T06:31:23.778375+0000","last_undegraded":"2023-11-24T06:31:23.778375+0000","last_fullsized":"2023-11-24T06:31:23.778375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:38:22.047882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.801230+0000","last_change":"2023-11-24T06:31:21.792451+0000","last_active":"2023-11-24T06:31:22.801230+0000","last_peered":"2023-11-24T06:31:22.801230+0000","last_clean":"2023-11-24T06:31:22.801230+0000","last_became_active":"2023-11-24T06:31:21.786873+0000","last_became_peered":"2023-11-24T06:31:21.786873+0000","last_unstale":"2023-11-24T06:31:22.801230+0000","last_undegraded":"2023-11-24T06:31:22.801230+0000","last_fullsized":"2023-11-24T06:31:22.801230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:40.073615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.813498+0000","last_change":"2023-11-24T06:31:22.807545+0000","last_active":"2023-11-24T06:31:22.813498+0000","last_peered":"2023-11-24T06:31:22.813498+0000","last_clean":"2023-11-24T06:31:22.813498+0000","last_became_active":"2023-11-24T06:31:22.807091+0000","last_became_peered":"2023-11-24T06:31:22.807091+0000","last_unstale":"2023-11-24T06:31:22.813498+0000","last_undegraded":"2023-11-24T06:31:22.813498+0000","last_fullsized":"2023-11-24T06:31:22.813498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:39.267136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.793421+0000","last_change":"2023-11-24T06:31:21.787100+0000","last_active":"2023-11-24T06:31:22.793421+0000","last_peered":"2023-11-24T06:31:22.793421+0000","last_clean":"2023-11-24T06:31:22.793421+0000","last_became_active":"2023-11-24T06:31:21.786710+0000","last_became_peered":"2023-11-24T06:31:21.786710+0000","last_unstale":"2023-11-24T06:31:22.793421+0000","last_undegraded":"2023-11-24T06:31:22.793421+0000","last_fullsized":"2023-11-24T06:31:22.793421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:27:31.623928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.815473+0000","last_change":"2023-11-24T06:31:22.813054+0000","last_active":"2023-11-24T06:31:22.815473+0000","last_peered":"2023-11-24T06:31:22.815473+0000","last_clean":"2023-11-24T06:31:22.815473+0000","last_became_active":"2023-11-24T06:31:22.812779+0000","last_became_peered":"2023-11-24T06:31:22.812779+0000","last_unstale":"2023-11-24T06:31:22.815473+0000","last_undegraded":"2023-11-24T06:31:22.815473+0000","last_fullsized":"2023-11-24T06:31:22.815473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:42.556182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"85'3","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.069237+0000","last_change":"2023-11-24T06:31:21.791641+0000","last_active":"2023-11-24T06:31:23.069237+0000","last_peered":"2023-11-24T06:31:23.069237+0000","last_clean":"2023-11-24T06:31:23.069237+0000","last_became_active":"2023-11-24T06:31:21.790014+0000","last_became_peered":"2023-11-24T06:31:21.790014+0000","last_unstale":"2023-11-24T06:31:23.069237+0000","last_undegraded":"2023-11-24T06:31:23.069237+0000","last_fullsized":"2023-11-24T06:31:23.069237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:52:24.164136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818789+0000","last_change":"2023-11-24T06:31:22.815053+0000","last_active":"2023-11-24T06:31:22.818789+0000","last_peered":"2023-11-24T06:31:22.818789+0000","last_clean":"2023-11-24T06:31:22.818789+0000","last_became_active":"2023-11-24T06:31:22.814566+0000","last_became_peered":"2023-11-24T06:31:22.814566+0000","last_unstale":"2023-11-24T06:31:22.818789+0000","last_undegraded":"2023-11-24T06:31:22.818789+0000","last_fullsized":"2023-11-24T06:31:22.818789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:24.195691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.077477+0000","last_change":"2023-11-24T06:31:21.788026+0000","last_active":"2023-11-24T06:31:23.077477+0000","last_peered":"2023-11-24T06:31:23.077477+0000","last_clean":"2023-11-24T06:31:23.077477+0000","last_became_active":"2023-11-24T06:31:21.786780+0000","last_became_peered":"2023-11-24T06:31:21.786780+0000","last_unstale":"2023-11-24T06:31:23.077477+0000","last_undegraded":"2023-11-24T06:31:23.077477+0000","last_fullsized":"2023-11-24T06:31:23.077477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:14.130062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.815003+0000","last_change":"2023-11-24T06:31:22.808421+0000","last_active":"2023-11-24T06:31:22.815003+0000","last_peered":"2023-11-24T06:31:22.815003+0000","last_clean":"2023-11-24T06:31:22.815003+0000","last_became_active":"2023-11-24T06:31:22.807941+0000","last_became_peered":"2023-11-24T06:31:22.807941+0000","last_unstale":"2023-11-24T06:31:22.815003+0000","last_undegraded":"2023-11-24T06:31:22.815003+0000","last_fullsized":"2023-11-24T06:31:22.815003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:36.387133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.066441+0000","last_change":"2023-11-24T06:31:21.778677+0000","last_active":"2023-11-24T06:31:23.066441+0000","last_peered":"2023-11-24T06:31:23.066441+0000","last_clean":"2023-11-24T06:31:23.066441+0000","last_became_active":"2023-11-24T06:31:21.778405+0000","last_became_peered":"2023-11-24T06:31:21.778405+0000","last_unstale":"2023-11-24T06:31:23.066441+0000","last_undegraded":"2023-11-24T06:31:23.066441+0000","last_fullsized":"2023-11-24T06:31:23.066441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:34.266769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782139+0000","last_change":"2023-11-24T06:31:22.815684+0000","last_active":"2023-11-24T06:31:23.782139+0000","last_peered":"2023-11-24T06:31:23.782139+0000","last_clean":"2023-11-24T06:31:23.782139+0000","last_became_active":"2023-11-24T06:31:22.815445+0000","last_became_peered":"2023-11-24T06:31:22.815445+0000","last_unstale":"2023-11-24T06:31:23.782139+0000","last_undegraded":"2023-11-24T06:31:23.782139+0000","last_fullsized":"2023-11-24T06:31:23.782139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:10:17.639377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.778027+0000","last_change":"2023-11-24T06:31:21.769523+0000","last_active":"2023-11-24T06:31:23.778027+0000","last_peered":"2023-11-24T06:31:23.778027+0000","last_clean":"2023-11-24T06:31:23.778027+0000","last_became_active":"2023-11-24T06:31:21.769213+0000","last_became_peered":"2023-11-24T06:31:21.769213+0000","last_unstale":"2023-11-24T06:31:23.778027+0000","last_undegraded":"2023-11-24T06:31:23.778027+0000","last_fullsized":"2023-11-24T06:31:23.778027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:17.365934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816296+0000","last_change":"2023-11-24T06:31:22.811064+0000","last_active":"2023-11-24T06:31:22.816296+0000","last_peered":"2023-11-24T06:31:22.816296+0000","last_clean":"2023-11-24T06:31:22.816296+0000","last_became_active":"2023-11-24T06:31:22.810708+0000","last_became_peered":"2023-11-24T06:31:22.810708+0000","last_unstale":"2023-11-24T06:31:22.816296+0000","last_undegraded":"2023-11-24T06:31:22.816296+0000","last_fullsized":"2023-11-24T06:31:22.816296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:40.398044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.789852+0000","last_change":"2023-11-24T06:31:21.792818+0000","last_active":"2023-11-24T06:31:22.789852+0000","last_peered":"2023-11-24T06:31:22.789852+0000","last_clean":"2023-11-24T06:31:22.789852+0000","last_became_active":"2023-11-24T06:31:21.791989+0000","last_became_peered":"2023-11-24T06:31:21.791989+0000","last_unstale":"2023-11-24T06:31:22.789852+0000","last_undegraded":"2023-11-24T06:31:22.789852+0000","last_fullsized":"2023-11-24T06:31:22.789852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:01.064522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.805447+0000","last_change":"2023-11-24T06:31:22.799081+0000","last_active":"2023-11-24T06:31:22.805447+0000","last_peered":"2023-11-24T06:31:22.805447+0000","last_clean":"2023-11-24T06:31:22.805447+0000","last_became_active":"2023-11-24T06:31:22.798766+0000","last_became_peered":"2023-11-24T06:31:22.798766+0000","last_unstale":"2023-11-24T06:31:22.805447+0000","last_undegraded":"2023-11-24T06:31:22.805447+0000","last_fullsized":"2023-11-24T06:31:22.805447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:28.625299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.072274+0000","last_change":"2023-11-24T06:31:21.778556+0000","last_active":"2023-11-24T06:31:23.072274+0000","last_peered":"2023-11-24T06:31:23.072274+0000","last_clean":"2023-11-24T06:31:23.072274+0000","last_became_active":"2023-11-24T06:31:21.778058+0000","last_became_peered":"2023-11-24T06:31:21.778058+0000","last_unstale":"2023-11-24T06:31:23.072274+0000","last_undegraded":"2023-11-24T06:31:23.072274+0000","last_fullsized":"2023-11-24T06:31:23.072274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:17:22.338340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.781660+0000","last_change":"2023-11-24T06:31:22.808707+0000","last_active":"2023-11-24T06:31:23.781660+0000","last_peered":"2023-11-24T06:31:23.781660+0000","last_clean":"2023-11-24T06:31:23.781660+0000","last_became_active":"2023-11-24T06:31:22.808392+0000","last_became_peered":"2023-11-24T06:31:22.808392+0000","last_unstale":"2023-11-24T06:31:23.781660+0000","last_undegraded":"2023-11-24T06:31:23.781660+0000","last_fullsized":"2023-11-24T06:31:23.781660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:35.142706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.074829+0000","last_change":"2023-11-24T06:31:21.790920+0000","last_active":"2023-11-24T06:31:23.074829+0000","last_peered":"2023-11-24T06:31:23.074829+0000","last_clean":"2023-11-24T06:31:23.074829+0000","last_became_active":"2023-11-24T06:31:21.790444+0000","last_became_peered":"2023-11-24T06:31:21.790444+0000","last_unstale":"2023-11-24T06:31:23.074829+0000","last_undegraded":"2023-11-24T06:31:23.074829+0000","last_fullsized":"2023-11-24T06:31:23.074829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:59.251577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.817819+0000","last_change":"2023-11-24T06:31:22.812940+0000","last_active":"2023-11-24T06:31:22.817819+0000","last_peered":"2023-11-24T06:31:22.817819+0000","last_clean":"2023-11-24T06:31:22.817819+0000","last_became_active":"2023-11-24T06:31:22.811511+0000","last_became_peered":"2023-11-24T06:31:22.811511+0000","last_unstale":"2023-11-24T06:31:22.817819+0000","last_undegraded":"2023-11-24T06:31:22.817819+0000","last_fullsized":"2023-11-24T06:31:22.817819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:25.408732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.069153+0000","last_change":"2023-11-24T06:31:21.788792+0000","last_active":"2023-11-24T06:31:23.069153+0000","last_peered":"2023-11-24T06:31:23.069153+0000","last_clean":"2023-11-24T06:31:23.069153+0000","last_became_active":"2023-11-24T06:31:21.788449+0000","last_became_peered":"2023-11-24T06:31:21.788449+0000","last_unstale":"2023-11-24T06:31:23.069153+0000","last_undegraded":"2023-11-24T06:31:23.069153+0000","last_fullsized":"2023-11-24T06:31:23.069153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.813118+0000","last_change":"2023-11-24T06:31:22.808518+0000","last_active":"2023-11-24T06:31:22.813118+0000","last_peered":"2023-11-24T06:31:22.813118+0000","last_clean":"2023-11-24T06:31:22.813118+0000","last_became_active":"2023-11-24T06:31:22.807365+0000","last_became_peered":"2023-11-24T06:31:22.807365+0000","last_unstale":"2023-11-24T06:31:22.813118+0000","last_undegraded":"2023-11-24T06:31:22.813118+0000","last_fullsized":"2023-11-24T06:31:22.813118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:27.745579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.806744+0000","last_change":"2023-11-24T06:31:21.789878+0000","last_active":"2023-11-24T06:31:22.806744+0000","last_peered":"2023-11-24T06:31:22.806744+0000","last_clean":"2023-11-24T06:31:22.806744+0000","last_became_active":"2023-11-24T06:31:21.789485+0000","last_became_peered":"2023-11-24T06:31:21.789485+0000","last_unstale":"2023-11-24T06:31:22.806744+0000","last_undegraded":"2023-11-24T06:31:22.806744+0000","last_fullsized":"2023-11-24T06:31:22.806744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:54:13.311618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775207+0000","last_change":"2023-11-24T06:31:22.807786+0000","last_active":"2023-11-24T06:31:23.775207+0000","last_peered":"2023-11-24T06:31:23.775207+0000","last_clean":"2023-11-24T06:31:23.775207+0000","last_became_active":"2023-11-24T06:31:22.806159+0000","last_became_peered":"2023-11-24T06:31:22.806159+0000","last_unstale":"2023-11-24T06:31:23.775207+0000","last_undegraded":"2023-11-24T06:31:23.775207+0000","last_fullsized":"2023-11-24T06:31:23.775207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:31:56.601361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"85'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.778075+0000","last_change":"2023-11-24T06:31:21.780033+0000","last_active":"2023-11-24T06:31:23.778075+0000","last_peered":"2023-11-24T06:31:23.778075+0000","last_clean":"2023-11-24T06:31:23.778075+0000","last_became_active":"2023-11-24T06:31:21.777249+0000","last_became_peered":"2023-11-24T06:31:21.777249+0000","last_unstale":"2023-11-24T06:31:23.778075+0000","last_undegraded":"2023-11-24T06:31:23.778075+0000","last_fullsized":"2023-11-24T06:31:23.778075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:11:59.099025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.811503+0000","last_change":"2023-11-24T06:31:22.806304+0000","last_active":"2023-11-24T06:31:22.811503+0000","last_peered":"2023-11-24T06:31:22.811503+0000","last_clean":"2023-11-24T06:31:22.811503+0000","last_became_active":"2023-11-24T06:31:22.805320+0000","last_became_peered":"2023-11-24T06:31:22.805320+0000","last_unstale":"2023-11-24T06:31:22.811503+0000","last_undegraded":"2023-11-24T06:31:22.811503+0000","last_fullsized":"2023-11-24T06:31:22.811503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:27:32.639784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775294+0000","last_change":"2023-11-24T06:31:21.779804+0000","last_active":"2023-11-24T06:31:23.775294+0000","last_peered":"2023-11-24T06:31:23.775294+0000","last_clean":"2023-11-24T06:31:23.775294+0000","last_became_active":"2023-11-24T06:31:21.779510+0000","last_became_peered":"2023-11-24T06:31:21.779510+0000","last_unstale":"2023-11-24T06:31:23.775294+0000","last_undegraded":"2023-11-24T06:31:23.775294+0000","last_fullsized":"2023-11-24T06:31:23.775294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:05.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774517+0000","last_change":"2023-11-24T06:31:22.812098+0000","last_active":"2023-11-24T06:31:23.774517+0000","last_peered":"2023-11-24T06:31:23.774517+0000","last_clean":"2023-11-24T06:31:23.774517+0000","last_became_active":"2023-11-24T06:31:22.811710+0000","last_became_peered":"2023-11-24T06:31:22.811710+0000","last_unstale":"2023-11-24T06:31:23.774517+0000","last_undegraded":"2023-11-24T06:31:23.774517+0000","last_fullsized":"2023-11-24T06:31:23.774517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:40.398044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774605+0000","last_change":"2023-11-24T06:31:21.777604+0000","last_active":"2023-11-24T06:31:23.774605+0000","last_peered":"2023-11-24T06:31:23.774605+0000","last_clean":"2023-11-24T06:31:23.774605+0000","last_became_active":"2023-11-24T06:31:21.777321+0000","last_became_peered":"2023-11-24T06:31:21.777321+0000","last_unstale":"2023-11-24T06:31:23.774605+0000","last_undegraded":"2023-11-24T06:31:23.774605+0000","last_fullsized":"2023-11-24T06:31:23.774605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:14.041150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816660+0000","last_change":"2023-11-24T06:31:22.812350+0000","last_active":"2023-11-24T06:31:22.816660+0000","last_peered":"2023-11-24T06:31:22.816660+0000","last_clean":"2023-11-24T06:31:22.816660+0000","last_became_active":"2023-11-24T06:31:22.812045+0000","last_became_peered":"2023-11-24T06:31:22.812045+0000","last_unstale":"2023-11-24T06:31:22.816660+0000","last_undegraded":"2023-11-24T06:31:22.816660+0000","last_fullsized":"2023-11-24T06:31:22.816660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:35.308671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.777748+0000","last_change":"2023-11-24T06:31:21.776712+0000","last_active":"2023-11-24T06:31:23.777748+0000","last_peered":"2023-11-24T06:31:23.777748+0000","last_clean":"2023-11-24T06:31:23.777748+0000","last_became_active":"2023-11-24T06:31:21.776425+0000","last_became_peered":"2023-11-24T06:31:21.776425+0000","last_unstale":"2023-11-24T06:31:23.777748+0000","last_undegraded":"2023-11-24T06:31:23.777748+0000","last_fullsized":"2023-11-24T06:31:23.777748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:53.789132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818572+0000","last_change":"2023-11-24T06:31:22.812786+0000","last_active":"2023-11-24T06:31:22.818572+0000","last_peered":"2023-11-24T06:31:22.818572+0000","last_clean":"2023-11-24T06:31:22.818572+0000","last_became_active":"2023-11-24T06:31:22.812492+0000","last_became_peered":"2023-11-24T06:31:22.812492+0000","last_unstale":"2023-11-24T06:31:22.818572+0000","last_undegraded":"2023-11-24T06:31:22.818572+0000","last_fullsized":"2023-11-24T06:31:22.818572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:52:25.179992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.069682+0000","last_change":"2023-11-24T06:31:21.771778+0000","last_active":"2023-11-24T06:31:23.069682+0000","last_peered":"2023-11-24T06:31:23.069682+0000","last_clean":"2023-11-24T06:31:23.069682+0000","last_became_active":"2023-11-24T06:31:21.771480+0000","last_became_peered":"2023-11-24T06:31:21.771480+0000","last_unstale":"2023-11-24T06:31:23.069682+0000","last_undegraded":"2023-11-24T06:31:23.069682+0000","last_fullsized":"2023-11-24T06:31:23.069682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:06.364496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782454+0000","last_change":"2023-11-24T06:31:22.809265+0000","last_active":"2023-11-24T06:31:23.782454+0000","last_peered":"2023-11-24T06:31:23.782454+0000","last_clean":"2023-11-24T06:31:23.782454+0000","last_became_active":"2023-11-24T06:31:22.808928+0000","last_became_peered":"2023-11-24T06:31:22.808928+0000","last_unstale":"2023-11-24T06:31:23.782454+0000","last_undegraded":"2023-11-24T06:31:23.782454+0000","last_fullsized":"2023-11-24T06:31:23.782454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:38:22.047882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.777533+0000","last_change":"2023-11-24T06:31:21.775650+0000","last_active":"2023-11-24T06:31:22.777533+0000","last_peered":"2023-11-24T06:31:22.777533+0000","last_clean":"2023-11-24T06:31:22.777533+0000","last_became_active":"2023-11-24T06:31:21.775245+0000","last_became_peered":"2023-11-24T06:31:21.775245+0000","last_unstale":"2023-11-24T06:31:22.777533+0000","last_undegraded":"2023-11-24T06:31:22.777533+0000","last_fullsized":"2023-11-24T06:31:22.777533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:03.232938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.819358+0000","last_change":"2023-11-24T06:31:22.817340+0000","last_active":"2023-11-24T06:31:22.819358+0000","last_peered":"2023-11-24T06:31:22.819358+0000","last_clean":"2023-11-24T06:31:22.819358+0000","last_became_active":"2023-11-24T06:31:22.817055+0000","last_became_peered":"2023-11-24T06:31:22.817055+0000","last_unstale":"2023-11-24T06:31:22.819358+0000","last_undegraded":"2023-11-24T06:31:22.819358+0000","last_fullsized":"2023-11-24T06:31:22.819358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:53.964055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.791742+0000","last_change":"2023-11-24T06:31:21.781153+0000","last_active":"2023-11-24T06:31:22.791742+0000","last_peered":"2023-11-24T06:31:22.791742+0000","last_clean":"2023-11-24T06:31:22.791742+0000","last_became_active":"2023-11-24T06:31:21.780827+0000","last_became_peered":"2023-11-24T06:31:21.780827+0000","last_unstale":"2023-11-24T06:31:22.791742+0000","last_undegraded":"2023-11-24T06:31:22.791742+0000","last_fullsized":"2023-11-24T06:31:22.791742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:39.795027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818778+0000","last_change":"2023-11-24T06:31:22.814845+0000","last_active":"2023-11-24T06:31:22.818778+0000","last_peered":"2023-11-24T06:31:22.818778+0000","last_clean":"2023-11-24T06:31:22.818778+0000","last_became_active":"2023-11-24T06:31:22.814214+0000","last_became_peered":"2023-11-24T06:31:22.814214+0000","last_unstale":"2023-11-24T06:31:22.818778+0000","last_undegraded":"2023-11-24T06:31:22.818778+0000","last_fullsized":"2023-11-24T06:31:22.818778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:40.810883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.800092+0000","last_change":"2023-11-24T06:31:21.784711+0000","last_active":"2023-11-24T06:31:22.800092+0000","last_peered":"2023-11-24T06:31:22.800092+0000","last_clean":"2023-11-24T06:31:22.800092+0000","last_became_active":"2023-11-24T06:31:21.784384+0000","last_became_peered":"2023-11-24T06:31:21.784384+0000","last_unstale":"2023-11-24T06:31:22.800092+0000","last_undegraded":"2023-11-24T06:31:22.800092+0000","last_fullsized":"2023-11-24T06:31:22.800092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:38.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.811303+0000","last_change":"2023-11-24T06:31:22.805954+0000","last_active":"2023-11-24T06:31:22.811303+0000","last_peered":"2023-11-24T06:31:22.811303+0000","last_clean":"2023-11-24T06:31:22.811303+0000","last_became_active":"2023-11-24T06:31:22.805655+0000","last_became_peered":"2023-11-24T06:31:22.805655+0000","last_unstale":"2023-11-24T06:31:22.811303+0000","last_undegraded":"2023-11-24T06:31:22.811303+0000","last_fullsized":"2023-11-24T06:31:22.811303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:34.125229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.805659+0000","last_change":"2023-11-24T06:31:21.778234+0000","last_active":"2023-11-24T06:31:22.805659+0000","last_peered":"2023-11-24T06:31:22.805659+0000","last_clean":"2023-11-24T06:31:22.805659+0000","last_became_active":"2023-11-24T06:31:21.777957+0000","last_became_peered":"2023-11-24T06:31:21.777957+0000","last_unstale":"2023-11-24T06:31:22.805659+0000","last_undegraded":"2023-11-24T06:31:22.805659+0000","last_fullsized":"2023-11-24T06:31:22.805659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:10.612562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.811571+0000","last_change":"2023-11-24T06:31:22.809537+0000","last_active":"2023-11-24T06:31:22.811571+0000","last_peered":"2023-11-24T06:31:22.811571+0000","last_clean":"2023-11-24T06:31:22.811571+0000","last_became_active":"2023-11-24T06:31:22.809212+0000","last_became_peered":"2023-11-24T06:31:22.809212+0000","last_unstale":"2023-11-24T06:31:22.811571+0000","last_undegraded":"2023-11-24T06:31:22.811571+0000","last_fullsized":"2023-11-24T06:31:22.811571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:11.168632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.795173+0000","last_change":"2023-11-24T06:31:21.787049+0000","last_active":"2023-11-24T06:31:22.795173+0000","last_peered":"2023-11-24T06:31:22.795173+0000","last_clean":"2023-11-24T06:31:22.795173+0000","last_became_active":"2023-11-24T06:31:21.786756+0000","last_became_peered":"2023-11-24T06:31:21.786756+0000","last_unstale":"2023-11-24T06:31:22.795173+0000","last_undegraded":"2023-11-24T06:31:22.795173+0000","last_fullsized":"2023-11-24T06:31:22.795173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:32.912388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.781357+0000","last_change":"2023-11-24T06:31:22.801840+0000","last_active":"2023-11-24T06:31:23.781357+0000","last_peered":"2023-11-24T06:31:23.781357+0000","last_clean":"2023-11-24T06:31:23.781357+0000","last_became_active":"2023-11-24T06:31:22.801534+0000","last_became_peered":"2023-11-24T06:31:22.801534+0000","last_unstale":"2023-11-24T06:31:23.781357+0000","last_undegraded":"2023-11-24T06:31:23.781357+0000","last_fullsized":"2023-11-24T06:31:23.781357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:24.978552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.793190+0000","last_change":"2023-11-24T06:31:21.788038+0000","last_active":"2023-11-24T06:31:22.793190+0000","last_peered":"2023-11-24T06:31:22.793190+0000","last_clean":"2023-11-24T06:31:22.793190+0000","last_became_active":"2023-11-24T06:31:21.785153+0000","last_became_peered":"2023-11-24T06:31:21.785153+0000","last_unstale":"2023-11-24T06:31:22.793190+0000","last_undegraded":"2023-11-24T06:31:22.793190+0000","last_fullsized":"2023-11-24T06:31:22.793190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:34.292815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.819393+0000","last_change":"2023-11-24T06:31:22.817823+0000","last_active":"2023-11-24T06:31:22.819393+0000","last_peered":"2023-11-24T06:31:22.819393+0000","last_clean":"2023-11-24T06:31:22.819393+0000","last_became_active":"2023-11-24T06:31:22.816930+0000","last_became_peered":"2023-11-24T06:31:22.816930+0000","last_unstale":"2023-11-24T06:31:22.819393+0000","last_undegraded":"2023-11-24T06:31:22.819393+0000","last_fullsized":"2023-11-24T06:31:22.819393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:33.928244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.790950+0000","last_change":"2023-11-24T06:31:21.790513+0000","last_active":"2023-11-24T06:31:22.790950+0000","last_peered":"2023-11-24T06:31:22.790950+0000","last_clean":"2023-11-24T06:31:22.790950+0000","last_became_active":"2023-11-24T06:31:21.789794+0000","last_became_peered":"2023-11-24T06:31:21.789794+0000","last_unstale":"2023-11-24T06:31:22.790950+0000","last_undegraded":"2023-11-24T06:31:22.790950+0000","last_fullsized":"2023-11-24T06:31:22.790950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:24.520463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.809607+0000","last_change":"2023-11-24T06:31:22.804403+0000","last_active":"2023-11-24T06:31:22.809607+0000","last_peered":"2023-11-24T06:31:22.809607+0000","last_clean":"2023-11-24T06:31:22.809607+0000","last_became_active":"2023-11-24T06:31:22.803445+0000","last_became_peered":"2023-11-24T06:31:22.803445+0000","last_unstale":"2023-11-24T06:31:22.809607+0000","last_undegraded":"2023-11-24T06:31:22.809607+0000","last_fullsized":"2023-11-24T06:31:22.809607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:18.192751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"85'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782044+0000","last_change":"2023-11-24T06:31:21.785003+0000","last_active":"2023-11-24T06:31:23.782044+0000","last_peered":"2023-11-24T06:31:23.782044+0000","last_clean":"2023-11-24T06:31:23.782044+0000","last_became_active":"2023-11-24T06:31:21.784691+0000","last_became_peered":"2023-11-24T06:31:21.784691+0000","last_unstale":"2023-11-24T06:31:23.782044+0000","last_undegraded":"2023-11-24T06:31:23.782044+0000","last_fullsized":"2023-11-24T06:31:23.782044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":233,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.779492+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:31:22.779492+0000","last_peered":"2023-11-24T06:31:22.779492+0000","last_clean":"2023-11-24T06:31:22.779492+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:31:22.779492+0000","last_undegraded":"2023-11-24T06:31:22.779492+0000","last_fullsized":"2023-11-24T06:31:22.779492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":1036861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1990000000000001,"15min":1.298},"min":{"1min":0.64700000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7310000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.071},{"interface":"front","average":{"1min":1.208,"5min":1.2769999999999999,"15min":1.2949999999999999},"min":{"1min":0.78400000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8440000000000001,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.5549999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0509999999999999,"15min":1.0209999999999999},"min":{"1min":0.69299999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.72,"5min":2.456,"15min":2.456},"last":1.526},{"interface":"front","average":{"1min":1.302,"5min":1.3240000000000001,"15min":1.371},"min":{"1min":0.75,"5min":0.495,"15min":0.495},"max":{"1min":1.962,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.3220000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.238,"15min":1.276},"min":{"1min":0.66800000000000004,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.657,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.296},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.194,"15min":1.2},"min":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6060000000000001,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.97199999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.339,"15min":1.395},"min":{"1min":0.85499999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.522,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.2230000000000001,"15min":1.2090000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9099999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.4019999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.423,"15min":1.5449999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.1859999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.5,"15min":1.5660000000000001},"min":{"1min":0.72199999999999998,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.966,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.4419999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3360000000000001,"15min":1.3919999999999999},"min":{"1min":0.77500000000000002,"5min":0.626,"15min":0.626},"max":{"1min":2.0990000000000002,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.163},{"interface":"front","average":{"1min":1.48,"5min":1.764,"15min":1.774},"min":{"1min":0.70699999999999996,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.4300000000000002,"5min":17.582999999999998,"15min":17.582999999999998},"last":1.4750000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.673,"15min":1.827},"min":{"1min":0.82299999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5419999999999998,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.367},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.5640000000000001,"15min":1.6739999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.9990000000000001,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.101}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":447117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0880000000000001,"15min":1.0980000000000001},"min":{"1min":0.82099999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.4870000000000001,"5min":1.829,"15min":1.829},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.1060000000000001,"15min":1.1100000000000001},"min":{"1min":0.66500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.605,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.0920000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.232,"15min":1.2170000000000001},"min":{"1min":0.73599999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6859999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.218,"5min":1.2030000000000001,"15min":1.2070000000000001},"min":{"1min":0.68400000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4850000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.875}]},{"osd":2,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1399999999999999,"15min":1.157},"min":{"1min":0.74099999999999999,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.704,"5min":3.597,"15min":3.597},"last":1.056},{"interface":"front","average":{"1min":1.252,"5min":1.278,"15min":1.3089999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.837,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.115}]},{"osd":3,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3009999999999999,"15min":1.3140000000000001},"min":{"1min":0.80500000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.631,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.995},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.89700000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.7509999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.1799999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.383,"15min":1.4330000000000001},"min":{"1min":0.73799999999999999,"5min":0.62,"15min":0.62},"max":{"1min":2.0880000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.399},{"interface":"front","average":{"1min":1.339,"5min":1.3720000000000001,"15min":1.393},"min":{"1min":0.88,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7609999999999999,"5min":3.04,"15min":3.04},"last":1.236}]},{"osd":5,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.365,"15min":1.375},"min":{"1min":0.90100000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7050000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.663},{"interface":"front","average":{"1min":1.55,"5min":1.5369999999999999,"15min":1.597},"min":{"1min":0.92200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9630000000000001,"5min":3.774,"15min":3.774},"last":1.458}]},{"osd":7,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.532,"15min":1.5549999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0409999999999999,"5min":3.403,"15min":3.403},"last":1.361},{"interface":"front","average":{"1min":1.464,"5min":1.488,"15min":1.508},"min":{"1min":1.0249999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.865,"5min":3.649,"15min":3.649},"last":1.7330000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":448057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.145,"15min":1.179},"min":{"1min":0.70699999999999996,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5129999999999999,"5min":1.903,"15min":1.903},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.25,"15min":1.3009999999999999},"min":{"1min":0.71599999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5349999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.1379999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.1279999999999999,"15min":1.1339999999999999},"min":{"1min":0.76200000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3879999999999999,"5min":1.708,"15min":1.708},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.196,"5min":1.167,"15min":1.121},"min":{"1min":0.74299999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6699999999999999,"5min":1.96,"15min":1.96},"last":1.29}]},{"osd":2,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.208,"15min":1.2050000000000001},"min":{"1min":0.77500000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.804,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.016},{"interface":"front","average":{"1min":1.268,"5min":1.306,"15min":1.296},"min":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.627,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.6890000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.341,"15min":1.377},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7829999999999999,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.3779999999999999,"15min":1.389},"min":{"1min":0.79500000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0019999999999998,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.954}]},{"osd":4,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3009999999999999,"15min":1.3280000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.2170000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4790000000000001,"15min":1.5289999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.411,"5min":3.085,"15min":3.085},"last":1.5229999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.464,"15min":1.4430000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.2490000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":2.032},{"interface":"front","average":{"1min":1.45,"5min":1.4710000000000001,"15min":1.4670000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.407,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.901}]},{"osd":7,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.623},"min":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.3809999999999998,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.804},{"interface":"front","average":{"1min":1.621,"5min":1.6699999999999999,"15min":1.7090000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.4820000000000002,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":2.089}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1044480,"data_stored":446423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.0980000000000001,"15min":1.081},"min":{"1min":0.879,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3859999999999999,"5min":1.982,"15min":1.982},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.099,"15min":1.115},"min":{"1min":0.80600000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.161,"15min":1.1180000000000001},"min":{"1min":0.81200000000000006,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.647,"5min":1.909,"15min":1.909},"last":0.875},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.242,"15min":1.2989999999999999},"min":{"1min":0.94199999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5980000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.206}]},{"osd":2,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.2290000000000001,"15min":1.2849999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.746,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.82899999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.659,"5min":2.254,"15min":2.254},"last":1.3320000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.248,"15min":1.3180000000000001},"min":{"1min":0.78100000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5409999999999999,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.284},{"interface":"front","average":{"1min":1.268,"5min":1.357,"15min":1.365},"min":{"1min":1.071,"5min":0.746,"15min":0.746},"max":{"1min":1.5089999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.48,"15min":1.514},"min":{"1min":1.4059999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7470000000000001,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.468,"15min":1.5429999999999999},"min":{"1min":0.91600000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.6619999999999999,"5min":3.52,"15min":3.52},"last":0.93500000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4390000000000001,"15min":1.47},"min":{"1min":0.91100000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.653,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.5209999999999999,"15min":1.5669999999999999},"min":{"1min":1.0129999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7929999999999999,"5min":2.452,"15min":2.452},"last":1.484}]},{"osd":7,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.5269999999999999,"15min":1.633},"min":{"1min":1.099,"5min":0.502,"15min":0.502},"max":{"1min":1.5249999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.5880000000000001,"15min":1.6040000000000001},"min":{"1min":1.1850000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.738,"5min":2.363,"15min":2.363},"last":1.3759999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1069056,"data_stored":454363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.38}]},{"osd":1,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.96799999999999997,"15min":0.96999999999999997},"min":{"1min":0.65800000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.46400000000000002},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0940000000000001,"15min":1.1140000000000001},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.857,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.58999999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6679999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.441},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.60399999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.488,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.327,"15min":1.383},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7190000000000001,"5min":3.081,"15min":3.081},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3440000000000001,"15min":1.3420000000000001},"min":{"1min":0.81000000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7909999999999999,"5min":2.758,"15min":2.758},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3540000000000001,"15min":1.3959999999999999},"min":{"1min":0.88900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6899999999999999,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.544,"15min":1.7230000000000001},"min":{"1min":0.91000000000000003,"5min":0.745,"15min":0.745},"max":{"1min":1.861,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.0229999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.80000000000000004,"5min":0.753,"15min":0.753},"max":{"1min":2.1480000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.51,"15min":1.5369999999999999},"min":{"1min":1.024,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.847,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.552,"15min":1.601},"min":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":4.641,"15min":4.641},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.528,"15min":1.5680000000000001},"min":{"1min":0.92500000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0299999999999998,"5min":3.798,"15min":3.798},"last":0.96599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":447005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.88100000000000001},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.173},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.99399999999999999},"min":{"1min":0.47899999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.2769999999999999,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.92700000000000005,"15min":0.95799999999999996},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.659,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.59799999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.524,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.077,"15min":1.1020000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.296},{"interface":"front","average":{"1min":0.998,"5min":1.069,"15min":1.1080000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.575,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.94799999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.397,"15min":1.4890000000000001},"min":{"1min":0.876,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6830000000000001,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.4059999999999999,"15min":1.4690000000000001},"min":{"1min":0.995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6850000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.0189999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.45,"15min":1.4630000000000001},"min":{"1min":0.97299999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0539999999999998,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.712},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4359999999999999,"15min":1.49},"min":{"1min":1.01,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.821,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.2589999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3819999999999999,"15min":1.4219999999999999},"min":{"1min":0.875,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.681,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.484,"15min":1.528},"min":{"1min":0.71399999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.804,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.6479999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.4550000000000001,"15min":1.5289999999999999},"min":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.873,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.5169999999999999,"15min":1.5349999999999999},"min":{"1min":0.94699999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.9359999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.7949999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":446617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.95599999999999996,"15min":1.0169999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.278,"5min":2.593,"15min":2.593},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.056,"5min":1.0229999999999999,"15min":1.0329999999999999},"min":{"1min":0.52500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.871,"15min":0.875},"min":{"1min":0.46700000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.35399999999999998},{"interface":"front","average":{"1min":1.075,"5min":1.022,"15min":1.012},"min":{"1min":0.61699999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.482,"5min":1.829,"15min":1.829},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1080000000000001,"15min":1.1220000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.556,"5min":1.766,"15min":1.766},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.226,"5min":1.0569999999999999,"15min":1.0780000000000001},"min":{"1min":0.67400000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.679,"5min":1.798,"15min":1.798},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.224,"15min":1.246},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4570000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4039999999999999,"15min":1.4410000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.6559999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.325,"15min":1.3120000000000001},"min":{"1min":1.0880000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.782,"5min":2.016,"15min":2.016},"last":1.147},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3819999999999999,"15min":1.3959999999999999},"min":{"1min":0.96499999999999997,"5min":0.877,"15min":0.877},"max":{"1min":1.7370000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.93799999999999994}]},{"osd":6,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.4319999999999999,"15min":1.4059999999999999},"min":{"1min":0.97899999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.117},{"interface":"front","average":{"1min":1.5,"5min":1.4430000000000001,"15min":1.4530000000000001},"min":{"1min":1.24,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.782,"5min":2.371,"15min":2.371},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.45,"15min":1.45},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.734,"5min":2.246,"15min":2.246},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.52,"15min":1.528},"min":{"1min":0.94499999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.087}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":1036825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84799999999999998,"15min":0.84999999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3420000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89800000000000002,"15min":0.94999999999999996},"min":{"1min":0.442,"5min":0.31,"15min":0.31},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.5329999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.98199999999999998,"15min":0.99199999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4370000000000001,"5min":1.784,"15min":1.784},"last":1.8080000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0720000000000001,"15min":1.121},"min":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6240000000000001,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.9410000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":0.94299999999999995},"min":{"1min":0.48299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5920000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0629999999999999,"15min":1.083},"min":{"1min":0.57999999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.8160000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.1930000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.226,"15min":1.2250000000000001},"min":{"1min":0.73699999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.758,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3240000000000001,"15min":1.3120000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.9350000000000001,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.585}]},{"osd":5,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3240000000000001,"15min":1.3340000000000001},"min":{"1min":0.83099999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.788,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.256,"15min":1.2729999999999999},"min":{"1min":0.78900000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.843,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.6379999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4390000000000001,"15min":1.4770000000000001},"min":{"1min":1.0009999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.9410000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.4330000000000001,"15min":1.498},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.839,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.7370000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.4359999999999999,"15min":1.452},"min":{"1min":0.85999999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9970000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":2.0750000000000002},{"interface":"front","average":{"1min":1.327,"5min":1.397,"15min":1.419},"min":{"1min":0.81299999999999994,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.891,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.875}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:31:27.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:31:27.734 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-11-24T06:31:27.731+0000 7fd9bd7d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:31:27.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.804+0000 7f8d770ca640 1 Processor -- start 2023-11-24T06:31:27.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.804+0000 7f8d770ca640 1 -- start start 2023-11-24T06:31:27.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.806+0000 7f8d770ca640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d700cc500 0x7f8d700cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.806+0000 7f8d770ca640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8d7006d160 0x7f8d7006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.806+0000 7f8d770ca640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d70070e50 con 0x7f8d70143c40 2023-11-24T06:31:27.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.806+0000 7f8d770ca640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700723c0 con 0x7f8d7006d160 2023-11-24T06:31:27.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.806+0000 7f8d770ca640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70144020 con 0x7f8d700cc500 2023-11-24T06:31:27.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.806+0000 7f8d760c8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8d70143c40 0x7f8d70071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55710/0 (socket says 172.21.15.16:55710) 2023-11-24T06:31:27.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.806+0000 7f8d760c8640 1 -- 172.21.15.16:0/1570920761 learned_addr learned my addr 172.21.15.16:0/1570920761 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d768c9640 1 --2- 172.21.15.16:0/1570920761 >> v2:172.21.15.173:3300/0 conn(0x7f8d7006d160 0x7f8d7006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d758c7640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d700cc500 0x7f8d700cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781400845 0 0) 0x7f8d70070e50 con 0x7f8d70143c40 2023-11-24T06:31:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d5c003520 con 0x7f8d70143c40 2023-11-24T06:31:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3550871391 0 0) 0x7f8d5c003520 con 0x7f8d70143c40 2023-11-24T06:31:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d700cc500 msgr2=0x7f8d700cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d700cc500 0x7f8d700cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 >> v2:172.21.15.173:3300/0 conn(0x7f8d7006d160 msgr2=0x7f8d7006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 --2- 172.21.15.16:0/1570920761 >> v2:172.21.15.173:3300/0 conn(0x7f8d7006d160 0x7f8d7006d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700ccea0 con 0x7f8d70143c40 2023-11-24T06:31:27.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f8d6c001670 con 0x7f8d70143c40 2023-11-24T06:31:27.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 >> v1:172.21.15.16:6789/0 conn(0x7f8d70143c40 legacy=0x7f8d70071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d700cc500 0x7f8d5c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c003ef0 0x7f8d5c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d5c004d00 0x7f8d5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700723c0 con 0x7f8d5c004d00 2023-11-24T06:31:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70144020 con 0x7f8d5c003ef0 2023-11-24T06:31:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.807+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c00ba80 con 0x7f8d700cc500 2023-11-24T06:31:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d768c9640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d5c004d00 0x7f8d5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d760c8640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d700cc500 0x7f8d5c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d758c7640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c003ef0 0x7f8d5c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d768c9640 1 -- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d700cc500 msgr2=0x7f8d5c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d768c9640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d700cc500 0x7f8d5c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d768c9640 1 -- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c003ef0 msgr2=0x7f8d5c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d768c9640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c003ef0 0x7f8d5c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d768c9640 1 -- 172.21.15.16:0/1570920761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c003520 con 0x7f8d5c004d00 2023-11-24T06:31:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d760c8640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d700cc500 0x7f8d5c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.808+0000 7f8d758c7640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c003ef0 0x7f8d5c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.809+0000 7f8d768c9640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d5c004d00 0x7f8d5c005180 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800e170 tx=0x7f8d5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338af96883cd49e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.809+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6802cc20 con 0x7f8d5c004d00 2023-11-24T06:31:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.809+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6800f040 con 0x7f8d5c004d00 2023-11-24T06:31:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.809+0000 7f8d750c6640 1 -- 172.21.15.16:0/1570920761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68034440 con 0x7f8d5c004d00 2023-11-24T06:31:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.810+0000 7f8d770ca640 1 -- 172.21.15.16:0/1570920761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d5c004d00 msgr2=0x7f8d5c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.810+0000 7f8d770ca640 1 --2- 172.21.15.16:0/1570920761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d5c004d00 0x7f8d5c005180 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800e170 tx=0x7f8d5c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.810+0000 7f8d768c9640 1 -- 172.21.15.16:0/1570920761 reap_dead start 2023-11-24T06:31:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.810+0000 7f8d770ca640 1 -- 172.21.15.16:0/1570920761 shutdown_connections 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.810+0000 7f8d770ca640 1 -- 172.21.15.16:0/1570920761 >> 172.21.15.16:0/1570920761 conn(0x7f8d7006a800 msgr2=0x7f8d7006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.810+0000 7f8d770ca640 1 -- 172.21.15.16:0/1570920761 shutdown_connections 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.810+0000 7f8d770ca640 1 -- 172.21.15.16:0/1570920761 wait complete. 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.812+0000 7f8d770ca640 1 Processor -- start 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.812+0000 7f8d770ca640 1 -- start start 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.813+0000 7f8d770ca640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c004d00 0x7f8d7013d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.813+0000 7f8d770ca640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d701f58d0 0x7f8d700ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.813+0000 7f8d770ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d701f42c0 0x7f8d700ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.813+0000 7f8d770ca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7013dbf0 con 0x7f8d701f42c0 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.813+0000 7f8d770ca640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701f4830 con 0x7f8d5c004d00 2023-11-24T06:31:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.813+0000 7f8d770ca640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701f4b10 con 0x7f8d701f58d0 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.813+0000 7f8d768c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d701f42c0 0x7f8d700ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.813+0000 7f8d768c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d701f42c0 0x7f8d700ac620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53930/0 (socket says 172.21.15.16:53930) 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.813+0000 7f8d768c9640 1 -- 172.21.15.16:0/2629894069 learned_addr learned my addr 172.21.15.16:0/2629894069 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d760c8640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c004d00 0x7f8d7013d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d758c7640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d701f58d0 0x7f8d700ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d768c9640 1 -- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d701f58d0 msgr2=0x7f8d700ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d768c9640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d701f58d0 0x7f8d700ac0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d768c9640 1 -- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c004d00 msgr2=0x7f8d7013d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d768c9640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c004d00 0x7f8d7013d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d768c9640 1 -- 172.21.15.16:0/2629894069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d701f4df0 con 0x7f8d701f42c0 2023-11-24T06:31:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d768c9640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d701f42c0 0x7f8d700ac620 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800e170 tx=0x7f8d68009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d66ffd640 1 -- 172.21.15.16:0/2629894069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68043070 con 0x7f8d701f42c0 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d66ffd640 1 -- 172.21.15.16:0/2629894069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6802de00 con 0x7f8d701f42c0 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.814+0000 7f8d66ffd640 1 -- 172.21.15.16:0/2629894069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68034d90 con 0x7f8d701f42c0 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.815+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d701f50d0 con 0x7f8d701f42c0 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.815+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d700caa70 con 0x7f8d701f42c0 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.816+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d700cac00 con 0x7f8d701f42c0 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.817+0000 7f8d66ffd640 1 -- 172.21.15.16:0/2629894069 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8d6803e440 con 0x7f8d701f42c0 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.818+0000 7f8d66ffd640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d5406b410 0x7f8d5406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.818+0000 7f8d66ffd640 1 -- 172.21.15.16:0/2629894069 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8d680abbf0 con 0x7f8d701f42c0 2023-11-24T06:31:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.819+0000 7f8d760c8640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d5406b410 0x7f8d5406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.820+0000 7f8d760c8640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d5406b410 0x7f8d5406d8d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001eb0 tx=0x7f8d6c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:27.822+0000 7f8d66ffd640 1 -- 172.21.15.16:0/2629894069 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d680764a0 con 0x7f8d701f42c0 2023-11-24T06:31:28.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.011+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d701f4c80 con 0x7f8d5406b410 2023-11-24T06:31:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.034+0000 7f8d66ffd640 1 -- 172.21.15.16:0/2629894069 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286885 (secure 0 0 0) 0x7f8d701f4c80 con 0x7f8d5406b410 2023-11-24T06:31:28.041 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:28.043 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:31:28.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.039+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d5406b410 msgr2=0x7f8d5406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.039+0000 7f8d770ca640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d5406b410 0x7f8d5406d8d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001eb0 tx=0x7f8d6c009040 comp rx=0 tx=0).stop 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.039+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d701f42c0 msgr2=0x7f8d700ac620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.039+0000 7f8d770ca640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d701f42c0 0x7f8d700ac620 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800e170 tx=0x7f8d68009750 comp rx=0 tx=0).stop 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.040+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 shutdown_connections 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.040+0000 7f8d770ca640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8d5406b410 0x7f8d5406d8d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.040+0000 7f8d770ca640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d701f42c0 0x7f8d700ac620 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.040+0000 7f8d770ca640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d701f58d0 0x7f8d700ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.040+0000 7f8d770ca640 1 --2- 172.21.15.16:0/2629894069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d5c004d00 0x7f8d7013d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.040+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 >> 172.21.15.16:0/2629894069 conn(0x7f8d7006a800 msgr2=0x7f8d700ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.040+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 shutdown_connections 2023-11-24T06:31:28.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.040+0000 7f8d770ca640 1 -- 172.21.15.16:0/2629894069 wait complete. 2023-11-24T06:31:28.064 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2023-11-24T06:31:26.195783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258892,"kb_used_data":9396,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715828,"statfs":{"total":773094113280,"available":772829007872,"internally_reserved":0,"allocated":9621504,"data_stored":4763268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.448834"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.813796+0000","last_change":"2023-11-24T06:31:22.809145+0000","last_active":"2023-11-24T06:31:22.813796+0000","last_peered":"2023-11-24T06:31:22.813796+0000","last_clean":"2023-11-24T06:31:22.813796+0000","last_became_active":"2023-11-24T06:31:22.808851+0000","last_became_peered":"2023-11-24T06:31:22.808851+0000","last_unstale":"2023-11-24T06:31:22.813796+0000","last_undegraded":"2023-11-24T06:31:22.813796+0000","last_fullsized":"2023-11-24T06:31:22.813796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:47.718070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816048+0000","last_change":"2023-11-24T06:31:22.812457+0000","last_active":"2023-11-24T06:31:22.816048+0000","last_peered":"2023-11-24T06:31:22.816048+0000","last_clean":"2023-11-24T06:31:22.816048+0000","last_became_active":"2023-11-24T06:31:22.812147+0000","last_became_peered":"2023-11-24T06:31:22.812147+0000","last_unstale":"2023-11-24T06:31:22.816048+0000","last_undegraded":"2023-11-24T06:31:22.816048+0000","last_fullsized":"2023-11-24T06:31:22.816048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:07.045761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.781784+0000","last_change":"2023-11-24T06:31:22.805000+0000","last_active":"2023-11-24T06:31:23.781784+0000","last_peered":"2023-11-24T06:31:23.781784+0000","last_clean":"2023-11-24T06:31:23.781784+0000","last_became_active":"2023-11-24T06:31:22.804696+0000","last_became_peered":"2023-11-24T06:31:22.804696+0000","last_unstale":"2023-11-24T06:31:23.781784+0000","last_undegraded":"2023-11-24T06:31:23.781784+0000","last_fullsized":"2023-11-24T06:31:23.781784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:12:00.114881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775146+0000","last_change":"2023-11-24T06:31:22.814662+0000","last_active":"2023-11-24T06:31:23.775146+0000","last_peered":"2023-11-24T06:31:23.775146+0000","last_clean":"2023-11-24T06:31:23.775146+0000","last_became_active":"2023-11-24T06:31:22.814120+0000","last_became_peered":"2023-11-24T06:31:22.814120+0000","last_unstale":"2023-11-24T06:31:23.775146+0000","last_undegraded":"2023-11-24T06:31:23.775146+0000","last_fullsized":"2023-11-24T06:31:23.775146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:35.282625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.814911+0000","last_change":"2023-11-24T06:31:22.808935+0000","last_active":"2023-11-24T06:31:22.814911+0000","last_peered":"2023-11-24T06:31:22.814911+0000","last_clean":"2023-11-24T06:31:22.814911+0000","last_became_active":"2023-11-24T06:31:22.808636+0000","last_became_peered":"2023-11-24T06:31:22.808636+0000","last_unstale":"2023-11-24T06:31:22.814911+0000","last_undegraded":"2023-11-24T06:31:22.814911+0000","last_fullsized":"2023-11-24T06:31:22.814911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:02.080378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816661+0000","last_change":"2023-11-24T06:31:22.814342+0000","last_active":"2023-11-24T06:31:22.816661+0000","last_peered":"2023-11-24T06:31:22.816661+0000","last_clean":"2023-11-24T06:31:22.816661+0000","last_became_active":"2023-11-24T06:31:22.812307+0000","last_became_peered":"2023-11-24T06:31:22.812307+0000","last_unstale":"2023-11-24T06:31:22.816661+0000","last_undegraded":"2023-11-24T06:31:22.816661+0000","last_fullsized":"2023-11-24T06:31:22.816661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:53.964055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774458+0000","last_change":"2023-11-24T06:31:22.800036+0000","last_active":"2023-11-24T06:31:23.774458+0000","last_peered":"2023-11-24T06:31:23.774458+0000","last_clean":"2023-11-24T06:31:23.774458+0000","last_became_active":"2023-11-24T06:31:22.799699+0000","last_became_peered":"2023-11-24T06:31:22.799699+0000","last_unstale":"2023-11-24T06:31:23.774458+0000","last_undegraded":"2023-11-24T06:31:23.774458+0000","last_fullsized":"2023-11-24T06:31:23.774458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:49.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.810703+0000","last_change":"2023-11-24T06:31:22.802557+0000","last_active":"2023-11-24T06:31:22.810703+0000","last_peered":"2023-11-24T06:31:22.810703+0000","last_clean":"2023-11-24T06:31:22.810703+0000","last_became_active":"2023-11-24T06:31:22.801079+0000","last_became_peered":"2023-11-24T06:31:22.801079+0000","last_unstale":"2023-11-24T06:31:22.810703+0000","last_undegraded":"2023-11-24T06:31:22.810703+0000","last_fullsized":"2023-11-24T06:31:22.810703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:24.195691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816628+0000","last_change":"2023-11-24T06:31:22.814013+0000","last_active":"2023-11-24T06:31:22.816628+0000","last_peered":"2023-11-24T06:31:22.816628+0000","last_clean":"2023-11-24T06:31:22.816628+0000","last_became_active":"2023-11-24T06:31:22.813304+0000","last_became_peered":"2023-11-24T06:31:22.813304+0000","last_unstale":"2023-11-24T06:31:22.816628+0000","last_undegraded":"2023-11-24T06:31:22.816628+0000","last_fullsized":"2023-11-24T06:31:22.816628+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:47.718070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.807987+0000","last_change":"2023-11-24T06:31:22.802610+0000","last_active":"2023-11-24T06:31:22.807987+0000","last_peered":"2023-11-24T06:31:22.807987+0000","last_clean":"2023-11-24T06:31:22.807987+0000","last_became_active":"2023-11-24T06:31:22.802319+0000","last_became_peered":"2023-11-24T06:31:22.802319+0000","last_unstale":"2023-11-24T06:31:22.807987+0000","last_undegraded":"2023-11-24T06:31:22.807987+0000","last_fullsized":"2023-11-24T06:31:22.807987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:50.981025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.798395+0000","last_change":"2023-11-24T06:31:22.793464+0000","last_active":"2023-11-24T06:31:22.798395+0000","last_peered":"2023-11-24T06:31:22.798395+0000","last_clean":"2023-11-24T06:31:22.798395+0000","last_became_active":"2023-11-24T06:31:22.792951+0000","last_became_peered":"2023-11-24T06:31:22.792951+0000","last_unstale":"2023-11-24T06:31:22.798395+0000","last_undegraded":"2023-11-24T06:31:22.798395+0000","last_fullsized":"2023-11-24T06:31:22.798395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:06.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.808974+0000","last_change":"2023-11-24T06:31:22.804463+0000","last_active":"2023-11-24T06:31:22.808974+0000","last_peered":"2023-11-24T06:31:22.808974+0000","last_clean":"2023-11-24T06:31:22.808974+0000","last_became_active":"2023-11-24T06:31:22.804161+0000","last_became_peered":"2023-11-24T06:31:22.804161+0000","last_unstale":"2023-11-24T06:31:22.808974+0000","last_undegraded":"2023-11-24T06:31:22.808974+0000","last_fullsized":"2023-11-24T06:31:22.808974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:34.125229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782051+0000","last_change":"2023-11-24T06:31:22.802795+0000","last_active":"2023-11-24T06:31:23.782051+0000","last_peered":"2023-11-24T06:31:23.782051+0000","last_clean":"2023-11-24T06:31:23.782051+0000","last_became_active":"2023-11-24T06:31:22.802522+0000","last_became_peered":"2023-11-24T06:31:22.802522+0000","last_unstale":"2023-11-24T06:31:23.782051+0000","last_undegraded":"2023-11-24T06:31:23.782051+0000","last_fullsized":"2023-11-24T06:31:23.782051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:18.381790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775098+0000","last_change":"2023-11-24T06:31:22.807462+0000","last_active":"2023-11-24T06:31:23.775098+0000","last_peered":"2023-11-24T06:31:23.775098+0000","last_clean":"2023-11-24T06:31:23.775098+0000","last_became_active":"2023-11-24T06:31:22.806195+0000","last_became_peered":"2023-11-24T06:31:22.806195+0000","last_unstale":"2023-11-24T06:31:23.775098+0000","last_undegraded":"2023-11-24T06:31:23.775098+0000","last_fullsized":"2023-11-24T06:31:23.775098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:32.005494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.777956+0000","last_change":"2023-11-24T06:31:22.793391+0000","last_active":"2023-11-24T06:31:23.777956+0000","last_peered":"2023-11-24T06:31:23.777956+0000","last_clean":"2023-11-24T06:31:23.777956+0000","last_became_active":"2023-11-24T06:31:22.792878+0000","last_became_peered":"2023-11-24T06:31:22.792878+0000","last_unstale":"2023-11-24T06:31:23.777956+0000","last_undegraded":"2023-11-24T06:31:23.777956+0000","last_fullsized":"2023-11-24T06:31:23.777956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:01.118116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.813160+0000","last_change":"2023-11-24T06:31:22.809923+0000","last_active":"2023-11-24T06:31:22.813160+0000","last_peered":"2023-11-24T06:31:22.813160+0000","last_clean":"2023-11-24T06:31:22.813160+0000","last_became_active":"2023-11-24T06:31:22.809351+0000","last_became_peered":"2023-11-24T06:31:22.809351+0000","last_unstale":"2023-11-24T06:31:22.813160+0000","last_undegraded":"2023-11-24T06:31:22.813160+0000","last_fullsized":"2023-11-24T06:31:22.813160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:30.966186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818630+0000","last_change":"2023-11-24T06:31:22.812999+0000","last_active":"2023-11-24T06:31:22.818630+0000","last_peered":"2023-11-24T06:31:22.818630+0000","last_clean":"2023-11-24T06:31:22.818630+0000","last_became_active":"2023-11-24T06:31:22.812697+0000","last_became_peered":"2023-11-24T06:31:22.812697+0000","last_unstale":"2023-11-24T06:31:22.818630+0000","last_undegraded":"2023-11-24T06:31:22.818630+0000","last_fullsized":"2023-11-24T06:31:22.818630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:53.964055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774884+0000","last_change":"2023-11-24T06:31:22.812199+0000","last_active":"2023-11-24T06:31:23.774884+0000","last_peered":"2023-11-24T06:31:23.774884+0000","last_clean":"2023-11-24T06:31:23.774884+0000","last_became_active":"2023-11-24T06:31:22.809074+0000","last_became_peered":"2023-11-24T06:31:22.809074+0000","last_unstale":"2023-11-24T06:31:23.774884+0000","last_undegraded":"2023-11-24T06:31:23.774884+0000","last_fullsized":"2023-11-24T06:31:23.774884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:13:07.045761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.812036+0000","last_change":"2023-11-24T06:31:22.810468+0000","last_active":"2023-11-24T06:31:22.812036+0000","last_peered":"2023-11-24T06:31:22.812036+0000","last_clean":"2023-11-24T06:31:22.812036+0000","last_became_active":"2023-11-24T06:31:22.810113+0000","last_became_peered":"2023-11-24T06:31:22.810113+0000","last_unstale":"2023-11-24T06:31:22.812036+0000","last_undegraded":"2023-11-24T06:31:22.812036+0000","last_fullsized":"2023-11-24T06:31:22.812036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:05.625519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775147+0000","last_change":"2023-11-24T06:31:22.814399+0000","last_active":"2023-11-24T06:31:23.775147+0000","last_peered":"2023-11-24T06:31:23.775147+0000","last_clean":"2023-11-24T06:31:23.775147+0000","last_became_active":"2023-11-24T06:31:22.813681+0000","last_became_peered":"2023-11-24T06:31:22.813681+0000","last_unstale":"2023-11-24T06:31:23.775147+0000","last_undegraded":"2023-11-24T06:31:23.775147+0000","last_fullsized":"2023-11-24T06:31:23.775147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:23.564974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774858+0000","last_change":"2023-11-24T06:31:22.815603+0000","last_active":"2023-11-24T06:31:23.774858+0000","last_peered":"2023-11-24T06:31:23.774858+0000","last_clean":"2023-11-24T06:31:23.774858+0000","last_became_active":"2023-11-24T06:31:22.815264+0000","last_became_peered":"2023-11-24T06:31:22.815264+0000","last_unstale":"2023-11-24T06:31:23.774858+0000","last_undegraded":"2023-11-24T06:31:23.774858+0000","last_fullsized":"2023-11-24T06:31:23.774858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:07.380352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.820290+0000","last_change":"2023-11-24T06:31:22.819388+0000","last_active":"2023-11-24T06:31:22.820290+0000","last_peered":"2023-11-24T06:31:22.820290+0000","last_clean":"2023-11-24T06:31:22.820290+0000","last_became_active":"2023-11-24T06:31:22.818851+0000","last_became_peered":"2023-11-24T06:31:22.818851+0000","last_unstale":"2023-11-24T06:31:22.820290+0000","last_undegraded":"2023-11-24T06:31:22.820290+0000","last_fullsized":"2023-11-24T06:31:22.820290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:51:30.186773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818905+0000","last_change":"2023-11-24T06:31:22.815168+0000","last_active":"2023-11-24T06:31:22.818905+0000","last_peered":"2023-11-24T06:31:22.818905+0000","last_clean":"2023-11-24T06:31:22.818905+0000","last_became_active":"2023-11-24T06:31:22.813528+0000","last_became_peered":"2023-11-24T06:31:22.813528+0000","last_unstale":"2023-11-24T06:31:22.818905+0000","last_undegraded":"2023-11-24T06:31:22.818905+0000","last_fullsized":"2023-11-24T06:31:22.818905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:50.076620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774531+0000","last_change":"2023-11-24T06:31:22.813482+0000","last_active":"2023-11-24T06:31:23.774531+0000","last_peered":"2023-11-24T06:31:23.774531+0000","last_clean":"2023-11-24T06:31:23.774531+0000","last_became_active":"2023-11-24T06:31:22.813023+0000","last_became_peered":"2023-11-24T06:31:22.813023+0000","last_unstale":"2023-11-24T06:31:23.774531+0000","last_undegraded":"2023-11-24T06:31:23.774531+0000","last_fullsized":"2023-11-24T06:31:23.774531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:04.248794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.808334+0000","last_change":"2023-11-24T06:31:22.803746+0000","last_active":"2023-11-24T06:31:22.808334+0000","last_peered":"2023-11-24T06:31:22.808334+0000","last_clean":"2023-11-24T06:31:22.808334+0000","last_became_active":"2023-11-24T06:31:22.803366+0000","last_became_peered":"2023-11-24T06:31:22.803366+0000","last_unstale":"2023-11-24T06:31:22.808334+0000","last_undegraded":"2023-11-24T06:31:22.808334+0000","last_fullsized":"2023-11-24T06:31:22.808334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:17:49.697828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782053+0000","last_change":"2023-11-24T06:31:22.800309+0000","last_active":"2023-11-24T06:31:23.782053+0000","last_peered":"2023-11-24T06:31:23.782053+0000","last_clean":"2023-11-24T06:31:23.782053+0000","last_became_active":"2023-11-24T06:31:22.799795+0000","last_became_peered":"2023-11-24T06:31:22.799795+0000","last_unstale":"2023-11-24T06:31:23.782053+0000","last_undegraded":"2023-11-24T06:31:23.782053+0000","last_fullsized":"2023-11-24T06:31:23.782053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:01.118116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.811060+0000","last_change":"2023-11-24T06:31:22.806492+0000","last_active":"2023-11-24T06:31:22.811060+0000","last_peered":"2023-11-24T06:31:22.811060+0000","last_clean":"2023-11-24T06:31:22.811060+0000","last_became_active":"2023-11-24T06:31:22.806138+0000","last_became_peered":"2023-11-24T06:31:22.806138+0000","last_unstale":"2023-11-24T06:31:22.811060+0000","last_undegraded":"2023-11-24T06:31:22.811060+0000","last_fullsized":"2023-11-24T06:31:22.811060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:50.076620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.817219+0000","last_change":"2023-11-24T06:31:22.810597+0000","last_active":"2023-11-24T06:31:22.817219+0000","last_peered":"2023-11-24T06:31:22.817219+0000","last_clean":"2023-11-24T06:31:22.817219+0000","last_became_active":"2023-11-24T06:31:22.810245+0000","last_became_peered":"2023-11-24T06:31:22.810245+0000","last_unstale":"2023-11-24T06:31:22.817219+0000","last_undegraded":"2023-11-24T06:31:22.817219+0000","last_fullsized":"2023-11-24T06:31:22.817219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:24.195691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.806003+0000","last_change":"2023-11-24T06:31:22.801112+0000","last_active":"2023-11-24T06:31:22.806003+0000","last_peered":"2023-11-24T06:31:22.806003+0000","last_clean":"2023-11-24T06:31:22.806003+0000","last_became_active":"2023-11-24T06:31:22.800423+0000","last_became_peered":"2023-11-24T06:31:22.800423+0000","last_unstale":"2023-11-24T06:31:22.806003+0000","last_undegraded":"2023-11-24T06:31:22.806003+0000","last_fullsized":"2023-11-24T06:31:22.806003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:42:23.564974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.819151+0000","last_change":"2023-11-24T06:31:22.815866+0000","last_active":"2023-11-24T06:31:22.819151+0000","last_peered":"2023-11-24T06:31:22.819151+0000","last_clean":"2023-11-24T06:31:22.819151+0000","last_became_active":"2023-11-24T06:31:22.814823+0000","last_became_peered":"2023-11-24T06:31:22.814823+0000","last_unstale":"2023-11-24T06:31:22.819151+0000","last_undegraded":"2023-11-24T06:31:22.819151+0000","last_fullsized":"2023-11-24T06:31:22.819151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:59:46.639287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818986+0000","last_change":"2023-11-24T06:31:22.816143+0000","last_active":"2023-11-24T06:31:22.818986+0000","last_peered":"2023-11-24T06:31:22.818986+0000","last_clean":"2023-11-24T06:31:22.818986+0000","last_became_active":"2023-11-24T06:31:22.815843+0000","last_became_peered":"2023-11-24T06:31:22.815843+0000","last_unstale":"2023-11-24T06:31:22.818986+0000","last_undegraded":"2023-11-24T06:31:22.818986+0000","last_fullsized":"2023-11-24T06:31:22.818986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:16:14.637441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818664+0000","last_change":"2023-11-24T06:31:22.812999+0000","last_active":"2023-11-24T06:31:22.818664+0000","last_peered":"2023-11-24T06:31:22.818664+0000","last_clean":"2023-11-24T06:31:22.818664+0000","last_became_active":"2023-11-24T06:31:22.812637+0000","last_became_peered":"2023-11-24T06:31:22.812637+0000","last_unstale":"2023-11-24T06:31:22.818664+0000","last_undegraded":"2023-11-24T06:31:22.818664+0000","last_fullsized":"2023-11-24T06:31:22.818664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:02.080378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818712+0000","last_change":"2023-11-24T06:31:22.814717+0000","last_active":"2023-11-24T06:31:22.818712+0000","last_peered":"2023-11-24T06:31:22.818712+0000","last_clean":"2023-11-24T06:31:22.818712+0000","last_became_active":"2023-11-24T06:31:22.813612+0000","last_became_peered":"2023-11-24T06:31:22.813612+0000","last_unstale":"2023-11-24T06:31:22.818712+0000","last_undegraded":"2023-11-24T06:31:22.818712+0000","last_fullsized":"2023-11-24T06:31:22.818712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:34.125229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"85'4","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.074524+0000","last_change":"2023-11-24T06:31:21.781696+0000","last_active":"2023-11-24T06:31:23.074524+0000","last_peered":"2023-11-24T06:31:23.074524+0000","last_clean":"2023-11-24T06:31:23.074524+0000","last_became_active":"2023-11-24T06:31:21.781341+0000","last_became_peered":"2023-11-24T06:31:21.781341+0000","last_unstale":"2023-11-24T06:31:23.074524+0000","last_undegraded":"2023-11-24T06:31:23.074524+0000","last_fullsized":"2023-11-24T06:31:23.074524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:46:00.242159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818718+0000","last_change":"2023-11-24T06:31:22.813808+0000","last_active":"2023-11-24T06:31:22.818718+0000","last_peered":"2023-11-24T06:31:22.818718+0000","last_clean":"2023-11-24T06:31:22.818718+0000","last_became_active":"2023-11-24T06:31:22.813452+0000","last_became_peered":"2023-11-24T06:31:22.813452+0000","last_unstale":"2023-11-24T06:31:22.818718+0000","last_undegraded":"2023-11-24T06:31:22.818718+0000","last_fullsized":"2023-11-24T06:31:22.818718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:15.145918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.067118+0000","last_change":"2023-11-24T06:31:21.784554+0000","last_active":"2023-11-24T06:31:23.067118+0000","last_peered":"2023-11-24T06:31:23.067118+0000","last_clean":"2023-11-24T06:31:23.067118+0000","last_became_active":"2023-11-24T06:31:21.783583+0000","last_became_peered":"2023-11-24T06:31:21.783583+0000","last_unstale":"2023-11-24T06:31:23.067118+0000","last_undegraded":"2023-11-24T06:31:23.067118+0000","last_fullsized":"2023-11-24T06:31:23.067118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:34.292815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.817795+0000","last_change":"2023-11-24T06:31:22.812711+0000","last_active":"2023-11-24T06:31:22.817795+0000","last_peered":"2023-11-24T06:31:22.817795+0000","last_clean":"2023-11-24T06:31:22.817795+0000","last_became_active":"2023-11-24T06:31:22.812410+0000","last_became_peered":"2023-11-24T06:31:22.812410+0000","last_unstale":"2023-11-24T06:31:22.817795+0000","last_undegraded":"2023-11-24T06:31:22.817795+0000","last_fullsized":"2023-11-24T06:31:22.817795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:27.745579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.065429+0000","last_change":"2023-11-24T06:31:21.778008+0000","last_active":"2023-11-24T06:31:23.065429+0000","last_peered":"2023-11-24T06:31:23.065429+0000","last_clean":"2023-11-24T06:31:23.065429+0000","last_became_active":"2023-11-24T06:31:21.777701+0000","last_became_peered":"2023-11-24T06:31:21.777701+0000","last_unstale":"2023-11-24T06:31:23.065429+0000","last_undegraded":"2023-11-24T06:31:23.065429+0000","last_fullsized":"2023-11-24T06:31:23.065429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:44.977659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782496+0000","last_change":"2023-11-24T06:31:22.804893+0000","last_active":"2023-11-24T06:31:23.782496+0000","last_peered":"2023-11-24T06:31:23.782496+0000","last_clean":"2023-11-24T06:31:23.782496+0000","last_became_active":"2023-11-24T06:31:22.804250+0000","last_became_peered":"2023-11-24T06:31:22.804250+0000","last_unstale":"2023-11-24T06:31:23.782496+0000","last_undegraded":"2023-11-24T06:31:23.782496+0000","last_fullsized":"2023-11-24T06:31:23.782496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:52:40.633471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.066805+0000","last_change":"2023-11-24T06:31:21.783413+0000","last_active":"2023-11-24T06:31:23.066805+0000","last_peered":"2023-11-24T06:31:23.066805+0000","last_clean":"2023-11-24T06:31:23.066805+0000","last_became_active":"2023-11-24T06:31:21.781374+0000","last_became_peered":"2023-11-24T06:31:21.781374+0000","last_unstale":"2023-11-24T06:31:23.066805+0000","last_undegraded":"2023-11-24T06:31:23.066805+0000","last_fullsized":"2023-11-24T06:31:23.066805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:41.406583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.809744+0000","last_change":"2023-11-24T06:31:22.806921+0000","last_active":"2023-11-24T06:31:22.809744+0000","last_peered":"2023-11-24T06:31:22.809744+0000","last_clean":"2023-11-24T06:31:22.809744+0000","last_became_active":"2023-11-24T06:31:22.805416+0000","last_became_peered":"2023-11-24T06:31:22.805416+0000","last_unstale":"2023-11-24T06:31:22.809744+0000","last_undegraded":"2023-11-24T06:31:22.809744+0000","last_fullsized":"2023-11-24T06:31:22.809744+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:32.005494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.067178+0000","last_change":"2023-11-24T06:31:21.788045+0000","last_active":"2023-11-24T06:31:23.067178+0000","last_peered":"2023-11-24T06:31:23.067178+0000","last_clean":"2023-11-24T06:31:23.067178+0000","last_became_active":"2023-11-24T06:31:21.787739+0000","last_became_peered":"2023-11-24T06:31:21.787739+0000","last_unstale":"2023-11-24T06:31:23.067178+0000","last_undegraded":"2023-11-24T06:31:23.067178+0000","last_fullsized":"2023-11-24T06:31:23.067178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:14.130062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.819001+0000","last_change":"2023-11-24T06:31:22.816709+0000","last_active":"2023-11-24T06:31:22.819001+0000","last_peered":"2023-11-24T06:31:22.819001+0000","last_clean":"2023-11-24T06:31:22.819001+0000","last_became_active":"2023-11-24T06:31:22.814530+0000","last_became_peered":"2023-11-24T06:31:22.814530+0000","last_unstale":"2023-11-24T06:31:22.819001+0000","last_undegraded":"2023-11-24T06:31:22.819001+0000","last_fullsized":"2023-11-24T06:31:22.819001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:27:32.639784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.067000+0000","last_change":"2023-11-24T06:31:21.787938+0000","last_active":"2023-11-24T06:31:23.067000+0000","last_peered":"2023-11-24T06:31:23.067000+0000","last_clean":"2023-11-24T06:31:23.067000+0000","last_became_active":"2023-11-24T06:31:21.787631+0000","last_became_peered":"2023-11-24T06:31:21.787631+0000","last_unstale":"2023-11-24T06:31:23.067000+0000","last_undegraded":"2023-11-24T06:31:23.067000+0000","last_fullsized":"2023-11-24T06:31:23.067000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:03:49.965169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.781174+0000","last_change":"2023-11-24T06:31:22.802217+0000","last_active":"2023-11-24T06:31:23.781174+0000","last_peered":"2023-11-24T06:31:23.781174+0000","last_clean":"2023-11-24T06:31:23.781174+0000","last_became_active":"2023-11-24T06:31:22.801806+0000","last_became_peered":"2023-11-24T06:31:22.801806+0000","last_unstale":"2023-11-24T06:31:23.781174+0000","last_undegraded":"2023-11-24T06:31:23.781174+0000","last_fullsized":"2023-11-24T06:31:23.781174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:55:27.893485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.808104+0000","last_change":"2023-11-24T06:31:21.786736+0000","last_active":"2023-11-24T06:31:22.808104+0000","last_peered":"2023-11-24T06:31:22.808104+0000","last_clean":"2023-11-24T06:31:22.808104+0000","last_became_active":"2023-11-24T06:31:21.786457+0000","last_became_peered":"2023-11-24T06:31:21.786457+0000","last_unstale":"2023-11-24T06:31:22.808104+0000","last_undegraded":"2023-11-24T06:31:22.808104+0000","last_fullsized":"2023-11-24T06:31:22.808104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:53.789132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.778075+0000","last_change":"2023-11-24T06:31:22.793185+0000","last_active":"2023-11-24T06:31:23.778075+0000","last_peered":"2023-11-24T06:31:23.778075+0000","last_clean":"2023-11-24T06:31:23.778075+0000","last_became_active":"2023-11-24T06:31:22.792441+0000","last_became_peered":"2023-11-24T06:31:22.792441+0000","last_unstale":"2023-11-24T06:31:23.778075+0000","last_undegraded":"2023-11-24T06:31:23.778075+0000","last_fullsized":"2023-11-24T06:31:23.778075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:10:17.639377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.776285+0000","last_change":"2023-11-24T06:31:21.785502+0000","last_active":"2023-11-24T06:31:22.776285+0000","last_peered":"2023-11-24T06:31:22.776285+0000","last_clean":"2023-11-24T06:31:22.776285+0000","last_became_active":"2023-11-24T06:31:21.785199+0000","last_became_peered":"2023-11-24T06:31:21.785199+0000","last_unstale":"2023-11-24T06:31:22.776285+0000","last_undegraded":"2023-11-24T06:31:22.776285+0000","last_fullsized":"2023-11-24T06:31:22.776285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:14.041150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.778375+0000","last_change":"2023-11-24T06:31:22.799283+0000","last_active":"2023-11-24T06:31:23.778375+0000","last_peered":"2023-11-24T06:31:23.778375+0000","last_clean":"2023-11-24T06:31:23.778375+0000","last_became_active":"2023-11-24T06:31:22.798927+0000","last_became_peered":"2023-11-24T06:31:22.798927+0000","last_unstale":"2023-11-24T06:31:23.778375+0000","last_undegraded":"2023-11-24T06:31:23.778375+0000","last_fullsized":"2023-11-24T06:31:23.778375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:38:22.047882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.801230+0000","last_change":"2023-11-24T06:31:21.792451+0000","last_active":"2023-11-24T06:31:22.801230+0000","last_peered":"2023-11-24T06:31:22.801230+0000","last_clean":"2023-11-24T06:31:22.801230+0000","last_became_active":"2023-11-24T06:31:21.786873+0000","last_became_peered":"2023-11-24T06:31:21.786873+0000","last_unstale":"2023-11-24T06:31:22.801230+0000","last_undegraded":"2023-11-24T06:31:22.801230+0000","last_fullsized":"2023-11-24T06:31:22.801230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:40.073615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.813498+0000","last_change":"2023-11-24T06:31:22.807545+0000","last_active":"2023-11-24T06:31:22.813498+0000","last_peered":"2023-11-24T06:31:22.813498+0000","last_clean":"2023-11-24T06:31:22.813498+0000","last_became_active":"2023-11-24T06:31:22.807091+0000","last_became_peered":"2023-11-24T06:31:22.807091+0000","last_unstale":"2023-11-24T06:31:22.813498+0000","last_undegraded":"2023-11-24T06:31:22.813498+0000","last_fullsized":"2023-11-24T06:31:22.813498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:39.267136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.793421+0000","last_change":"2023-11-24T06:31:21.787100+0000","last_active":"2023-11-24T06:31:22.793421+0000","last_peered":"2023-11-24T06:31:22.793421+0000","last_clean":"2023-11-24T06:31:22.793421+0000","last_became_active":"2023-11-24T06:31:21.786710+0000","last_became_peered":"2023-11-24T06:31:21.786710+0000","last_unstale":"2023-11-24T06:31:22.793421+0000","last_undegraded":"2023-11-24T06:31:22.793421+0000","last_fullsized":"2023-11-24T06:31:22.793421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:27:31.623928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.815473+0000","last_change":"2023-11-24T06:31:22.813054+0000","last_active":"2023-11-24T06:31:22.815473+0000","last_peered":"2023-11-24T06:31:22.815473+0000","last_clean":"2023-11-24T06:31:22.815473+0000","last_became_active":"2023-11-24T06:31:22.812779+0000","last_became_peered":"2023-11-24T06:31:22.812779+0000","last_unstale":"2023-11-24T06:31:22.815473+0000","last_undegraded":"2023-11-24T06:31:22.815473+0000","last_fullsized":"2023-11-24T06:31:22.815473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:42.556182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"85'3","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.069237+0000","last_change":"2023-11-24T06:31:21.791641+0000","last_active":"2023-11-24T06:31:23.069237+0000","last_peered":"2023-11-24T06:31:23.069237+0000","last_clean":"2023-11-24T06:31:23.069237+0000","last_became_active":"2023-11-24T06:31:21.790014+0000","last_became_peered":"2023-11-24T06:31:21.790014+0000","last_unstale":"2023-11-24T06:31:23.069237+0000","last_undegraded":"2023-11-24T06:31:23.069237+0000","last_fullsized":"2023-11-24T06:31:23.069237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:52:24.164136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818789+0000","last_change":"2023-11-24T06:31:22.815053+0000","last_active":"2023-11-24T06:31:22.818789+0000","last_peered":"2023-11-24T06:31:22.818789+0000","last_clean":"2023-11-24T06:31:22.818789+0000","last_became_active":"2023-11-24T06:31:22.814566+0000","last_became_peered":"2023-11-24T06:31:22.814566+0000","last_unstale":"2023-11-24T06:31:22.818789+0000","last_undegraded":"2023-11-24T06:31:22.818789+0000","last_fullsized":"2023-11-24T06:31:22.818789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:24.195691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.077477+0000","last_change":"2023-11-24T06:31:21.788026+0000","last_active":"2023-11-24T06:31:23.077477+0000","last_peered":"2023-11-24T06:31:23.077477+0000","last_clean":"2023-11-24T06:31:23.077477+0000","last_became_active":"2023-11-24T06:31:21.786780+0000","last_became_peered":"2023-11-24T06:31:21.786780+0000","last_unstale":"2023-11-24T06:31:23.077477+0000","last_undegraded":"2023-11-24T06:31:23.077477+0000","last_fullsized":"2023-11-24T06:31:23.077477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:14.130062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.815003+0000","last_change":"2023-11-24T06:31:22.808421+0000","last_active":"2023-11-24T06:31:22.815003+0000","last_peered":"2023-11-24T06:31:22.815003+0000","last_clean":"2023-11-24T06:31:22.815003+0000","last_became_active":"2023-11-24T06:31:22.807941+0000","last_became_peered":"2023-11-24T06:31:22.807941+0000","last_unstale":"2023-11-24T06:31:22.815003+0000","last_undegraded":"2023-11-24T06:31:22.815003+0000","last_fullsized":"2023-11-24T06:31:22.815003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:36.387133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.066441+0000","last_change":"2023-11-24T06:31:21.778677+0000","last_active":"2023-11-24T06:31:23.066441+0000","last_peered":"2023-11-24T06:31:23.066441+0000","last_clean":"2023-11-24T06:31:23.066441+0000","last_became_active":"2023-11-24T06:31:21.778405+0000","last_became_peered":"2023-11-24T06:31:21.778405+0000","last_unstale":"2023-11-24T06:31:23.066441+0000","last_undegraded":"2023-11-24T06:31:23.066441+0000","last_fullsized":"2023-11-24T06:31:23.066441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:34.266769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782139+0000","last_change":"2023-11-24T06:31:22.815684+0000","last_active":"2023-11-24T06:31:23.782139+0000","last_peered":"2023-11-24T06:31:23.782139+0000","last_clean":"2023-11-24T06:31:23.782139+0000","last_became_active":"2023-11-24T06:31:22.815445+0000","last_became_peered":"2023-11-24T06:31:22.815445+0000","last_unstale":"2023-11-24T06:31:23.782139+0000","last_undegraded":"2023-11-24T06:31:23.782139+0000","last_fullsized":"2023-11-24T06:31:23.782139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:10:17.639377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.778027+0000","last_change":"2023-11-24T06:31:21.769523+0000","last_active":"2023-11-24T06:31:23.778027+0000","last_peered":"2023-11-24T06:31:23.778027+0000","last_clean":"2023-11-24T06:31:23.778027+0000","last_became_active":"2023-11-24T06:31:21.769213+0000","last_became_peered":"2023-11-24T06:31:21.769213+0000","last_unstale":"2023-11-24T06:31:23.778027+0000","last_undegraded":"2023-11-24T06:31:23.778027+0000","last_fullsized":"2023-11-24T06:31:23.778027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:13:17.365934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816296+0000","last_change":"2023-11-24T06:31:22.811064+0000","last_active":"2023-11-24T06:31:22.816296+0000","last_peered":"2023-11-24T06:31:22.816296+0000","last_clean":"2023-11-24T06:31:22.816296+0000","last_became_active":"2023-11-24T06:31:22.810708+0000","last_became_peered":"2023-11-24T06:31:22.810708+0000","last_unstale":"2023-11-24T06:31:22.816296+0000","last_undegraded":"2023-11-24T06:31:22.816296+0000","last_fullsized":"2023-11-24T06:31:22.816296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:40.398044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.789852+0000","last_change":"2023-11-24T06:31:21.792818+0000","last_active":"2023-11-24T06:31:22.789852+0000","last_peered":"2023-11-24T06:31:22.789852+0000","last_clean":"2023-11-24T06:31:22.789852+0000","last_became_active":"2023-11-24T06:31:21.791989+0000","last_became_peered":"2023-11-24T06:31:21.791989+0000","last_unstale":"2023-11-24T06:31:22.789852+0000","last_undegraded":"2023-11-24T06:31:22.789852+0000","last_fullsized":"2023-11-24T06:31:22.789852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:01.064522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.805447+0000","last_change":"2023-11-24T06:31:22.799081+0000","last_active":"2023-11-24T06:31:22.805447+0000","last_peered":"2023-11-24T06:31:22.805447+0000","last_clean":"2023-11-24T06:31:22.805447+0000","last_became_active":"2023-11-24T06:31:22.798766+0000","last_became_peered":"2023-11-24T06:31:22.798766+0000","last_unstale":"2023-11-24T06:31:22.805447+0000","last_undegraded":"2023-11-24T06:31:22.805447+0000","last_fullsized":"2023-11-24T06:31:22.805447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:28.625299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.072274+0000","last_change":"2023-11-24T06:31:21.778556+0000","last_active":"2023-11-24T06:31:23.072274+0000","last_peered":"2023-11-24T06:31:23.072274+0000","last_clean":"2023-11-24T06:31:23.072274+0000","last_became_active":"2023-11-24T06:31:21.778058+0000","last_became_peered":"2023-11-24T06:31:21.778058+0000","last_unstale":"2023-11-24T06:31:23.072274+0000","last_undegraded":"2023-11-24T06:31:23.072274+0000","last_fullsized":"2023-11-24T06:31:23.072274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:17:22.338340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.781660+0000","last_change":"2023-11-24T06:31:22.808707+0000","last_active":"2023-11-24T06:31:23.781660+0000","last_peered":"2023-11-24T06:31:23.781660+0000","last_clean":"2023-11-24T06:31:23.781660+0000","last_became_active":"2023-11-24T06:31:22.808392+0000","last_became_peered":"2023-11-24T06:31:22.808392+0000","last_unstale":"2023-11-24T06:31:23.781660+0000","last_undegraded":"2023-11-24T06:31:23.781660+0000","last_fullsized":"2023-11-24T06:31:23.781660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:16:35.142706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.074829+0000","last_change":"2023-11-24T06:31:21.790920+0000","last_active":"2023-11-24T06:31:23.074829+0000","last_peered":"2023-11-24T06:31:23.074829+0000","last_clean":"2023-11-24T06:31:23.074829+0000","last_became_active":"2023-11-24T06:31:21.790444+0000","last_became_peered":"2023-11-24T06:31:21.790444+0000","last_unstale":"2023-11-24T06:31:23.074829+0000","last_undegraded":"2023-11-24T06:31:23.074829+0000","last_fullsized":"2023-11-24T06:31:23.074829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:56:59.251577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.817819+0000","last_change":"2023-11-24T06:31:22.812940+0000","last_active":"2023-11-24T06:31:22.817819+0000","last_peered":"2023-11-24T06:31:22.817819+0000","last_clean":"2023-11-24T06:31:22.817819+0000","last_became_active":"2023-11-24T06:31:22.811511+0000","last_became_peered":"2023-11-24T06:31:22.811511+0000","last_unstale":"2023-11-24T06:31:22.817819+0000","last_undegraded":"2023-11-24T06:31:22.817819+0000","last_fullsized":"2023-11-24T06:31:22.817819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:25.408732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.069153+0000","last_change":"2023-11-24T06:31:21.788792+0000","last_active":"2023-11-24T06:31:23.069153+0000","last_peered":"2023-11-24T06:31:23.069153+0000","last_clean":"2023-11-24T06:31:23.069153+0000","last_became_active":"2023-11-24T06:31:21.788449+0000","last_became_peered":"2023-11-24T06:31:21.788449+0000","last_unstale":"2023-11-24T06:31:23.069153+0000","last_undegraded":"2023-11-24T06:31:23.069153+0000","last_fullsized":"2023-11-24T06:31:23.069153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.813118+0000","last_change":"2023-11-24T06:31:22.808518+0000","last_active":"2023-11-24T06:31:22.813118+0000","last_peered":"2023-11-24T06:31:22.813118+0000","last_clean":"2023-11-24T06:31:22.813118+0000","last_became_active":"2023-11-24T06:31:22.807365+0000","last_became_peered":"2023-11-24T06:31:22.807365+0000","last_unstale":"2023-11-24T06:31:22.813118+0000","last_undegraded":"2023-11-24T06:31:22.813118+0000","last_fullsized":"2023-11-24T06:31:22.813118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:27.745579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.806744+0000","last_change":"2023-11-24T06:31:21.789878+0000","last_active":"2023-11-24T06:31:22.806744+0000","last_peered":"2023-11-24T06:31:22.806744+0000","last_clean":"2023-11-24T06:31:22.806744+0000","last_became_active":"2023-11-24T06:31:21.789485+0000","last_became_peered":"2023-11-24T06:31:21.789485+0000","last_unstale":"2023-11-24T06:31:22.806744+0000","last_undegraded":"2023-11-24T06:31:22.806744+0000","last_fullsized":"2023-11-24T06:31:22.806744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:54:13.311618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775207+0000","last_change":"2023-11-24T06:31:22.807786+0000","last_active":"2023-11-24T06:31:23.775207+0000","last_peered":"2023-11-24T06:31:23.775207+0000","last_clean":"2023-11-24T06:31:23.775207+0000","last_became_active":"2023-11-24T06:31:22.806159+0000","last_became_peered":"2023-11-24T06:31:22.806159+0000","last_unstale":"2023-11-24T06:31:23.775207+0000","last_undegraded":"2023-11-24T06:31:23.775207+0000","last_fullsized":"2023-11-24T06:31:23.775207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:31:56.601361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"85'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.778075+0000","last_change":"2023-11-24T06:31:21.780033+0000","last_active":"2023-11-24T06:31:23.778075+0000","last_peered":"2023-11-24T06:31:23.778075+0000","last_clean":"2023-11-24T06:31:23.778075+0000","last_became_active":"2023-11-24T06:31:21.777249+0000","last_became_peered":"2023-11-24T06:31:21.777249+0000","last_unstale":"2023-11-24T06:31:23.778075+0000","last_undegraded":"2023-11-24T06:31:23.778075+0000","last_fullsized":"2023-11-24T06:31:23.778075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:11:59.099025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.811503+0000","last_change":"2023-11-24T06:31:22.806304+0000","last_active":"2023-11-24T06:31:22.811503+0000","last_peered":"2023-11-24T06:31:22.811503+0000","last_clean":"2023-11-24T06:31:22.811503+0000","last_became_active":"2023-11-24T06:31:22.805320+0000","last_became_peered":"2023-11-24T06:31:22.805320+0000","last_unstale":"2023-11-24T06:31:22.811503+0000","last_undegraded":"2023-11-24T06:31:22.811503+0000","last_fullsized":"2023-11-24T06:31:22.811503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:27:32.639784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.775294+0000","last_change":"2023-11-24T06:31:21.779804+0000","last_active":"2023-11-24T06:31:23.775294+0000","last_peered":"2023-11-24T06:31:23.775294+0000","last_clean":"2023-11-24T06:31:23.775294+0000","last_became_active":"2023-11-24T06:31:21.779510+0000","last_became_peered":"2023-11-24T06:31:21.779510+0000","last_unstale":"2023-11-24T06:31:23.775294+0000","last_undegraded":"2023-11-24T06:31:23.775294+0000","last_fullsized":"2023-11-24T06:31:23.775294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:43:05.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774517+0000","last_change":"2023-11-24T06:31:22.812098+0000","last_active":"2023-11-24T06:31:23.774517+0000","last_peered":"2023-11-24T06:31:23.774517+0000","last_clean":"2023-11-24T06:31:23.774517+0000","last_became_active":"2023-11-24T06:31:22.811710+0000","last_became_peered":"2023-11-24T06:31:22.811710+0000","last_unstale":"2023-11-24T06:31:23.774517+0000","last_undegraded":"2023-11-24T06:31:23.774517+0000","last_fullsized":"2023-11-24T06:31:23.774517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:40.398044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.774605+0000","last_change":"2023-11-24T06:31:21.777604+0000","last_active":"2023-11-24T06:31:23.774605+0000","last_peered":"2023-11-24T06:31:23.774605+0000","last_clean":"2023-11-24T06:31:23.774605+0000","last_became_active":"2023-11-24T06:31:21.777321+0000","last_became_peered":"2023-11-24T06:31:21.777321+0000","last_unstale":"2023-11-24T06:31:23.774605+0000","last_undegraded":"2023-11-24T06:31:23.774605+0000","last_fullsized":"2023-11-24T06:31:23.774605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:55:14.041150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.816660+0000","last_change":"2023-11-24T06:31:22.812350+0000","last_active":"2023-11-24T06:31:22.816660+0000","last_peered":"2023-11-24T06:31:22.816660+0000","last_clean":"2023-11-24T06:31:22.816660+0000","last_became_active":"2023-11-24T06:31:22.812045+0000","last_became_peered":"2023-11-24T06:31:22.812045+0000","last_unstale":"2023-11-24T06:31:22.816660+0000","last_undegraded":"2023-11-24T06:31:22.816660+0000","last_fullsized":"2023-11-24T06:31:22.816660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:35.308671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.777748+0000","last_change":"2023-11-24T06:31:21.776712+0000","last_active":"2023-11-24T06:31:23.777748+0000","last_peered":"2023-11-24T06:31:23.777748+0000","last_clean":"2023-11-24T06:31:23.777748+0000","last_became_active":"2023-11-24T06:31:21.776425+0000","last_became_peered":"2023-11-24T06:31:21.776425+0000","last_unstale":"2023-11-24T06:31:23.777748+0000","last_undegraded":"2023-11-24T06:31:23.777748+0000","last_fullsized":"2023-11-24T06:31:23.777748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:53.789132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818572+0000","last_change":"2023-11-24T06:31:22.812786+0000","last_active":"2023-11-24T06:31:22.818572+0000","last_peered":"2023-11-24T06:31:22.818572+0000","last_clean":"2023-11-24T06:31:22.818572+0000","last_became_active":"2023-11-24T06:31:22.812492+0000","last_became_peered":"2023-11-24T06:31:22.812492+0000","last_unstale":"2023-11-24T06:31:22.818572+0000","last_undegraded":"2023-11-24T06:31:22.818572+0000","last_fullsized":"2023-11-24T06:31:22.818572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:52:25.179992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.069682+0000","last_change":"2023-11-24T06:31:21.771778+0000","last_active":"2023-11-24T06:31:23.069682+0000","last_peered":"2023-11-24T06:31:23.069682+0000","last_clean":"2023-11-24T06:31:23.069682+0000","last_became_active":"2023-11-24T06:31:21.771480+0000","last_became_peered":"2023-11-24T06:31:21.771480+0000","last_unstale":"2023-11-24T06:31:23.069682+0000","last_undegraded":"2023-11-24T06:31:23.069682+0000","last_fullsized":"2023-11-24T06:31:23.069682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:06.364496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782454+0000","last_change":"2023-11-24T06:31:22.809265+0000","last_active":"2023-11-24T06:31:23.782454+0000","last_peered":"2023-11-24T06:31:23.782454+0000","last_clean":"2023-11-24T06:31:23.782454+0000","last_became_active":"2023-11-24T06:31:22.808928+0000","last_became_peered":"2023-11-24T06:31:22.808928+0000","last_unstale":"2023-11-24T06:31:23.782454+0000","last_undegraded":"2023-11-24T06:31:23.782454+0000","last_fullsized":"2023-11-24T06:31:23.782454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:38:22.047882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.777533+0000","last_change":"2023-11-24T06:31:21.775650+0000","last_active":"2023-11-24T06:31:22.777533+0000","last_peered":"2023-11-24T06:31:22.777533+0000","last_clean":"2023-11-24T06:31:22.777533+0000","last_became_active":"2023-11-24T06:31:21.775245+0000","last_became_peered":"2023-11-24T06:31:21.775245+0000","last_unstale":"2023-11-24T06:31:22.777533+0000","last_undegraded":"2023-11-24T06:31:22.777533+0000","last_fullsized":"2023-11-24T06:31:22.777533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:03.232938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.819358+0000","last_change":"2023-11-24T06:31:22.817340+0000","last_active":"2023-11-24T06:31:22.819358+0000","last_peered":"2023-11-24T06:31:22.819358+0000","last_clean":"2023-11-24T06:31:22.819358+0000","last_became_active":"2023-11-24T06:31:22.817055+0000","last_became_peered":"2023-11-24T06:31:22.817055+0000","last_unstale":"2023-11-24T06:31:22.819358+0000","last_undegraded":"2023-11-24T06:31:22.819358+0000","last_fullsized":"2023-11-24T06:31:22.819358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:53.964055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.791742+0000","last_change":"2023-11-24T06:31:21.781153+0000","last_active":"2023-11-24T06:31:22.791742+0000","last_peered":"2023-11-24T06:31:22.791742+0000","last_clean":"2023-11-24T06:31:22.791742+0000","last_became_active":"2023-11-24T06:31:21.780827+0000","last_became_peered":"2023-11-24T06:31:21.780827+0000","last_unstale":"2023-11-24T06:31:22.791742+0000","last_undegraded":"2023-11-24T06:31:22.791742+0000","last_fullsized":"2023-11-24T06:31:22.791742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:39.795027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.818778+0000","last_change":"2023-11-24T06:31:22.814845+0000","last_active":"2023-11-24T06:31:22.818778+0000","last_peered":"2023-11-24T06:31:22.818778+0000","last_clean":"2023-11-24T06:31:22.818778+0000","last_became_active":"2023-11-24T06:31:22.814214+0000","last_became_peered":"2023-11-24T06:31:22.814214+0000","last_unstale":"2023-11-24T06:31:22.818778+0000","last_undegraded":"2023-11-24T06:31:22.818778+0000","last_fullsized":"2023-11-24T06:31:22.818778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:40.810883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.800092+0000","last_change":"2023-11-24T06:31:21.784711+0000","last_active":"2023-11-24T06:31:22.800092+0000","last_peered":"2023-11-24T06:31:22.800092+0000","last_clean":"2023-11-24T06:31:22.800092+0000","last_became_active":"2023-11-24T06:31:21.784384+0000","last_became_peered":"2023-11-24T06:31:21.784384+0000","last_unstale":"2023-11-24T06:31:22.800092+0000","last_undegraded":"2023-11-24T06:31:22.800092+0000","last_fullsized":"2023-11-24T06:31:22.800092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:38.251280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.811303+0000","last_change":"2023-11-24T06:31:22.805954+0000","last_active":"2023-11-24T06:31:22.811303+0000","last_peered":"2023-11-24T06:31:22.811303+0000","last_clean":"2023-11-24T06:31:22.811303+0000","last_became_active":"2023-11-24T06:31:22.805655+0000","last_became_peered":"2023-11-24T06:31:22.805655+0000","last_unstale":"2023-11-24T06:31:22.811303+0000","last_undegraded":"2023-11-24T06:31:22.811303+0000","last_fullsized":"2023-11-24T06:31:22.811303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:34.125229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.805659+0000","last_change":"2023-11-24T06:31:21.778234+0000","last_active":"2023-11-24T06:31:22.805659+0000","last_peered":"2023-11-24T06:31:22.805659+0000","last_clean":"2023-11-24T06:31:22.805659+0000","last_became_active":"2023-11-24T06:31:21.777957+0000","last_became_peered":"2023-11-24T06:31:21.777957+0000","last_unstale":"2023-11-24T06:31:22.805659+0000","last_undegraded":"2023-11-24T06:31:22.805659+0000","last_fullsized":"2023-11-24T06:31:22.805659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:10.612562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.811571+0000","last_change":"2023-11-24T06:31:22.809537+0000","last_active":"2023-11-24T06:31:22.811571+0000","last_peered":"2023-11-24T06:31:22.811571+0000","last_clean":"2023-11-24T06:31:22.811571+0000","last_became_active":"2023-11-24T06:31:22.809212+0000","last_became_peered":"2023-11-24T06:31:22.809212+0000","last_unstale":"2023-11-24T06:31:22.811571+0000","last_undegraded":"2023-11-24T06:31:22.811571+0000","last_fullsized":"2023-11-24T06:31:22.811571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:11.168632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.795173+0000","last_change":"2023-11-24T06:31:21.787049+0000","last_active":"2023-11-24T06:31:22.795173+0000","last_peered":"2023-11-24T06:31:22.795173+0000","last_clean":"2023-11-24T06:31:22.795173+0000","last_became_active":"2023-11-24T06:31:21.786756+0000","last_became_peered":"2023-11-24T06:31:21.786756+0000","last_unstale":"2023-11-24T06:31:22.795173+0000","last_undegraded":"2023-11-24T06:31:22.795173+0000","last_fullsized":"2023-11-24T06:31:22.795173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:32.912388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.781357+0000","last_change":"2023-11-24T06:31:22.801840+0000","last_active":"2023-11-24T06:31:23.781357+0000","last_peered":"2023-11-24T06:31:23.781357+0000","last_clean":"2023-11-24T06:31:23.781357+0000","last_became_active":"2023-11-24T06:31:22.801534+0000","last_became_peered":"2023-11-24T06:31:22.801534+0000","last_unstale":"2023-11-24T06:31:23.781357+0000","last_undegraded":"2023-11-24T06:31:23.781357+0000","last_fullsized":"2023-11-24T06:31:23.781357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:24.978552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.793190+0000","last_change":"2023-11-24T06:31:21.788038+0000","last_active":"2023-11-24T06:31:22.793190+0000","last_peered":"2023-11-24T06:31:22.793190+0000","last_clean":"2023-11-24T06:31:22.793190+0000","last_became_active":"2023-11-24T06:31:21.785153+0000","last_became_peered":"2023-11-24T06:31:21.785153+0000","last_unstale":"2023-11-24T06:31:22.793190+0000","last_undegraded":"2023-11-24T06:31:22.793190+0000","last_fullsized":"2023-11-24T06:31:22.793190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:34.292815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.819393+0000","last_change":"2023-11-24T06:31:22.817823+0000","last_active":"2023-11-24T06:31:22.819393+0000","last_peered":"2023-11-24T06:31:22.819393+0000","last_clean":"2023-11-24T06:31:22.819393+0000","last_became_active":"2023-11-24T06:31:22.816930+0000","last_became_peered":"2023-11-24T06:31:22.816930+0000","last_unstale":"2023-11-24T06:31:22.819393+0000","last_undegraded":"2023-11-24T06:31:22.819393+0000","last_fullsized":"2023-11-24T06:31:22.819393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:33.928244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.790950+0000","last_change":"2023-11-24T06:31:21.790513+0000","last_active":"2023-11-24T06:31:22.790950+0000","last_peered":"2023-11-24T06:31:22.790950+0000","last_clean":"2023-11-24T06:31:22.790950+0000","last_became_active":"2023-11-24T06:31:21.789794+0000","last_became_peered":"2023-11-24T06:31:21.789794+0000","last_unstale":"2023-11-24T06:31:22.790950+0000","last_undegraded":"2023-11-24T06:31:22.790950+0000","last_fullsized":"2023-11-24T06:31:22.790950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:24.520463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.809607+0000","last_change":"2023-11-24T06:31:22.804403+0000","last_active":"2023-11-24T06:31:22.809607+0000","last_peered":"2023-11-24T06:31:22.809607+0000","last_clean":"2023-11-24T06:31:22.809607+0000","last_became_active":"2023-11-24T06:31:22.803445+0000","last_became_peered":"2023-11-24T06:31:22.803445+0000","last_unstale":"2023-11-24T06:31:22.809607+0000","last_undegraded":"2023-11-24T06:31:22.809607+0000","last_fullsized":"2023-11-24T06:31:22.809607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:21.742660+0000","last_clean_scrub_stamp":"2023-11-24T06:31:21.742660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:18.192751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"85'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:31:23.782044+0000","last_change":"2023-11-24T06:31:21.785003+0000","last_active":"2023-11-24T06:31:23.782044+0000","last_peered":"2023-11-24T06:31:23.782044+0000","last_clean":"2023-11-24T06:31:23.782044+0000","last_became_active":"2023-11-24T06:31:21.784691+0000","last_became_peered":"2023-11-24T06:31:21.784691+0000","last_unstale":"2023-11-24T06:31:23.782044+0000","last_undegraded":"2023-11-24T06:31:23.782044+0000","last_fullsized":"2023-11-24T06:31:23.782044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:31:20.726804+0000","last_clean_scrub_stamp":"2023-11-24T06:31:20.726804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:13.065573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":233,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T06:31:22.779492+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:31:22.779492+0000","last_peered":"2023-11-24T06:31:22.779492+0000","last_clean":"2023-11-24T06:31:22.779492+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:31:22.779492+0000","last_undegraded":"2023-11-24T06:31:22.779492+0000","last_fullsized":"2023-11-24T06:31:22.779492+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":1036861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1990000000000001,"15min":1.298},"min":{"1min":0.64700000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.7310000000000001,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.071},{"interface":"front","average":{"1min":1.208,"5min":1.2769999999999999,"15min":1.2949999999999999},"min":{"1min":0.78400000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8440000000000001,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.5549999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0509999999999999,"15min":1.0209999999999999},"min":{"1min":0.69299999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.72,"5min":2.456,"15min":2.456},"last":1.526},{"interface":"front","average":{"1min":1.302,"5min":1.3240000000000001,"15min":1.371},"min":{"1min":0.75,"5min":0.495,"15min":0.495},"max":{"1min":1.962,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.3220000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.238,"15min":1.276},"min":{"1min":0.66800000000000004,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.657,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.296},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.194,"15min":1.2},"min":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6060000000000001,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.97199999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.339,"15min":1.395},"min":{"1min":0.85499999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.522,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.2230000000000001,"15min":1.2090000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9099999999999999,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.4019999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.423,"15min":1.5449999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.1859999999999999,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.5,"15min":1.5660000000000001},"min":{"1min":0.72199999999999998,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.966,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.4419999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3360000000000001,"15min":1.3919999999999999},"min":{"1min":0.77500000000000002,"5min":0.626,"15min":0.626},"max":{"1min":2.0990000000000002,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.163},{"interface":"front","average":{"1min":1.48,"5min":1.764,"15min":1.774},"min":{"1min":0.70699999999999996,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.4300000000000002,"5min":17.582999999999998,"15min":17.582999999999998},"last":1.4750000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.673,"15min":1.827},"min":{"1min":0.82299999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.5419999999999998,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.367},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.5640000000000001,"15min":1.6739999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.9990000000000001,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.101}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":447117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0880000000000001,"15min":1.0980000000000001},"min":{"1min":0.82099999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.4870000000000001,"5min":1.829,"15min":1.829},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.1060000000000001,"15min":1.1100000000000001},"min":{"1min":0.66500000000000004,"5min":0.629,"15min":0.629},"max":{"1min":1.605,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.0920000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.232,"15min":1.2170000000000001},"min":{"1min":0.73599999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6859999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.218,"5min":1.2030000000000001,"15min":1.2070000000000001},"min":{"1min":0.68400000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4850000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.875}]},{"osd":2,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1399999999999999,"15min":1.157},"min":{"1min":0.74099999999999999,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.704,"5min":3.597,"15min":3.597},"last":1.056},{"interface":"front","average":{"1min":1.252,"5min":1.278,"15min":1.3089999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.837,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.115}]},{"osd":3,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3009999999999999,"15min":1.3140000000000001},"min":{"1min":0.80500000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.631,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.995},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.89700000000000002,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.7509999999999999,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.1799999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.383,"15min":1.4330000000000001},"min":{"1min":0.73799999999999999,"5min":0.62,"15min":0.62},"max":{"1min":2.0880000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.399},{"interface":"front","average":{"1min":1.339,"5min":1.3720000000000001,"15min":1.393},"min":{"1min":0.88,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7609999999999999,"5min":3.04,"15min":3.04},"last":1.236}]},{"osd":5,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.365,"15min":1.375},"min":{"1min":0.90100000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7050000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.663},{"interface":"front","average":{"1min":1.55,"5min":1.5369999999999999,"15min":1.597},"min":{"1min":0.92200000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9630000000000001,"5min":3.774,"15min":3.774},"last":1.458}]},{"osd":7,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.532,"15min":1.5549999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.0409999999999999,"5min":3.403,"15min":3.403},"last":1.361},{"interface":"front","average":{"1min":1.464,"5min":1.488,"15min":1.508},"min":{"1min":1.0249999999999999,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.865,"5min":3.649,"15min":3.649},"last":1.7330000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":448057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.145,"15min":1.179},"min":{"1min":0.70699999999999996,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5129999999999999,"5min":1.903,"15min":1.903},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.25,"15min":1.3009999999999999},"min":{"1min":0.71599999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.5349999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.1379999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.1279999999999999,"15min":1.1339999999999999},"min":{"1min":0.76200000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3879999999999999,"5min":1.708,"15min":1.708},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.196,"5min":1.167,"15min":1.121},"min":{"1min":0.74299999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6699999999999999,"5min":1.96,"15min":1.96},"last":1.29}]},{"osd":2,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.208,"15min":1.2050000000000001},"min":{"1min":0.77500000000000002,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.804,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.016},{"interface":"front","average":{"1min":1.268,"5min":1.306,"15min":1.296},"min":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.627,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.6890000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.341,"15min":1.377},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7829999999999999,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.3779999999999999,"15min":1.389},"min":{"1min":0.79500000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0019999999999998,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.954}]},{"osd":4,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3009999999999999,"15min":1.3280000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.2170000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4790000000000001,"15min":1.5289999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.411,"5min":3.085,"15min":3.085},"last":1.5229999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.464,"15min":1.4430000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.2490000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":2.032},{"interface":"front","average":{"1min":1.45,"5min":1.4710000000000001,"15min":1.4670000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.407,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.901}]},{"osd":7,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.623},"min":{"1min":0.61199999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.3809999999999998,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.804},{"interface":"front","average":{"1min":1.621,"5min":1.6699999999999999,"15min":1.7090000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.4820000000000002,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":2.089}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1044480,"data_stored":446423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.0980000000000001,"15min":1.081},"min":{"1min":0.879,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3859999999999999,"5min":1.982,"15min":1.982},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.099,"15min":1.115},"min":{"1min":0.80600000000000005,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.81699999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.161,"15min":1.1180000000000001},"min":{"1min":0.81200000000000006,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.647,"5min":1.909,"15min":1.909},"last":0.875},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.242,"15min":1.2989999999999999},"min":{"1min":0.94199999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.5980000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.206}]},{"osd":2,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.2290000000000001,"15min":1.2849999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.746,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.82899999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.659,"5min":2.254,"15min":2.254},"last":1.3320000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.248,"15min":1.3180000000000001},"min":{"1min":0.78100000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5409999999999999,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":1.284},{"interface":"front","average":{"1min":1.268,"5min":1.357,"15min":1.365},"min":{"1min":1.071,"5min":0.746,"15min":0.746},"max":{"1min":1.5089999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.48,"15min":1.514},"min":{"1min":1.4059999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7470000000000001,"5min":2.5950000000000002,"15min":2.5950000000000002},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.468,"15min":1.5429999999999999},"min":{"1min":0.91600000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.6619999999999999,"5min":3.52,"15min":3.52},"last":0.93500000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4390000000000001,"15min":1.47},"min":{"1min":0.91100000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.653,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.5209999999999999,"15min":1.5669999999999999},"min":{"1min":1.0129999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7929999999999999,"5min":2.452,"15min":2.452},"last":1.484}]},{"osd":7,"last update":"Fri Nov 24 06:30:54 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.5269999999999999,"15min":1.633},"min":{"1min":1.099,"5min":0.502,"15min":0.502},"max":{"1min":1.5249999999999999,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.5880000000000001,"15min":1.6040000000000001},"min":{"1min":1.1850000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.738,"5min":2.363,"15min":2.363},"last":1.3759999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1069056,"data_stored":454363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.38}]},{"osd":1,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.96799999999999997,"15min":0.96999999999999997},"min":{"1min":0.65800000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.46400000000000002},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0940000000000001,"15min":1.1140000000000001},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.857,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.58999999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6679999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.441},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.60399999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.488,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.327,"15min":1.383},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7190000000000001,"5min":3.081,"15min":3.081},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3440000000000001,"15min":1.3420000000000001},"min":{"1min":0.81000000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7909999999999999,"5min":2.758,"15min":2.758},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3540000000000001,"15min":1.3959999999999999},"min":{"1min":0.88900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6899999999999999,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.544,"15min":1.7230000000000001},"min":{"1min":0.91000000000000003,"5min":0.745,"15min":0.745},"max":{"1min":1.861,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.0229999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.80000000000000004,"5min":0.753,"15min":0.753},"max":{"1min":2.1480000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.51,"15min":1.5369999999999999},"min":{"1min":1.024,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.847,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.552,"15min":1.601},"min":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":4.641,"15min":4.641},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.528,"15min":1.5680000000000001},"min":{"1min":0.92500000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0299999999999998,"5min":3.798,"15min":3.798},"last":0.96599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":447005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.88100000000000001},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.173},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.99399999999999999},"min":{"1min":0.47899999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.2769999999999999,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.92700000000000005,"15min":0.95799999999999996},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.659,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.59799999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.524,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.077,"15min":1.1020000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.296},{"interface":"front","average":{"1min":0.998,"5min":1.069,"15min":1.1080000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.575,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.94799999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.397,"15min":1.4890000000000001},"min":{"1min":0.876,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6830000000000001,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.4059999999999999,"15min":1.4690000000000001},"min":{"1min":0.995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6850000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.0189999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.45,"15min":1.4630000000000001},"min":{"1min":0.97299999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0539999999999998,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.712},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4359999999999999,"15min":1.49},"min":{"1min":1.01,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.821,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.2589999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3819999999999999,"15min":1.4219999999999999},"min":{"1min":0.875,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.681,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.484,"15min":1.528},"min":{"1min":0.71399999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.804,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.6479999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.4550000000000001,"15min":1.5289999999999999},"min":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.873,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.5169999999999999,"15min":1.5349999999999999},"min":{"1min":0.94699999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.9359999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.7949999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":446617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.95599999999999996,"15min":1.0169999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.278,"5min":2.593,"15min":2.593},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.056,"5min":1.0229999999999999,"15min":1.0329999999999999},"min":{"1min":0.52500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.871,"15min":0.875},"min":{"1min":0.46700000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.35399999999999998},{"interface":"front","average":{"1min":1.075,"5min":1.022,"15min":1.012},"min":{"1min":0.61699999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.482,"5min":1.829,"15min":1.829},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1080000000000001,"15min":1.1220000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.556,"5min":1.766,"15min":1.766},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.226,"5min":1.0569999999999999,"15min":1.0780000000000001},"min":{"1min":0.67400000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.679,"5min":1.798,"15min":1.798},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.224,"15min":1.246},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4570000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4039999999999999,"15min":1.4410000000000001},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.6559999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.325,"15min":1.3120000000000001},"min":{"1min":1.0880000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.782,"5min":2.016,"15min":2.016},"last":1.147},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3819999999999999,"15min":1.3959999999999999},"min":{"1min":0.96499999999999997,"5min":0.877,"15min":0.877},"max":{"1min":1.7370000000000001,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.93799999999999994}]},{"osd":6,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.4319999999999999,"15min":1.4059999999999999},"min":{"1min":0.97899999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.117},{"interface":"front","average":{"1min":1.5,"5min":1.4430000000000001,"15min":1.4530000000000001},"min":{"1min":1.24,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.782,"5min":2.371,"15min":2.371},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.45,"15min":1.45},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.734,"5min":2.246,"15min":2.246},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.52,"15min":1.528},"min":{"1min":0.94499999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.087}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":1036825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84799999999999998,"15min":0.84999999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.3420000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89800000000000002,"15min":0.94999999999999996},"min":{"1min":0.442,"5min":0.31,"15min":0.31},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.5329999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.98199999999999998,"15min":0.99199999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4370000000000001,"5min":1.784,"15min":1.784},"last":1.8080000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0720000000000001,"15min":1.121},"min":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.6240000000000001,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.9410000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.91400000000000003,"15min":0.94299999999999995},"min":{"1min":0.48299999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5920000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0629999999999999,"15min":1.083},"min":{"1min":0.57999999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.8160000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.1930000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.226,"15min":1.2250000000000001},"min":{"1min":0.73699999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.758,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3240000000000001,"15min":1.3120000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.9350000000000001,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.585}]},{"osd":5,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3240000000000001,"15min":1.3340000000000001},"min":{"1min":0.83099999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.788,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.256,"15min":1.2729999999999999},"min":{"1min":0.78900000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.843,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.6379999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4390000000000001,"15min":1.4770000000000001},"min":{"1min":1.0009999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.9410000000000001,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.4330000000000001,"15min":1.498},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.839,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.7370000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:30:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.4359999999999999,"15min":1.452},"min":{"1min":0.85999999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9970000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":2.0750000000000002},{"interface":"front","average":{"1min":1.327,"5min":1.397,"15min":1.419},"min":{"1min":0.81299999999999994,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.891,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.875}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:31:28.069 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:31:28.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:31:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.341+0000 7ff34d727640 1 Processor -- start 2023-11-24T06:31:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.341+0000 7ff34d727640 1 -- start start 2023-11-24T06:31:28.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.343+0000 7ff34d727640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff34815aba0 0x7ff34806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.343+0000 7ff34d727640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815e100 0x7ff3480b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.343+0000 7ff34d727640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34806dd60 con 0x7ff34806d900 2023-11-24T06:31:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.343+0000 7ff34d727640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34815e570 con 0x7ff34815aba0 2023-11-24T06:31:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.343+0000 7ff34d727640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480ad810 con 0x7ff34815e100 2023-11-24T06:31:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.343+0000 7ff3477fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff34806d900 0x7ff34815d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55722/0 (socket says 172.21.15.16:55722) 2023-11-24T06:31:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.343+0000 7ff3477fe640 1 -- 172.21.15.16:0/1727160302 learned_addr learned my addr 172.21.15.16:0/1727160302 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.343+0000 7ff347fff640 1 --2- 172.21.15.16:0/1727160302 >> v2:172.21.15.16:3300/0 conn(0x7ff34815aba0 0x7ff34806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.344+0000 7ff34cf26640 1 --2- 172.21.15.16:0/1727160302 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815e100 0x7ff3480b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.344+0000 7ff347fff640 1 -- 172.21.15.16:0/1727160302 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815e100 msgr2=0x7ff3480b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:28.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.344+0000 7ff347fff640 1 --2- 172.21.15.16:0/1727160302 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815e100 0x7ff3480b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.344+0000 7ff347fff640 1 -- 172.21.15.16:0/1727160302 >> v1:172.21.15.16:6789/0 conn(0x7ff34806d900 legacy=0x7ff34815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.344+0000 7ff347fff640 1 -- 172.21.15.16:0/1727160302 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3480ad380 con 0x7ff34815aba0 2023-11-24T06:31:28.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.345+0000 7ff347fff640 1 --2- 172.21.15.16:0/1727160302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34815aba0 0x7ff34806d3c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff3380097d0 tx=0x7ff33802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a56086ee32ffc39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:28.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.345+0000 7ff346ffd640 1 -- 172.21.15.16:0/1727160302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33802d040 con 0x7ff34815aba0 2023-11-24T06:31:28.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.345+0000 7ff346ffd640 1 -- 172.21.15.16:0/1727160302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff338031810 con 0x7ff34815aba0 2023-11-24T06:31:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.345+0000 7ff346ffd640 1 -- 172.21.15.16:0/1727160302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338031ba0 con 0x7ff34815aba0 2023-11-24T06:31:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.346+0000 7ff34d727640 1 -- 172.21.15.16:0/1727160302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34815aba0 msgr2=0x7ff34806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.346+0000 7ff34d727640 1 --2- 172.21.15.16:0/1727160302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34815aba0 0x7ff34806d3c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff3380097d0 tx=0x7ff33802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.346+0000 7ff34d727640 1 -- 172.21.15.16:0/1727160302 shutdown_connections 2023-11-24T06:31:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.346+0000 7ff34d727640 1 --2- 172.21.15.16:0/1727160302 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815e100 0x7ff3480b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.346+0000 7ff34d727640 1 --2- 172.21.15.16:0/1727160302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34815aba0 0x7ff34806d3c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.346+0000 7ff34d727640 1 -- 172.21.15.16:0/1727160302 >> 172.21.15.16:0/1727160302 conn(0x7ff3480b0b70 msgr2=0x7ff3480b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.346+0000 7ff34d727640 1 -- 172.21.15.16:0/1727160302 shutdown_connections 2023-11-24T06:31:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.346+0000 7ff34d727640 1 -- 172.21.15.16:0/1727160302 wait complete. 2023-11-24T06:31:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.348+0000 7ff34d727640 1 Processor -- start 2023-11-24T06:31:28.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.348+0000 7ff34d727640 1 -- start start 2023-11-24T06:31:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.349+0000 7ff34d727640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34806d900 0x7ff348144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.349+0000 7ff34d727640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815aba0 0x7ff3480cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.349+0000 7ff34d727640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff34815e100 0x7ff3480cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.349+0000 7ff34d727640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480ccbb0 con 0x7ff34806d900 2023-11-24T06:31:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.349+0000 7ff34d727640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480cd160 con 0x7ff34815e100 2023-11-24T06:31:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.349+0000 7ff34d727640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3480cd440 con 0x7ff34815aba0 2023-11-24T06:31:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34806d900 0x7ff348144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34806d900 0x7ff348144540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53952/0 (socket says 172.21.15.16:53952) 2023-11-24T06:31:28.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff347fff640 1 -- 172.21.15.16:0/399310931 learned_addr learned my addr 172.21.15.16:0/399310931 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:28.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff3477fe640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815aba0 0x7ff3480cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff34cf26640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff34815e100 0x7ff3480cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff347fff640 1 -- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815aba0 msgr2=0x7ff3480cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff347fff640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815aba0 0x7ff3480cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff347fff640 1 -- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff34815e100 msgr2=0x7ff3480cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff347fff640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff34815e100 0x7ff3480cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff347fff640 1 -- 172.21.15.16:0/399310931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338009400 con 0x7ff34806d900 2023-11-24T06:31:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.350+0000 7ff347fff640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34806d900 0x7ff348144540 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff33802b8f0 tx=0x7ff3380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.351+0000 7ff344ff9640 1 -- 172.21.15.16:0/399310931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33803ace0 con 0x7ff34806d900 2023-11-24T06:31:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.351+0000 7ff344ff9640 1 -- 172.21.15.16:0/399310931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33803ae80 con 0x7ff34806d900 2023-11-24T06:31:28.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.351+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3480cd720 con 0x7ff34806d900 2023-11-24T06:31:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.351+0000 7ff344ff9640 1 -- 172.21.15.16:0/399310931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338044630 con 0x7ff34806d900 2023-11-24T06:31:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.351+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3480cce50 con 0x7ff34806d900 2023-11-24T06:31:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.352+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3480cdc40 con 0x7ff34806d900 2023-11-24T06:31:28.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.353+0000 7ff344ff9640 1 -- 172.21.15.16:0/399310931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff33804d450 con 0x7ff34806d900 2023-11-24T06:31:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.354+0000 7ff344ff9640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff33406b3e0 0x7ff33406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.354+0000 7ff344ff9640 1 -- 172.21.15.16:0/399310931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff3380ac540 con 0x7ff34806d900 2023-11-24T06:31:28.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.355+0000 7ff3477fe640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff33406b3e0 0x7ff33406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.356+0000 7ff3477fe640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff33406b3e0 0x7ff33406d8a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff3481608a0 tx=0x7ff33c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:28.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.359+0000 7ff344ff9640 1 -- 172.21.15.16:0/399310931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3480cdc40 con 0x7ff34806d900 2023-11-24T06:31:28.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.629+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff348067450 con 0x7ff34806d900 2023-11-24T06:31:28.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.633+0000 7ff344ff9640 1 -- 172.21.15.16:0/399310931 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff3380350e0 con 0x7ff34806d900 2023-11-24T06:31:28.637 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-11-24T06:31:28.637 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQD1QWBleGvrOhAA5Xn8CeMxrMmu7OhfHUEiRA== 2023-11-24T06:31:28.637 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2023-11-24T06:31:28.637 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2023-11-24T06:31:28.637 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:31:28.637 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-11-24T06:31:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff33406b3e0 msgr2=0x7ff33406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff33406b3e0 0x7ff33406d8a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff3481608a0 tx=0x7ff33c009040 comp rx=0 tx=0).stop 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34806d900 msgr2=0x7ff348144540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34806d900 0x7ff348144540 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff33802b8f0 tx=0x7ff3380046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 shutdown_connections 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff33406b3e0 0x7ff33406d8a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff34815e100 0x7ff3480cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff34815aba0 0x7ff3480cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 --2- 172.21.15.16:0/399310931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34806d900 0x7ff348144540 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 >> 172.21.15.16:0/399310931 conn(0x7ff3480b0b70 msgr2=0x7ff348071100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 shutdown_connections 2023-11-24T06:31:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.636+0000 7ff34d727640 1 -- 172.21.15.16:0/399310931 wait complete. 2023-11-24T06:31:28.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.876+0000 7fe46eda9640 1 Processor -- start 2023-11-24T06:31:28.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.877+0000 7fe46eda9640 1 -- start start 2023-11-24T06:31:28.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.878+0000 7fe46eda9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46813e360 0x7fe4680cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.878+0000 7fe46eda9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe46815f9c0 0x7fe4680cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.878+0000 7fe46eda9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46815c5f0 con 0x7fe46813df80 2023-11-24T06:31:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.878+0000 7fe46eda9640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46806c470 con 0x7fe46815f9c0 2023-11-24T06:31:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.878+0000 7fe46eda9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46806c750 con 0x7fe46813e360 2023-11-24T06:31:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.879+0000 7fe46cb45640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe46813df80 0x7fe46815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55726/0 (socket says 172.21.15.16:55726) 2023-11-24T06:31:28.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.879+0000 7fe46cb45640 1 -- 172.21.15.16:0/703148089 learned_addr learned my addr 172.21.15.16:0/703148089 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.879+0000 7fe467fff640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46813e360 0x7fe4680cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.879+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176125861 0 0) 0x7fe46815c5f0 con 0x7fe46813df80 2023-11-24T06:31:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.879+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe450003520 con 0x7fe46813df80 2023-11-24T06:31:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.879+0000 7fe46d346640 1 --2- 172.21.15.16:0/703148089 >> v2:172.21.15.173:3300/0 conn(0x7fe46815f9c0 0x7fe4680cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.879+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2085377807 0 0) 0x7fe450003520 con 0x7fe46813df80 2023-11-24T06:31:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46813e360 msgr2=0x7fe4680cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46813e360 0x7fe4680cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 >> v2:172.21.15.173:3300/0 conn(0x7fe46815f9c0 msgr2=0x7fe4680cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 --2- 172.21.15.16:0/703148089 >> v2:172.21.15.173:3300/0 conn(0x7fe46815f9c0 0x7fe4680cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46806ca30 con 0x7fe46813df80 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fe458001670 con 0x7fe46813df80 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 >> v1:172.21.15.16:6789/0 conn(0x7fe46813df80 legacy=0x7fe46815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe450003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46813e360 0x7fe4500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe450004d00 0x7fe450005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46806c470 con 0x7fe450003970 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46806c750 con 0x7fe450004d00 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45000b9b0 con 0x7fe46813e360 2023-11-24T06:31:28.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe46d346640 1 --2- 172.21.15.16:0/703148089 >> v2:172.21.15.173:3300/0 conn(0x7fe46815f9c0 0x7fe4680cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:31:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe46cb45640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe450003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.880+0000 7fe467fff640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46813e360 0x7fe4500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.881+0000 7fe46d346640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe450004d00 0x7fe450005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.881+0000 7fe46cb45640 1 -- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46813e360 msgr2=0x7fe4500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.881+0000 7fe46cb45640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe46813e360 0x7fe4500047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.881+0000 7fe46cb45640 1 -- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe450004d00 msgr2=0x7fe450005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.881+0000 7fe46cb45640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe450004d00 0x7fe450005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.881+0000 7fe46cb45640 1 -- 172.21.15.16:0/703148089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450003520 con 0x7fe450003970 2023-11-24T06:31:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.882+0000 7fe46cb45640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe450003d90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe45800c290 tx=0x7fe4500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284e037b12559f32 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.882+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458005ad0 con 0x7fe450003970 2023-11-24T06:31:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.882+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe458005c30 con 0x7fe450003970 2023-11-24T06:31:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.882+0000 7fe4677fe640 1 -- 172.21.15.16:0/703148089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45802c8d0 con 0x7fe450003970 2023-11-24T06:31:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.883+0000 7fe46eda9640 1 -- 172.21.15.16:0/703148089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 msgr2=0x7fe450003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.883+0000 7fe46eda9640 1 --2- 172.21.15.16:0/703148089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe450003d90 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe45800c290 tx=0x7fe4500057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.883+0000 7fe46d346640 1 -- 172.21.15.16:0/703148089 reap_dead start 2023-11-24T06:31:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.883+0000 7fe46eda9640 1 -- 172.21.15.16:0/703148089 shutdown_connections 2023-11-24T06:31:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.883+0000 7fe46eda9640 1 -- 172.21.15.16:0/703148089 >> 172.21.15.16:0/703148089 conn(0x7fe46806a800 msgr2=0x7fe46806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.883+0000 7fe46eda9640 1 -- 172.21.15.16:0/703148089 shutdown_connections 2023-11-24T06:31:28.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.883+0000 7fe46eda9640 1 -- 172.21.15.16:0/703148089 wait complete. 2023-11-24T06:31:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.885+0000 7fe46eda9640 1 Processor -- start 2023-11-24T06:31:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.885+0000 7fe46eda9640 1 -- start start 2023-11-24T06:31:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46eda9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe46813cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46eda9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe468154330 0x7fe468154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46eda9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe468146cc0 0x7fe468147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46eda9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680ae3c0 con 0x7fe450003970 2023-11-24T06:31:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46eda9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe468147680 con 0x7fe468154330 2023-11-24T06:31:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46eda9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe468154cd0 con 0x7fe468146cc0 2023-11-24T06:31:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46cb45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe46813cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46cb45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe46813cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53972/0 (socket says 172.21.15.16:53972) 2023-11-24T06:31:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46cb45640 1 -- 172.21.15.16:0/3401746343 learned_addr learned my addr 172.21.15.16:0/3401746343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46cb45640 1 -- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe468146cc0 msgr2=0x7fe468147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46d346640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe468146cc0 0x7fe468147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46cb45640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe468146cc0 0x7fe468147140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46cb45640 1 -- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe468154330 msgr2=0x7fe468154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:28.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.886+0000 7fe46cb45640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe468154330 0x7fe468154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.887+0000 7fe46cb45640 1 -- 172.21.15.16:0/3401746343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe468153540 con 0x7fe450003970 2023-11-24T06:31:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.887+0000 7fe46cb45640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe46813cd70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe45802c470 tx=0x7fe458000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.887+0000 7fe4657fa640 1 -- 172.21.15.16:0/3401746343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458039070 con 0x7fe450003970 2023-11-24T06:31:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.887+0000 7fe4657fa640 1 -- 172.21.15.16:0/3401746343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45802dd70 con 0x7fe450003970 2023-11-24T06:31:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.887+0000 7fe4657fa640 1 -- 172.21.15.16:0/3401746343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4580072b0 con 0x7fe450003970 2023-11-24T06:31:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.887+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4681478a0 con 0x7fe450003970 2023-11-24T06:31:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.887+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe468144ca0 con 0x7fe450003970 2023-11-24T06:31:28.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.888+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe468067000 con 0x7fe450003970 2023-11-24T06:31:28.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.889+0000 7fe4657fa640 1 -- 172.21.15.16:0/3401746343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe458032070 con 0x7fe450003970 2023-11-24T06:31:28.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.890+0000 7fe4657fa640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe45406b3e0 0x7fe45406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:28.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.890+0000 7fe4657fa640 1 -- 172.21.15.16:0/3401746343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe45803c040 con 0x7fe450003970 2023-11-24T06:31:28.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.891+0000 7fe467fff640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe45406b3e0 0x7fe45406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:28.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.892+0000 7fe467fff640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe45406b3e0 0x7fe45406d8a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe46815f0f0 tx=0x7fe45c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:28.896+0000 7fe4657fa640 1 -- 172.21.15.16:0/3401746343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe458075a60 con 0x7fe450003970 2023-11-24T06:31:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.128+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4680675d0 con 0x7fe450003970 2023-11-24T06:31:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.129+0000 7fe4657fa640 1 -- 172.21.15.16:0/3401746343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fe458045020 con 0x7fe450003970 2023-11-24T06:31:29.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:29.133 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3235524397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":3235524397},{"type":"v1","addr":"172.21.15.173:6837","nonce":3235524397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/593524744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":593524744},{"type":"v1","addr":"172.21.15.173:6836","nonce":593524744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7173,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/825050165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":825050165},{"type":"v1","addr":"172.21.15.16:6834","nonce":825050165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:31:22.237194+0000","modified":"2023-11-24T06:31:25.583915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2773872585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2773872585},{"type":"v1","addr":"172.21.15.16:6835","nonce":2773872585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:31:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 53 2023-11-24T06:31:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe45406b3e0 msgr2=0x7fe45406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe45406b3e0 0x7fe45406d8a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe46815f0f0 tx=0x7fe45c008040 comp rx=0 tx=0).stop 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 msgr2=0x7fe46813cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe46813cd70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe45802c470 tx=0x7fe458000fe0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 shutdown_connections 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe45406b3e0 0x7fe45406d8a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe468146cc0 0x7fe468147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe468154330 0x7fe468154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 --2- 172.21.15.16:0/3401746343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe450003970 0x7fe46813cd70 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 >> 172.21.15.16:0/3401746343 conn(0x7fe46806a800 msgr2=0x7fe4680b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 shutdown_connections 2023-11-24T06:31:29.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.132+0000 7fe46eda9640 1 -- 172.21.15.16:0/3401746343 wait complete. 2023-11-24T06:31:29.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:31:22.237194+0000', 'modified': '2023-11-24T06:31:25.583915+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7176}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7176': {'gid': 7176, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/2773872585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 2773872585}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2773872585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:31:29.558 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:31:29.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:31:29.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.790+0000 7f01ee46d640 1 Processor -- start 2023-11-24T06:31:29.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.790+0000 7f01ee46d640 1 -- start start 2023-11-24T06:31:29.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.792+0000 7f01ee46d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f01e80b3f30 0x7f01e80d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.792+0000 7f01ee46d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e80d0a10 con 0x7f01e80d0100 2023-11-24T06:31:29.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.792+0000 7f01ee46d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e80cf080 con 0x7f01e8160920 2023-11-24T06:31:29.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.792+0000 7f01ee46d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e80c6be0 con 0x7f01e80b3f30 2023-11-24T06:31:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.792+0000 7f01ecc6a640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f01e80d0100 0x7f01e80d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55728/0 (socket says 172.21.15.16:55728) 2023-11-24T06:31:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.792+0000 7f01ecc6a640 1 -- 172.21.15.16:0/3390250999 learned_addr learned my addr 172.21.15.16:0/3390250999 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01edc6c640 1 --2- 172.21.15.16:0/3390250999 >> v2:172.21.15.173:3300/0 conn(0x7f01e80b3f30 0x7f01e80d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296594692 0 0) 0x7f01e80d0a10 con 0x7f01e80d0100 2023-11-24T06:31:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d4003520 con 0x7f01e80d0100 2023-11-24T06:31:29.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3858928172 0 0) 0x7f01d4003520 con 0x7f01e80d0100 2023-11-24T06:31:29.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 >> v2:172.21.15.173:3300/0 conn(0x7f01e80b3f30 msgr2=0x7f01e80d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 --2- 172.21.15.16:0/3390250999 >> v2:172.21.15.173:3300/0 conn(0x7f01e80b3f30 0x7f01e80d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 >> v1:172.21.15.173:6789/0 conn(0x7f01e8160920 legacy=0x7f01e80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80ce7e0 con 0x7f01e80d0100 2023-11-24T06:31:29.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f01dc001670 con 0x7f01e80d0100 2023-11-24T06:31:29.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 >> v1:172.21.15.16:6789/0 conn(0x7f01e80d0100 legacy=0x7f01e80d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01d40047c0 0x7f01d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01d4005180 0x7f01d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80c6be0 con 0x7f01d4003970 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d400c020 con 0x7f01d4005180 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.793+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d400c1e0 con 0x7f01d40047c0 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.794+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.794+0000 7f01edc6c640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01d4005180 0x7f01d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.794+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01d40047c0 0x7f01d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.794+0000 7f01ed46b640 1 -- 172.21.15.16:0/3390250999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01d40047c0 msgr2=0x7f01d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.794+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01d40047c0 0x7f01d4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.794+0000 7f01ed46b640 1 -- 172.21.15.16:0/3390250999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01d4005180 msgr2=0x7f01d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.794+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01d4005180 0x7f01d400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.794+0000 7f01ed46b640 1 -- 172.21.15.16:0/3390250999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d4003520 con 0x7f01d4003970 2023-11-24T06:31:29.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.794+0000 7f01edc6c640 1 -- 172.21.15.16:0/3390250999 reap_dead start 2023-11-24T06:31:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.795+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01d4003d90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f01e4001750 tx=0x7f01d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9409171071707fa9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.795+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40052d0 con 0x7f01d4003970 2023-11-24T06:31:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.795+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e402e050 con 0x7f01d4003970 2023-11-24T06:31:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.795+0000 7f01dbfff640 1 -- 172.21.15.16:0/3390250999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4003830 con 0x7f01d4003970 2023-11-24T06:31:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.796+0000 7f01ee46d640 1 -- 172.21.15.16:0/3390250999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 msgr2=0x7f01d4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.796+0000 7f01ee46d640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01d4003d90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f01e4001750 tx=0x7f01d40057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.796+0000 7f01ee46d640 1 -- 172.21.15.16:0/3390250999 shutdown_connections 2023-11-24T06:31:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.796+0000 7f01ee46d640 1 --2- 172.21.15.16:0/3390250999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01d4003d90 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.796+0000 7f01ee46d640 1 -- 172.21.15.16:0/3390250999 >> 172.21.15.16:0/3390250999 conn(0x7f01e80d3090 msgr2=0x7f01e80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.796+0000 7f01ee46d640 1 -- 172.21.15.16:0/3390250999 shutdown_connections 2023-11-24T06:31:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.796+0000 7f01ee46d640 1 -- 172.21.15.16:0/3390250999 wait complete. 2023-11-24T06:31:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.798+0000 7f01ee46d640 1 Processor -- start 2023-11-24T06:31:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.798+0000 7f01ee46d640 1 -- start start 2023-11-24T06:31:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.799+0000 7f01ee46d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01e80b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.799+0000 7f01ee46d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01e80c8bb0 0x7f01e8150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.799+0000 7f01ee46d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01e80c9030 0x7f01e8150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.799+0000 7f01ee46d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e8151910 con 0x7f01d4003970 2023-11-24T06:31:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.799+0000 7f01ee46d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80cad40 con 0x7f01e80c9030 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.799+0000 7f01ee46d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80b7a20 con 0x7f01e80c8bb0 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.799+0000 7f01ed46b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01e80b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01ed46b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01e80b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53984/0 (socket says 172.21.15.16:53984) 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01ed46b640 1 -- 172.21.15.16:0/283878050 learned_addr learned my addr 172.21.15.16:0/283878050 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01e80c8bb0 0x7f01e8150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01edc6c640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01e80c9030 0x7f01e8150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01ed46b640 1 -- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01e80c8bb0 msgr2=0x7f01e8150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01ed46b640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01e80c8bb0 0x7f01e8150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01ed46b640 1 -- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01e80c9030 msgr2=0x7f01e8150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01ed46b640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01e80c9030 0x7f01e8150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01ed46b640 1 -- 172.21.15.16:0/283878050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e80b7d00 con 0x7f01d4003970 2023-11-24T06:31:29.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.800+0000 7f01ed46b640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01e80b8500 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f01e402bc00 tx=0x7f01e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:29.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.801+0000 7f01d9ffb640 1 -- 172.21.15.16:0/283878050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40076c0 con 0x7f01d4003970 2023-11-24T06:31:29.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.801+0000 7f01d9ffb640 1 -- 172.21.15.16:0/283878050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4007a30 con 0x7f01d4003970 2023-11-24T06:31:29.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.801+0000 7f01d9ffb640 1 -- 172.21.15.16:0/283878050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4006be0 con 0x7f01d4003970 2023-11-24T06:31:29.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.801+0000 7f01ee46d640 1 -- 172.21.15.16:0/283878050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e80b7fe0 con 0x7f01d4003970 2023-11-24T06:31:29.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.801+0000 7f01ee46d640 1 -- 172.21.15.16:0/283878050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e80b8a40 con 0x7f01d4003970 2023-11-24T06:31:29.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.803+0000 7f01d9ffb640 1 -- 172.21.15.16:0/283878050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f01e4006d40 con 0x7f01d4003970 2023-11-24T06:31:29.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.804+0000 7f01d9ffb640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01cc06b3e0 0x7f01cc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.804+0000 7f01d9ffb640 1 -- 172.21.15.16:0/283878050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f01e40abde0 con 0x7f01d4003970 2023-11-24T06:31:29.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.805+0000 7f01f2269740 1 Processor -- start 2023-11-24T06:31:29.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.805+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01cc06b3e0 0x7f01cc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.805+0000 7f01f2269740 1 -- start start 2023-11-24T06:31:29.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01f2269740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f059070 0x55ff6f059450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01f2269740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f058270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01f2269740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f0587b0 0x55ff6f09c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01f2269740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6efb7880 con 0x55ff6f053d70 2023-11-24T06:31:29.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01f2269740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6efc6ae0 con 0x55ff6f0587b0 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01f2269740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff6efc41d0 con 0x55ff6f059070 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01cc06b3e0 0x7f01cc06d8a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f01dc001200 tx=0x7f01dc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01ecc6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f058270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01ecc6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f058270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53988/0 (socket says 172.21.15.16:53988) 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01ecc6a640 1 -- 172.21.15.16:0/3978777765 learned_addr learned my addr 172.21.15.16:0/3978777765 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01edc6c640 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f0587b0 0x55ff6f09c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f059070 0x55ff6f059450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01ecc6a640 1 -- 172.21.15.16:0/3978777765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f059070 msgr2=0x55ff6f059450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f059070 0x55ff6f059450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01ecc6a640 1 -- 172.21.15.16:0/3978777765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f0587b0 msgr2=0x55ff6f09c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.806+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f0587b0 0x55ff6f09c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.807+0000 7f01ecc6a640 1 -- 172.21.15.16:0/3978777765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6ef87860 con 0x55ff6f053d70 2023-11-24T06:31:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.807+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f059070 0x55ff6f059450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.807+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f058270 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f01dc007180 tx=0x7f01dc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c352f211eba03b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.807+0000 7f01caffd640 1 -- 172.21.15.16:0/3978777765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00c050 con 0x55ff6f053d70 2023-11-24T06:31:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.807+0000 7f01caffd640 1 -- 172.21.15.16:0/3978777765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc00d040 con 0x55ff6f053d70 2023-11-24T06:31:29.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.807+0000 7f01caffd640 1 -- 172.21.15.16:0/3978777765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00eb70 con 0x55ff6f053d70 2023-11-24T06:31:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.808+0000 7f01f2269740 1 -- 172.21.15.16:0/3978777765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 msgr2=0x55ff6f058270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.808+0000 7f01f2269740 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f058270 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f01dc007180 tx=0x7f01dc011000 comp rx=0 tx=0).stop 2023-11-24T06:31:29.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.808+0000 7f01f2269740 1 -- 172.21.15.16:0/3978777765 shutdown_connections 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.808+0000 7f01f2269740 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f0587b0 0x55ff6f09c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.808+0000 7f01f2269740 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f058270 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.808+0000 7f01f2269740 1 --2- 172.21.15.16:0/3978777765 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f059070 0x55ff6f059450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.808+0000 7f01f2269740 1 -- 172.21.15.16:0/3978777765 >> 172.21.15.16:0/3978777765 conn(0x55ff6f01ac70 msgr2=0x55ff6f09eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.808+0000 7f01f2269740 1 -- 172.21.15.16:0/3978777765 shutdown_connections 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.808+0000 7f01f2269740 1 -- 172.21.15.16:0/3978777765 wait complete. 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.809+0000 7f01f2269740 1 -- start start 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.809+0000 7f01f2269740 1 Processor -- start 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01f2269740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f0a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01f2269740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f081e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01f2269740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 0x55ff6f0823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01f2269740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6efbfac0 con 0x55ff6f053d70 2023-11-24T06:31:29.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01f2269740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6ef5c2f0 con 0x55ff6f059070 2023-11-24T06:31:29.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01f2269740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff6ef79c70 con 0x55ff6f0587b0 2023-11-24T06:31:29.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01f2269740 10 client.0 _ll_register_callbacks cb 0x55ff6ef7e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:31:29.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01ed46b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f0a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01ed46b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f0a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53990/0 (socket says 172.21.15.16:53990) 2023-11-24T06:31:29.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.810+0000 7f01ed46b640 1 -- 172.21.15.16:0/3813978080 learned_addr learned my addr 172.21.15.16:0/3813978080 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:29.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f081e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01edc6c640 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 0x55ff6f0823d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01ed46b640 1 -- 172.21.15.16:0/3813978080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 msgr2=0x55ff6f081e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f081e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01ed46b640 1 -- 172.21.15.16:0/3813978080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 msgr2=0x55ff6f0823d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 0x55ff6f0823d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01ed46b640 1 -- 172.21.15.16:0/3813978080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e80ce7e0 con 0x55ff6f053d70 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01edc6c640 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 0x55ff6f0823d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f0a3c40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f01e404b000 tx=0x7f01e407a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139730c007a6ad3f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01c3fff640 1 -- 172.21.15.16:0/3813978080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40a9890 con 0x55ff6f053d70 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01c3fff640 1 -- 172.21.15.16:0/3813978080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f01e4082500 con 0x55ff6f053d70 2023-11-24T06:31:29.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.811+0000 7f01f2269740 1 -- 172.21.15.16:0/3813978080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff6efb8aa0 con 0x55ff6f053d70 2023-11-24T06:31:29.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01c3fff640 1 -- 172.21.15.16:0/3813978080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01e40796f0 con 0x55ff6f053d70 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01f2269740 10 client.7323 fetch_fsmap learned FSMap version 53 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01f2269740 1 -- 172.21.15.16:0/3813978080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55ff6efbe140 con 0x55ff6f053d70 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01c3fff640 1 -- 172.21.15.16:0/3813978080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff6efb8aa0 con 0x55ff6f053d70 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01c3fff640 1 -- 172.21.15.16:0/3813978080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01e404d020 con 0x55ff6f053d70 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01f2269740 10 client.7323 fetch_fsmap finished waiting for FSMap version 53 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01f2269740 10 client.7323 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01f2269740 20 client.7323 populate_metadata read hostname 'smithi016' 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01f2269740 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] conn(0x55ff6f0b2200 0x55ff6f0bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01f2269740 4 client.7323 mds_command: new command op to 7176 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:31:29.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01f2269740 1 -- 172.21.15.16:0/3813978080 --> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff6efb87c0 con 0x55ff6f0b2200 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.812+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] conn(0x55ff6f0b2200 0x55ff6f0bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.813+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] conn(0x55ff6f0b2200 0x55ff6f0bbbf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da7223d30d56c48d server_cookie=8f2cedf260c23e01 in_seq=0 out_seq=0 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.813+0000 7f01c3fff640 10 client.7323 ms_handle_connect on v2:172.21.15.16:6833/2773872585 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01c3fff640 1 -- 172.21.15.16:0/3813978080 <== mds.0 v2:172.21.15.16:6833/2773872585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff6efb87c0 con 0x55ff6f0b2200 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01c3fff640 10 client.7323 handle_command_reply: tid=0 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 client.7323 shutdown 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 -- 172.21.15.16:0/3813978080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 msgr2=0x55ff6f0a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f0a3c40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f01e404b000 tx=0x7f01e407a980 comp rx=0 tx=0).stop 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 -- 172.21.15.16:0/3813978080 shutdown_connections 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] conn(0x55ff6f0b2200 0x55ff6f0bbbf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 0x55ff6f0823d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f081e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 --2- 172.21.15.16:0/3813978080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f0a3c40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01c3fff640 0 client.7323 ms_handle_reset on v2:172.21.15.16:6833/2773872585 2023-11-24T06:31:29.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 -- 172.21.15.16:0/3813978080 >> 172.21.15.16:0/3813978080 conn(0x55ff6f01ac70 msgr2=0x55ff6f052dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:29.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 -- 172.21.15.16:0/3813978080 shutdown_connections 2023-11-24T06:31:29.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.814+0000 7f01f2269740 1 -- 172.21.15.16:0/3813978080 wait complete. 2023-11-24T06:31:29.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.815+0000 7f01f2269740 20 client.7323 trim_cache size 0 max 16384 2023-11-24T06:31:29.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.832+0000 7f01f2269740 1 Processor -- start 2023-11-24T06:31:29.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.832+0000 7f01f2269740 1 -- start start 2023-11-24T06:31:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.833+0000 7f01f2269740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f053d70 0x55ff6f09f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.833+0000 7f01f2269740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f09ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.833+0000 7f01f2269740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f059070 0x55ff6f052dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.833+0000 7f01f2269740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6ed22f20 con 0x55ff6f059070 2023-11-24T06:31:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.833+0000 7f01f2269740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6f059690 con 0x55ff6f053d70 2023-11-24T06:31:29.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.833+0000 7f01f2269740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff6f0585f0 con 0x55ff6f0587b0 2023-11-24T06:31:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.833+0000 7f01edc6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f059070 0x55ff6f052dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.833+0000 7f01edc6c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f059070 0x55ff6f052dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54004/0 (socket says 172.21.15.16:54004) 2023-11-24T06:31:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.833+0000 7f01edc6c640 1 -- 172.21.15.16:0/3161060647 learned_addr learned my addr 172.21.15.16:0/3161060647 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.834+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f053d70 0x55ff6f09f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.834+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f09ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.834+0000 7f01edc6c640 1 -- 172.21.15.16:0/3161060647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 msgr2=0x55ff6f09ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.834+0000 7f01edc6c640 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f09ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.834+0000 7f01edc6c640 1 -- 172.21.15.16:0/3161060647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f053d70 msgr2=0x55ff6f09f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.834+0000 7f01edc6c640 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f053d70 0x55ff6f09f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.834+0000 7f01edc6c640 1 -- 172.21.15.16:0/3161060647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff6ef5bb80 con 0x55ff6f059070 2023-11-24T06:31:29.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.834+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f053d70 0x55ff6f09f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:31:29.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.834+0000 7f01edc6c640 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f059070 0x55ff6f052dd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009f90 tx=0x7f01e0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818e225d2eadd651 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:29.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01cb7fe640 1 -- 172.21.15.16:0/3161060647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e000e040 con 0x55ff6f059070 2023-11-24T06:31:29.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01cb7fe640 1 -- 172.21.15.16:0/3161060647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e0015ea0 con 0x55ff6f059070 2023-11-24T06:31:29.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01cb7fe640 1 -- 172.21.15.16:0/3161060647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0003bd0 con 0x55ff6f059070 2023-11-24T06:31:29.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01f2269740 1 -- 172.21.15.16:0/3161060647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f059070 msgr2=0x55ff6f052dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01f2269740 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f059070 0x55ff6f052dd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009f90 tx=0x7f01e0009750 comp rx=0 tx=0).stop 2023-11-24T06:31:29.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01f2269740 1 -- 172.21.15.16:0/3161060647 shutdown_connections 2023-11-24T06:31:29.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01f2269740 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f059070 0x55ff6f052dd0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01f2269740 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f09ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01f2269740 1 --2- 172.21.15.16:0/3161060647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f053d70 0x55ff6f09f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.835+0000 7f01f2269740 1 -- 172.21.15.16:0/3161060647 >> 172.21.15.16:0/3161060647 conn(0x55ff6f01ac70 msgr2=0x55ff6f057960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:29.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.836+0000 7f01f2269740 1 -- 172.21.15.16:0/3161060647 shutdown_connections 2023-11-24T06:31:29.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.836+0000 7f01f2269740 1 -- 172.21.15.16:0/3161060647 wait complete. 2023-11-24T06:31:29.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.836+0000 7f01f2269740 1 -- start start 2023-11-24T06:31:29.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.836+0000 7f01f2269740 1 Processor -- start 2023-11-24T06:31:29.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.837+0000 7f01f2269740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f07fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.837+0000 7f01f2269740 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f07e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.837+0000 7f01f2269740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 0x55ff6f09c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.837+0000 7f01f2269740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6f09d340 con 0x55ff6f053d70 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.837+0000 7f01f2269740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6f086630 con 0x55ff6f059070 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.837+0000 7f01f2269740 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff6f05b380 con 0x55ff6f0587b0 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01f2269740 10 client.0 _ll_register_callbacks cb 0x7f01e40765b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01ed46b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f07fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01ed46b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f07fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54014/0 (socket says 172.21.15.16:54014) 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01ed46b640 1 -- 172.21.15.16:0/3504809086 learned_addr learned my addr 172.21.15.16:0/3504809086 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01ed46b640 1 -- 172.21.15.16:0/3504809086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 msgr2=0x55ff6f07e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f07e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01ed46b640 1 -- 172.21.15.16:0/3504809086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 msgr2=0x55ff6f09c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 0x55ff6f09c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01ed46b640 1 -- 172.21.15.16:0/3504809086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e0009cc0 con 0x55ff6f053d70 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01ed46b640 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f07fdc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f01e4078550 tx=0x7f01e4007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953846406e39b4c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.838+0000 7f01c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01c3fff640 1 -- 172.21.15.16:0/3504809086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4030430 con 0x55ff6f053d70 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01c3fff640 1 -- 172.21.15.16:0/3504809086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f01e4082500 con 0x55ff6f053d70 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01f2269740 1 -- 172.21.15.16:0/3504809086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01dc00d040 con 0x55ff6f053d70 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01c3fff640 1 -- 172.21.15.16:0/3504809086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01e407a200 con 0x55ff6f053d70 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01f2269740 10 client.7329 fetch_fsmap learned FSMap version 53 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01f2269740 1 -- 172.21.15.16:0/3504809086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55ff6ef81170 con 0x55ff6f053d70 2023-11-24T06:31:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01c3fff640 1 -- 172.21.15.16:0/3504809086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc00d040 con 0x55ff6f053d70 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01c3fff640 1 -- 172.21.15.16:0/3504809086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01e4091d20 con 0x55ff6f053d70 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01f2269740 10 client.7329 fetch_fsmap finished waiting for FSMap version 53 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01f2269740 10 client.7329 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01f2269740 20 client.7329 populate_metadata read hostname 'smithi016' 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01f2269740 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] conn(0x55ff6f09f0a0 0x55ff6f09c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01f2269740 4 client.7329 mds_command: new command op to 7176 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01f2269740 1 -- 172.21.15.16:0/3504809086 --> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ff6ecc06c0 con 0x55ff6f09f0a0 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.839+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] conn(0x55ff6f09f0a0 0x55ff6f09c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01ecc6a640 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] conn(0x55ff6f09f0a0 0x55ff6f09c9d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f4c93233b6f1b1d server_cookie=5400238fda3e5463 in_seq=0 out_seq=0 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01c3fff640 10 client.7329 ms_handle_connect on v2:172.21.15.16:6833/2773872585 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01c3fff640 1 -- 172.21.15.16:0/3504809086 <== mds.0 v2:172.21.15.16:6833/2773872585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ff6ecc06c0 con 0x55ff6f09f0a0 2023-11-24T06:31:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01c3fff640 10 client.7329 handle_command_reply: tid=0 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01f2269740 1 client.7329 shutdown 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01f2269740 1 -- 172.21.15.16:0/3504809086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 msgr2=0x55ff6f07fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01f2269740 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f07fdc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f01e4078550 tx=0x7f01e4007c20 comp rx=0 tx=0).stop 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01f2269740 1 -- 172.21.15.16:0/3504809086 shutdown_connections 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01f2269740 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.16:6833/2773872585,v1:172.21.15.16:6835/2773872585] conn(0x55ff6f09f0a0 0x55ff6f09c9d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01f2269740 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff6f059070 0x55ff6f09c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01f2269740 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ff6f0587b0 0x55ff6f07e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01f2269740 1 --2- 172.21.15.16:0/3504809086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ff6f053d70 0x55ff6f07fdc0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01c3fff640 0 client.7329 ms_handle_reset on v2:172.21.15.16:6833/2773872585 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.840+0000 7f01f2269740 1 -- 172.21.15.16:0/3504809086 >> 172.21.15.16:0/3504809086 conn(0x55ff6f01ac70 msgr2=0x55ff6f080590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.841+0000 7f01f2269740 1 -- 172.21.15.16:0/3504809086 shutdown_connections 2023-11-24T06:31:29.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.841+0000 7f01f2269740 1 -- 172.21.15.16:0/3504809086 wait complete. 2023-11-24T06:31:29.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.841+0000 7f01f2269740 20 client.7329 trim_cache size 0 max 16384 2023-11-24T06:31:29.848 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:31:29.848 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "6dc1668b-58b0-47dd-b8dc-4164e5a228ab", 2023-11-24T06:31:29.848 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-11-24T06:31:29.848 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7176, 2023-11-24T06:31:29.849 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-11-24T06:31:29.849 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-11-24T06:31:29.849 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-11-24T06:31:29.849 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.7918492090000004, 2023-11-24T06:31:29.849 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 53, 2023-11-24T06:31:29.849 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 86, 2023-11-24T06:31:29.849 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 86, 2023-11-24T06:31:29.849 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 12.858685163000001 2023-11-24T06:31:29.849 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:31:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.841+0000 7f01ee46d640 1 -- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01cc06b3e0 msgr2=0x7f01cc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.841+0000 7f01ee46d640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01cc06b3e0 0x7f01cc06d8a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f01dc001200 tx=0x7f01dc009040 comp rx=0 tx=0).stop 2023-11-24T06:31:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.841+0000 7f01ee46d640 1 -- 172.21.15.16:0/283878050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 msgr2=0x7f01e80b8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.841+0000 7f01ee46d640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01e80b8500 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f01e402bc00 tx=0x7f01e4001120 comp rx=0 tx=0).stop 2023-11-24T06:31:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.842+0000 7f01ee46d640 1 -- 172.21.15.16:0/283878050 shutdown_connections 2023-11-24T06:31:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.842+0000 7f01ee46d640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f01cc06b3e0 0x7f01cc06d8a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.842+0000 7f01ee46d640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01e80c9030 0x7f01e8150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.842+0000 7f01ee46d640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01e80c8bb0 0x7f01e8150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.842+0000 7f01ee46d640 1 --2- 172.21.15.16:0/283878050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01d4003970 0x7f01e80b8500 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.842+0000 7f01ee46d640 1 -- 172.21.15.16:0/283878050 >> 172.21.15.16:0/283878050 conn(0x7f01e80d3090 msgr2=0x7f01e80d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.842+0000 7f01ee46d640 1 -- 172.21.15.16:0/283878050 shutdown_connections 2023-11-24T06:31:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:29.842+0000 7f01ee46d640 1 -- 172.21.15.16:0/283878050 wait complete. 2023-11-24T06:31:29.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:31:30.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.083+0000 7fa714fb3640 1 Processor -- start 2023-11-24T06:31:30.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.083+0000 7fa714fb3640 1 -- start start 2023-11-24T06:31:30.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.085+0000 7fa714fb3640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa710071410 0x7fa710146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:30.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.085+0000 7fa714fb3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa71006c7c0 0x7fa7100717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:30.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.085+0000 7fa714fb3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71006cc20 con 0x7fa710071d30 2023-11-24T06:31:30.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.085+0000 7fa714fb3640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71006cec0 con 0x7fa710071410 2023-11-24T06:31:30.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.085+0000 7fa714fb3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71006d1a0 con 0x7fa71006c7c0 2023-11-24T06:31:30.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.085+0000 7fa70ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa710071d30 0x7fa7100b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55730/0 (socket says 172.21.15.16:55730) 2023-11-24T06:31:30.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.085+0000 7fa70ffff640 1 -- 172.21.15.16:0/3357898373 learned_addr learned my addr 172.21.15.16:0/3357898373 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70effd640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa71006c7c0 0x7fa7100717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70f7fe640 1 --2- 172.21.15.16:0/3357898373 >> v2:172.21.15.173:3300/0 conn(0x7fa710071410 0x7fa710146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731313726 0 0) 0x7fa71006cc20 con 0x7fa710071d30 2023-11-24T06:31:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8003520 con 0x7fa710071d30 2023-11-24T06:31:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1927612888 0 0) 0x7fa6f8003520 con 0x7fa710071d30 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa71006c7c0 msgr2=0x7fa7100717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa71006c7c0 0x7fa7100717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 >> v2:172.21.15.173:3300/0 conn(0x7fa710071410 msgr2=0x7fa710146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 --2- 172.21.15.16:0/3357898373 >> v2:172.21.15.173:3300/0 conn(0x7fa710071410 0x7fa710146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71006d480 con 0x7fa710071d30 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fa7080019f0 con 0x7fa710071d30 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 >> v1:172.21.15.16:6789/0 conn(0x7fa710071d30 legacy=0x7fa7100b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa6f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6f80047c0 0x7fa6f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa71006c7c0 0x7fa6f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71006cec0 con 0x7fa6f8003970 2023-11-24T06:31:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71006d1a0 con 0x7fa6f80047c0 2023-11-24T06:31:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.086+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f800b980 con 0x7fa71006c7c0 2023-11-24T06:31:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.087+0000 7fa70f7fe640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa6f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.087+0000 7fa70ffff640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa71006c7c0 0x7fa6f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.087+0000 7fa70f7fe640 1 -- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa71006c7c0 msgr2=0x7fa6f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.087+0000 7fa70effd640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6f80047c0 0x7fa6f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.087+0000 7fa70f7fe640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa71006c7c0 0x7fa6f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.087+0000 7fa70f7fe640 1 -- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6f80047c0 msgr2=0x7fa6f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.087+0000 7fa70f7fe640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6f80047c0 0x7fa6f8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.087+0000 7fa70f7fe640 1 -- 172.21.15.16:0/3357898373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8003520 con 0x7fa6f8003970 2023-11-24T06:31:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.089+0000 7fa70f7fe640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa6f8003d90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa700009ac0 tx=0x7fa6f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5008d66aa847d2fa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.089+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70002c710 con 0x7fa6f8003970 2023-11-24T06:31:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.089+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70002c870 con 0x7fa6f8003970 2023-11-24T06:31:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.089+0000 7fa70e7fc640 1 -- 172.21.15.16:0/3357898373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70002cc00 con 0x7fa6f8003970 2023-11-24T06:31:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.089+0000 7fa714fb3640 1 -- 172.21.15.16:0/3357898373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 msgr2=0x7fa6f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.089+0000 7fa714fb3640 1 --2- 172.21.15.16:0/3357898373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa6f8003d90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa700009ac0 tx=0x7fa6f80057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.089+0000 7fa70ffff640 1 -- 172.21.15.16:0/3357898373 reap_dead start 2023-11-24T06:31:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.089+0000 7fa714fb3640 1 -- 172.21.15.16:0/3357898373 shutdown_connections 2023-11-24T06:31:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.089+0000 7fa714fb3640 1 -- 172.21.15.16:0/3357898373 >> 172.21.15.16:0/3357898373 conn(0x7fa71006a800 msgr2=0x7fa71006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.090+0000 7fa714fb3640 1 -- 172.21.15.16:0/3357898373 shutdown_connections 2023-11-24T06:31:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.090+0000 7fa714fb3640 1 -- 172.21.15.16:0/3357898373 wait complete. 2023-11-24T06:31:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.092+0000 7fa714fb3640 1 Processor -- start 2023-11-24T06:31:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.092+0000 7fa714fb3640 1 -- start start 2023-11-24T06:31:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.093+0000 7fa714fb3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa710133e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.093+0000 7fa714fb3640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7100c3d70 0x7fa7100c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.093+0000 7fa714fb3640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7101e59e0 0x7fa7101e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.093+0000 7fa714fb3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71012e970 con 0x7fa6f8003970 2023-11-24T06:31:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.093+0000 7fa714fb3640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100c4730 con 0x7fa7100c3d70 2023-11-24T06:31:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.093+0000 7fa714fb3640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71012e010 con 0x7fa7101e59e0 2023-11-24T06:31:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.093+0000 7fa70f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa710133e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.093+0000 7fa70f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa710133e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54026/0 (socket says 172.21.15.16:54026) 2023-11-24T06:31:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.093+0000 7fa70f7fe640 1 -- 172.21.15.16:0/3783923952 learned_addr learned my addr 172.21.15.16:0/3783923952 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa70ffff640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7101e59e0 0x7fa7101e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa70f7fe640 1 -- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7101e59e0 msgr2=0x7fa7101e5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa70f7fe640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7101e59e0 0x7fa7101e5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa70f7fe640 1 -- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7100c3d70 msgr2=0x7fa7100c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa70effd640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7100c3d70 0x7fa7100c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa70f7fe640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7100c3d70 0x7fa7100c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa70f7fe640 1 -- 172.21.15.16:0/3783923952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71012e2b0 con 0x7fa6f8003970 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa70effd640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7100c3d70 0x7fa7100c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa70f7fe640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa710133e60 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa70002c2b0 tx=0x7fa700004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa6f7fff640 1 -- 172.21.15.16:0/3783923952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700003810 con 0x7fa6f8003970 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.094+0000 7fa6f7fff640 1 -- 172.21.15.16:0/3783923952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70002de90 con 0x7fa6f8003970 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.095+0000 7fa6f7fff640 1 -- 172.21.15.16:0/3783923952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700003b70 con 0x7fa6f8003970 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.095+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71012e590 con 0x7fa6f8003970 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.095+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7101343a0 con 0x7fa6f8003970 2023-11-24T06:31:30.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.096+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa710067450 con 0x7fa6f8003970 2023-11-24T06:31:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.098+0000 7fa6f7fff640 1 -- 172.21.15.16:0/3783923952 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fa700003cd0 con 0x7fa6f8003970 2023-11-24T06:31:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.098+0000 7fa6f7fff640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa6fc06b410 0x7fa6fc06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.099+0000 7fa6f7fff640 1 -- 172.21.15.16:0/3783923952 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa7000abeb0 con 0x7fa6f8003970 2023-11-24T06:31:30.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.099+0000 7fa70effd640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa6fc06b410 0x7fa6fc06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:30.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.100+0000 7fa70effd640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa6fc06b410 0x7fa6fc06d8d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa71012de40 tx=0x7fa704008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:30.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.103+0000 7fa6f7fff640 1 -- 172.21.15.16:0/3783923952 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa700076840 con 0x7fa6f8003970 2023-11-24T06:31:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.334+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa71006b3f0 con 0x7fa6f8003970 2023-11-24T06:31:30.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.335+0000 7fa6f7fff640 1 -- 172.21.15.16:0/3783923952 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa700044060 con 0x7fa6f8003970 2023-11-24T06:31:30.339 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:31:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa6fc06b410 msgr2=0x7fa6fc06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa6fc06b410 0x7fa6fc06d8d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa71012de40 tx=0x7fa704008040 comp rx=0 tx=0).stop 2023-11-24T06:31:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 msgr2=0x7fa710133e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:30.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa710133e60 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa70002c2b0 tx=0x7fa700004730 comp rx=0 tx=0).stop 2023-11-24T06:31:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 shutdown_connections 2023-11-24T06:31:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fa6fc06b410 0x7fa6fc06d8d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa7101e59e0 0x7fa7101e5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7100c3d70 0x7fa7100c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 --2- 172.21.15.16:0/3783923952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6f8003970 0x7fa710133e60 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 >> 172.21.15.16:0/3783923952 conn(0x7fa71006a800 msgr2=0x7fa7100b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 shutdown_connections 2023-11-24T06:31:30.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:30.338+0000 7fa714fb3640 1 -- 172.21.15.16:0/3783923952 wait complete. 2023-11-24T06:31:30.359 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:31:30.360 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:31:30.360 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-11-24T06:31:30.360 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:31:30.360 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:31:30.360 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:31:30.360 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:31:30.360 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:31:30.360 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:31:30.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:30.361 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-11-24T06:31:30.417 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:31:30.417 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:31:30.417 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:31:30.417 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:30.417 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:30.418 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42053sec preferred_lft 42053sec 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:31:30.419 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8cfd:d2ff:fee1:5b18/64 scope link 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 8e:89:e0:fc:a3:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8c89:e0ff:fefc:a3fb/64 scope link 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether ea:e3:8b:37:17:ee brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::e8e3:8bff:fe37:17ee/64 scope link 2023-11-24T06:31:30.420 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:30.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:30.421 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:31:30.521 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-11-24T06:31:30.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:30.526 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-11-24T06:31:30.581 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-11-24T06:31:30.582 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:31:30.582 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:31:30.582 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:31:30.582 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:31:30.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:30.582 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-11-24T06:31:30.583 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-11-24T06:31:30.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:31:30.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:31:30.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:31:30.583 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-11-24T06:31:30.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:30.787 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-24T06:31:30.845 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:31:30.845 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:31:30.845 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:31:30.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:30.846 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-11-24T06:31:30.948 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:31:30.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:31:30.949 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:31:31.009 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:31:31.010 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:31:31.010 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:31:31.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:31:31.168 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:31:31.225 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:31:31.225 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:31:31.225 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:31:31.225 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:31:31.226 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:31:31.226 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:31:31.226 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:31:31.226 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:31:31.226 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:31:31.226 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:31:31.226 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:31:31.227 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:31:31.227 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:31:31.227 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:31:31.227 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:31:31.227 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:31:31.227 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:31:31.228 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:31:31.228 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:31:31.228 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:31:31.228 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:31:31.228 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:31:31.228 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:31:31.228 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:31:31.228 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:31:31.229 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:31:31.229 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:31:31.229 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:31:31.229 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:31:31.229 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:31:31.229 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:31:31.229 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:31:31.230 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:31:31.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:31.231 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:31:31.287 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:31:31.288 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:31:31.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:31:31.378 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:31:31.427 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:31:31.427 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:31:31.427 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:31:31.427 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:31:31.427 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:31:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:31:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:31:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:31:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:31:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:31:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:31:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:31:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:31:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:31:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:31:31.430 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:31:31.430 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:31:31.430 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:31:31.430 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:31:31.430 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:31:31.430 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:31:31.430 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:31:31.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:31.432 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:31:31.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-11-24T06:31:31.491+0000 7f1cdbed6480 -1 init, newargv = 0x5567aa132a90 newargc=15 2023-11-24T06:31:31.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[114892]: starting ceph client 2023-11-24T06:31:31.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[114892]: starting fuse 2023-11-24T06:31:32.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-11-24T06:31:32.688 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:31:32.750 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-11-24T06:31:32.751 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-11-24T06:31:32.752 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-11-24T06:31:32.752 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-11-24T06:31:32.752 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-11-24T06:31:32.752 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-11-24T06:31:32.752 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-11-24T06:31:32.753 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-11-24T06:31:32.753 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-11-24T06:31:32.753 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-11-24T06:31:32.753 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-11-24T06:31:32.753 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:31:32.754 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:31:32.755 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:31:32.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:32.756 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-11-24T06:31:32.815 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-11-24T06:31:32.815 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:31:32.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:31:32.816 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:31:32.817 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-11-24T06:31:32.818 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-11-24T06:31:32.818 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-11-24T06:31:32.818 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:31:32.818 DEBUG:teuthology.orchestra.run.smithi016:> 2023-11-24T06:31:32.818 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-11-24T06:31:32.818 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-11-24T06:31:33.007 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.114892.asok 2023-11-24T06:31:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:31:33.025 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.114892.asok 2023-11-24T06:31:33.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:33.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.114892.asok status 2023-11-24T06:31:33.279 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:31:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-11-24T06:31:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:31:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:31:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "114892", 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7341, 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-11-24T06:31:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 7341 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3463577644 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-11-24T06:31:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-11-24T06:31:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:31:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3463577644 2023-11-24T06:31:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-11-24T06:31:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.7341 192.168.0.1:0/3463577644", 2023-11-24T06:31:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/3463577644", 2023-11-24T06:31:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-11-24T06:31:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 53, 2023-11-24T06:31:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 86, 2023-11-24T06:31:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:31:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-11-24T06:31:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-11-24T06:31:33.283 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:31:33.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:33.295 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-11-24T06:31:33.295 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:31:33.354 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-11-24T06:31:33.355 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:31:33.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:33.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:31:33.458 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:31:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.679+0000 7f84bb59e640 1 Processor -- start 2023-11-24T06:31:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.679+0000 7f84bb59e640 1 -- start start 2023-11-24T06:31:33.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.681+0000 7f84bb59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f84bc071f00 0x7f84bc146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:33.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.681+0000 7f84bb59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84bc0b2e10 con 0x7f84bc06c7c0 2023-11-24T06:31:33.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.681+0000 7f84bb59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84bc06cc10 con 0x7f84bc071410 2023-11-24T06:31:33.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.681+0000 7f84bb59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84bc06cec0 con 0x7f84bc071f00 2023-11-24T06:31:33.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.681+0000 7f84bad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f84bc06c7c0 0x7f84bc166cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52038/0 (socket says 172.21.15.16:52038) 2023-11-24T06:31:33.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.681+0000 7f84bad9d640 1 -- 172.21.15.16:0/810175432 learned_addr learned my addr 172.21.15.16:0/810175432 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b3fff640 1 --2- 172.21.15.16:0/810175432 >> v2:172.21.15.173:3300/0 conn(0x7f84bc071f00 0x7f84bc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021814110 0 0) 0x7f84bc0b2e10 con 0x7f84bc06c7c0 2023-11-24T06:31:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84a8003520 con 0x7f84bc06c7c0 2023-11-24T06:31:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3090771327 0 0) 0x7f84a8003520 con 0x7f84bc06c7c0 2023-11-24T06:31:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 >> v2:172.21.15.173:3300/0 conn(0x7f84bc071f00 msgr2=0x7f84bc146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 --2- 172.21.15.16:0/810175432 >> v2:172.21.15.173:3300/0 conn(0x7f84bc071f00 0x7f84bc146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 >> v1:172.21.15.173:6789/0 conn(0x7f84bc071410 legacy=0x7f84bc0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc06d1a0 con 0x7f84bc06c7c0 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f84b40019f0 con 0x7f84bc06c7c0 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 >> v1:172.21.15.16:6789/0 conn(0x7f84bc06c7c0 legacy=0x7f84bc166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84a8003970 0x7f84a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a80047c0 0x7f84a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84a8005180 0x7f84a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc06cec0 con 0x7f84a80047c0 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a800c120 con 0x7f84a8003970 2023-11-24T06:31:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.682+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a800c2e0 con 0x7f84a8005180 2023-11-24T06:31:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.683+0000 7f84b3fff640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a80047c0 0x7f84a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.683+0000 7f84b3fff640 1 -- 172.21.15.16:0/810175432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84a8005180 msgr2=0x7f84a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.683+0000 7f84b3fff640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84a8005180 0x7f84a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.683+0000 7f84ba59c640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84a8003970 0x7f84a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.683+0000 7f84b3fff640 1 -- 172.21.15.16:0/810175432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84a8003970 msgr2=0x7f84a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.683+0000 7f84b3fff640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84a8003970 0x7f84a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.683+0000 7f84b3fff640 1 -- 172.21.15.16:0/810175432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a8003520 con 0x7f84a80047c0 2023-11-24T06:31:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.683+0000 7f84bad9d640 1 -- 172.21.15.16:0/810175432 reap_dead start 2023-11-24T06:31:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.683+0000 7f84ba59c640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84a8003970 0x7f84a8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.684+0000 7f84b3fff640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a80047c0 0x7f84a8004c40 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009ac0 tx=0x7f84a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8309858e401a9700 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.685+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a402c6d0 con 0x7f84a80047c0 2023-11-24T06:31:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.685+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a402c830 con 0x7f84a80047c0 2023-11-24T06:31:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.685+0000 7f84b9d9b640 1 -- 172.21.15.16:0/810175432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a402cbc0 con 0x7f84a80047c0 2023-11-24T06:31:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.685+0000 7f84bb59e640 1 -- 172.21.15.16:0/810175432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a80047c0 msgr2=0x7f84a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.685+0000 7f84bb59e640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a80047c0 0x7f84a8004c40 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009ac0 tx=0x7f84a80057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.686+0000 7f84bb59e640 1 -- 172.21.15.16:0/810175432 shutdown_connections 2023-11-24T06:31:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.686+0000 7f84bb59e640 1 --2- 172.21.15.16:0/810175432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a80047c0 0x7f84a8004c40 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.686+0000 7f84bb59e640 1 -- 172.21.15.16:0/810175432 >> 172.21.15.16:0/810175432 conn(0x7f84bc06a800 msgr2=0x7f84bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.686+0000 7f84bb59e640 1 -- 172.21.15.16:0/810175432 shutdown_connections 2023-11-24T06:31:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.686+0000 7f84bb59e640 1 -- 172.21.15.16:0/810175432 wait complete. 2023-11-24T06:31:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.688+0000 7f84bb59e640 1 Processor -- start 2023-11-24T06:31:33.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.688+0000 7f84bb59e640 1 -- start start 2023-11-24T06:31:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84bb59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84a80047c0 0x7f84bc132b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84bb59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84bc1e8140 0x7f84bc1e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84bb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc1e63b0 0x7f84bc1e6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84bb59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1e8b00 con 0x7f84bc1e63b0 2023-11-24T06:31:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84bb59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc133050 con 0x7f84bc1e8140 2023-11-24T06:31:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84bb59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1e70a0 con 0x7f84a80047c0 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84bad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc1e63b0 0x7f84bc1e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84bad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc1e63b0 0x7f84bc1e6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51288/0 (socket says 172.21.15.16:51288) 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84bad9d640 1 -- 172.21.15.16:0/1187201174 learned_addr learned my addr 172.21.15.16:0/1187201174 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84b3fff640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84bc1e8140 0x7f84bc1e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.689+0000 7f84ba59c640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84a80047c0 0x7f84bc132b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84bad9d640 1 -- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84a80047c0 msgr2=0x7f84bc132b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84bad9d640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84a80047c0 0x7f84bc132b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84bad9d640 1 -- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84bc1e8140 msgr2=0x7f84bc1e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84bad9d640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84bc1e8140 0x7f84bc1e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84bad9d640 1 -- 172.21.15.16:0/1187201174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc1e7380 con 0x7f84bc1e63b0 2023-11-24T06:31:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84bad9d640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc1e63b0 0x7f84bc1e6830 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f84b4001560 tx=0x7f84b400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84b2ffd640 1 -- 172.21.15.16:0/1187201174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400a610 con 0x7f84bc1e63b0 2023-11-24T06:31:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84b2ffd640 1 -- 172.21.15.16:0/1187201174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b400a790 con 0x7f84bc1e63b0 2023-11-24T06:31:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84b2ffd640 1 -- 172.21.15.16:0/1187201174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400ab40 con 0x7f84bc1e63b0 2023-11-24T06:31:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.690+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc1e7660 con 0x7f84bc1e63b0 2023-11-24T06:31:33.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.691+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc1e7c20 con 0x7f84bc1e63b0 2023-11-24T06:31:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.692+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc067450 con 0x7f84bc1e63b0 2023-11-24T06:31:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.693+0000 7f84b2ffd640 1 -- 172.21.15.16:0/1187201174 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f84b400aca0 con 0x7f84bc1e63b0 2023-11-24T06:31:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.694+0000 7f84b2ffd640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f849c06b3e0 0x7f849c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:33.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.694+0000 7f84b2ffd640 1 -- 172.21.15.16:0/1187201174 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f84b40899b0 con 0x7f84bc1e63b0 2023-11-24T06:31:33.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.695+0000 7f84ba59c640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f849c06b3e0 0x7f849c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:33.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.696+0000 7f84ba59c640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f849c06b3e0 0x7f849c06d8a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f84ac000c00 tx=0x7f84ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:33.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.699+0000 7f84b2ffd640 1 -- 172.21.15.16:0/1187201174 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84b4054260 con 0x7f84bc1e63b0 2023-11-24T06:31:33.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.933+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f84bc0acbd0 con 0x7f84bc1e63b0 2023-11-24T06:31:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.934+0000 7f84b2ffd640 1 -- 172.21.15.16:0/1187201174 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f84b401a020 con 0x7f84bc1e63b0 2023-11-24T06:31:33.937 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:31:33.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f849c06b3e0 msgr2=0x7f849c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f849c06b3e0 0x7f849c06d8a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f84ac000c00 tx=0x7f84ac009040 comp rx=0 tx=0).stop 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc1e63b0 msgr2=0x7f84bc1e6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc1e63b0 0x7f84bc1e6830 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f84b4001560 tx=0x7f84b400cf00 comp rx=0 tx=0).stop 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 shutdown_connections 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f849c06b3e0 0x7f849c06d8a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc1e63b0 0x7f84bc1e6830 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84bc1e8140 0x7f84bc1e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 --2- 172.21.15.16:0/1187201174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84a80047c0 0x7f84bc132b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 >> 172.21.15.16:0/1187201174 conn(0x7f84bc06a800 msgr2=0x7f84bc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 shutdown_connections 2023-11-24T06:31:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:33.937+0000 7f84bb59e640 1 -- 172.21.15.16:0/1187201174 wait complete. 2023-11-24T06:31:33.960 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:31:33.960 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:31:33.960 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2023-11-24T06:31:33.960 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:31:33.960 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:31:33.961 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:31:33.961 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:31:33.961 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:31:33.961 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:31:33.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:33.961 DEBUG:teuthology.orchestra.run.smithi173:> ip addr 2023-11-24T06:31:34.021 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:31:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:31:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:31:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2023-11-24T06:31:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:34.021 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f0 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: altname enp7s0f1 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft 42110sec preferred_lft 42110sec 2023-11-24T06:31:34.022 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::c4e1:59ff:feae:2f56/64 scope link 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ea:3b:9f:a2:9b:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:31:34.023 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::e83b:9fff:fea2:9ba6/64 scope link 2023-11-24T06:31:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:34.024 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:31:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 66:de:a3:e0:a8:27 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:31:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::64de:a3ff:fee0:a827/64 scope link 2023-11-24T06:31:34.024 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:31:34.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:34.025 DEBUG:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:31:34.117 INFO:teuthology.orchestra.run.smithi173.stdout:1 2023-11-24T06:31:34.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:34.121 DEBUG:teuthology.orchestra.run.smithi173:> route 2023-11-24T06:31:34.172 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2023-11-24T06:31:34.173 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:31:34.173 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:31:34.173 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:31:34.173 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:31:34.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:34.173 DEBUG:teuthology.orchestra.run.smithi173:> (cd / && exec bash -c ' 2023-11-24T06:31:34.174 DEBUG:teuthology.orchestra.run.smithi173:> set -e 2023-11-24T06:31:34.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:31:34.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:31:34.174 DEBUG:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:31:34.174 DEBUG:teuthology.orchestra.run.smithi173:> ') 2023-11-24T06:31:34.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:34.353 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2023-11-24T06:31:34.405 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:31:34.406 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:31:34.406 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:31:34.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:34.407 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2023-11-24T06:31:34.505 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:31:34.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:31:34.506 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:31:34.559 INFO:teuthology.orchestra.run.smithi173.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:31:34.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:31:34.560 DEBUG:teuthology.orchestra.run.smithi173:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:31:34.611 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:31:34.704 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:31:34.761 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:31:34.761 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:31:34.761 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:31:34.761 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:31:34.761 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:31:34.762 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:31:34.763 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:31:34.764 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:31:34.764 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:31:34.764 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:31:34.764 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:31:34.764 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:31:34.764 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:31:34.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:34.765 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:31:34.823 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:31:34.823 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:31:34.829 DEBUG:teuthology.orchestra.run.smithi173:> sudo modprobe fuse 2023-11-24T06:31:34.911 DEBUG:teuthology.orchestra.run.smithi173:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:31:34.955 INFO:teuthology.orchestra.run.smithi173.stdout:/proc 2023-11-24T06:31:34.955 INFO:teuthology.orchestra.run.smithi173.stdout:/sys 2023-11-24T06:31:34.955 INFO:teuthology.orchestra.run.smithi173.stdout:/dev 2023-11-24T06:31:34.956 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/security 2023-11-24T06:31:34.956 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/shm 2023-11-24T06:31:34.956 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/pts 2023-11-24T06:31:34.956 INFO:teuthology.orchestra.run.smithi173.stdout:/run 2023-11-24T06:31:34.956 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/cgroup 2023-11-24T06:31:34.956 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/pstore 2023-11-24T06:31:34.956 INFO:teuthology.orchestra.run.smithi173.stdout:/ 2023-11-24T06:31:34.957 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/selinux 2023-11-24T06:31:34.957 INFO:teuthology.orchestra.run.smithi173.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:31:34.957 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/hugepages 2023-11-24T06:31:34.957 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/mqueue 2023-11-24T06:31:34.957 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/debug 2023-11-24T06:31:34.957 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/tracing 2023-11-24T06:31:34.957 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:31:34.958 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/fs/fuse/connections 2023-11-24T06:31:34.958 INFO:teuthology.orchestra.run.smithi173.stdout:/sys/kernel/config 2023-11-24T06:31:34.958 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:31:34.958 INFO:teuthology.orchestra.run.smithi173.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:31:34.958 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:31:34.958 INFO:teuthology.orchestra.run.smithi173.stdout:/run/user/1000 2023-11-24T06:31:34.958 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph 2023-11-24T06:31:34.959 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:31:34.959 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:31:34.959 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:31:34.959 INFO:teuthology.orchestra.run.smithi173.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:31:34.959 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns 2023-11-24T06:31:34.959 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:31:34.959 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:31:34.959 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:31:34.959 INFO:teuthology.orchestra.run.smithi173.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:31:34.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:34.961 DEBUG:teuthology.orchestra.run.smithi173:> ls /sys/fs/fuse/connections 2023-11-24T06:31:35.015 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:2023-11-24T06:31:35.012+0000 7f3d86429480 -1 init, newargv = 0x563f95e68ac0 newargc=15 2023-11-24T06:31:35.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[98231]: starting ceph client 2023-11-24T06:31:35.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[98231]: starting fuse 2023-11-24T06:31:35.046 INFO:teuthology.orchestra.run.smithi173.stdout:39 2023-11-24T06:31:35.047 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-11-24T06:31:35.047 DEBUG:teuthology.orchestra.run.smithi173:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:31:35.047 DEBUG:teuthology.orchestra.run.smithi173:> import glob 2023-11-24T06:31:35.047 DEBUG:teuthology.orchestra.run.smithi173:> import re 2023-11-24T06:31:35.047 DEBUG:teuthology.orchestra.run.smithi173:> import os 2023-11-24T06:31:35.047 DEBUG:teuthology.orchestra.run.smithi173:> import subprocess 2023-11-24T06:31:35.047 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:31:35.047 DEBUG:teuthology.orchestra.run.smithi173:> def _find_admin_socket(client_name): 2023-11-24T06:31:35.047 DEBUG:teuthology.orchestra.run.smithi173:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> files = glob.glob(asok_path) 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> # Given a non-glob path, it better be there 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> if "*" not in asok_path: 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> assert(len(files) == 1) 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> return files[0] 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> for f in files: 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:31:35.048 DEBUG:teuthology.orchestra.run.smithi173:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:31:35.049 DEBUG:teuthology.orchestra.run.smithi173:> contents = proc_f.read() 2023-11-24T06:31:35.049 DEBUG:teuthology.orchestra.run.smithi173:> if mountpoint in contents: 2023-11-24T06:31:35.049 DEBUG:teuthology.orchestra.run.smithi173:> return f 2023-11-24T06:31:35.049 DEBUG:teuthology.orchestra.run.smithi173:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:31:35.049 DEBUG:teuthology.orchestra.run.smithi173:> 2023-11-24T06:31:35.049 DEBUG:teuthology.orchestra.run.smithi173:> print(_find_admin_socket("client.1")) 2023-11-24T06:31:35.049 DEBUG:teuthology.orchestra.run.smithi173:> ' 2023-11-24T06:31:35.245 INFO:teuthology.orchestra.run.smithi173.stdout:/var/run/ceph/ceph-client.1.98231.asok 2023-11-24T06:31:35.249 INFO:teuthology.orchestra.run.smithi173.stderr:2023-11-24T06:31:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:31:35.262 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.98231.asok 2023-11-24T06:31:35.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:35.263 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.98231.asok status 2023-11-24T06:31:35.503 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-11-24T06:31:35.503 INFO:teuthology.orchestra.run.smithi173.stdout: "metadata": { 2023-11-24T06:31:35.503 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:31:35.503 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:31:35.503 INFO:teuthology.orchestra.run.smithi173.stdout: "entity_id": "1", 2023-11-24T06:31:35.503 INFO:teuthology.orchestra.run.smithi173.stdout: "hostname": "smithi173", 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "pid": "98231", 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "root": "/" 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_count": 0, 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "dentry_pinned_count": 0, 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "id": 5504, 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "inst": { 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "name": { 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "client", 2023-11-24T06:31:35.504 INFO:teuthology.orchestra.run.smithi173.stdout: "num": 5504 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 804102422 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": { 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: "type": "v1", 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:31:35.505 INFO:teuthology.orchestra.run.smithi173.stdout: "nonce": 804102422 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout: "inst_str": "client.5504 192.168.0.1:0/804102422", 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout: "addr_str": "192.168.0.1:0/804102422", 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout: "inode_count": 1, 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_epoch": 53, 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch": 86, 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout: "blocklisted": false, 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout: "fs_name": "cephfs" 2023-11-24T06:31:35.506 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-11-24T06:31:35.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:35.518 DEBUG:teuthology.orchestra.run.smithi173:> stat --file-system '--printf=%T 2023-11-24T06:31:35.518 DEBUG:teuthology.orchestra.run.smithi173:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:31:35.576 INFO:teuthology.orchestra.run.smithi173.stdout:fuseblk 2023-11-24T06:31:35.576 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:31:35.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:35.576 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:31:35.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:31:35.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.908+0000 7f34dbfff640 1 Processor -- start 2023-11-24T06:31:35.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.908+0000 7f34dbfff640 1 -- start start 2023-11-24T06:31:35.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.910+0000 7f34dbfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f34dc13cfa0 0x7f34dc15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:35.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.910+0000 7f34dbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34dc13fcb0 0x7f34dc0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:35.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.910+0000 7f34dbfff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc0cdec0 con 0x7f34dc0cd530 2023-11-24T06:31:35.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.910+0000 7f34dbfff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc0ce1a0 con 0x7f34dc13fcb0 2023-11-24T06:31:35.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.910+0000 7f34dbfff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc0ce480 con 0x7f34dc13cfa0 2023-11-24T06:31:35.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.911+0000 7f34db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34dc13fcb0 0x7f34dc0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:35.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.911+0000 7f34db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34dc13fcb0 0x7f34dc0b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51290/0 (socket says 172.21.15.16:51290) 2023-11-24T06:31:35.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.911+0000 7f34db7fe640 1 -- 172.21.15.16:0/3955568343 learned_addr learned my addr 172.21.15.16:0/3955568343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:35.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.911+0000 7f34daffd640 1 --2- 172.21.15.16:0/3955568343 >> v2:172.21.15.173:3300/0 conn(0x7f34dc13cfa0 0x7f34dc15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:35.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.911+0000 7f34db7fe640 1 -- 172.21.15.16:0/3955568343 >> v2:172.21.15.173:3300/0 conn(0x7f34dc13cfa0 msgr2=0x7f34dc15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:35.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.911+0000 7f34db7fe640 1 --2- 172.21.15.16:0/3955568343 >> v2:172.21.15.173:3300/0 conn(0x7f34dc13cfa0 0x7f34dc15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.912+0000 7f34db7fe640 1 -- 172.21.15.16:0/3955568343 >> v1:172.21.15.173:6789/0 conn(0x7f34dc0cd530 legacy=0x7f34dc13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.912+0000 7f34db7fe640 1 -- 172.21.15.16:0/3955568343 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0ce760 con 0x7f34dc13fcb0 2023-11-24T06:31:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.913+0000 7f34db7fe640 1 --2- 172.21.15.16:0/3955568343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13fcb0 0x7f34dc0b2040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f34dc06cf00 tx=0x7f34d402e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3eb4ef9cb723c8b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.913+0000 7f34d9ffb640 1 -- 172.21.15.16:0/3955568343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d402ea90 con 0x7f34dc13fcb0 2023-11-24T06:31:35.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.913+0000 7f34d9ffb640 1 -- 172.21.15.16:0/3955568343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d402ebf0 con 0x7f34dc13fcb0 2023-11-24T06:31:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.913+0000 7f34d9ffb640 1 -- 172.21.15.16:0/3955568343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4009600 con 0x7f34dc13fcb0 2023-11-24T06:31:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.914+0000 7f34dbfff640 1 -- 172.21.15.16:0/3955568343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13fcb0 msgr2=0x7f34dc0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.914+0000 7f34dbfff640 1 --2- 172.21.15.16:0/3955568343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13fcb0 0x7f34dc0b2040 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f34dc06cf00 tx=0x7f34d402e0f0 comp rx=0 tx=0).stop 2023-11-24T06:31:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.914+0000 7f34dbfff640 1 -- 172.21.15.16:0/3955568343 shutdown_connections 2023-11-24T06:31:35.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.914+0000 7f34dbfff640 1 --2- 172.21.15.16:0/3955568343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13fcb0 0x7f34dc0b2040 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.914+0000 7f34dbfff640 1 --2- 172.21.15.16:0/3955568343 >> v2:172.21.15.173:3300/0 conn(0x7f34dc13cfa0 0x7f34dc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.914+0000 7f34dbfff640 1 -- 172.21.15.16:0/3955568343 >> 172.21.15.16:0/3955568343 conn(0x7f34dc06a800 msgr2=0x7f34dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.914+0000 7f34dbfff640 1 -- 172.21.15.16:0/3955568343 shutdown_connections 2023-11-24T06:31:35.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.914+0000 7f34dbfff640 1 -- 172.21.15.16:0/3955568343 wait complete. 2023-11-24T06:31:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.916+0000 7f34dbfff640 1 Processor -- start 2023-11-24T06:31:35.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.916+0000 7f34dbfff640 1 -- start start 2023-11-24T06:31:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.917+0000 7f34dbfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34dc0cd530 0x7f34dc0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.917+0000 7f34dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13cfa0 0x7f34dc0b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.917+0000 7f34dbfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34dc13fcb0 0x7f34dc13d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:35.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.917+0000 7f34dbfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc13f220 con 0x7f34dc13cfa0 2023-11-24T06:31:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.917+0000 7f34dbfff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc13e610 con 0x7f34dc13fcb0 2023-11-24T06:31:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.917+0000 7f34dbfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc13e8f0 con 0x7f34dc0cd530 2023-11-24T06:31:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.917+0000 7f34da7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13cfa0 0x7f34dc0b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.917+0000 7f34da7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13cfa0 0x7f34dc0b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51304/0 (socket says 172.21.15.16:51304) 2023-11-24T06:31:35.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.917+0000 7f34da7fc640 1 -- 172.21.15.16:0/2646013671 learned_addr learned my addr 172.21.15.16:0/2646013671 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34daffd640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34dc0cd530 0x7f34dc0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34da7fc640 1 -- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34dc0cd530 msgr2=0x7f34dc0afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34da7fc640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34dc0cd530 0x7f34dc0afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34db7fe640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34dc13fcb0 0x7f34dc13d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:35.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34da7fc640 1 -- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34dc13fcb0 msgr2=0x7f34dc13d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34da7fc640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34dc13fcb0 0x7f34dc13d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34da7fc640 1 -- 172.21.15.16:0/2646013671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d400bd40 con 0x7f34dc13cfa0 2023-11-24T06:31:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34da7fc640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13cfa0 0x7f34dc0b0370 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f34d000c450 tx=0x7f34d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34c37fe640 1 -- 172.21.15.16:0/2646013671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d00052d0 con 0x7f34dc13cfa0 2023-11-24T06:31:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34c37fe640 1 -- 172.21.15.16:0/2646013671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d0005430 con 0x7f34dc13cfa0 2023-11-24T06:31:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34c37fe640 1 -- 172.21.15.16:0/2646013671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0012870 con 0x7f34dc13cfa0 2023-11-24T06:31:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.918+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc13ebd0 con 0x7f34dc13cfa0 2023-11-24T06:31:35.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.919+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc13f6b0 con 0x7f34dc13cfa0 2023-11-24T06:31:35.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.919+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc13f3d0 con 0x7f34dc13cfa0 2023-11-24T06:31:35.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.921+0000 7f34c37fe640 1 -- 172.21.15.16:0/2646013671 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f34d00129d0 con 0x7f34dc13cfa0 2023-11-24T06:31:35.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.922+0000 7f34c37fe640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c806b3e0 0x7f34c806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.922+0000 7f34c37fe640 1 -- 172.21.15.16:0/2646013671 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f34d0089c10 con 0x7f34dc13cfa0 2023-11-24T06:31:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.923+0000 7f34daffd640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c806b3e0 0x7f34c806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:35.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.924+0000 7f34daffd640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c806b3e0 0x7f34c806d8a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f34cc006fd0 tx=0x7f34cc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:35.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:35.927+0000 7f34c37fe640 1 -- 172.21.15.16:0/2646013671 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34dc13f3d0 con 0x7f34dc13cfa0 2023-11-24T06:31:36.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:36.162+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f34dc0b2500 con 0x7f34dc13cfa0 2023-11-24T06:31:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.140+0000 7f34c37fe640 1 -- 172.21.15.16:0/2646013671 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 136+0+0 (secure 0 0 0) 0x7f34d00543f0 con 0x7f34dc13cfa0 2023-11-24T06:31:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c806b3e0 msgr2=0x7f34c806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c806b3e0 0x7f34c806d8a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f34cc006fd0 tx=0x7f34cc008040 comp rx=0 tx=0).stop 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13cfa0 msgr2=0x7f34dc0b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13cfa0 0x7f34dc0b0370 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f34d000c450 tx=0x7f34d000c910 comp rx=0 tx=0).stop 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 shutdown_connections 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f34c806b3e0 0x7f34c806d8a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34dc13fcb0 0x7f34dc13d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34dc13cfa0 0x7f34dc0b0370 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 --2- 172.21.15.16:0/2646013671 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34dc0cd530 0x7f34dc0afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 >> 172.21.15.16:0/2646013671 conn(0x7f34dc06a800 msgr2=0x7f34dc0aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 shutdown_connections 2023-11-24T06:31:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.144+0000 7f34dbfff640 1 -- 172.21.15.16:0/2646013671 wait complete. 2023-11-24T06:31:37.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:31:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.400+0000 7fcd0a0d1640 1 Processor -- start 2023-11-24T06:31:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.400+0000 7fcd0a0d1640 1 -- start start 2023-11-24T06:31:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.401+0000 7fcd0a0d1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fcd0415d410 0x7fcd0406c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.401+0000 7fcd0a0d1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd0406c640 0x7fcd040cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.401+0000 7fcd0a0d1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd040b1e00 con 0x7fcd040d0dc0 2023-11-24T06:31:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.401+0000 7fcd0a0d1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd040b1b20 con 0x7fcd0406c640 2023-11-24T06:31:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.401+0000 7fcd0a0d1640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd0406b620 con 0x7fcd0415d410 2023-11-24T06:31:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd090cf640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd040d0dc0 0x7fcd040d11a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52040/0 (socket says 172.21.15.16:52040) 2023-11-24T06:31:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd090cf640 1 -- 172.21.15.16:0/672727099 learned_addr learned my addr 172.21.15.16:0/672727099 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd098d0640 1 --2- 172.21.15.16:0/672727099 >> v2:172.21.15.16:3300/0 conn(0x7fcd0406c640 0x7fcd040cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd088ce640 1 --2- 172.21.15.16:0/672727099 >> v2:172.21.15.173:3300/0 conn(0x7fcd0415d410 0x7fcd0406c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd03fff640 1 -- 172.21.15.16:0/672727099 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008598598 0 0) 0x7fcd040b1e00 con 0x7fcd040d0dc0 2023-11-24T06:31:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd03fff640 1 -- 172.21.15.16:0/672727099 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccec003520 con 0x7fcd040d0dc0 2023-11-24T06:31:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd098d0640 1 -- 172.21.15.16:0/672727099 >> v2:172.21.15.173:3300/0 conn(0x7fcd0415d410 msgr2=0x7fcd0406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd098d0640 1 --2- 172.21.15.16:0/672727099 >> v2:172.21.15.173:3300/0 conn(0x7fcd0415d410 0x7fcd0406c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd098d0640 1 -- 172.21.15.16:0/672727099 >> v1:172.21.15.16:6789/0 conn(0x7fcd040d0dc0 legacy=0x7fcd040d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.402+0000 7fcd098d0640 1 -- 172.21.15.16:0/672727099 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0406b900 con 0x7fcd0406c640 2023-11-24T06:31:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.404+0000 7fcd098d0640 1 --2- 172.21.15.16:0/672727099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd0406c640 0x7fcd040cf1d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcd040ce980 tx=0x7fccfc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9db8fed9456c93 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.404+0000 7fcd03fff640 1 -- 172.21.15.16:0/672727099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc02e070 con 0x7fcd0406c640 2023-11-24T06:31:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.404+0000 7fcd03fff640 1 -- 172.21.15.16:0/672727099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc0315a0 con 0x7fcd0406c640 2023-11-24T06:31:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.404+0000 7fcd03fff640 1 -- 172.21.15.16:0/672727099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc031930 con 0x7fcd0406c640 2023-11-24T06:31:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.404+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/672727099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd0406c640 msgr2=0x7fcd040cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.404+0000 7fcd0a0d1640 1 --2- 172.21.15.16:0/672727099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd0406c640 0x7fcd040cf1d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcd040ce980 tx=0x7fccfc02b840 comp rx=0 tx=0).stop 2023-11-24T06:31:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.404+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/672727099 shutdown_connections 2023-11-24T06:31:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.404+0000 7fcd0a0d1640 1 --2- 172.21.15.16:0/672727099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd0406c640 0x7fcd040cf1d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.405+0000 7fcd0a0d1640 1 --2- 172.21.15.16:0/672727099 >> v2:172.21.15.173:3300/0 conn(0x7fcd0415d410 0x7fcd0406c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.405+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/672727099 >> 172.21.15.16:0/672727099 conn(0x7fcd04099a60 msgr2=0x7fcd04099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.405+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/672727099 shutdown_connections 2023-11-24T06:31:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.405+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/672727099 wait complete. 2023-11-24T06:31:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.406+0000 7fcd0a0d1640 1 Processor -- start 2023-11-24T06:31:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.407+0000 7fcd0a0d1640 1 -- start start 2023-11-24T06:31:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd0a0d1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd0406c640 0x7fcd040c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd0a0d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd040d0dc0 0x7fcd04153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd0a0d1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0415d410 0x7fcd040b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd0a0d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040c8fb0 con 0x7fcd040d0dc0 2023-11-24T06:31:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd0a0d1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040c8470 con 0x7fcd0415d410 2023-11-24T06:31:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd0a0d1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040c8710 con 0x7fcd0406c640 2023-11-24T06:31:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd088ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd040d0dc0 0x7fcd04153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd088ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd040d0dc0 0x7fcd04153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51314/0 (socket says 172.21.15.16:51314) 2023-11-24T06:31:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd088ce640 1 -- 172.21.15.16:0/262871432 learned_addr learned my addr 172.21.15.16:0/262871432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd090cf640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd0406c640 0x7fcd040c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd088ce640 1 -- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd0406c640 msgr2=0x7fcd040c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd088ce640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd0406c640 0x7fcd040c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd088ce640 1 -- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0415d410 msgr2=0x7fcd040b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd088ce640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0415d410 0x7fcd040b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.408+0000 7fcd088ce640 1 -- 172.21.15.16:0/262871432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc009400 con 0x7fcd040d0dc0 2023-11-24T06:31:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.409+0000 7fcd090cf640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd0406c640 0x7fcd040c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.409+0000 7fcd088ce640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd040d0dc0 0x7fcd04153290 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fccf800d800 tx=0x7fccf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.409+0000 7fcd01ffb640 1 -- 172.21.15.16:0/262871432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf800b6c0 con 0x7fcd040d0dc0 2023-11-24T06:31:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.409+0000 7fcd01ffb640 1 -- 172.21.15.16:0/262871432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf800b820 con 0x7fcd040d0dc0 2023-11-24T06:31:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.409+0000 7fcd01ffb640 1 -- 172.21.15.16:0/262871432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf800bbb0 con 0x7fcd040d0dc0 2023-11-24T06:31:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.409+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd040c89f0 con 0x7fcd040d0dc0 2023-11-24T06:31:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.409+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd040c94b0 con 0x7fcd040d0dc0 2023-11-24T06:31:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.410+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd040c9640 con 0x7fcd040d0dc0 2023-11-24T06:31:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.412+0000 7fcd01ffb640 1 -- 172.21.15.16:0/262871432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fccf800bd10 con 0x7fcd040d0dc0 2023-11-24T06:31:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.413+0000 7fcd01ffb640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fccf006b410 0x7fccf006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.413+0000 7fcd01ffb640 1 -- 172.21.15.16:0/262871432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fccf808a580 con 0x7fcd040d0dc0 2023-11-24T06:31:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.413+0000 7fcd090cf640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fccf006b410 0x7fccf006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.415+0000 7fcd090cf640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fccf006b410 0x7fccf006d8d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fccf4001560 tx=0x7fccf4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.418+0000 7fcd01ffb640 1 -- 172.21.15.16:0/262871432 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fccf8054df0 con 0x7fcd040d0dc0 2023-11-24T06:31:37.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.636+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd040cbb80 con 0x7fccf006b410 2023-11-24T06:31:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.640+0000 7fcd01ffb640 1 -- 172.21.15.16:0/262871432 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcd040cbb80 con 0x7fccf006b410 2023-11-24T06:31:37.642 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-11-24T06:31:37.642 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-11-24T06:31:37.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephfs" 2023-11-24T06:31:37.643 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-11-24T06:31:37.643 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-11-24T06:31:37.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fccf006b410 msgr2=0x7fccf006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fccf006b410 0x7fccf006d8d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fccf4001560 tx=0x7fccf4009040 comp rx=0 tx=0).stop 2023-11-24T06:31:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd040d0dc0 msgr2=0x7fcd04153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd040d0dc0 0x7fcd04153290 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fccf800d800 tx=0x7fccf800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 shutdown_connections 2023-11-24T06:31:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fccf006b410 0x7fccf006d8d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0415d410 0x7fcd040b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd040d0dc0 0x7fcd04153290 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 --2- 172.21.15.16:0/262871432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd0406c640 0x7fcd040c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.641+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 >> 172.21.15.16:0/262871432 conn(0x7fcd04099a60 msgr2=0x7fcd040cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:37.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.642+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 shutdown_connections 2023-11-24T06:31:37.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.642+0000 7fcd0a0d1640 1 -- 172.21.15.16:0/262871432 wait complete. 2023-11-24T06:31:37.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:31:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.898+0000 7f2883209640 1 Processor -- start 2023-11-24T06:31:37.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.899+0000 7f2883209640 1 -- start start 2023-11-24T06:31:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.900+0000 7f2883209640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f287c0ae5a0 0x7f287c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.900+0000 7f2883209640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f287c0b3ae0 0x7f287c06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.900+0000 7f2883209640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f287c06d120 0x7f287c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.900+0000 7f2883209640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c071450 con 0x7f287c0ae5a0 2023-11-24T06:31:37.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.900+0000 7f2883209640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c070c40 con 0x7f287c06d120 2023-11-24T06:31:37.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.900+0000 7f2883209640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c06c470 con 0x7f287c0b3ae0 2023-11-24T06:31:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.901+0000 7f2880fa5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f287c0ae5a0 0x7f287c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.901+0000 7f2880fa5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f287c0ae5a0 0x7f287c15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51316/0 (socket says 172.21.15.16:51316) 2023-11-24T06:31:37.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.901+0000 7f2880fa5640 1 -- 172.21.15.16:0/400917294 learned_addr learned my addr 172.21.15.16:0/400917294 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.901+0000 7f287bfff640 1 --2- 172.21.15.16:0/400917294 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f287c0b3ae0 0x7f287c06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.901+0000 7f28817a6640 1 --2- 172.21.15.16:0/400917294 >> v2:172.21.15.173:3300/0 conn(0x7f287c06d120 0x7f287c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.902+0000 7f2880fa5640 1 -- 172.21.15.16:0/400917294 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f287c0b3ae0 msgr2=0x7f287c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.902+0000 7f2880fa5640 1 --2- 172.21.15.16:0/400917294 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f287c0b3ae0 0x7f287c06cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.902+0000 7f2880fa5640 1 -- 172.21.15.16:0/400917294 >> v2:172.21.15.173:3300/0 conn(0x7f287c06d120 msgr2=0x7f287c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:37.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.902+0000 7f2880fa5640 1 --2- 172.21.15.16:0/400917294 >> v2:172.21.15.173:3300/0 conn(0x7f287c06d120 0x7f287c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.902+0000 7f2880fa5640 1 -- 172.21.15.16:0/400917294 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c06c750 con 0x7f287c0ae5a0 2023-11-24T06:31:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.903+0000 7f2880fa5640 1 --2- 172.21.15.16:0/400917294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0ae5a0 0x7f287c15e430 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f287c06e230 tx=0x7f286c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51cdf4d80a340f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f287b7fe640 1 -- 172.21.15.16:0/400917294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c02ca80 con 0x7f287c0ae5a0 2023-11-24T06:31:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f287b7fe640 1 -- 172.21.15.16:0/400917294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c02cbe0 con 0x7f287c0ae5a0 2023-11-24T06:31:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f287b7fe640 1 -- 172.21.15.16:0/400917294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c0343f0 con 0x7f287c0ae5a0 2023-11-24T06:31:37.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f2883209640 1 -- 172.21.15.16:0/400917294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0ae5a0 msgr2=0x7f287c15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f2883209640 1 --2- 172.21.15.16:0/400917294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0ae5a0 0x7f287c15e430 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f287c06e230 tx=0x7f286c02bee0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f2883209640 1 -- 172.21.15.16:0/400917294 shutdown_connections 2023-11-24T06:31:37.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f2883209640 1 --2- 172.21.15.16:0/400917294 >> v2:172.21.15.173:3300/0 conn(0x7f287c06d120 0x7f287c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f2883209640 1 --2- 172.21.15.16:0/400917294 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f287c0b3ae0 0x7f287c06cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f2883209640 1 --2- 172.21.15.16:0/400917294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0ae5a0 0x7f287c15e430 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.904+0000 7f2883209640 1 -- 172.21.15.16:0/400917294 >> 172.21.15.16:0/400917294 conn(0x7f287c06a800 msgr2=0x7f287c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.905+0000 7f2883209640 1 -- 172.21.15.16:0/400917294 shutdown_connections 2023-11-24T06:31:37.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.905+0000 7f2883209640 1 -- 172.21.15.16:0/400917294 wait complete. 2023-11-24T06:31:37.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.906+0000 7f2883209640 1 Processor -- start 2023-11-24T06:31:37.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.907+0000 7f2883209640 1 -- start start 2023-11-24T06:31:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f2883209640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f287c06d120 0x7f287c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f2883209640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f287c0ae5a0 0x7f287c15d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f2883209640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b3ae0 0x7f287c15dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f2883209640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0ccdd0 con 0x7f287c0b3ae0 2023-11-24T06:31:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f2883209640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0cbd30 con 0x7f287c0ae5a0 2023-11-24T06:31:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f2883209640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0cc010 con 0x7f287c06d120 2023-11-24T06:31:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f28817a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b3ae0 0x7f287c15dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f28817a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b3ae0 0x7f287c15dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51320/0 (socket says 172.21.15.16:51320) 2023-11-24T06:31:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f28817a6640 1 -- 172.21.15.16:0/3329410635 learned_addr learned my addr 172.21.15.16:0/3329410635 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f28817a6640 1 -- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f287c06d120 msgr2=0x7f287c15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f28817a6640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f287c06d120 0x7f287c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f28817a6640 1 -- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f287c0ae5a0 msgr2=0x7f287c15d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f28817a6640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f287c0ae5a0 0x7f287c15d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.908+0000 7f28817a6640 1 -- 172.21.15.16:0/3329410635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c009b30 con 0x7f287c0b3ae0 2023-11-24T06:31:37.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.909+0000 7f28817a6640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b3ae0 0x7f287c15dd60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f287400d800 tx=0x7f287400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:37.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.909+0000 7f28797fa640 1 -- 172.21.15.16:0/3329410635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28740096c0 con 0x7f287c0b3ae0 2023-11-24T06:31:37.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.909+0000 7f28797fa640 1 -- 172.21.15.16:0/3329410635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2874009840 con 0x7f287c0b3ae0 2023-11-24T06:31:37.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.909+0000 7f28797fa640 1 -- 172.21.15.16:0/3329410635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874009bf0 con 0x7f287c0b3ae0 2023-11-24T06:31:37.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.909+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c0cc2f0 con 0x7f287c0b3ae0 2023-11-24T06:31:37.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.909+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c0cc8b0 con 0x7f287c0b3ae0 2023-11-24T06:31:37.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.910+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c0675d0 con 0x7f287c0b3ae0 2023-11-24T06:31:37.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.911+0000 7f28797fa640 1 -- 172.21.15.16:0/3329410635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2874009d50 con 0x7f287c0b3ae0 2023-11-24T06:31:37.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.912+0000 7f28797fa640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f285c06b3e0 0x7f285c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:37.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.913+0000 7f28797fa640 1 -- 172.21.15.16:0/3329410635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f287408a420 con 0x7f287c0b3ae0 2023-11-24T06:31:37.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.913+0000 7f2880fa5640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f285c06b3e0 0x7f285c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:37.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.914+0000 7f2880fa5640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f285c06b3e0 0x7f285c06d8a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f286c035000 tx=0x7f286c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:37.917+0000 7f28797fa640 1 -- 172.21.15.16:0/3329410635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2874054cd0 con 0x7f287c0b3ae0 2023-11-24T06:31:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.111+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f287c156310 con 0x7f287c0b3ae0 2023-11-24T06:31:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.112+0000 7f28797fa640 1 -- 172.21.15.16:0/3329410635 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f28740582f0 con 0x7f287c0b3ae0 2023-11-24T06:31:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.114+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f285c06b3e0 msgr2=0x7f285c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.114+0000 7f2883209640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f285c06b3e0 0x7f285c06d8a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f286c035000 tx=0x7f286c02e000 comp rx=0 tx=0).stop 2023-11-24T06:31:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b3ae0 msgr2=0x7f287c15dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b3ae0 0x7f287c15dd60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f287400d800 tx=0x7f287400dcc0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 shutdown_connections 2023-11-24T06:31:38.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f285c06b3e0 0x7f285c06d8a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287c0b3ae0 0x7f287c15dd60 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f287c0ae5a0 0x7f287c15d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 --2- 172.21.15.16:0/3329410635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f287c06d120 0x7f287c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 >> 172.21.15.16:0/3329410635 conn(0x7f287c06a800 msgr2=0x7f287c15a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:38.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 shutdown_connections 2023-11-24T06:31:38.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.115+0000 7f2883209640 1 -- 172.21.15.16:0/3329410635 wait complete. 2023-11-24T06:31:38.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp5166 2023-11-24T06:31:38.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.366+0000 7f2c08a89640 1 Processor -- start 2023-11-24T06:31:38.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.366+0000 7f2c08a89640 1 -- start start 2023-11-24T06:31:38.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.368+0000 7f2c08a89640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2c0413e360 0x7f2c040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.368+0000 7f2c08a89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c0415f9c0 0x7f2c040cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.368+0000 7f2c08a89640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0415c5f0 con 0x7f2c0413df80 2023-11-24T06:31:38.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.368+0000 7f2c08a89640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0406c470 con 0x7f2c0415f9c0 2023-11-24T06:31:38.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.368+0000 7f2c08a89640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0406c750 con 0x7f2c0413e360 2023-11-24T06:31:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.368+0000 7f2c02d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c0415f9c0 0x7f2c040cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.368+0000 7f2c02d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c0415f9c0 0x7f2c040cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51322/0 (socket says 172.21.15.16:51322) 2023-11-24T06:31:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.368+0000 7f2c02d9d640 1 -- 172.21.15.16:0/2771726879 learned_addr learned my addr 172.21.15.16:0/2771726879 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.369+0000 7f2bfbfff640 1 --2- 172.21.15.16:0/2771726879 >> v2:172.21.15.173:3300/0 conn(0x7f2c0413e360 0x7f2c040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.369+0000 7f2c02d9d640 1 -- 172.21.15.16:0/2771726879 >> v2:172.21.15.173:3300/0 conn(0x7f2c0413e360 msgr2=0x7f2c040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.369+0000 7f2c02d9d640 1 --2- 172.21.15.16:0/2771726879 >> v2:172.21.15.173:3300/0 conn(0x7f2c0413e360 0x7f2c040cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.369+0000 7f2c02d9d640 1 -- 172.21.15.16:0/2771726879 >> v1:172.21.15.16:6789/0 conn(0x7f2c0413df80 legacy=0x7f2c0415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.369+0000 7f2c02d9d640 1 -- 172.21.15.16:0/2771726879 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0406ca30 con 0x7f2c0415f9c0 2023-11-24T06:31:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.370+0000 7f2c02d9d640 1 --2- 172.21.15.16:0/2771726879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 0x7f2c040cf7d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f2c0406ded0 tx=0x7f2bfc02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2552dc7030ec43a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.370+0000 7f2c01d9b640 1 -- 172.21.15.16:0/2771726879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc02cb10 con 0x7f2c0415f9c0 2023-11-24T06:31:38.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.370+0000 7f2c01d9b640 1 -- 172.21.15.16:0/2771726879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc02cc70 con 0x7f2c0415f9c0 2023-11-24T06:31:38.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.370+0000 7f2c01d9b640 1 -- 172.21.15.16:0/2771726879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0343f0 con 0x7f2c0415f9c0 2023-11-24T06:31:38.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.371+0000 7f2c08a89640 1 -- 172.21.15.16:0/2771726879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 msgr2=0x7f2c040cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.371+0000 7f2c08a89640 1 --2- 172.21.15.16:0/2771726879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 0x7f2c040cf7d0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f2c0406ded0 tx=0x7f2bfc02bf70 comp rx=0 tx=0).stop 2023-11-24T06:31:38.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.371+0000 7f2c08a89640 1 -- 172.21.15.16:0/2771726879 shutdown_connections 2023-11-24T06:31:38.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.371+0000 7f2c08a89640 1 --2- 172.21.15.16:0/2771726879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 0x7f2c040cf7d0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.371+0000 7f2c08a89640 1 --2- 172.21.15.16:0/2771726879 >> v2:172.21.15.173:3300/0 conn(0x7f2c0413e360 0x7f2c040cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.371+0000 7f2c08a89640 1 -- 172.21.15.16:0/2771726879 >> 172.21.15.16:0/2771726879 conn(0x7f2c0406a800 msgr2=0x7f2c0406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.371+0000 7f2c08a89640 1 -- 172.21.15.16:0/2771726879 shutdown_connections 2023-11-24T06:31:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.371+0000 7f2c08a89640 1 -- 172.21.15.16:0/2771726879 wait complete. 2023-11-24T06:31:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.373+0000 7f2c08a89640 1 Processor -- start 2023-11-24T06:31:38.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.373+0000 7f2c08a89640 1 -- start start 2023-11-24T06:31:38.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2c08a89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0413df80 0x7f2c0413d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2c08a89640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0413e360 0x7f2c04136f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2c08a89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 0x7f2c04135aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2c08a89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c04137490 con 0x7f2c0415f9c0 2023-11-24T06:31:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2c08a89640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c04137c60 con 0x7f2c0413e360 2023-11-24T06:31:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2c08a89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c041376a0 con 0x7f2c0413df80 2023-11-24T06:31:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2c02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 0x7f2c04135aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2c02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 0x7f2c04135aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51334/0 (socket says 172.21.15.16:51334) 2023-11-24T06:31:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2c02d9d640 1 -- 172.21.15.16:0/4053108360 learned_addr learned my addr 172.21.15.16:0/4053108360 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.374+0000 7f2bfbfff640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0413e360 0x7f2c04136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2c0259c640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0413df80 0x7f2c0413d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2c02d9d640 1 -- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0413df80 msgr2=0x7f2c0413d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2c02d9d640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0413df80 0x7f2c0413d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2c02d9d640 1 -- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0413e360 msgr2=0x7f2c04136f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2c02d9d640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0413e360 0x7f2c04136f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2c02d9d640 1 -- 172.21.15.16:0/4053108360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc009b30 con 0x7f2c0415f9c0 2023-11-24T06:31:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2c0259c640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0413df80 0x7f2c0413d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2c02d9d640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 0x7f2c04135aa0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc036040 tx=0x7f2bfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:38.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2bfaffd640 1 -- 172.21.15.16:0/4053108360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc009510 con 0x7f2c0415f9c0 2023-11-24T06:31:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2bfaffd640 1 -- 172.21.15.16:0/4053108360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc03e450 con 0x7f2c0415f9c0 2023-11-24T06:31:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2bfaffd640 1 -- 172.21.15.16:0/4053108360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc03e7e0 con 0x7f2c0415f9c0 2023-11-24T06:31:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.375+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c04137980 con 0x7f2c0415f9c0 2023-11-24T06:31:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.376+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0414e6e0 con 0x7f2c0415f9c0 2023-11-24T06:31:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.377+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0414e260 con 0x7f2c0415f9c0 2023-11-24T06:31:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.378+0000 7f2bfaffd640 1 -- 172.21.15.16:0/4053108360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2bfc03e940 con 0x7f2c0415f9c0 2023-11-24T06:31:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.378+0000 7f2bfaffd640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2be406b3e0 0x7f2be406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.378+0000 7f2bfaffd640 1 -- 172.21.15.16:0/4053108360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2bfc0ad110 con 0x7f2c0415f9c0 2023-11-24T06:31:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.379+0000 7f2c0259c640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2be406b3e0 0x7f2be406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.380+0000 7f2c0259c640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2be406b3e0 0x7f2be406d8a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4000f60 tx=0x7f2bf4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:38.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.383+0000 7f2bfaffd640 1 -- 172.21.15.16:0/4053108360 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2bfc0dfe90 con 0x7f2c0415f9c0 2023-11-24T06:31:38.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.601+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp5166", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c0406cff0 con 0x7f2be406b3e0 2023-11-24T06:31:38.621 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:31:38.618+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:31:38.621 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:31:38.618+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:31:38.621 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:31:38.618+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:31:38.621 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:31:38.618+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:31:38.621 INFO:tasks.ceph.mgr.x.smithi173.stderr:2023-11-24T06:31:38.618+0000 7fd2b5afe640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:31:38.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.651+0000 7f2bfaffd640 1 -- 172.21.15.16:0/4053108360 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2bf4001320 con 0x7f2be406b3e0 2023-11-24T06:31:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2be406b3e0 msgr2=0x7f2be406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2be406b3e0 0x7f2be406d8a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4000f60 tx=0x7f2bf4009040 comp rx=0 tx=0).stop 2023-11-24T06:31:38.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 msgr2=0x7f2c04135aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 0x7f2c04135aa0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc036040 tx=0x7f2bfc004a40 comp rx=0 tx=0).stop 2023-11-24T06:31:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 shutdown_connections 2023-11-24T06:31:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f2be406b3e0 0x7f2be406d8a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c0415f9c0 0x7f2c04135aa0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c0413e360 0x7f2c04136f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 --2- 172.21.15.16:0/4053108360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c0413df80 0x7f2c0413d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 >> 172.21.15.16:0/4053108360 conn(0x7f2c0406a800 msgr2=0x7f2c040cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:38.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.654+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 shutdown_connections 2023-11-24T06:31:38.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.655+0000 7f2c08a89640 1 -- 172.21.15.16:0/4053108360 wait complete. 2023-11-24T06:31:38.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol8466 --group_name subvol_grp5166 2023-11-24T06:31:38.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.904+0000 7f5c5dc9b640 1 Processor -- start 2023-11-24T06:31:38.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.904+0000 7f5c5dc9b640 1 -- start start 2023-11-24T06:31:38.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.906+0000 7f5c5dc9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c58143ae0 0x7f5c580cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.906+0000 7f5c5dc9b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5c5815e3b0 0x7f5c580702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.906+0000 7f5c5dc9b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580cfc30 con 0x7f5c580cf7d0 2023-11-24T06:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.906+0000 7f5c5dc9b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c580b1ee0 con 0x7f5c5815e3b0 2023-11-24T06:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.906+0000 7f5c5dc9b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b2780 con 0x7f5c58143ae0 2023-11-24T06:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.907+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c58143ae0 0x7f5c580cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.907+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c58143ae0 0x7f5c580cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:44642/0 (socket says 172.21.15.16:44642) 2023-11-24T06:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.907+0000 7f5c577fe640 1 -- 172.21.15.16:0/1364983756 learned_addr learned my addr 172.21.15.16:0/1364983756 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.907+0000 7f5c57fff640 1 --2- 172.21.15.16:0/1364983756 >> v2:172.21.15.173:3300/0 conn(0x7f5c5815e3b0 0x7f5c580702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.908+0000 7f5c577fe640 1 -- 172.21.15.16:0/1364983756 >> v2:172.21.15.173:3300/0 conn(0x7f5c5815e3b0 msgr2=0x7f5c580702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.908+0000 7f5c577fe640 1 --2- 172.21.15.16:0/1364983756 >> v2:172.21.15.173:3300/0 conn(0x7f5c5815e3b0 0x7f5c580702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.908+0000 7f5c577fe640 1 -- 172.21.15.16:0/1364983756 >> v1:172.21.15.173:6789/0 conn(0x7f5c580cf7d0 legacy=0x7f5c5815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.908+0000 7f5c577fe640 1 -- 172.21.15.16:0/1364983756 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5815e990 con 0x7f5c58143ae0 2023-11-24T06:31:38.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.909+0000 7f5c577fe640 1 --2- 172.21.15.16:0/1364983756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c58143ae0 0x7f5c580cf290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009e50 tx=0x7f5c4802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660f1f7b3d7cbb7c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c567fc640 1 -- 172.21.15.16:0/1364983756 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4802cb00 con 0x7f5c58143ae0 2023-11-24T06:31:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c567fc640 1 -- 172.21.15.16:0/1364983756 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4802cc60 con 0x7f5c58143ae0 2023-11-24T06:31:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c567fc640 1 -- 172.21.15.16:0/1364983756 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480343f0 con 0x7f5c58143ae0 2023-11-24T06:31:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/1364983756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c58143ae0 msgr2=0x7f5c580cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c5dc9b640 1 --2- 172.21.15.16:0/1364983756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c58143ae0 0x7f5c580cf290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009e50 tx=0x7f5c4802bf60 comp rx=0 tx=0).stop 2023-11-24T06:31:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/1364983756 shutdown_connections 2023-11-24T06:31:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c5dc9b640 1 --2- 172.21.15.16:0/1364983756 >> v2:172.21.15.173:3300/0 conn(0x7f5c5815e3b0 0x7f5c580702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c5dc9b640 1 --2- 172.21.15.16:0/1364983756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c58143ae0 0x7f5c580cf290 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/1364983756 >> 172.21.15.16:0/1364983756 conn(0x7f5c5806a800 msgr2=0x7f5c5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:38.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/1364983756 shutdown_connections 2023-11-24T06:31:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.910+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/1364983756 wait complete. 2023-11-24T06:31:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.912+0000 7f5c5dc9b640 1 Processor -- start 2023-11-24T06:31:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.912+0000 7f5c5dc9b640 1 -- start start 2023-11-24T06:31:38.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.913+0000 7f5c5dc9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c580cf7d0 0x7f5c580afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.913+0000 7f5c5dc9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c58143ae0 0x7f5c58141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.913+0000 7f5c5dc9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c5815e3b0 0x7f5c58158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.913+0000 7f5c5dc9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580ae890 con 0x7f5c58143ae0 2023-11-24T06:31:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.913+0000 7f5c5dc9b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58159ea0 con 0x7f5c5815e3b0 2023-11-24T06:31:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.913+0000 7f5c5dc9b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b0160 con 0x7f5c580cf7d0 2023-11-24T06:31:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c58143ae0 0x7f5c58141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c58143ae0 0x7f5c58141900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51346/0 (socket says 172.21.15.16:51346) 2023-11-24T06:31:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c56ffd640 1 -- 172.21.15.16:0/2024902536 learned_addr learned my addr 172.21.15.16:0/2024902536 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c577fe640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c580cf7d0 0x7f5c580afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c57fff640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c5815e3b0 0x7f5c58158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c56ffd640 1 -- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c580cf7d0 msgr2=0x7f5c580afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c580cf7d0 0x7f5c580afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c56ffd640 1 -- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c5815e3b0 msgr2=0x7f5c58158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:38.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c5815e3b0 0x7f5c58158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c56ffd640 1 -- 172.21.15.16:0/2024902536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009b30 con 0x7f5c58143ae0 2023-11-24T06:31:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c57fff640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c5815e3b0 0x7f5c58158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.914+0000 7f5c577fe640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c580cf7d0 0x7f5c580afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.915+0000 7f5c56ffd640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c58143ae0 0x7f5c58141900 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c400 tx=0x7f5c4000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.915+0000 7f5c4ffff640 1 -- 172.21.15.16:0/2024902536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c400052d0 con 0x7f5c58143ae0 2023-11-24T06:31:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.915+0000 7f5c4ffff640 1 -- 172.21.15.16:0/2024902536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c40005450 con 0x7f5c58143ae0 2023-11-24T06:31:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.915+0000 7f5c4ffff640 1 -- 172.21.15.16:0/2024902536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40007260 con 0x7f5c58143ae0 2023-11-24T06:31:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.915+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c580ce650 con 0x7f5c58143ae0 2023-11-24T06:31:38.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.915+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c580cde50 con 0x7f5c58143ae0 2023-11-24T06:31:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.916+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c580ac940 con 0x7f5c58143ae0 2023-11-24T06:31:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.917+0000 7f5c4ffff640 1 -- 172.21.15.16:0/2024902536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5c400073c0 con 0x7f5c58143ae0 2023-11-24T06:31:38.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.918+0000 7f5c4ffff640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5c3806b3e0 0x7f5c3806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.918+0000 7f5c4ffff640 1 -- 172.21.15.16:0/2024902536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5c400892b0 con 0x7f5c58143ae0 2023-11-24T06:31:38.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.919+0000 7f5c577fe640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5c3806b3e0 0x7f5c3806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:38.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.920+0000 7f5c577fe640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5c3806b3e0 0x7f5c3806d8a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5c48035000 tx=0x7f5c48030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:38.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:38.922+0000 7f5c4ffff640 1 -- 172.21.15.16:0/2024902536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c580ac940 con 0x7f5c58143ae0 2023-11-24T06:31:39.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.145+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol8466", "group_name": "subvol_grp5166", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c580675d0 con 0x7f5c3806b3e0 2023-11-24T06:31:39.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.211+0000 7f5c4ffff640 1 -- 172.21.15.16:0/2024902536 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5c580675d0 con 0x7f5c3806b3e0 2023-11-24T06:31:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5c3806b3e0 msgr2=0x7f5c3806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5c3806b3e0 0x7f5c3806d8a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5c48035000 tx=0x7f5c48030000 comp rx=0 tx=0).stop 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c58143ae0 msgr2=0x7f5c58141900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c58143ae0 0x7f5c58141900 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c400 tx=0x7f5c4000c8c0 comp rx=0 tx=0).stop 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 shutdown_connections 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5c3806b3e0 0x7f5c3806d8a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c5815e3b0 0x7f5c58158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c58143ae0 0x7f5c58141900 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 --2- 172.21.15.16:0/2024902536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c580cf7d0 0x7f5c580afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 >> 172.21.15.16:0/2024902536 conn(0x7f5c5806a800 msgr2=0x7f5c580b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 shutdown_connections 2023-11-24T06:31:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:39.214+0000 7f5c5dc9b640 1 -- 172.21.15.16:0/2024902536 wait complete. 2023-11-24T06:31:43.672 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:31:43.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:31:43.801 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:31:43.932 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:31:43.932 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:31:43.932 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:43.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:44.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.228+0000 7fab9889b640 1 Processor -- start 2023-11-24T06:31:44.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.228+0000 7fab9889b640 1 -- start start 2023-11-24T06:31:44.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.229+0000 7fab9889b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab940b3ae0 0x7fab9415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.229+0000 7fab9889b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab94070c40 con 0x7fab940ae5a0 2023-11-24T06:31:44.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.229+0000 7fab9889b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9406c470 con 0x7fab940cf290 2023-11-24T06:31:44.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.229+0000 7fab9889b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab94071450 con 0x7fab940b3ae0 2023-11-24T06:31:44.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.230+0000 7fab91d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab940b3ae0 0x7fab9415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.230+0000 7fab9259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fab940ae5a0 0x7fab9406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57540/0 (socket says 172.21.15.16:57540) 2023-11-24T06:31:44.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.230+0000 7fab9259c640 1 -- 172.21.15.16:0/2298003480 learned_addr learned my addr 172.21.15.16:0/2298003480 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:44.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.230+0000 7fab91d9b640 1 -- 172.21.15.16:0/2298003480 >> v1:172.21.15.173:6789/0 conn(0x7fab940cf290 legacy=0x7fab940b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:44.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.230+0000 7fab91d9b640 1 -- 172.21.15.16:0/2298003480 >> v1:172.21.15.16:6789/0 conn(0x7fab940ae5a0 legacy=0x7fab9406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.230+0000 7fab91d9b640 1 -- 172.21.15.16:0/2298003480 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9406c750 con 0x7fab940b3ae0 2023-11-24T06:31:44.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.232+0000 7fab91d9b640 1 --2- 172.21.15.16:0/2298003480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 0x7fab9415e430 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a793e953a129e5b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:44.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.232+0000 7fab9159a640 1 -- 172.21.15.16:0/2298003480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c02ccf0 con 0x7fab940b3ae0 2023-11-24T06:31:44.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.232+0000 7fab9159a640 1 -- 172.21.15.16:0/2298003480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c02ce50 con 0x7fab940b3ae0 2023-11-24T06:31:44.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.232+0000 7fab9159a640 1 -- 172.21.15.16:0/2298003480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c034660 con 0x7fab940b3ae0 2023-11-24T06:31:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.233+0000 7fab9889b640 1 -- 172.21.15.16:0/2298003480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 msgr2=0x7fab9415e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.233+0000 7fab9889b640 1 --2- 172.21.15.16:0/2298003480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 0x7fab9415e430 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c02bf40 comp rx=0 tx=0).stop 2023-11-24T06:31:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.233+0000 7fab9889b640 1 -- 172.21.15.16:0/2298003480 shutdown_connections 2023-11-24T06:31:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.233+0000 7fab9889b640 1 --2- 172.21.15.16:0/2298003480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 0x7fab9415e430 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.233+0000 7fab9889b640 1 -- 172.21.15.16:0/2298003480 >> 172.21.15.16:0/2298003480 conn(0x7fab9406a800 msgr2=0x7fab9406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.233+0000 7fab9889b640 1 -- 172.21.15.16:0/2298003480 shutdown_connections 2023-11-24T06:31:44.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.233+0000 7fab9889b640 1 -- 172.21.15.16:0/2298003480 wait complete. 2023-11-24T06:31:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.235+0000 7fab9889b640 1 Processor -- start 2023-11-24T06:31:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.235+0000 7fab9889b640 1 -- start start 2023-11-24T06:31:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab9889b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab940ae5a0 0x7fab94152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab9889b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 0x7fab940b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab9889b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab940cf290 0x7fab941593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab9889b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab941569a0 con 0x7fab940b3ae0 2023-11-24T06:31:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab9889b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab94156c80 con 0x7fab940ae5a0 2023-11-24T06:31:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab9889b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab94156f60 con 0x7fab940cf290 2023-11-24T06:31:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab91d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 0x7fab940b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab91d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 0x7fab940b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40664/0 (socket says 172.21.15.16:40664) 2023-11-24T06:31:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab91d9b640 1 -- 172.21.15.16:0/3005396822 learned_addr learned my addr 172.21.15.16:0/3005396822 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.236+0000 7fab9259c640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab940ae5a0 0x7fab94152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab91d9b640 1 -- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab940cf290 msgr2=0x7fab941593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab92d9d640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab940cf290 0x7fab941593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab91d9b640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab940cf290 0x7fab941593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab91d9b640 1 -- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab940ae5a0 msgr2=0x7fab94152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab91d9b640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab940ae5a0 0x7fab94152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab91d9b640 1 -- 172.21.15.16:0/3005396822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c009a70 con 0x7fab940b3ae0 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab91d9b640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 0x7fab940b0020 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fab7c02dfd0 tx=0x7fab7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab8affd640 1 -- 172.21.15.16:0/3005396822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c03e610 con 0x7fab940b3ae0 2023-11-24T06:31:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab8affd640 1 -- 172.21.15.16:0/3005396822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c03e790 con 0x7fab940b3ae0 2023-11-24T06:31:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab8affd640 1 -- 172.21.15.16:0/3005396822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c03eb40 con 0x7fab940b3ae0 2023-11-24T06:31:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.237+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab94157240 con 0x7fab940b3ae0 2023-11-24T06:31:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.238+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab941f2ce0 con 0x7fab940b3ae0 2023-11-24T06:31:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.239+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab940675d0 con 0x7fab940b3ae0 2023-11-24T06:31:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.240+0000 7fab8affd640 1 -- 172.21.15.16:0/3005396822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fab7c03b050 con 0x7fab940b3ae0 2023-11-24T06:31:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.240+0000 7fab8affd640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab7406b470 0x7fab7406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.241+0000 7fab8affd640 1 -- 172.21.15.16:0/3005396822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5766+0+0 (secure 0 0 0) 0x7fab7c0acb60 con 0x7fab940b3ae0 2023-11-24T06:31:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.241+0000 7fab9259c640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab7406b470 0x7fab7406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.242+0000 7fab9259c640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab7406b470 0x7fab7406d930 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fab84000f60 tx=0x7fab84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:44.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.245+0000 7fab8affd640 1 -- 172.21.15.16:0/3005396822 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab7c07adb0 con 0x7fab940b3ae0 2023-11-24T06:31:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.443+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab9415a150 con 0x7fab940b3ae0 2023-11-24T06:31:44.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.445+0000 7fab8affd640 1 -- 172.21.15.16:0/3005396822 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16654 (secure 0 0 0) 0x7fab7c07a940 con 0x7fab940b3ae0 2023-11-24T06:31:44.448 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:44.448 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":87,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:44.176703+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:31:20.522140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:31:21.257255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.447+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab7406b470 msgr2=0x7fab7406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab7406b470 0x7fab7406d930 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fab84000f60 tx=0x7fab84009040 comp rx=0 tx=0).stop 2023-11-24T06:31:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 msgr2=0x7fab940b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 0x7fab940b0020 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fab7c02dfd0 tx=0x7fab7c004a40 comp rx=0 tx=0).stop 2023-11-24T06:31:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 shutdown_connections 2023-11-24T06:31:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fab7406b470 0x7fab7406d930 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab940cf290 0x7fab941593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab940b3ae0 0x7fab940b0020 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 --2- 172.21.15.16:0/3005396822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab940ae5a0 0x7fab94152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 >> 172.21.15.16:0/3005396822 conn(0x7fab9406a800 msgr2=0x7fab9415aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 shutdown_connections 2023-11-24T06:31:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.448+0000 7fab9889b640 1 -- 172.21.15.16:0/3005396822 wait complete. 2023-11-24T06:31:44.469 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:31:20.522140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:31:21.257255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:31:44.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:31:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.701+0000 7ff244850640 1 Processor -- start 2023-11-24T06:31:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.702+0000 7ff244850640 1 -- start start 2023-11-24T06:31:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.703+0000 7ff244850640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2400cf9a0 0x7ff2400cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.703+0000 7ff244850640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 0x7ff2400b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff244850640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2400d0320 con 0x7ff24015d4b0 2023-11-24T06:31:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff244850640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2400d1c60 con 0x7ff2400cf9a0 2023-11-24T06:31:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff244850640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400b2730 con 0x7ff2400b31d0 2023-11-24T06:31:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff23effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2400cf9a0 0x7ff2400cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff23effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2400cf9a0 0x7ff2400cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40666/0 (socket says 172.21.15.16:40666) 2023-11-24T06:31:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff23effd640 1 -- 172.21.15.16:0/289640406 learned_addr learned my addr 172.21.15.16:0/289640406 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff23ffff640 1 --2- 172.21.15.16:0/289640406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 0x7ff2400b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff23effd640 1 -- 172.21.15.16:0/289640406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 msgr2=0x7ff2400b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff23effd640 1 --2- 172.21.15.16:0/289640406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 0x7ff2400b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.704+0000 7ff23effd640 1 -- 172.21.15.16:0/289640406 >> v1:172.21.15.173:6789/0 conn(0x7ff24015d4b0 legacy=0x7ff2400d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.705+0000 7ff23effd640 1 -- 172.21.15.16:0/289640406 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2400d1ec0 con 0x7ff2400cf9a0 2023-11-24T06:31:44.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.706+0000 7ff23effd640 1 --2- 172.21.15.16:0/289640406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 0x7ff2400cfde0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff228009870 tx=0x7ff22802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6531bf73b933dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:44.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.706+0000 7ff23e7fc640 1 -- 172.21.15.16:0/289640406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228030840 con 0x7ff2400cf9a0 2023-11-24T06:31:44.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.706+0000 7ff23e7fc640 1 -- 172.21.15.16:0/289640406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2280309a0 con 0x7ff2400cf9a0 2023-11-24T06:31:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.706+0000 7ff23e7fc640 1 -- 172.21.15.16:0/289640406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228030d30 con 0x7ff2400cf9a0 2023-11-24T06:31:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.707+0000 7ff244850640 1 -- 172.21.15.16:0/289640406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 msgr2=0x7ff2400cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.707+0000 7ff244850640 1 --2- 172.21.15.16:0/289640406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 0x7ff2400cfde0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff228009870 tx=0x7ff22802b980 comp rx=0 tx=0).stop 2023-11-24T06:31:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.707+0000 7ff244850640 1 -- 172.21.15.16:0/289640406 shutdown_connections 2023-11-24T06:31:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.707+0000 7ff244850640 1 --2- 172.21.15.16:0/289640406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 0x7ff2400b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.707+0000 7ff244850640 1 --2- 172.21.15.16:0/289640406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 0x7ff2400cfde0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.707+0000 7ff244850640 1 -- 172.21.15.16:0/289640406 >> 172.21.15.16:0/289640406 conn(0x7ff2400d3000 msgr2=0x7ff2400d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.707+0000 7ff244850640 1 -- 172.21.15.16:0/289640406 shutdown_connections 2023-11-24T06:31:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.707+0000 7ff244850640 1 -- 172.21.15.16:0/289640406 wait complete. 2023-11-24T06:31:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.709+0000 7ff244850640 1 Processor -- start 2023-11-24T06:31:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.709+0000 7ff244850640 1 -- start start 2023-11-24T06:31:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.710+0000 7ff244850640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 0x7ff240154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.710+0000 7ff244850640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 0x7ff24009f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.710+0000 7ff244850640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff24015d4b0 0x7ff2400ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.710+0000 7ff244850640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24009e2e0 con 0x7ff2400cf9a0 2023-11-24T06:31:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.710+0000 7ff244850640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400a0180 con 0x7ff24015d4b0 2023-11-24T06:31:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.710+0000 7ff244850640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400a0460 con 0x7ff2400b31d0 2023-11-24T06:31:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.710+0000 7ff23effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 0x7ff24009f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.710+0000 7ff23effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 0x7ff24009f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40672/0 (socket says 172.21.15.16:40672) 2023-11-24T06:31:44.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.710+0000 7ff23effd640 1 -- 172.21.15.16:0/1016749861 learned_addr learned my addr 172.21.15.16:0/1016749861 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff23f7fe640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 0x7ff240154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff23effd640 1 -- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 msgr2=0x7ff240154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff23effd640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 0x7ff240154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff23ffff640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff24015d4b0 0x7ff2400ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff23effd640 1 -- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff24015d4b0 msgr2=0x7ff2400ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff23effd640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff24015d4b0 0x7ff2400ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff23effd640 1 -- 172.21.15.16:0/1016749861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2280094d0 con 0x7ff2400cf9a0 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff23ffff640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff24015d4b0 0x7ff2400ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff23effd640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 0x7ff24009f6a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff228000c00 tx=0x7ff228004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff237fff640 1 -- 172.21.15.16:0/1016749861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228039a90 con 0x7ff2400cf9a0 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff237fff640 1 -- 172.21.15.16:0/1016749861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff228039bf0 con 0x7ff2400cf9a0 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff237fff640 1 -- 172.21.15.16:0/1016749861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228043410 con 0x7ff2400cf9a0 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.711+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2400a0740 con 0x7ff2400cf9a0 2023-11-24T06:31:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.712+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2401548c0 con 0x7ff2400cf9a0 2023-11-24T06:31:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.713+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2400b7000 con 0x7ff2400cf9a0 2023-11-24T06:31:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.714+0000 7ff237fff640 1 -- 172.21.15.16:0/1016749861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff22802f390 con 0x7ff2400cf9a0 2023-11-24T06:31:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.715+0000 7ff237fff640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff22006b470 0x7ff22006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.715+0000 7ff237fff640 1 -- 172.21.15.16:0/1016749861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5766+0+0 (secure 0 0 0) 0x7ff2280ac480 con 0x7ff2400cf9a0 2023-11-24T06:31:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.716+0000 7ff23f7fe640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff22006b470 0x7ff22006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.717+0000 7ff23f7fe640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff22006b470 0x7ff22006d930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff230007fd0 tx=0x7ff230009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:44.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.719+0000 7ff237fff640 1 -- 172.21.15.16:0/1016749861 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2400b7000 con 0x7ff2400cf9a0 2023-11-24T06:31:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.916+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2400b7000 con 0x7ff2400cf9a0 2023-11-24T06:31:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.917+0000 7ff237fff640 1 -- 172.21.15.16:0/1016749861 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2400b7000 con 0x7ff2400cf9a0 2023-11-24T06:31:44.920 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:31:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.920+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff22006b470 msgr2=0x7ff22006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.920+0000 7ff244850640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff22006b470 0x7ff22006d930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff230007fd0 tx=0x7ff230009040 comp rx=0 tx=0).stop 2023-11-24T06:31:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.920+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 msgr2=0x7ff24009f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.920+0000 7ff244850640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 0x7ff24009f6a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff228000c00 tx=0x7ff228004730 comp rx=0 tx=0).stop 2023-11-24T06:31:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.920+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 shutdown_connections 2023-11-24T06:31:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.920+0000 7ff244850640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff22006b470 0x7ff22006d930 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.920+0000 7ff244850640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff24015d4b0 0x7ff2400ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.920+0000 7ff244850640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2400cf9a0 0x7ff24009f6a0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.921+0000 7ff244850640 1 --2- 172.21.15.16:0/1016749861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2400b31d0 0x7ff240154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.921+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 >> 172.21.15.16:0/1016749861 conn(0x7ff2400d3000 msgr2=0x7ff2400b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.921+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 shutdown_connections 2023-11-24T06:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:44.921+0000 7ff244850640 1 -- 172.21.15.16:0/1016749861 wait complete. 2023-11-24T06:31:44.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:31:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.162+0000 7f702e267640 1 Processor -- start 2023-11-24T06:31:45.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.163+0000 7f702e267640 1 -- start start 2023-11-24T06:31:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.164+0000 7f702e267640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f70280adf20 0x7f7028070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.165+0000 7f702e267640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f702815bf10 0x7f70280cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.165+0000 7f702e267640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702815f4e0 con 0x7f70280cc980 2023-11-24T06:31:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.165+0000 7f702e267640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f702815f970 con 0x7f70280adf20 2023-11-24T06:31:45.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.165+0000 7f702e267640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70280cce10 con 0x7f702815bf10 2023-11-24T06:31:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.165+0000 7f702ca64640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f702815bf10 0x7f70280cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.165+0000 7f702d265640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f70280adf20 0x7f7028070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.165+0000 7f702da66640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f70280cc980 0x7f7028143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.16:36152/0 (socket says 172.21.15.16:36152) 2023-11-24T06:31:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.165+0000 7f702da66640 1 -- 172.21.15.16:0/1846623890 learned_addr learned my addr 172.21.15.16:0/1846623890 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:45.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.166+0000 7f702d265640 1 -- 172.21.15.16:0/1846623890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f702815bf10 msgr2=0x7f70280cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.166+0000 7f702d265640 1 --2- 172.21.15.16:0/1846623890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f702815bf10 0x7f70280cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.166+0000 7f702d265640 1 -- 172.21.15.16:0/1846623890 >> v1:172.21.15.173:6789/0 conn(0x7f70280cc980 legacy=0x7f7028143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.166+0000 7f702d265640 1 -- 172.21.15.16:0/1846623890 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028071130 con 0x7f70280adf20 2023-11-24T06:31:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.168+0000 7f702d265640 1 --2- 172.21.15.16:0/1846623890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70280adf20 0x7f7028070a30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7024009d90 tx=0x7f702402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a982c892ec7a997 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.168+0000 7f701ffff640 1 -- 172.21.15.16:0/1846623890 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702402cbc0 con 0x7f70280adf20 2023-11-24T06:31:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.168+0000 7f701ffff640 1 -- 172.21.15.16:0/1846623890 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702402cd20 con 0x7f70280adf20 2023-11-24T06:31:45.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.168+0000 7f701ffff640 1 -- 172.21.15.16:0/1846623890 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024034660 con 0x7f70280adf20 2023-11-24T06:31:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.169+0000 7f702e267640 1 -- 172.21.15.16:0/1846623890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70280adf20 msgr2=0x7f7028070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.169+0000 7f702e267640 1 --2- 172.21.15.16:0/1846623890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70280adf20 0x7f7028070a30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7024009d90 tx=0x7f702402be10 comp rx=0 tx=0).stop 2023-11-24T06:31:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.169+0000 7f702e267640 1 -- 172.21.15.16:0/1846623890 shutdown_connections 2023-11-24T06:31:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.169+0000 7f702e267640 1 --2- 172.21.15.16:0/1846623890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f702815bf10 0x7f70280cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.169+0000 7f702e267640 1 --2- 172.21.15.16:0/1846623890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70280adf20 0x7f7028070a30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.169+0000 7f702e267640 1 -- 172.21.15.16:0/1846623890 >> 172.21.15.16:0/1846623890 conn(0x7f702806a7d0 msgr2=0x7f702806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:45.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.169+0000 7f702e267640 1 -- 172.21.15.16:0/1846623890 shutdown_connections 2023-11-24T06:31:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.169+0000 7f702e267640 1 -- 172.21.15.16:0/1846623890 wait complete. 2023-11-24T06:31:45.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.172+0000 7f702e267640 1 Processor -- start 2023-11-24T06:31:45.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.172+0000 7f702e267640 1 -- start start 2023-11-24T06:31:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.173+0000 7f702e267640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70280adf20 0x7f7028143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.173+0000 7f702e267640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70280cc980 0x7f702815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.173+0000 7f702e267640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702815bf10 0x7f70281e5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.173+0000 7f702e267640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70281421a0 con 0x7f702815bf10 2023-11-24T06:31:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.173+0000 7f702e267640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7028142480 con 0x7f70280cc980 2023-11-24T06:31:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.173+0000 7f702e267640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7028142760 con 0x7f70280adf20 2023-11-24T06:31:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.173+0000 7f702da66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702815bf10 0x7f70281e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.173+0000 7f702da66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702815bf10 0x7f70281e5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40686/0 (socket says 172.21.15.16:40686) 2023-11-24T06:31:45.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702da66640 1 -- 172.21.15.16:0/4005668934 learned_addr learned my addr 172.21.15.16:0/4005668934 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702ca64640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70280cc980 0x7f702815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702d265640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70280adf20 0x7f7028143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702da66640 1 -- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70280adf20 msgr2=0x7f7028143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702da66640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70280adf20 0x7f7028143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702da66640 1 -- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70280cc980 msgr2=0x7f702815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702da66640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70280cc980 0x7f702815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702da66640 1 -- 172.21.15.16:0/4005668934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7024009a70 con 0x7f702815bf10 2023-11-24T06:31:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702d265640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70280adf20 0x7f7028143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.174+0000 7f702da66640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702815bf10 0x7f70281e5240 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f702000c420 tx=0x7f702000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.175+0000 7f701dffb640 1 -- 172.21.15.16:0/4005668934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020005ce0 con 0x7f702815bf10 2023-11-24T06:31:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.175+0000 7f701dffb640 1 -- 172.21.15.16:0/4005668934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7020005e40 con 0x7f702815bf10 2023-11-24T06:31:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.175+0000 7f701dffb640 1 -- 172.21.15.16:0/4005668934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70200072d0 con 0x7f702815bf10 2023-11-24T06:31:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.175+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7028142a40 con 0x7f702815bf10 2023-11-24T06:31:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.175+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702815ae50 con 0x7f702815bf10 2023-11-24T06:31:45.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.176+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7028142f60 con 0x7f702815bf10 2023-11-24T06:31:45.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.177+0000 7f701dffb640 1 -- 172.21.15.16:0/4005668934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f7020007430 con 0x7f702815bf10 2023-11-24T06:31:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.178+0000 7f701dffb640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f700c06b4a0 0x7f700c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.179+0000 7f701dffb640 1 -- 172.21.15.16:0/4005668934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f7020089600 con 0x7f702815bf10 2023-11-24T06:31:45.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.179+0000 7f702d265640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f700c06b4a0 0x7f700c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.181+0000 7f702d265640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f700c06b4a0 0x7f700c06d960 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7024036000 tx=0x7f7024030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.183+0000 7f701dffb640 1 -- 172.21.15.16:0/4005668934 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7020053da0 con 0x7f702815bf10 2023-11-24T06:31:45.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.387+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7028067450 con 0x7f702815bf10 2023-11-24T06:31:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.388+0000 7f701dffb640 1 -- 172.21.15.16:0/4005668934 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70200573c0 con 0x7f702815bf10 2023-11-24T06:31:45.391 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:31:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f700c06b4a0 msgr2=0x7f700c06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f700c06b4a0 0x7f700c06d960 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7024036000 tx=0x7f7024030000 comp rx=0 tx=0).stop 2023-11-24T06:31:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702815bf10 msgr2=0x7f70281e5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702815bf10 0x7f70281e5240 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f702000c420 tx=0x7f702000c8e0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 shutdown_connections 2023-11-24T06:31:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f700c06b4a0 0x7f700c06d960 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f702815bf10 0x7f70281e5240 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f70280cc980 0x7f702815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 --2- 172.21.15.16:0/4005668934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f70280adf20 0x7f7028143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 >> 172.21.15.16:0/4005668934 conn(0x7f702806a7d0 msgr2=0x7f702815b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 shutdown_connections 2023-11-24T06:31:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.391+0000 7f702e267640 1 -- 172.21.15.16:0/4005668934 wait complete. 2023-11-24T06:31:45.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:31:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.636+0000 7fc2c88c8640 1 Processor -- start 2023-11-24T06:31:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.636+0000 7fc2c88c8640 1 -- start start 2023-11-24T06:31:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.638+0000 7fc2c88c8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c41505b0 0x7fc2c40b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.638+0000 7fc2c88c8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2c415a690 0x7fc2c406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.638+0000 7fc2c88c8640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c406d930 con 0x7fc2c406d500 2023-11-24T06:31:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.638+0000 7fc2c88c8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c40b1e30 con 0x7fc2c415a690 2023-11-24T06:31:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.638+0000 7fc2c88c8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40cdc10 con 0x7fc2c41505b0 2023-11-24T06:31:45.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.639+0000 7fc2bbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2c415a690 0x7fc2c406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.639+0000 7fc2bbfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2c415a690 0x7fc2c406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40698/0 (socket says 172.21.15.16:40698) 2023-11-24T06:31:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.639+0000 7fc2bbfff640 1 -- 172.21.15.16:0/4027665897 learned_addr learned my addr 172.21.15.16:0/4027665897 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.639+0000 7fc2c2d9d640 1 --2- 172.21.15.16:0/4027665897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c41505b0 0x7fc2c40b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.639+0000 7fc2bbfff640 1 -- 172.21.15.16:0/4027665897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c41505b0 msgr2=0x7fc2c40b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.639+0000 7fc2bbfff640 1 --2- 172.21.15.16:0/4027665897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c41505b0 0x7fc2c40b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.639+0000 7fc2bbfff640 1 -- 172.21.15.16:0/4027665897 >> v1:172.21.15.173:6789/0 conn(0x7fc2c406d500 legacy=0x7fc2c415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.639+0000 7fc2bbfff640 1 -- 172.21.15.16:0/4027665897 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c40b20a0 con 0x7fc2c415a690 2023-11-24T06:31:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.640+0000 7fc2bbfff640 1 --2- 172.21.15.16:0/4027665897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c415a690 0x7fc2c406cfc0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc2c406dbd0 tx=0x7fc2ac02de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f9c7742944b0dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.641+0000 7fc2c259c640 1 -- 172.21.15.16:0/4027665897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac02ec10 con 0x7fc2c415a690 2023-11-24T06:31:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.641+0000 7fc2c259c640 1 -- 172.21.15.16:0/4027665897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac032070 con 0x7fc2c415a690 2023-11-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.641+0000 7fc2c259c640 1 -- 172.21.15.16:0/4027665897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac038400 con 0x7fc2c415a690 2023-11-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.641+0000 7fc2c88c8640 1 -- 172.21.15.16:0/4027665897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c415a690 msgr2=0x7fc2c406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.641+0000 7fc2c88c8640 1 --2- 172.21.15.16:0/4027665897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c415a690 0x7fc2c406cfc0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc2c406dbd0 tx=0x7fc2ac02de60 comp rx=0 tx=0).stop 2023-11-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.642+0000 7fc2c88c8640 1 -- 172.21.15.16:0/4027665897 shutdown_connections 2023-11-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.642+0000 7fc2c88c8640 1 --2- 172.21.15.16:0/4027665897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c415a690 0x7fc2c406cfc0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.642+0000 7fc2c88c8640 1 --2- 172.21.15.16:0/4027665897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c41505b0 0x7fc2c40b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.642+0000 7fc2c88c8640 1 -- 172.21.15.16:0/4027665897 >> 172.21.15.16:0/4027665897 conn(0x7fc2c406a7d0 msgr2=0x7fc2c406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.642+0000 7fc2c88c8640 1 -- 172.21.15.16:0/4027665897 shutdown_connections 2023-11-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.642+0000 7fc2c88c8640 1 -- 172.21.15.16:0/4027665897 wait complete. 2023-11-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.643+0000 7fc2c88c8640 1 Processor -- start 2023-11-24T06:31:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.644+0000 7fc2c88c8640 1 -- start start 2023-11-24T06:31:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.645+0000 7fc2c88c8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2c406d500 0x7fc2c414cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.645+0000 7fc2c88c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c41505b0 0x7fc2c4151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.645+0000 7fc2c88c8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c415a690 0x7fc2c41589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.645+0000 7fc2c88c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41e50d0 con 0x7fc2c41505b0 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.645+0000 7fc2c88c8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c414d610 con 0x7fc2c406d500 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.645+0000 7fc2c88c8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c414d8f0 con 0x7fc2c415a690 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.645+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c41505b0 0x7fc2c4151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.645+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c41505b0 0x7fc2c4151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40712/0 (socket says 172.21.15.16:40712) 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.645+0000 7fc2bbfff640 1 -- 172.21.15.16:0/2799152360 learned_addr learned my addr 172.21.15.16:0/2799152360 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.646+0000 7fc2c2d9d640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2c406d500 0x7fc2c414cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.646+0000 7fc2c359e640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c415a690 0x7fc2c41589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.646+0000 7fc2bbfff640 1 -- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c415a690 msgr2=0x7fc2c41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.646+0000 7fc2bbfff640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c415a690 0x7fc2c41589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.646+0000 7fc2bbfff640 1 -- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2c406d500 msgr2=0x7fc2c414cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.646+0000 7fc2bbfff640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2c406d500 0x7fc2c414cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.646+0000 7fc2bbfff640 1 -- 172.21.15.16:0/2799152360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac00b760 con 0x7fc2c41505b0 2023-11-24T06:31:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.646+0000 7fc2c2d9d640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2c406d500 0x7fc2c414cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:31:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.646+0000 7fc2bbfff640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c41505b0 0x7fc2c4151230 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac000c00 tx=0x7fc2ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.647+0000 7fc2bb7fe640 1 -- 172.21.15.16:0/2799152360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac006dd0 con 0x7fc2c41505b0 2023-11-24T06:31:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.647+0000 7fc2bb7fe640 1 -- 172.21.15.16:0/2799152360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac038d60 con 0x7fc2c41505b0 2023-11-24T06:31:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.647+0000 7fc2bb7fe640 1 -- 172.21.15.16:0/2799152360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac042610 con 0x7fc2c41505b0 2023-11-24T06:31:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.647+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c414dbd0 con 0x7fc2c41505b0 2023-11-24T06:31:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.647+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c414d300 con 0x7fc2c41505b0 2023-11-24T06:31:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.648+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c41e5560 con 0x7fc2c41505b0 2023-11-24T06:31:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.649+0000 7fc2bb7fe640 1 -- 172.21.15.16:0/2799152360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fc2ac042770 con 0x7fc2c41505b0 2023-11-24T06:31:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.650+0000 7fc2bb7fe640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc2a406b470 0x7fc2a406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.650+0000 7fc2bb7fe640 1 -- 172.21.15.16:0/2799152360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7fc2ac0ac680 con 0x7fc2c41505b0 2023-11-24T06:31:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.650+0000 7fc2c2d9d640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc2a406b470 0x7fc2a406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.651+0000 7fc2c2d9d640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc2a406b470 0x7fc2a406d930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc2c41f6c70 tx=0x7fc2b4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.654+0000 7fc2bb7fe640 1 -- 172.21.15.16:0/2799152360 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2c41e5560 con 0x7fc2c41505b0 2023-11-24T06:31:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.851+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2c41e5560 con 0x7fc2c41505b0 2023-11-24T06:31:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.851+0000 7fc2bb7fe640 1 -- 172.21.15.16:0/2799152360 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2c41e5560 con 0x7fc2c41505b0 2023-11-24T06:31:45.854 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:31:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc2a406b470 msgr2=0x7fc2a406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc2a406b470 0x7fc2a406d930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc2c41f6c70 tx=0x7fc2b4008040 comp rx=0 tx=0).stop 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c41505b0 msgr2=0x7fc2c4151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c41505b0 0x7fc2c4151230 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac000c00 tx=0x7fc2ac0046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 shutdown_connections 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc2a406b470 0x7fc2a406d930 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2c415a690 0x7fc2c41589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2c41505b0 0x7fc2c4151230 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 --2- 172.21.15.16:0/2799152360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2c406d500 0x7fc2c414cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 >> 172.21.15.16:0/2799152360 conn(0x7fc2c406a7d0 msgr2=0x7fc2c406abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 shutdown_connections 2023-11-24T06:31:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:45.854+0000 7fc2c88c8640 1 -- 172.21.15.16:0/2799152360 wait complete. 2023-11-24T06:31:45.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:31:46.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.092+0000 7fba0e573640 1 Processor -- start 2023-11-24T06:31:46.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.092+0000 7fba0e573640 1 -- start start 2023-11-24T06:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.093+0000 7fba0e573640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155540 0x7fba080aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.094+0000 7fba0e573640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba08071b90 con 0x7fba0815b470 2023-11-24T06:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.094+0000 7fba0e573640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba080af1e0 con 0x7fba08071290 2023-11-24T06:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.094+0000 7fba0e573640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba08155920 con 0x7fba08155540 2023-11-24T06:31:46.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.094+0000 7fba0cd70640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fba0815b470 0x7fba08070b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57544/0 (socket says 172.21.15.16:57544) 2023-11-24T06:31:46.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.094+0000 7fba0cd70640 1 -- 172.21.15.16:0/1666667974 learned_addr learned my addr 172.21.15.16:0/1666667974 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:46.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.094+0000 7fba0d571640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155540 0x7fba080aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.094+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963781113 0 0) 0x7fba08071b90 con 0x7fba0815b470 2023-11-24T06:31:46.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9f4003520 con 0x7fba0815b470 2023-11-24T06:31:46.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2185555243 0 0) 0x7fb9f4003520 con 0x7fba0815b470 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155540 msgr2=0x7fba080aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155540 0x7fba080aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 >> v1:172.21.15.173:6789/0 conn(0x7fba08071290 legacy=0x7fba080b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0815b9e0 con 0x7fba0815b470 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fb9f8001b70 con 0x7fba0815b470 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 >> v1:172.21.15.16:6789/0 conn(0x7fba0815b470 legacy=0x7fba08070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f40039a0 0x7fb9f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f40047c0 0x7fb9f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155540 0x7fb9f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba08155920 con 0x7fb9f40047c0 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f400bab0 con 0x7fb9f40039a0 2023-11-24T06:31:46.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f400bc70 con 0x7fba08155540 2023-11-24T06:31:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.095+0000 7fba0cd70640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f40047c0 0x7fb9f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.096+0000 7fba0d571640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f40039a0 0x7fb9f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.096+0000 7fba0dd72640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155540 0x7fb9f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.096+0000 7fba0cd70640 1 -- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155540 msgr2=0x7fb9f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.096+0000 7fba0cd70640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155540 0x7fb9f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.096+0000 7fba0cd70640 1 -- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f40039a0 msgr2=0x7fb9f4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.096+0000 7fba0cd70640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f40039a0 0x7fb9f4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.096+0000 7fba0cd70640 1 -- 172.21.15.16:0/1666667974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f4003520 con 0x7fb9f40047c0 2023-11-24T06:31:46.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.096+0000 7fba0dd72640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155540 0x7fb9f40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:46.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.097+0000 7fba0cd70640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f40047c0 0x7fb9f4004c40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c520 tx=0x7fb9f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b860760ca9a095 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:46.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.097+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f802ce40 con 0x7fb9f40047c0 2023-11-24T06:31:46.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.097+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8005b20 con 0x7fb9f40047c0 2023-11-24T06:31:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.097+0000 7fb9fffff640 1 -- 172.21.15.16:0/1666667974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8007070 con 0x7fb9f40047c0 2023-11-24T06:31:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.098+0000 7fba0e573640 1 -- 172.21.15.16:0/1666667974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f40047c0 msgr2=0x7fb9f4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.098+0000 7fba0e573640 1 --2- 172.21.15.16:0/1666667974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f40047c0 0x7fb9f4004c40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c520 tx=0x7fb9f40057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.098+0000 7fba0dd72640 1 -- 172.21.15.16:0/1666667974 reap_dead start 2023-11-24T06:31:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.098+0000 7fba0e573640 1 -- 172.21.15.16:0/1666667974 shutdown_connections 2023-11-24T06:31:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.098+0000 7fba0e573640 1 -- 172.21.15.16:0/1666667974 >> 172.21.15.16:0/1666667974 conn(0x7fba080cc1d0 msgr2=0x7fba080cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.098+0000 7fba0e573640 1 -- 172.21.15.16:0/1666667974 shutdown_connections 2023-11-24T06:31:46.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.098+0000 7fba0e573640 1 -- 172.21.15.16:0/1666667974 wait complete. 2023-11-24T06:31:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.100+0000 7fba0e573640 1 Processor -- start 2023-11-24T06:31:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.100+0000 7fba0e573640 1 -- start start 2023-11-24T06:31:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0e573640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f40047c0 0x7fba08142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0e573640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0815acd0 0x7fba081584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0e573640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155ac0 0x7fba0806d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0e573640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba08158060 con 0x7fba0815acd0 2023-11-24T06:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0e573640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0815b1c0 con 0x7fb9f40047c0 2023-11-24T06:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0e573640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba080cee60 con 0x7fba08155ac0 2023-11-24T06:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0cd70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0815acd0 0x7fba081584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0cd70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0815acd0 0x7fba081584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40732/0 (socket says 172.21.15.16:40732) 2023-11-24T06:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0cd70640 1 -- 172.21.15.16:0/2094231684 learned_addr learned my addr 172.21.15.16:0/2094231684 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:46.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0dd72640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155ac0 0x7fba0806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.101+0000 7fba0cd70640 1 -- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155ac0 msgr2=0x7fba0806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fba0d571640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f40047c0 0x7fba08142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fba0cd70640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155ac0 0x7fba0806d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fba0cd70640 1 -- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f40047c0 msgr2=0x7fba08142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fba0cd70640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f40047c0 0x7fba08142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fba0cd70640 1 -- 172.21.15.16:0/2094231684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0814f630 con 0x7fba0815acd0 2023-11-24T06:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fba0cd70640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0815acd0 0x7fba081584f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb9f802c7e0 tx=0x7fb9f8001f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fb9fdffb640 1 -- 172.21.15.16:0/2094231684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8007480 con 0x7fba0815acd0 2023-11-24T06:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fb9fdffb640 1 -- 172.21.15.16:0/2094231684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f80075e0 con 0x7fba0815acd0 2023-11-24T06:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba080ce8a0 con 0x7fba0815acd0 2023-11-24T06:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fb9fdffb640 1 -- 172.21.15.16:0/2094231684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8007950 con 0x7fba0815acd0 2023-11-24T06:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.102+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba08155f40 con 0x7fba0815acd0 2023-11-24T06:31:46.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.104+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba080cefd0 con 0x7fba0815acd0 2023-11-24T06:31:46.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.105+0000 7fb9fdffb640 1 -- 172.21.15.16:0/2094231684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fb9f8032070 con 0x7fba0815acd0 2023-11-24T06:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.105+0000 7fb9fdffb640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9ec06b470 0x7fb9ec06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.106+0000 7fb9fdffb640 1 -- 172.21.15.16:0/2094231684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7fb9f80ab8b0 con 0x7fba0815acd0 2023-11-24T06:31:46.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.106+0000 7fba0d571640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9ec06b470 0x7fb9ec06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.107+0000 7fba0d571640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9ec06b470 0x7fb9ec06d930 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fba04009950 tx=0x7fba04008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:46.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.110+0000 7fb9fdffb640 1 -- 172.21.15.16:0/2094231684 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9f80760e0 con 0x7fba0815acd0 2023-11-24T06:31:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.322+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fba08067b60 con 0x7fba0815acd0 2023-11-24T06:31:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.323+0000 7fb9fdffb640 1 -- 172.21.15.16:0/2094231684 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb9f8045080 con 0x7fba0815acd0 2023-11-24T06:31:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:31:46.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.325+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9ec06b470 msgr2=0x7fb9ec06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.325+0000 7fba0e573640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9ec06b470 0x7fb9ec06d930 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fba04009950 tx=0x7fba04008040 comp rx=0 tx=0).stop 2023-11-24T06:31:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.325+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0815acd0 msgr2=0x7fba081584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.325+0000 7fba0e573640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0815acd0 0x7fba081584f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb9f802c7e0 tx=0x7fb9f8001f00 comp rx=0 tx=0).stop 2023-11-24T06:31:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.326+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 shutdown_connections 2023-11-24T06:31:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.326+0000 7fba0e573640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9ec06b470 0x7fb9ec06d930 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.326+0000 7fba0e573640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba08155ac0 0x7fba0806d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.326+0000 7fba0e573640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0815acd0 0x7fba081584f0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.326+0000 7fba0e573640 1 --2- 172.21.15.16:0/2094231684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f40047c0 0x7fba08142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.326+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 >> 172.21.15.16:0/2094231684 conn(0x7fba080cc1d0 msgr2=0x7fba080b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.326+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 shutdown_connections 2023-11-24T06:31:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.326+0000 7fba0e573640 1 -- 172.21.15.16:0/2094231684 wait complete. 2023-11-24T06:31:46.346 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-11-24T06:31:46.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:46.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:31:46.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[114892]: fuse finished with error 0 and tester_r 0 2023-11-24T06:31:46.447 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:31:46.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:46.447 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:31:46.503 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:31:46.503 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:46.564 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:46.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.797+0000 7f54cacf8640 1 Processor -- start 2023-11-24T06:31:46.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.797+0000 7f54cacf8640 1 -- start start 2023-11-24T06:31:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.799+0000 7f54cacf8640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f54c40cc500 0x7f54c40cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.799+0000 7f54cacf8640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54c406d160 0x7f54c406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.799+0000 7f54cacf8640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c4070e50 con 0x7f54c4143c40 2023-11-24T06:31:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.799+0000 7f54cacf8640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c40723c0 con 0x7f54c406d160 2023-11-24T06:31:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.799+0000 7f54cacf8640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c40cced0 con 0x7f54c40cc500 2023-11-24T06:31:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.800+0000 7f54ca4f7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54c406d160 0x7f54c406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.800+0000 7f54ca4f7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f54c406d160 0x7f54c406d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40734/0 (socket says 172.21.15.16:40734) 2023-11-24T06:31:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.800+0000 7f54ca4f7640 1 -- 172.21.15.16:0/3605181996 learned_addr learned my addr 172.21.15.16:0/3605181996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.800+0000 7f54ca4f7640 1 -- 172.21.15.16:0/3605181996 >> v2:172.21.15.173:3300/0 conn(0x7f54c40cc500 msgr2=0x7f54c40cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.800+0000 7f54ca4f7640 1 --2- 172.21.15.16:0/3605181996 >> v2:172.21.15.173:3300/0 conn(0x7f54c40cc500 0x7f54c40cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.800+0000 7f54ca4f7640 1 -- 172.21.15.16:0/3605181996 >> v1:172.21.15.173:6789/0 conn(0x7f54c4143c40 legacy=0x7f54c4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.800+0000 7f54ca4f7640 1 -- 172.21.15.16:0/3605181996 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c4144020 con 0x7f54c406d160 2023-11-24T06:31:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.801+0000 7f54ca4f7640 1 --2- 172.21.15.16:0/3605181996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c406d160 0x7f54c406d560 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f54c40b44a0 tx=0x7f54bc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6142e6cf597450d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.802+0000 7f54c8cf4640 1 -- 172.21.15.16:0/3605181996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc02c9f0 con 0x7f54c406d160 2023-11-24T06:31:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.802+0000 7f54c8cf4640 1 -- 172.21.15.16:0/3605181996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc02cb50 con 0x7f54c406d160 2023-11-24T06:31:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.802+0000 7f54c8cf4640 1 -- 172.21.15.16:0/3605181996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc0343f0 con 0x7f54c406d160 2023-11-24T06:31:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.802+0000 7f54cacf8640 1 -- 172.21.15.16:0/3605181996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c406d160 msgr2=0x7f54c406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.802+0000 7f54cacf8640 1 --2- 172.21.15.16:0/3605181996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c406d160 0x7f54c406d560 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f54c40b44a0 tx=0x7f54bc02be50 comp rx=0 tx=0).stop 2023-11-24T06:31:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.803+0000 7f54cacf8640 1 -- 172.21.15.16:0/3605181996 shutdown_connections 2023-11-24T06:31:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.803+0000 7f54cacf8640 1 --2- 172.21.15.16:0/3605181996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c406d160 0x7f54c406d560 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.803+0000 7f54cacf8640 1 --2- 172.21.15.16:0/3605181996 >> v2:172.21.15.173:3300/0 conn(0x7f54c40cc500 0x7f54c40cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.803+0000 7f54cacf8640 1 -- 172.21.15.16:0/3605181996 >> 172.21.15.16:0/3605181996 conn(0x7f54c406a800 msgr2=0x7f54c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.803+0000 7f54cacf8640 1 -- 172.21.15.16:0/3605181996 shutdown_connections 2023-11-24T06:31:46.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.803+0000 7f54cacf8640 1 -- 172.21.15.16:0/3605181996 wait complete. 2023-11-24T06:31:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.805+0000 7f54cacf8640 1 Processor -- start 2023-11-24T06:31:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.805+0000 7f54cacf8640 1 -- start start 2023-11-24T06:31:46.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.806+0000 7f54cacf8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54c406d160 0x7f54c40b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.806+0000 7f54cacf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c40cc500 0x7f54c40c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.806+0000 7f54cacf8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54c4143c40 0x7f54c415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.806+0000 7f54cacf8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40b2b70 con 0x7f54c40cc500 2023-11-24T06:31:46.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.806+0000 7f54cacf8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40b1fa0 con 0x7f54c406d160 2023-11-24T06:31:46.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.806+0000 7f54cacf8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40b2240 con 0x7f54c4143c40 2023-11-24T06:31:46.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.806+0000 7f54c94f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c40cc500 0x7f54c40c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.806+0000 7f54c94f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c40cc500 0x7f54c40c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40742/0 (socket says 172.21.15.16:40742) 2023-11-24T06:31:46.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.806+0000 7f54c94f5640 1 -- 172.21.15.16:0/2990364604 learned_addr learned my addr 172.21.15.16:0/2990364604 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.807+0000 7f54c9cf6640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54c406d160 0x7f54c40b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.807+0000 7f54ca4f7640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54c4143c40 0x7f54c415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.807+0000 7f54c94f5640 1 -- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54c4143c40 msgr2=0x7f54c415a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.807+0000 7f54c94f5640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54c4143c40 0x7f54c415a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.807+0000 7f54c94f5640 1 -- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54c406d160 msgr2=0x7f54c40b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.807+0000 7f54c94f5640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54c406d160 0x7f54c40b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.807+0000 7f54c94f5640 1 -- 172.21.15.16:0/2990364604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc009b30 con 0x7f54c40cc500 2023-11-24T06:31:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.807+0000 7f54c94f5640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c40cc500 0x7f54c40c3d70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f54b400b4a0 tx=0x7f54b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.807+0000 7f54ba7fc640 1 -- 172.21.15.16:0/2990364604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4010610 con 0x7f54c40cc500 2023-11-24T06:31:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.808+0000 7f54ba7fc640 1 -- 172.21.15.16:0/2990364604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b4010770 con 0x7f54c40cc500 2023-11-24T06:31:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.808+0000 7f54ba7fc640 1 -- 172.21.15.16:0/2990364604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4010b20 con 0x7f54c40cc500 2023-11-24T06:31:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.808+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c40b2520 con 0x7f54c40cc500 2023-11-24T06:31:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.808+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c40c42b0 con 0x7f54c40cc500 2023-11-24T06:31:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.809+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c40cc1e0 con 0x7f54c40cc500 2023-11-24T06:31:46.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.810+0000 7f54ba7fc640 1 -- 172.21.15.16:0/2990364604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f54b4010c80 con 0x7f54c40cc500 2023-11-24T06:31:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.811+0000 7f54ba7fc640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54a806b470 0x7f54a806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.811+0000 7f54ba7fc640 1 -- 172.21.15.16:0/2990364604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f54b4089e60 con 0x7f54c40cc500 2023-11-24T06:31:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.812+0000 7f54c9cf6640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54a806b470 0x7f54a806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:46.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.814+0000 7f54c9cf6640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54a806b470 0x7f54a806d930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f54c0006fd0 tx=0x7f54c0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:46.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:46.817+0000 7f54ba7fc640 1 -- 172.21.15.16:0/2990364604 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54b4054550 con 0x7f54c40cc500 2023-11-24T06:31:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.016+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54c4067450 con 0x7f54c40cc500 2023-11-24T06:31:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.019+0000 7f54ba7fc640 1 -- 172.21.15.16:0/2990364604 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16654 (secure 0 0 0) 0x7f54b401e090 con 0x7f54c40cc500 2023-11-24T06:31:47.021 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:47.021 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:45.189909+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:31:20.522140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:31:21.257255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.021+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54a806b470 msgr2=0x7f54a806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.021+0000 7f54cacf8640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54a806b470 0x7f54a806d930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f54c0006fd0 tx=0x7f54c0008040 comp rx=0 tx=0).stop 2023-11-24T06:31:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.021+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c40cc500 msgr2=0x7f54c40c3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.021+0000 7f54cacf8640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c40cc500 0x7f54c40c3d70 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f54b400b4a0 tx=0x7f54b400b960 comp rx=0 tx=0).stop 2023-11-24T06:31:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.022+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 shutdown_connections 2023-11-24T06:31:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.022+0000 7f54cacf8640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f54a806b470 0x7f54a806d930 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.022+0000 7f54cacf8640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54c4143c40 0x7f54c415a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.022+0000 7f54cacf8640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54c40cc500 0x7f54c40c3d70 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.022+0000 7f54cacf8640 1 --2- 172.21.15.16:0/2990364604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54c406d160 0x7f54c40b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.022+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 >> 172.21.15.16:0/2990364604 conn(0x7f54c406a800 msgr2=0x7f54c406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.022+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 shutdown_connections 2023-11-24T06:31:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.022+0000 7f54cacf8640 1 -- 172.21.15.16:0/2990364604 wait complete. 2023-11-24T06:31:47.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:31:20.522140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:31:21.257255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:31:47.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:31:47.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.266+0000 7ff82f051640 1 Processor -- start 2023-11-24T06:31:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.266+0000 7ff82f051640 1 -- start start 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff82f051640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8280cf9a0 0x7ff82815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff82f051640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82815e1e0 0x7ff8280702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff82f051640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8280b1ee0 con 0x7ff828143ae0 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff82f051640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8280b2780 con 0x7ff8280cf9a0 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff82f051640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82815e990 con 0x7ff82815e1e0 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff827fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8280cf9a0 0x7ff82815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff827fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8280cf9a0 0x7ff82815dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40748/0 (socket says 172.21.15.16:40748) 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff827fff640 1 -- 172.21.15.16:0/1437095021 learned_addr learned my addr 172.21.15.16:0/1437095021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff82d5ee640 1 --2- 172.21.15.16:0/1437095021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82815e1e0 0x7ff8280702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff827fff640 1 -- 172.21.15.16:0/1437095021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82815e1e0 msgr2=0x7ff8280702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff827fff640 1 --2- 172.21.15.16:0/1437095021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82815e1e0 0x7ff8280702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff827fff640 1 -- 172.21.15.16:0/1437095021 >> v1:172.21.15.173:6789/0 conn(0x7ff828143ae0 legacy=0x7ff8280cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.268+0000 7ff827fff640 1 -- 172.21.15.16:0/1437095021 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8280b2130 con 0x7ff8280cf9a0 2023-11-24T06:31:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.270+0000 7ff827fff640 1 --2- 172.21.15.16:0/1437095021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8280cf9a0 0x7ff82815dca0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff814009870 tx=0x7ff81402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b5b295c2eb77ed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.270+0000 7ff8277fe640 1 -- 172.21.15.16:0/1437095021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814031640 con 0x7ff8280cf9a0 2023-11-24T06:31:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.270+0000 7ff8277fe640 1 -- 172.21.15.16:0/1437095021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8140317a0 con 0x7ff8280cf9a0 2023-11-24T06:31:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.270+0000 7ff8277fe640 1 -- 172.21.15.16:0/1437095021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814031b30 con 0x7ff8280cf9a0 2023-11-24T06:31:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.271+0000 7ff82f051640 1 -- 172.21.15.16:0/1437095021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8280cf9a0 msgr2=0x7ff82815dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.271+0000 7ff82f051640 1 --2- 172.21.15.16:0/1437095021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8280cf9a0 0x7ff82815dca0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff814009870 tx=0x7ff81402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.271+0000 7ff82f051640 1 -- 172.21.15.16:0/1437095021 shutdown_connections 2023-11-24T06:31:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.271+0000 7ff82f051640 1 --2- 172.21.15.16:0/1437095021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82815e1e0 0x7ff8280702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.271+0000 7ff82f051640 1 --2- 172.21.15.16:0/1437095021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8280cf9a0 0x7ff82815dca0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.271+0000 7ff82f051640 1 -- 172.21.15.16:0/1437095021 >> 172.21.15.16:0/1437095021 conn(0x7ff82806a800 msgr2=0x7ff82806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.271+0000 7ff82f051640 1 -- 172.21.15.16:0/1437095021 shutdown_connections 2023-11-24T06:31:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.271+0000 7ff82f051640 1 -- 172.21.15.16:0/1437095021 wait complete. 2023-11-24T06:31:47.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.273+0000 7ff82f051640 1 Processor -- start 2023-11-24T06:31:47.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.273+0000 7ff82f051640 1 -- start start 2023-11-24T06:31:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.274+0000 7ff82f051640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8280cf9a0 0x7ff828159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.274+0000 7ff82f051640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff828143ae0 0x7ff8280c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.274+0000 7ff82f051640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815e1e0 0x7ff8280afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.274+0000 7ff82f051640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8280c6670 con 0x7ff82815e1e0 2023-11-24T06:31:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.274+0000 7ff82f051640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8280c55d0 con 0x7ff8280cf9a0 2023-11-24T06:31:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.274+0000 7ff82f051640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8280c58b0 con 0x7ff828143ae0 2023-11-24T06:31:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.274+0000 7ff82d5ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815e1e0 0x7ff8280afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.274+0000 7ff82d5ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815e1e0 0x7ff8280afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40760/0 (socket says 172.21.15.16:40760) 2023-11-24T06:31:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.274+0000 7ff82d5ee640 1 -- 172.21.15.16:0/3938380029 learned_addr learned my addr 172.21.15.16:0/3938380029 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.275+0000 7ff827fff640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff828143ae0 0x7ff8280c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.275+0000 7ff82cded640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8280cf9a0 0x7ff828159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.275+0000 7ff82d5ee640 1 -- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff828143ae0 msgr2=0x7ff8280c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.275+0000 7ff82d5ee640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff828143ae0 0x7ff8280c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.275+0000 7ff82d5ee640 1 -- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8280cf9a0 msgr2=0x7ff828159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.275+0000 7ff82d5ee640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8280cf9a0 0x7ff828159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.275+0000 7ff82d5ee640 1 -- 172.21.15.16:0/3938380029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8140094d0 con 0x7ff82815e1e0 2023-11-24T06:31:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.275+0000 7ff82d5ee640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815e1e0 0x7ff8280afcd0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff82000cc30 tx=0x7ff82000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.276+0000 7ff8257fa640 1 -- 172.21.15.16:0/3938380029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82000ce40 con 0x7ff82815e1e0 2023-11-24T06:31:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.276+0000 7ff8257fa640 1 -- 172.21.15.16:0/3938380029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff820003990 con 0x7ff82815e1e0 2023-11-24T06:31:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.276+0000 7ff8257fa640 1 -- 172.21.15.16:0/3938380029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820003d20 con 0x7ff82815e1e0 2023-11-24T06:31:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.276+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8280c5b90 con 0x7ff82815e1e0 2023-11-24T06:31:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.276+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8280c6150 con 0x7ff82815e1e0 2023-11-24T06:31:47.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.277+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8280675d0 con 0x7ff82815e1e0 2023-11-24T06:31:47.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.278+0000 7ff8257fa640 1 -- 172.21.15.16:0/3938380029 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff820003e80 con 0x7ff82815e1e0 2023-11-24T06:31:47.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.279+0000 7ff8257fa640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff80c06b470 0x7ff80c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.279+0000 7ff8257fa640 1 -- 172.21.15.16:0/3938380029 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7ff82008a6a0 con 0x7ff82815e1e0 2023-11-24T06:31:47.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.280+0000 7ff82cded640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff80c06b470 0x7ff80c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.281+0000 7ff82cded640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff80c06b470 0x7ff80c06d930 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff81c004ea0 tx=0x7ff81c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.284+0000 7ff8257fa640 1 -- 172.21.15.16:0/3938380029 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff820054c80 con 0x7ff82815e1e0 2023-11-24T06:31:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.484+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff82806c810 con 0x7ff82815e1e0 2023-11-24T06:31:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.485+0000 7ff8257fa640 1 -- 172.21.15.16:0/3938380029 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff82001d0a0 con 0x7ff82815e1e0 2023-11-24T06:31:47.488 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:31:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff80c06b470 msgr2=0x7ff80c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff80c06b470 0x7ff80c06d930 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff81c004ea0 tx=0x7ff81c009040 comp rx=0 tx=0).stop 2023-11-24T06:31:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815e1e0 msgr2=0x7ff8280afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815e1e0 0x7ff8280afcd0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff82000cc30 tx=0x7ff82000a5c0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 shutdown_connections 2023-11-24T06:31:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff80c06b470 0x7ff80c06d930 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff82815e1e0 0x7ff8280afcd0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff828143ae0 0x7ff8280c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 --2- 172.21.15.16:0/3938380029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8280cf9a0 0x7ff828159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 >> 172.21.15.16:0/3938380029 conn(0x7ff82806a800 msgr2=0x7ff82806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:47.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 shutdown_connections 2023-11-24T06:31:47.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.488+0000 7ff82f051640 1 -- 172.21.15.16:0/3938380029 wait complete. 2023-11-24T06:31:47.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:31:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.741+0000 7f5824abb640 1 Processor -- start 2023-11-24T06:31:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.742+0000 7f5824abb640 1 -- start start 2023-11-24T06:31:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.743+0000 7f5824abb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58200cf9a0 0x7f58200cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.743+0000 7f5824abb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f58200b31d0 0x7f58200b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.743+0000 7f5824abb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58200d0320 con 0x7f582015d4b0 2023-11-24T06:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.743+0000 7f5824abb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58200d1c60 con 0x7f58200b31d0 2023-11-24T06:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.743+0000 7f5824abb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58200b2730 con 0x7f58200cf9a0 2023-11-24T06:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.743+0000 7f581f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f582015d4b0 0x7f58200d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57546/0 (socket says 172.21.15.16:57546) 2023-11-24T06:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.744+0000 7f581f7fe640 1 -- 172.21.15.16:0/137195118 learned_addr learned my addr 172.21.15.16:0/137195118 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.744+0000 7f5817fff640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58200cf9a0 0x7f58200cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.744+0000 7f581ffff640 1 --2- 172.21.15.16:0/137195118 >> v2:172.21.15.173:3300/0 conn(0x7f58200b31d0 0x7f58200b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.744+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320112929 0 0) 0x7f58200d0320 con 0x7f582015d4b0 2023-11-24T06:31:47.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.744+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f580c003520 con 0x7f582015d4b0 2023-11-24T06:31:47.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.744+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3696765287 0 0) 0x7f580c003520 con 0x7f582015d4b0 2023-11-24T06:31:47.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.744+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58200cf9a0 msgr2=0x7f58200cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:47.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.744+0000 7f581effd640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58200cf9a0 0x7f58200cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.744+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 >> v2:172.21.15.173:3300/0 conn(0x7f58200b31d0 msgr2=0x7f58200b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:47.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 --2- 172.21.15.16:0/137195118 >> v2:172.21.15.173:3300/0 conn(0x7f58200b31d0 0x7f58200b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200d1ec0 con 0x7f582015d4b0 2023-11-24T06:31:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f5810001670 con 0x7f582015d4b0 2023-11-24T06:31:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 >> v1:172.21.15.16:6789/0 conn(0x7f582015d4b0 legacy=0x7f58200d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f580c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58200cf9a0 0x7f580c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f580c004d00 0x7f580c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58200d1c60 con 0x7f580c003970 2023-11-24T06:31:47.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58200b2730 con 0x7f580c004d00 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c00b980 con 0x7f58200cf9a0 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581ffff640 1 --2- 172.21.15.16:0/137195118 >> v2:172.21.15.173:3300/0 conn(0x7f58200b31d0 0x7f58200b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581f7fe640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f580c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f5817fff640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58200cf9a0 0x7f580c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581f7fe640 1 -- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58200cf9a0 msgr2=0x7f580c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581f7fe640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58200cf9a0 0x7f580c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581f7fe640 1 -- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f580c004d00 msgr2=0x7f580c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581f7fe640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f580c004d00 0x7f580c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.745+0000 7f581f7fe640 1 -- 172.21.15.16:0/137195118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c003520 con 0x7f580c003970 2023-11-24T06:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.746+0000 7f581f7fe640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f580c003d90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f581000c270 tx=0x7f580c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a8687bd027bad7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.747+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810005ad0 con 0x7f580c003970 2023-11-24T06:31:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.747+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810005c50 con 0x7f580c003970 2023-11-24T06:31:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.747+0000 7f581effd640 1 -- 172.21.15.16:0/137195118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581002c8a0 con 0x7f580c003970 2023-11-24T06:31:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.747+0000 7f5824abb640 1 -- 172.21.15.16:0/137195118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 msgr2=0x7f580c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.747+0000 7f5824abb640 1 --2- 172.21.15.16:0/137195118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f580c003d90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f581000c270 tx=0x7f580c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.747+0000 7f581ffff640 1 -- 172.21.15.16:0/137195118 reap_dead start 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.747+0000 7f5824abb640 1 -- 172.21.15.16:0/137195118 shutdown_connections 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.748+0000 7f5824abb640 1 -- 172.21.15.16:0/137195118 >> 172.21.15.16:0/137195118 conn(0x7f58200d3000 msgr2=0x7f58200d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.748+0000 7f5824abb640 1 -- 172.21.15.16:0/137195118 shutdown_connections 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.748+0000 7f5824abb640 1 -- 172.21.15.16:0/137195118 wait complete. 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.749+0000 7f5824abb640 1 Processor -- start 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.750+0000 7f5824abb640 1 -- start start 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.750+0000 7f5824abb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f58200b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f5824abb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5820158940 0x7f5820151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f5824abb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5820160820 0x7f58200ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f5824abb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5820151a50 con 0x7f580c003970 2023-11-24T06:31:47.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f5824abb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5820158f00 con 0x7f5820160820 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f5824abb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58201591e0 con 0x7f5820158940 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f58200b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f581f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f58200b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40780/0 (socket says 172.21.15.16:40780) 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f581f7fe640 1 -- 172.21.15.16:0/1254925635 learned_addr learned my addr 172.21.15.16:0/1254925635 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f5817fff640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5820158940 0x7f5820151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f581f7fe640 1 -- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5820158940 msgr2=0x7f5820151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f581ffff640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5820160820 0x7f58200ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f581f7fe640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5820158940 0x7f5820151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f581f7fe640 1 -- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5820160820 msgr2=0x7f58200ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f581f7fe640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5820160820 0x7f58200ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.751+0000 7f581f7fe640 1 -- 172.21.15.16:0/1254925635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58201594c0 con 0x7f580c003970 2023-11-24T06:31:47.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.752+0000 7f5817fff640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5820158940 0x7f5820151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.752+0000 7f581ffff640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5820160820 0x7f58200ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.752+0000 7f581f7fe640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f58200b5bc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f581002c440 tx=0x7f5810001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.752+0000 7f581cff9640 1 -- 172.21.15.16:0/1254925635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810039070 con 0x7f580c003970 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.752+0000 7f581cff9640 1 -- 172.21.15.16:0/1254925635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581002dd40 con 0x7f580c003970 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.752+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58201597a0 con 0x7f580c003970 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.752+0000 7f581cff9640 1 -- 172.21.15.16:0/1254925635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581003c040 con 0x7f580c003970 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.752+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5820160ca0 con 0x7f580c003970 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.754+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5820067450 con 0x7f580c003970 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.754+0000 7f581cff9640 1 -- 172.21.15.16:0/1254925635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f5810032070 con 0x7f580c003970 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.755+0000 7f581cff9640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f580006b470 0x7f580006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.756+0000 7f581cff9640 1 -- 172.21.15.16:0/1254925635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f58100ab8a0 con 0x7f580c003970 2023-11-24T06:31:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.756+0000 7f5817fff640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f580006b470 0x7f580006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:47.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.757+0000 7f5817fff640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f580006b470 0x7f580006d930 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f58200cd9d0 tx=0x7f5808008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:47.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.760+0000 7f581cff9640 1 -- 172.21.15.16:0/1254925635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58100760f0 con 0x7f580c003970 2023-11-24T06:31:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.954+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f582015aeb0 con 0x7f580c003970 2023-11-24T06:31:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.955+0000 7f581cff9640 1 -- 172.21.15.16:0/1254925635 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5810079710 con 0x7f580c003970 2023-11-24T06:31:47.958 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f580006b470 msgr2=0x7f580006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f580006b470 0x7f580006d930 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f58200cd9d0 tx=0x7f5808008040 comp rx=0 tx=0).stop 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 msgr2=0x7f58200b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f58200b5bc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f581002c440 tx=0x7f5810001a00 comp rx=0 tx=0).stop 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 shutdown_connections 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f580006b470 0x7f580006d930 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5820160820 0x7f58200ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5820158940 0x7f5820151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 --2- 172.21.15.16:0/1254925635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c003970 0x7f58200b5bc0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 >> 172.21.15.16:0/1254925635 conn(0x7f58200d3000 msgr2=0x7f58200b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 shutdown_connections 2023-11-24T06:31:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:47.958+0000 7f5824abb640 1 -- 172.21.15.16:0/1254925635 wait complete. 2023-11-24T06:31:47.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:31:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.211+0000 7f82a71b0640 1 Processor -- start 2023-11-24T06:31:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.211+0000 7f82a71b0640 1 -- start start 2023-11-24T06:31:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.213+0000 7f82a71b0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a0150760 0x7f82a0072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.213+0000 7f82a71b0640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f82a00b32e0 0x7f82a015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.213+0000 7f82a71b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a015dc80 con 0x7f82a015d300 2023-11-24T06:31:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.213+0000 7f82a71b0640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00b3770 con 0x7f82a00b32e0 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.213+0000 7f82a71b0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0071190 con 0x7f82a0150760 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.214+0000 7f8297fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f82a015d300 0x7f82a00b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57562/0 (socket says 172.21.15.16:57562) 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.214+0000 7f8297fff640 1 -- 172.21.15.16:0/420994428 learned_addr learned my addr 172.21.15.16:0/420994428 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.214+0000 7f82a574d640 1 --2- 172.21.15.16:0/420994428 >> v2:172.21.15.173:3300/0 conn(0x7f82a00b32e0 0x7f82a015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.214+0000 7f82a4f4c640 1 --2- 172.21.15.16:0/420994428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a0150760 0x7f82a0072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.214+0000 7f82977fe640 1 -- 172.21.15.16:0/420994428 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018740480 0 0) 0x7f82a015dc80 con 0x7f82a015d300 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.215+0000 7f82977fe640 1 -- 172.21.15.16:0/420994428 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8288003520 con 0x7f82a015d300 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.215+0000 7f82a574d640 1 -- 172.21.15.16:0/420994428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a0150760 msgr2=0x7f82a0072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.215+0000 7f82a574d640 1 --2- 172.21.15.16:0/420994428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a0150760 0x7f82a0072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.215+0000 7f82977fe640 1 -- 172.21.15.16:0/420994428 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 900282035 0 0) 0x7f8288003520 con 0x7f82a015d300 2023-11-24T06:31:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.215+0000 7f82a574d640 1 -- 172.21.15.16:0/420994428 >> v1:172.21.15.16:6789/0 conn(0x7f82a015d300 legacy=0x7f82a00b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.215+0000 7f82a574d640 1 -- 172.21.15.16:0/420994428 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a0071470 con 0x7f82a00b32e0 2023-11-24T06:31:48.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.216+0000 7f82a574d640 1 --2- 172.21.15.16:0/420994428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a00b32e0 0x7f82a015d740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8298009d90 tx=0x7f829802df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff84ca4ed0051cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:48.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.217+0000 7f82977fe640 1 -- 172.21.15.16:0/420994428 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290001670 con 0x7f82a00b32e0 2023-11-24T06:31:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.217+0000 7f82977fe640 1 -- 172.21.15.16:0/420994428 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829802eab0 con 0x7f82a00b32e0 2023-11-24T06:31:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.217+0000 7f82977fe640 1 -- 172.21.15.16:0/420994428 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829802ee40 con 0x7f82a00b32e0 2023-11-24T06:31:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.217+0000 7f82a71b0640 1 -- 172.21.15.16:0/420994428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a00b32e0 msgr2=0x7f82a015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.218+0000 7f82a71b0640 1 --2- 172.21.15.16:0/420994428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a00b32e0 0x7f82a015d740 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8298009d90 tx=0x7f829802df80 comp rx=0 tx=0).stop 2023-11-24T06:31:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.218+0000 7f82a71b0640 1 -- 172.21.15.16:0/420994428 shutdown_connections 2023-11-24T06:31:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.218+0000 7f82a71b0640 1 --2- 172.21.15.16:0/420994428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a00b32e0 0x7f82a015d740 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.218+0000 7f82a71b0640 1 --2- 172.21.15.16:0/420994428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a0150760 0x7f82a0072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.218+0000 7f82a71b0640 1 -- 172.21.15.16:0/420994428 >> 172.21.15.16:0/420994428 conn(0x7f82a006a800 msgr2=0x7f82a006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.218+0000 7f82a71b0640 1 -- 172.21.15.16:0/420994428 shutdown_connections 2023-11-24T06:31:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.218+0000 7f82a71b0640 1 -- 172.21.15.16:0/420994428 wait complete. 2023-11-24T06:31:48.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.220+0000 7f82a71b0640 1 Processor -- start 2023-11-24T06:31:48.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.220+0000 7f82a71b0640 1 -- start start 2023-11-24T06:31:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.222+0000 7f82a71b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00b32e0 0x7f82a01e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.222+0000 7f82a71b0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0150760 0x7f82a01396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.222+0000 7f82a71b0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a015d300 0x7f82a00b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.222+0000 7f82a71b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a014bce0 con 0x7f82a00b32e0 2023-11-24T06:31:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.222+0000 7f82a71b0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a01e4380 con 0x7f82a0150760 2023-11-24T06:31:48.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.222+0000 7f82a71b0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a014b690 con 0x7f82a015d300 2023-11-24T06:31:48.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.222+0000 7f82a4f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00b32e0 0x7f82a01e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.222+0000 7f82a4f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00b32e0 0x7f82a01e3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40794/0 (socket says 172.21.15.16:40794) 2023-11-24T06:31:48.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.222+0000 7f82a4f4c640 1 -- 172.21.15.16:0/3555477862 learned_addr learned my addr 172.21.15.16:0/3555477862 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:48.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82a574d640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a015d300 0x7f82a00b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f8297fff640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0150760 0x7f82a01396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82a4f4c640 1 -- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a015d300 msgr2=0x7f82a00b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82a4f4c640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a015d300 0x7f82a00b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82a4f4c640 1 -- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0150760 msgr2=0x7f82a01396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82a4f4c640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0150760 0x7f82a01396f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82a4f4c640 1 -- 172.21.15.16:0/3555477862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8298009a70 con 0x7f82a00b32e0 2023-11-24T06:31:48.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82a574d640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a015d300 0x7f82a00b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f8297fff640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0150760 0x7f82a01396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82a4f4c640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00b32e0 0x7f82a01e3b30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e930 tx=0x7f829c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82957fa640 1 -- 172.21.15.16:0/3555477862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c009690 con 0x7f82a00b32e0 2023-11-24T06:31:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82957fa640 1 -- 172.21.15.16:0/3555477862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c00ca00 con 0x7f82a00b32e0 2023-11-24T06:31:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82957fa640 1 -- 172.21.15.16:0/3555477862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00cdb0 con 0x7f82a00b32e0 2023-11-24T06:31:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.223+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a014b970 con 0x7f82a00b32e0 2023-11-24T06:31:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.224+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a0139c30 con 0x7f82a00b32e0 2023-11-24T06:31:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.225+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a00675d0 con 0x7f82a00b32e0 2023-11-24T06:31:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.226+0000 7f82957fa640 1 -- 172.21.15.16:0/3555477862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f829c013470 con 0x7f82a00b32e0 2023-11-24T06:31:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.227+0000 7f82957fa640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f828c06b470 0x7f828c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.227+0000 7f82957fa640 1 -- 172.21.15.16:0/3555477862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f829c08a490 con 0x7f82a00b32e0 2023-11-24T06:31:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.227+0000 7f8297fff640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f828c06b470 0x7f828c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.229+0000 7f8297fff640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f828c06b470 0x7f828c06d930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f82900011e0 tx=0x7f8290009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:48.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.233+0000 7f82957fa640 1 -- 172.21.15.16:0/3555477862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f829c054c40 con 0x7f82a00b32e0 2023-11-24T06:31:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.439+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82a00b3e60 con 0x7f82a00b32e0 2023-11-24T06:31:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.440+0000 7f82957fa640 1 -- 172.21.15.16:0/3555477862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f829c058260 con 0x7f82a00b32e0 2023-11-24T06:31:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:31:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.442+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f828c06b470 msgr2=0x7f828c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.442+0000 7f82a71b0640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f828c06b470 0x7f828c06d930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f82900011e0 tx=0x7f8290009040 comp rx=0 tx=0).stop 2023-11-24T06:31:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.442+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00b32e0 msgr2=0x7f82a01e3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.442+0000 7f82a71b0640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00b32e0 0x7f82a01e3b30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e930 tx=0x7f829c00edf0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.443+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 shutdown_connections 2023-11-24T06:31:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.443+0000 7f82a71b0640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f828c06b470 0x7f828c06d930 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.443+0000 7f82a71b0640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82a015d300 0x7f82a00b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.443+0000 7f82a71b0640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82a0150760 0x7f82a01396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.443+0000 7f82a71b0640 1 --2- 172.21.15.16:0/3555477862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00b32e0 0x7f82a01e3b30 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.443+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 >> 172.21.15.16:0/3555477862 conn(0x7f82a006a800 msgr2=0x7f82a006d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.443+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 shutdown_connections 2023-11-24T06:31:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.443+0000 7f82a71b0640 1 -- 172.21.15.16:0/3555477862 wait complete. 2023-11-24T06:31:48.466 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:31:48.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.690+0000 7ff813b73640 1 Processor -- start 2023-11-24T06:31:48.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.690+0000 7ff813b73640 1 -- start start 2023-11-24T06:31:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.691+0000 7ff813b73640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff80c13df80 0x7ff80c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.691+0000 7ff813b73640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff80c0cf9a0 0x7ff80c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.691+0000 7ff813b73640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c06c470 con 0x7ff80c13e360 2023-11-24T06:31:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.691+0000 7ff813b73640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c06c750 con 0x7ff80c13df80 2023-11-24T06:31:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.691+0000 7ff813b73640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c06ca30 con 0x7ff80c0cf9a0 2023-11-24T06:31:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.692+0000 7ff81190f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff80c13df80 0x7ff80c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.692+0000 7ff81190f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff80c13df80 0x7ff80c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40808/0 (socket says 172.21.15.16:40808) 2023-11-24T06:31:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.692+0000 7ff81190f640 1 -- 172.21.15.16:0/1517868237 learned_addr learned my addr 172.21.15.16:0/1517868237 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.692+0000 7ff81190f640 1 -- 172.21.15.16:0/1517868237 >> v2:172.21.15.173:3300/0 conn(0x7ff80c0cf9a0 msgr2=0x7ff80c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.692+0000 7ff81190f640 1 --2- 172.21.15.16:0/1517868237 >> v2:172.21.15.173:3300/0 conn(0x7ff80c0cf9a0 0x7ff80c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.692+0000 7ff81190f640 1 -- 172.21.15.16:0/1517868237 >> v1:172.21.15.173:6789/0 conn(0x7ff80c13e360 legacy=0x7ff80c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.692+0000 7ff81190f640 1 -- 172.21.15.16:0/1517868237 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c06cd10 con 0x7ff80c13df80 2023-11-24T06:31:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.693+0000 7ff81190f640 1 --2- 172.21.15.16:0/1517868237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 0x7ff80c15bee0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0ccb10 tx=0x7ff80802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5d63c8690ece27 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff81090d640 1 -- 172.21.15.16:0/1517868237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808031870 con 0x7ff80c13df80 2023-11-24T06:31:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff81090d640 1 -- 172.21.15.16:0/1517868237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8080319d0 con 0x7ff80c13df80 2023-11-24T06:31:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff81090d640 1 -- 172.21.15.16:0/1517868237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808031d60 con 0x7ff80c13df80 2023-11-24T06:31:48.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff813b73640 1 -- 172.21.15.16:0/1517868237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 msgr2=0x7ff80c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff813b73640 1 --2- 172.21.15.16:0/1517868237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 0x7ff80c15bee0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff80c0ccb10 tx=0x7ff80802b910 comp rx=0 tx=0).stop 2023-11-24T06:31:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff813b73640 1 -- 172.21.15.16:0/1517868237 shutdown_connections 2023-11-24T06:31:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff813b73640 1 --2- 172.21.15.16:0/1517868237 >> v2:172.21.15.173:3300/0 conn(0x7ff80c0cf9a0 0x7ff80c0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff813b73640 1 --2- 172.21.15.16:0/1517868237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 0x7ff80c15bee0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff813b73640 1 -- 172.21.15.16:0/1517868237 >> 172.21.15.16:0/1517868237 conn(0x7ff80c06a800 msgr2=0x7ff80c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff813b73640 1 -- 172.21.15.16:0/1517868237 shutdown_connections 2023-11-24T06:31:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.694+0000 7ff813b73640 1 -- 172.21.15.16:0/1517868237 wait complete. 2023-11-24T06:31:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.696+0000 7ff813b73640 1 Processor -- start 2023-11-24T06:31:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.696+0000 7ff813b73640 1 -- start start 2023-11-24T06:31:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.697+0000 7ff813b73640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c0cf9a0 0x7ff80c1356c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.697+0000 7ff813b73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 0x7ff80c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.697+0000 7ff813b73640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff80c13e360 0x7ff80c0afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.697+0000 7ff813b73640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c1f67b0 con 0x7ff80c13df80 2023-11-24T06:31:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.697+0000 7ff813b73640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c1f7330 con 0x7ff80c0cf9a0 2023-11-24T06:31:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.697+0000 7ff813b73640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c135c00 con 0x7ff80c13e360 2023-11-24T06:31:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.697+0000 7ff81110e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 0x7ff80c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.697+0000 7ff81110e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 0x7ff80c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40822/0 (socket says 172.21.15.16:40822) 2023-11-24T06:31:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.697+0000 7ff81110e640 1 -- 172.21.15.16:0/398500624 learned_addr learned my addr 172.21.15.16:0/398500624 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.698+0000 7ff81190f640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c0cf9a0 0x7ff80c1356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.698+0000 7ff81110e640 1 -- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff80c13e360 msgr2=0x7ff80c0afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.698+0000 7ff812110640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff80c13e360 0x7ff80c0afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.698+0000 7ff81110e640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff80c13e360 0x7ff80c0afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.698+0000 7ff81110e640 1 -- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c0cf9a0 msgr2=0x7ff80c1356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.698+0000 7ff81110e640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c0cf9a0 0x7ff80c1356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.698+0000 7ff81110e640 1 -- 172.21.15.16:0/398500624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080094d0 con 0x7ff80c13df80 2023-11-24T06:31:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.698+0000 7ff81110e640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 0x7ff80c0702c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c430 tx=0x7ff7fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.699+0000 7ff8027fc640 1 -- 172.21.15.16:0/398500624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0035f0 con 0x7ff80c13df80 2023-11-24T06:31:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.699+0000 7ff8027fc640 1 -- 172.21.15.16:0/398500624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc003750 con 0x7ff80c13df80 2023-11-24T06:31:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.699+0000 7ff8027fc640 1 -- 172.21.15.16:0/398500624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc003b00 con 0x7ff80c13df80 2023-11-24T06:31:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.699+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c1f6a00 con 0x7ff80c13df80 2023-11-24T06:31:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.699+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c1f6fc0 con 0x7ff80c13df80 2023-11-24T06:31:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.700+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c0675d0 con 0x7ff80c13df80 2023-11-24T06:31:48.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.701+0000 7ff8027fc640 1 -- 172.21.15.16:0/398500624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff7fc003c60 con 0x7ff80c13df80 2023-11-24T06:31:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.702+0000 7ff8027fc640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff7f006b470 0x7ff7f006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.702+0000 7ff8027fc640 1 -- 172.21.15.16:0/398500624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7ff7fc089440 con 0x7ff80c13df80 2023-11-24T06:31:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.702+0000 7ff81190f640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff7f006b470 0x7ff7f006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:48.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.704+0000 7ff81190f640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff7f006b470 0x7ff7f006d930 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff808036000 tx=0x7ff80803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:48.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.706+0000 7ff8027fc640 1 -- 172.21.15.16:0/398500624 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff7fc053c00 con 0x7ff80c13df80 2023-11-24T06:31:48.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.913+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff80c06d640 con 0x7ff80c13df80 2023-11-24T06:31:48.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.915+0000 7ff8027fc640 1 -- 172.21.15.16:0/398500624 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff7fc057220 con 0x7ff80c13df80 2023-11-24T06:31:48.917 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-11-24T06:31:48.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff7f006b470 msgr2=0x7ff7f006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff7f006b470 0x7ff7f006d930 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff808036000 tx=0x7ff80803c040 comp rx=0 tx=0).stop 2023-11-24T06:31:48.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 msgr2=0x7ff80c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:48.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 0x7ff80c0702c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c430 tx=0x7ff7fc00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 shutdown_connections 2023-11-24T06:31:48.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff7f006b470 0x7ff7f006d930 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff80c13e360 0x7ff80c0afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff80c13df80 0x7ff80c0702c0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 --2- 172.21.15.16:0/398500624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff80c0cf9a0 0x7ff80c1356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:48.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.917+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 >> 172.21.15.16:0/398500624 conn(0x7ff80c06a800 msgr2=0x7ff80c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:48.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.918+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 shutdown_connections 2023-11-24T06:31:48.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:48.918+0000 7ff813b73640 1 -- 172.21.15.16:0/398500624 wait complete. 2023-11-24T06:31:48.940 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi173.front.sepia.ceph.com... 2023-11-24T06:31:48.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:48.941 DEBUG:teuthology.orchestra.run.smithi173:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:31:49.037 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi173.stderr:ceph-fuse[98231]: fuse finished with error 0 and tester_r 0 2023-11-24T06:31:49.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:31:49.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:49.042 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:31:49.221 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:31:49.221 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:49.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:31:49.285 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:31:49.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:49.286 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:31:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:31:49.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:31:49.342 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:31:49.342 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:49.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:31:49.402 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:31:49.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:49.403 DEBUG:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:31:49.456 INFO:teuthology.orchestra.run.smithi173.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:31:49.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:31:49.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:49.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.691+0000 7fd2c91ef640 1 Processor -- start 2023-11-24T06:31:49.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.691+0000 7fd2c91ef640 1 -- start start 2023-11-24T06:31:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.693+0000 7fd2c91ef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c40cc500 0x7fd2c40cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.693+0000 7fd2c91ef640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2c406d160 0x7fd2c406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.693+0000 7fd2c91ef640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c4070e50 con 0x7fd2c4143c40 2023-11-24T06:31:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.693+0000 7fd2c91ef640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c40723c0 con 0x7fd2c406d160 2023-11-24T06:31:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.693+0000 7fd2c91ef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4144020 con 0x7fd2c40cc500 2023-11-24T06:31:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.694+0000 7fd2c89ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2c406d160 0x7fd2c406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.694+0000 7fd2c89ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2c406d160 0x7fd2c406d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40828/0 (socket says 172.21.15.16:40828) 2023-11-24T06:31:49.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.694+0000 7fd2c89ee640 1 -- 172.21.15.16:0/3040115474 learned_addr learned my addr 172.21.15.16:0/3040115474 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.694+0000 7fd2bbfff640 1 --2- 172.21.15.16:0/3040115474 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c40cc500 0x7fd2c40cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.694+0000 7fd2c89ee640 1 -- 172.21.15.16:0/3040115474 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c40cc500 msgr2=0x7fd2c40cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.694+0000 7fd2c89ee640 1 --2- 172.21.15.16:0/3040115474 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c40cc500 0x7fd2c40cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.694+0000 7fd2c89ee640 1 -- 172.21.15.16:0/3040115474 >> v1:172.21.15.173:6789/0 conn(0x7fd2c4143c40 legacy=0x7fd2c4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:49.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.694+0000 7fd2c89ee640 1 -- 172.21.15.16:0/3040115474 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c40ccea0 con 0x7fd2c406d160 2023-11-24T06:31:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.695+0000 7fd2c89ee640 1 --2- 172.21.15.16:0/3040115474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c406d160 0x7fd2c406d590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e50 tx=0x7fd2bc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f0267e991f3de4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.696+0000 7fd2c37fe640 1 -- 172.21.15.16:0/3040115474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc02ca70 con 0x7fd2c406d160 2023-11-24T06:31:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.696+0000 7fd2c37fe640 1 -- 172.21.15.16:0/3040115474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc02cbd0 con 0x7fd2c406d160 2023-11-24T06:31:49.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.696+0000 7fd2c37fe640 1 -- 172.21.15.16:0/3040115474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0343f0 con 0x7fd2c406d160 2023-11-24T06:31:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.697+0000 7fd2c91ef640 1 -- 172.21.15.16:0/3040115474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c406d160 msgr2=0x7fd2c406d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.697+0000 7fd2c91ef640 1 --2- 172.21.15.16:0/3040115474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c406d160 0x7fd2c406d590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009e50 tx=0x7fd2bc02bed0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.697+0000 7fd2c91ef640 1 -- 172.21.15.16:0/3040115474 shutdown_connections 2023-11-24T06:31:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.697+0000 7fd2c91ef640 1 --2- 172.21.15.16:0/3040115474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c406d160 0x7fd2c406d590 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.697+0000 7fd2c91ef640 1 --2- 172.21.15.16:0/3040115474 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c40cc500 0x7fd2c40cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.697+0000 7fd2c91ef640 1 -- 172.21.15.16:0/3040115474 >> 172.21.15.16:0/3040115474 conn(0x7fd2c406a800 msgr2=0x7fd2c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.697+0000 7fd2c91ef640 1 -- 172.21.15.16:0/3040115474 shutdown_connections 2023-11-24T06:31:49.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.697+0000 7fd2c91ef640 1 -- 172.21.15.16:0/3040115474 wait complete. 2023-11-24T06:31:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.699+0000 7fd2c91ef640 1 Processor -- start 2023-11-24T06:31:49.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.699+0000 7fd2c91ef640 1 -- start start 2023-11-24T06:31:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2c91ef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c406d160 0x7fd2c41e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2c91ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c40cc500 0x7fd2c40adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2c91ef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c4143c40 0x7fd2c41e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2c91ef640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c40b3ee0 con 0x7fd2c40cc500 2023-11-24T06:31:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2c91ef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41e0ac0 con 0x7fd2c406d160 2023-11-24T06:31:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2c91ef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41e0da0 con 0x7fd2c4143c40 2023-11-24T06:31:49.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c40cc500 0x7fd2c40adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c40cc500 0x7fd2c40adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40844/0 (socket says 172.21.15.16:40844) 2023-11-24T06:31:49.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2bbfff640 1 -- 172.21.15.16:0/4241409482 learned_addr learned my addr 172.21.15.16:0/4241409482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2c3fff640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c406d160 0x7fd2c41e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2c89ee640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c4143c40 0x7fd2c41e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2bbfff640 1 -- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c4143c40 msgr2=0x7fd2c41e5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:49.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.700+0000 7fd2bbfff640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c4143c40 0x7fd2c41e5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2bbfff640 1 -- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c406d160 msgr2=0x7fd2c41e03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2bbfff640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c406d160 0x7fd2c41e03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2bbfff640 1 -- 172.21.15.16:0/4241409482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc009b30 con 0x7fd2c40cc500 2023-11-24T06:31:49.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2c3fff640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c406d160 0x7fd2c41e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2bbfff640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c40cc500 0x7fd2c40adb90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00d800 tx=0x7fd2ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2c17fa640 1 -- 172.21.15.16:0/4241409482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac00b6c0 con 0x7fd2c40cc500 2023-11-24T06:31:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2c17fa640 1 -- 172.21.15.16:0/4241409482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac00b820 con 0x7fd2c40cc500 2023-11-24T06:31:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c41e1080 con 0x7fd2c40cc500 2023-11-24T06:31:49.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2c17fa640 1 -- 172.21.15.16:0/4241409482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac00bbb0 con 0x7fd2c40cc500 2023-11-24T06:31:49.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.701+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c41e1640 con 0x7fd2c40cc500 2023-11-24T06:31:49.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.702+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c41e1800 con 0x7fd2c40cc500 2023-11-24T06:31:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.704+0000 7fd2c17fa640 1 -- 172.21.15.16:0/4241409482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fd2ac00bd10 con 0x7fd2c40cc500 2023-11-24T06:31:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.704+0000 7fd2c17fa640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd2a406b470 0x7fd2a406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.705+0000 7fd2c17fa640 1 -- 172.21.15.16:0/4241409482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7fd2ac08a630 con 0x7fd2c40cc500 2023-11-24T06:31:49.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.705+0000 7fd2c3fff640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd2a406b470 0x7fd2a406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:49.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.706+0000 7fd2c3fff640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd2a406b470 0x7fd2a406d930 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4000f60 tx=0x7fd2b4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:49.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.709+0000 7fd2c17fa640 1 -- 172.21.15.16:0/4241409482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2ac054e60 con 0x7fd2c40cc500 2023-11-24T06:31:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.940+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c40ce860 con 0x7fd2c40cc500 2023-11-24T06:31:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.941+0000 7fd2c17fa640 1 -- 172.21.15.16:0/4241409482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fd2ac058480 con 0x7fd2c40cc500 2023-11-24T06:31:49.944 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:49.944 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3235524397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":3235524397},{"type":"v1","addr":"172.21.15.173:6837","nonce":3235524397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/593524744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":593524744},{"type":"v1","addr":"172.21.15.173:6836","nonce":593524744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7173,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/825050165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":825050165},{"type":"v1","addr":"172.21.15.16:6834","nonce":825050165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:31:22.237194+0000","modified":"2023-11-24T06:31:25.583915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2773872585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2773872585},{"type":"v1","addr":"172.21.15.16:6835","nonce":2773872585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:31:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 54 2023-11-24T06:31:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd2a406b470 msgr2=0x7fd2a406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd2a406b470 0x7fd2a406d930 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4000f60 tx=0x7fd2b4009040 comp rx=0 tx=0).stop 2023-11-24T06:31:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c40cc500 msgr2=0x7fd2c40adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c40cc500 0x7fd2c40adb90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00d800 tx=0x7fd2ac00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 shutdown_connections 2023-11-24T06:31:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd2a406b470 0x7fd2a406d930 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2c4143c40 0x7fd2c41e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2c40cc500 0x7fd2c40adb90 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 --2- 172.21.15.16:0/4241409482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2c406d160 0x7fd2c41e03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 >> 172.21.15.16:0/4241409482 conn(0x7fd2c406a800 msgr2=0x7fd2c406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 shutdown_connections 2023-11-24T06:31:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:49.945+0000 7fd2c91ef640 1 -- 172.21.15.16:0/4241409482 wait complete. 2023-11-24T06:31:49.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.192+0000 7f36aa29b640 1 Processor -- start 2023-11-24T06:31:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.192+0000 7f36aa29b640 1 -- start start 2023-11-24T06:31:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.193+0000 7f36aa29b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 0x7f36a40ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.193+0000 7f36aa29b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36a40d0560 0x7f36a4161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.193+0000 7f36aa29b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a40cf4c0 con 0x7f36a40d2260 2023-11-24T06:31:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.193+0000 7f36aa29b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a40d2ef0 con 0x7f36a40d0560 2023-11-24T06:31:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.193+0000 7f36aa29b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a40c9c50 con 0x7f36a40c9870 2023-11-24T06:31:50.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.193+0000 7f36a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36a40d0560 0x7f36a4161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.193+0000 7f36a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36a40d0560 0x7f36a4161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40852/0 (socket says 172.21.15.16:40852) 2023-11-24T06:31:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.193+0000 7f36a37fe640 1 -- 172.21.15.16:0/2062869996 learned_addr learned my addr 172.21.15.16:0/2062869996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.194+0000 7f36a3fff640 1 --2- 172.21.15.16:0/2062869996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 0x7f36a40ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.194+0000 7f36a37fe640 1 -- 172.21.15.16:0/2062869996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 msgr2=0x7f36a40ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.194+0000 7f36a37fe640 1 --2- 172.21.15.16:0/2062869996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 0x7f36a40ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.194+0000 7f36a37fe640 1 -- 172.21.15.16:0/2062869996 >> v1:172.21.15.173:6789/0 conn(0x7f36a40d2260 legacy=0x7f36a40d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.194+0000 7f36a37fe640 1 -- 172.21.15.16:0/2062869996 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40cf7a0 con 0x7f36a40d0560 2023-11-24T06:31:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.195+0000 7f36a37fe640 1 --2- 172.21.15.16:0/2062869996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d0560 0x7f36a4161030 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3698009870 tx=0x7f369802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e340e4bfe5d3d918 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.195+0000 7f36a2ffd640 1 -- 172.21.15.16:0/2062869996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3698031640 con 0x7f36a40d0560 2023-11-24T06:31:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.195+0000 7f36a2ffd640 1 -- 172.21.15.16:0/2062869996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36980317a0 con 0x7f36a40d0560 2023-11-24T06:31:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.195+0000 7f36a2ffd640 1 -- 172.21.15.16:0/2062869996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3698031b30 con 0x7f36a40d0560 2023-11-24T06:31:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.196+0000 7f36aa29b640 1 -- 172.21.15.16:0/2062869996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d0560 msgr2=0x7f36a4161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.196+0000 7f36aa29b640 1 --2- 172.21.15.16:0/2062869996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d0560 0x7f36a4161030 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3698009870 tx=0x7f369802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.196+0000 7f36aa29b640 1 -- 172.21.15.16:0/2062869996 shutdown_connections 2023-11-24T06:31:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.196+0000 7f36aa29b640 1 --2- 172.21.15.16:0/2062869996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d0560 0x7f36a4161030 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.196+0000 7f36aa29b640 1 --2- 172.21.15.16:0/2062869996 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 0x7f36a40ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.196+0000 7f36aa29b640 1 -- 172.21.15.16:0/2062869996 >> 172.21.15.16:0/2062869996 conn(0x7f36a409c560 msgr2=0x7f36a40d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.196+0000 7f36aa29b640 1 -- 172.21.15.16:0/2062869996 shutdown_connections 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.196+0000 7f36aa29b640 1 -- 172.21.15.16:0/2062869996 wait complete. 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.198+0000 7f36aa29b640 1 Processor -- start 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.198+0000 7f36aa29b640 1 -- start start 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.199+0000 7f36aa29b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 0x7f36a415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.199+0000 7f36aa29b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a40d0560 0x7f36a40cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.199+0000 7f36aa29b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d2260 0x7f36a415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.199+0000 7f36aa29b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a41f72a0 con 0x7f36a40d2260 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.199+0000 7f36aa29b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a41f6690 con 0x7f36a40d0560 2023-11-24T06:31:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.199+0000 7f36aa29b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a41f6970 con 0x7f36a40c9870 2023-11-24T06:31:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.199+0000 7f36a8838640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d2260 0x7f36a415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a8838640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d2260 0x7f36a415f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40856/0 (socket says 172.21.15.16:40856) 2023-11-24T06:31:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a8838640 1 -- 172.21.15.16:0/1971097960 learned_addr learned my addr 172.21.15.16:0/1971097960 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a3fff640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 0x7f36a415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a37fe640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a40d0560 0x7f36a40cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a8838640 1 -- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 msgr2=0x7f36a415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a8838640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 0x7f36a415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a8838640 1 -- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a40d0560 msgr2=0x7f36a40cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a8838640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a40d0560 0x7f36a40cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a8838640 1 -- 172.21.15.16:0/1971097960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36980094d0 con 0x7f36a40d2260 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.200+0000 7f36a8838640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d2260 0x7f36a415f4a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f369c00c700 tx=0x7f369c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.201+0000 7f36a0ff9640 1 -- 172.21.15.16:0/1971097960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00aa50 con 0x7f36a40d2260 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.201+0000 7f36a0ff9640 1 -- 172.21.15.16:0/1971097960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c00abb0 con 0x7f36a40d2260 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.201+0000 7f36a0ff9640 1 -- 172.21.15.16:0/1971097960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00ce10 con 0x7f36a40d2260 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.201+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a41f6c50 con 0x7f36a40d2260 2023-11-24T06:31:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.201+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a415e660 con 0x7f36a40d2260 2023-11-24T06:31:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.202+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a4066640 con 0x7f36a40d2260 2023-11-24T06:31:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.203+0000 7f36a0ff9640 1 -- 172.21.15.16:0/1971097960 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f369c012420 con 0x7f36a40d2260 2023-11-24T06:31:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.204+0000 7f36a0ff9640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f369006b470 0x7f369006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.204+0000 7f36a0ff9640 1 -- 172.21.15.16:0/1971097960 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f369c08a5e0 con 0x7f36a40d2260 2023-11-24T06:31:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.205+0000 7f36a3fff640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f369006b470 0x7f369006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.206+0000 7f36a3fff640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f369006b470 0x7f369006d930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3694006fd0 tx=0x7f3694008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.208+0000 7f36a0ff9640 1 -- 172.21.15.16:0/1971097960 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f369c054da0 con 0x7f36a40d2260 2023-11-24T06:31:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.404+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36a40675d0 con 0x7f36a40d2260 2023-11-24T06:31:50.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.407+0000 7f36a0ff9640 1 -- 172.21.15.16:0/1971097960 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16654 (secure 0 0 0) 0x7f369c003eb0 con 0x7f36a40d2260 2023-11-24T06:31:50.411 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:50.411 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:45.189909+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:31:20.522140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:31:21.257255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.410+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f369006b470 msgr2=0x7f369006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.410+0000 7f36aa29b640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f369006b470 0x7f369006d930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3694006fd0 tx=0x7f3694008040 comp rx=0 tx=0).stop 2023-11-24T06:31:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d2260 msgr2=0x7f36a415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d2260 0x7f36a415f4a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f369c00c700 tx=0x7f369c00cbc0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 shutdown_connections 2023-11-24T06:31:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f369006b470 0x7f369006d930 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36a40d2260 0x7f36a415f4a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a40d0560 0x7f36a40cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 --2- 172.21.15.16:0/1971097960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a40c9870 0x7f36a415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 >> 172.21.15.16:0/1971097960 conn(0x7f36a409c560 msgr2=0x7f36a40b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 shutdown_connections 2023-11-24T06:31:50.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.411+0000 7f36aa29b640 1 -- 172.21.15.16:0/1971097960 wait complete. 2023-11-24T06:31:50.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:26:50.170210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:31:20.522140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:31:21.257255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:31:50.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:31:50.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.665+0000 7f84eac1a640 1 Processor -- start 2023-11-24T06:31:50.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.665+0000 7f84eac1a640 1 -- start start 2023-11-24T06:31:50.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.667+0000 7f84eac1a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f84e415b090 0x7f84e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.667+0000 7f84eac1a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e406de60 con 0x7f84e406da20 2023-11-24T06:31:50.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.667+0000 7f84eac1a640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e4157ed0 con 0x7f84e4157af0 2023-11-24T06:31:50.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.667+0000 7f84eac1a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e40b2fd0 con 0x7f84e415b090 2023-11-24T06:31:50.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.667+0000 7f84ea419640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f84e406da20 0x7f84e40b3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57576/0 (socket says 172.21.15.16:57576) 2023-11-24T06:31:50.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.667+0000 7f84ea419640 1 -- 172.21.15.16:0/1408232745 learned_addr learned my addr 172.21.15.16:0/1408232745 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.667+0000 7f84e9417640 1 --2- 172.21.15.16:0/1408232745 >> v2:172.21.15.173:3300/0 conn(0x7f84e415b090 0x7f84e406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378505214 0 0) 0x7f84e406de60 con 0x7f84e406da20 2023-11-24T06:31:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84d0003520 con 0x7f84e406da20 2023-11-24T06:31:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3967721467 0 0) 0x7f84d0003520 con 0x7f84e406da20 2023-11-24T06:31:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 >> v2:172.21.15.173:3300/0 conn(0x7f84e415b090 msgr2=0x7f84e406d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 --2- 172.21.15.16:0/1408232745 >> v2:172.21.15.173:3300/0 conn(0x7f84e415b090 0x7f84e406d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 >> v1:172.21.15.173:6789/0 conn(0x7f84e4157af0 legacy=0x7f84e415da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e415e250 con 0x7f84e406da20 2023-11-24T06:31:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f84dc0019f0 con 0x7f84e406da20 2023-11-24T06:31:50.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 >> v1:172.21.15.16:6789/0 conn(0x7f84e406da20 legacy=0x7f84e40b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d00039c0 0x7f84d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d00047c0 0x7f84d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d0005180 0x7f84d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40b2fd0 con 0x7f84d0005180 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d000c080 con 0x7f84d00047c0 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.668+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d000c240 con 0x7f84d00039c0 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.669+0000 7f84ea419640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d0005180 0x7f84d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.669+0000 7f84e9417640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d00047c0 0x7f84d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.669+0000 7f84e9c18640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d00039c0 0x7f84d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.669+0000 7f84ea419640 1 -- 172.21.15.16:0/1408232745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d00039c0 msgr2=0x7f84d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.669+0000 7f84ea419640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d00039c0 0x7f84d0003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.669+0000 7f84ea419640 1 -- 172.21.15.16:0/1408232745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d00047c0 msgr2=0x7f84d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.669+0000 7f84ea419640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d00047c0 0x7f84d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.669+0000 7f84ea419640 1 -- 172.21.15.16:0/1408232745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d0003520 con 0x7f84d0005180 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.670+0000 7f84ea419640 1 -- 172.21.15.16:0/1408232745 reap_dead start 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.670+0000 7f84ea419640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d0005180 0x7f84d000b980 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00a760 tx=0x7f84d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4548236270f6efa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.671+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc034630 con 0x7f84d0005180 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.671+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc02f070 con 0x7f84d0005180 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.671+0000 7f84e8c16640 1 -- 172.21.15.16:0/1408232745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc034a10 con 0x7f84d0005180 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.671+0000 7f84eac1a640 1 -- 172.21.15.16:0/1408232745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d0005180 msgr2=0x7f84d000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.671+0000 7f84eac1a640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d0005180 0x7f84d000b980 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00a760 tx=0x7f84d00057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.671+0000 7f84eac1a640 1 -- 172.21.15.16:0/1408232745 shutdown_connections 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.671+0000 7f84eac1a640 1 --2- 172.21.15.16:0/1408232745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84d0005180 0x7f84d000b980 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.671+0000 7f84eac1a640 1 -- 172.21.15.16:0/1408232745 >> 172.21.15.16:0/1408232745 conn(0x7f84e406bb20 msgr2=0x7f84e40cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.671+0000 7f84eac1a640 1 -- 172.21.15.16:0/1408232745 shutdown_connections 2023-11-24T06:31:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.672+0000 7f84eac1a640 1 -- 172.21.15.16:0/1408232745 wait complete. 2023-11-24T06:31:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.673+0000 7f84eac1a640 1 Processor -- start 2023-11-24T06:31:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.673+0000 7f84eac1a640 1 -- start start 2023-11-24T06:31:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.674+0000 7f84eac1a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d0005180 0x7f84e40b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.674+0000 7f84eac1a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e40ce690 0x7f84e40ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.674+0000 7f84eac1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40caee0 0x7f84e40cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.674+0000 7f84eac1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40cc090 con 0x7f84e40caee0 2023-11-24T06:31:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.674+0000 7f84eac1a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40cb410 con 0x7f84e40ce690 2023-11-24T06:31:50.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.674+0000 7f84eac1a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40cb6f0 con 0x7f84d0005180 2023-11-24T06:31:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84ea419640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40caee0 0x7f84e40cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84ea419640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40caee0 0x7f84e40cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40868/0 (socket says 172.21.15.16:40868) 2023-11-24T06:31:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84ea419640 1 -- 172.21.15.16:0/2648479679 learned_addr learned my addr 172.21.15.16:0/2648479679 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84e9c18640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d0005180 0x7f84e40b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84e9417640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e40ce690 0x7f84e40ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84ea419640 1 -- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d0005180 msgr2=0x7f84e40b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84ea419640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d0005180 0x7f84e40b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84ea419640 1 -- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e40ce690 msgr2=0x7f84e40ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84ea419640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e40ce690 0x7f84e40ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.675+0000 7f84ea419640 1 -- 172.21.15.16:0/2648479679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40cb9d0 con 0x7f84e40caee0 2023-11-24T06:31:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.676+0000 7f84ea419640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40caee0 0x7f84e40cd5a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f84dc002230 tx=0x7f84dc001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.676+0000 7f84da7fc640 1 -- 172.21.15.16:0/2648479679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc001930 con 0x7f84e40caee0 2023-11-24T06:31:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.676+0000 7f84da7fc640 1 -- 172.21.15.16:0/2648479679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc0345b0 con 0x7f84e40caee0 2023-11-24T06:31:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.676+0000 7f84da7fc640 1 -- 172.21.15.16:0/2648479679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004da0 con 0x7f84e40caee0 2023-11-24T06:31:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.676+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e40cbcb0 con 0x7f84e40caee0 2023-11-24T06:31:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.676+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e40b3ee0 con 0x7f84e40caee0 2023-11-24T06:31:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.677+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e4067b60 con 0x7f84e40caee0 2023-11-24T06:31:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.678+0000 7f84da7fc640 1 -- 172.21.15.16:0/2648479679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f84dc031030 con 0x7f84e40caee0 2023-11-24T06:31:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.679+0000 7f84da7fc640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f84c806b470 0x7f84c806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.679+0000 7f84da7fc640 1 -- 172.21.15.16:0/2648479679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f84dc0aafd0 con 0x7f84e40caee0 2023-11-24T06:31:50.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.680+0000 7f84e9c18640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f84c806b470 0x7f84c806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:50.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.681+0000 7f84e9c18640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f84c806b470 0x7f84c806d930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f84e0000c00 tx=0x7f84e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.684+0000 7f84da7fc640 1 -- 172.21.15.16:0/2648479679 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84dc0755b0 con 0x7f84e40caee0 2023-11-24T06:31:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.879+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84e40b3ee0 con 0x7f84e40caee0 2023-11-24T06:31:50.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.880+0000 7f84da7fc640 1 -- 172.21.15.16:0/2648479679 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84e40b3ee0 con 0x7f84e40caee0 2023-11-24T06:31:50.882 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-11-24T06:31:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f84c806b470 msgr2=0x7f84c806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f84c806b470 0x7f84c806d930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f84e0000c00 tx=0x7f84e0009040 comp rx=0 tx=0).stop 2023-11-24T06:31:50.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40caee0 msgr2=0x7f84e40cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40caee0 0x7f84e40cd5a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f84dc002230 tx=0x7f84dc001cb0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 shutdown_connections 2023-11-24T06:31:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f84c806b470 0x7f84c806d930 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84e40caee0 0x7f84e40cd5a0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84e40ce690 0x7f84e40ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 --2- 172.21.15.16:0/2648479679 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d0005180 0x7f84e40b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 >> 172.21.15.16:0/2648479679 conn(0x7f84e406bb20 msgr2=0x7f84e41561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:50.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.883+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 shutdown_connections 2023-11-24T06:31:50.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:50.884+0000 7f84eac1a640 1 -- 172.21.15.16:0/2648479679 wait complete. 2023-11-24T06:31:50.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:31:51.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.136+0000 7ffa6e8da640 1 Processor -- start 2023-11-24T06:31:51.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.136+0000 7ffa6e8da640 1 -- start start 2023-11-24T06:31:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.138+0000 7ffa6e8da640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa68071e40 0x7ffa680ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.138+0000 7ffa6e8da640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 0x7ffa6815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.138+0000 7ffa6e8da640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6815e140 con 0x7ffa68143810 2023-11-24T06:31:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.138+0000 7ffa6e8da640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa68143bf0 con 0x7ffa68071e40 2023-11-24T06:31:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.138+0000 7ffa6e8da640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680722f0 con 0x7ffa6815d770 2023-11-24T06:31:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.138+0000 7ffa6d0d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa68071e40 0x7ffa680ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.138+0000 7ffa6d8d8640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffa68143810 0x7ffa6815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57578/0 (socket says 172.21.15.16:57578) 2023-11-24T06:31:51.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.138+0000 7ffa6d8d8640 1 -- 172.21.15.16:0/3227426672 learned_addr learned my addr 172.21.15.16:0/3227426672 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:51.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.139+0000 7ffa6e0d9640 1 --2- 172.21.15.16:0/3227426672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 0x7ffa6815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.139+0000 7ffa6d0d7640 1 -- 172.21.15.16:0/3227426672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 msgr2=0x7ffa6815dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:51.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.139+0000 7ffa6d0d7640 1 --2- 172.21.15.16:0/3227426672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 0x7ffa6815dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.139+0000 7ffa6d0d7640 1 -- 172.21.15.16:0/3227426672 >> v1:172.21.15.16:6789/0 conn(0x7ffa68143810 legacy=0x7ffa6815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.139+0000 7ffa6d0d7640 1 -- 172.21.15.16:0/3227426672 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6806c730 con 0x7ffa68071e40 2023-11-24T06:31:51.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.139+0000 7ffa6c8d6640 1 -- 172.21.15.16:0/3227426672 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275341999 0 0) 0x7ffa6815e140 con 0x7ffa68143810 2023-11-24T06:31:51.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.140+0000 7ffa6d0d7640 1 --2- 172.21.15.16:0/3227426672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68071e40 0x7ffa680ae170 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ffa580097a0 tx=0x7ffa5802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6310893d9903d223 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:51.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.140+0000 7ffa6c8d6640 1 -- 172.21.15.16:0/3227426672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5802f050 con 0x7ffa68071e40 2023-11-24T06:31:51.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.140+0000 7ffa6c8d6640 1 -- 172.21.15.16:0/3227426672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58031700 con 0x7ffa68071e40 2023-11-24T06:31:51.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.140+0000 7ffa6c8d6640 1 -- 172.21.15.16:0/3227426672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58031a90 con 0x7ffa68071e40 2023-11-24T06:31:51.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.141+0000 7ffa6e8da640 1 -- 172.21.15.16:0/3227426672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68071e40 msgr2=0x7ffa680ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.141+0000 7ffa6e8da640 1 --2- 172.21.15.16:0/3227426672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68071e40 0x7ffa680ae170 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ffa580097a0 tx=0x7ffa5802b7a0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.141+0000 7ffa6e8da640 1 -- 172.21.15.16:0/3227426672 shutdown_connections 2023-11-24T06:31:51.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.141+0000 7ffa6e8da640 1 --2- 172.21.15.16:0/3227426672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 0x7ffa6815dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.141+0000 7ffa6e8da640 1 --2- 172.21.15.16:0/3227426672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68071e40 0x7ffa680ae170 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.141+0000 7ffa6e8da640 1 -- 172.21.15.16:0/3227426672 >> 172.21.15.16:0/3227426672 conn(0x7ffa6806a7d0 msgr2=0x7ffa6806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:51.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.141+0000 7ffa6e8da640 1 -- 172.21.15.16:0/3227426672 shutdown_connections 2023-11-24T06:31:51.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.141+0000 7ffa6e8da640 1 -- 172.21.15.16:0/3227426672 wait complete. 2023-11-24T06:31:51.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.143+0000 7ffa6e8da640 1 Processor -- start 2023-11-24T06:31:51.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.143+0000 7ffa6e8da640 1 -- start start 2023-11-24T06:31:51.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.144+0000 7ffa6e8da640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68071e40 0x7ffa680cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.144+0000 7ffa6e8da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68143810 0x7ffa681f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.144+0000 7ffa6e8da640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 0x7ffa681efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.144+0000 7ffa6e8da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa681f2b90 con 0x7ffa68143810 2023-11-24T06:31:51.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.144+0000 7ffa6e8da640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680cdb60 con 0x7ffa68071e40 2023-11-24T06:31:51.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.144+0000 7ffa6e8da640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa681f2d80 con 0x7ffa6815d770 2023-11-24T06:31:51.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6d0d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68143810 0x7ffa681f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6d0d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68143810 0x7ffa681f6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40886/0 (socket says 172.21.15.16:40886) 2023-11-24T06:31:51.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6d0d7640 1 -- 172.21.15.16:0/2722405115 learned_addr learned my addr 172.21.15.16:0/2722405115 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:51.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6e0d9640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 0x7ffa681efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6d0d7640 1 -- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 msgr2=0x7ffa681efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6d8d8640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68071e40 0x7ffa680cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6d0d7640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 0x7ffa681efe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6d0d7640 1 -- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68071e40 msgr2=0x7ffa680cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6d0d7640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68071e40 0x7ffa680cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6d0d7640 1 -- 172.21.15.16:0/2722405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58009400 con 0x7ffa68143810 2023-11-24T06:31:51.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.145+0000 7ffa6e0d9640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 0x7ffa681efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:51.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.146+0000 7ffa6d0d7640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68143810 0x7ffa681f6b20 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ffa58036040 tx=0x7ffa580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:51.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.146+0000 7ffa5e7fc640 1 -- 172.21.15.16:0/2722405115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580339b0 con 0x7ffa68143810 2023-11-24T06:31:51.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.146+0000 7ffa5e7fc640 1 -- 172.21.15.16:0/2722405115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58033b30 con 0x7ffa68143810 2023-11-24T06:31:51.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.146+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa681f3060 con 0x7ffa68143810 2023-11-24T06:31:51.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.146+0000 7ffa5e7fc640 1 -- 172.21.15.16:0/2722405115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5803c400 con 0x7ffa68143810 2023-11-24T06:31:51.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.146+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa681f7060 con 0x7ffa68143810 2023-11-24T06:31:51.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.148+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa68067450 con 0x7ffa68143810 2023-11-24T06:31:51.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.148+0000 7ffa5e7fc640 1 -- 172.21.15.16:0/2722405115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ffa5803a040 con 0x7ffa68143810 2023-11-24T06:31:51.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.150+0000 7ffa5e7fc640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa4c06b470 0x7ffa4c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.150+0000 7ffa5e7fc640 1 -- 172.21.15.16:0/2722405115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7ffa580ab690 con 0x7ffa68143810 2023-11-24T06:31:51.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.150+0000 7ffa6d8d8640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa4c06b470 0x7ffa4c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.151+0000 7ffa6d8d8640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa4c06b470 0x7ffa4c06d930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffa64007fd0 tx=0x7ffa64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:51.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.154+0000 7ffa5e7fc640 1 -- 172.21.15.16:0/2722405115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa58075d90 con 0x7ffa68143810 2023-11-24T06:31:51.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.350+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa680b1f40 con 0x7ffa68143810 2023-11-24T06:31:51.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.351+0000 7ffa5e7fc640 1 -- 172.21.15.16:0/2722405115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa580793b0 con 0x7ffa68143810 2023-11-24T06:31:51.353 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-11-24T06:31:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.353+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa4c06b470 msgr2=0x7ffa4c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.353+0000 7ffa6e8da640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa4c06b470 0x7ffa4c06d930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffa64007fd0 tx=0x7ffa64009040 comp rx=0 tx=0).stop 2023-11-24T06:31:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.353+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68143810 msgr2=0x7ffa681f6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.353+0000 7ffa6e8da640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68143810 0x7ffa681f6b20 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ffa58036040 tx=0x7ffa580046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.354+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 shutdown_connections 2023-11-24T06:31:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.354+0000 7ffa6e8da640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa4c06b470 0x7ffa4c06d930 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.354+0000 7ffa6e8da640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa6815d770 0x7ffa681efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.354+0000 7ffa6e8da640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa68143810 0x7ffa681f6b20 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.354+0000 7ffa6e8da640 1 --2- 172.21.15.16:0/2722405115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa68071e40 0x7ffa680cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.354+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 >> 172.21.15.16:0/2722405115 conn(0x7ffa6806a7d0 msgr2=0x7ffa6806abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.354+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 shutdown_connections 2023-11-24T06:31:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.354+0000 7ffa6e8da640 1 -- 172.21.15.16:0/2722405115 wait complete. 2023-11-24T06:31:51.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:31:51.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.594+0000 7ffbb7059640 1 Processor -- start 2023-11-24T06:31:51.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.595+0000 7ffbb7059640 1 -- start start 2023-11-24T06:31:51.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.596+0000 7ffbb7059640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb00adf20 0x7ffbb0070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.596+0000 7ffbb7059640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ffbb015bf10 0x7ffbb00cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.596+0000 7ffbb7059640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb015f4e0 con 0x7ffbb00cc980 2023-11-24T06:31:51.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.596+0000 7ffbb7059640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb015f970 con 0x7ffbb015bf10 2023-11-24T06:31:51.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.596+0000 7ffbb7059640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00cce10 con 0x7ffbb00adf20 2023-11-24T06:31:51.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.597+0000 7ffbb6858640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffbb00cc980 0x7ffbb0143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57594/0 (socket says 172.21.15.16:57594) 2023-11-24T06:31:51.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.597+0000 7ffbb6858640 1 -- 172.21.15.16:0/2678280847 learned_addr learned my addr 172.21.15.16:0/2678280847 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.597+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037071656 0 0) 0x7ffbb015f4e0 con 0x7ffbb00cc980 2023-11-24T06:31:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.597+0000 7ffbb6057640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb00adf20 0x7ffbb0070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.597+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb9c003520 con 0x7ffbb00cc980 2023-11-24T06:31:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.597+0000 7ffbb5856640 1 --2- 172.21.15.16:0/2678280847 >> v2:172.21.15.173:3300/0 conn(0x7ffbb015bf10 0x7ffbb00cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892913173 0 0) 0x7ffb9c003520 con 0x7ffbb00cc980 2023-11-24T06:31:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb00adf20 msgr2=0x7ffbb0070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb00adf20 0x7ffbb0070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 >> v2:172.21.15.173:3300/0 conn(0x7ffbb015bf10 msgr2=0x7ffbb00cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 --2- 172.21.15.16:0/2678280847 >> v2:172.21.15.173:3300/0 conn(0x7ffbb015bf10 0x7ffbb00cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb0071130 con 0x7ffbb00cc980 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7ffba80019f0 con 0x7ffbb00cc980 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 >> v1:172.21.15.16:6789/0 conn(0x7ffbb00cc980 legacy=0x7ffbb0143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb9c003970 0x7ffb9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c0047c0 0x7ffb9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb9c005180 0x7ffb9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb015f970 con 0x7ffb9c005180 2023-11-24T06:31:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00cce10 con 0x7ffb9c0047c0 2023-11-24T06:31:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c00c050 con 0x7ffb9c003970 2023-11-24T06:31:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb6858640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb9c005180 0x7ffb9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb6057640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb9c003970 0x7ffb9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.598+0000 7ffbb5856640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c0047c0 0x7ffb9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.599+0000 7ffbb6858640 1 -- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb9c003970 msgr2=0x7ffb9c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.599+0000 7ffbb6858640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffb9c003970 0x7ffb9c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.599+0000 7ffbb6858640 1 -- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c0047c0 msgr2=0x7ffb9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.599+0000 7ffbb6858640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb9c0047c0 0x7ffb9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.599+0000 7ffbb6858640 1 -- 172.21.15.16:0/2678280847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c003520 con 0x7ffb9c005180 2023-11-24T06:31:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.600+0000 7ffbb6858640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb9c005180 0x7ffb9c00b980 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ffba800a840 tx=0x7ffb9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab77e917ed16464 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.600+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8034630 con 0x7ffb9c005180 2023-11-24T06:31:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.600+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba802f070 con 0x7ffb9c005180 2023-11-24T06:31:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.600+0000 7ffbb5055640 1 -- 172.21.15.16:0/2678280847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8034a30 con 0x7ffb9c005180 2023-11-24T06:31:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.601+0000 7ffbb7059640 1 -- 172.21.15.16:0/2678280847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb9c005180 msgr2=0x7ffb9c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.601+0000 7ffbb7059640 1 --2- 172.21.15.16:0/2678280847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb9c005180 0x7ffb9c00b980 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ffba800a840 tx=0x7ffb9c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.601+0000 7ffbb6858640 1 -- 172.21.15.16:0/2678280847 reap_dead start 2023-11-24T06:31:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.601+0000 7ffbb7059640 1 -- 172.21.15.16:0/2678280847 shutdown_connections 2023-11-24T06:31:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.601+0000 7ffbb7059640 1 -- 172.21.15.16:0/2678280847 >> 172.21.15.16:0/2678280847 conn(0x7ffbb006a7d0 msgr2=0x7ffbb006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.601+0000 7ffbb7059640 1 -- 172.21.15.16:0/2678280847 shutdown_connections 2023-11-24T06:31:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.601+0000 7ffbb7059640 1 -- 172.21.15.16:0/2678280847 wait complete. 2023-11-24T06:31:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.603+0000 7ffbb7059640 1 Processor -- start 2023-11-24T06:31:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.603+0000 7ffbb7059640 1 -- start start 2023-11-24T06:31:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.604+0000 7ffbb7059640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb0141a10 0x7ffbb00c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.604+0000 7ffbb7059640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0141df0 0x7ffbb00abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.604+0000 7ffbb7059640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01422b0 0x7ffbb00ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.604+0000 7ffbb7059640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00c4670 con 0x7ffbb0141a10 2023-11-24T06:31:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.604+0000 7ffbb7059640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00c4950 con 0x7ffbb0141df0 2023-11-24T06:31:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.604+0000 7ffbb7059640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00c4c30 con 0x7ffbb01422b0 2023-11-24T06:31:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.604+0000 7ffbb6057640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb0141a10 0x7ffbb00c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.604+0000 7ffbb6057640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb0141a10 0x7ffbb00c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40906/0 (socket says 172.21.15.16:40906) 2023-11-24T06:31:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.604+0000 7ffbb6057640 1 -- 172.21.15.16:0/1653923778 learned_addr learned my addr 172.21.15.16:0/1653923778 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffbb6858640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01422b0 0x7ffbb00ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffbb5856640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0141df0 0x7ffbb00abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffbb6057640 1 -- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01422b0 msgr2=0x7ffbb00ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffbb6057640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01422b0 0x7ffbb00ac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffbb6057640 1 -- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0141df0 msgr2=0x7ffbb00abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffbb6057640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0141df0 0x7ffbb00abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffbb6057640 1 -- 172.21.15.16:0/1653923778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00c4f10 con 0x7ffbb0141a10 2023-11-24T06:31:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffbb5856640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0141df0 0x7ffbb00abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffbb6057640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb0141a10 0x7ffbb00c3d00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00f530 tx=0x7ffbac00f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffba2ffd640 1 -- 172.21.15.16:0/1653923778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac004020 con 0x7ffbb0141a10 2023-11-24T06:31:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.605+0000 7ffba2ffd640 1 -- 172.21.15.16:0/1653923778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac004180 con 0x7ffbb0141a10 2023-11-24T06:31:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.606+0000 7ffba2ffd640 1 -- 172.21.15.16:0/1653923778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac014670 con 0x7ffbb0141a10 2023-11-24T06:31:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.606+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb00c51f0 con 0x7ffbb0141a10 2023-11-24T06:31:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.606+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb00c4420 con 0x7ffbb0141a10 2023-11-24T06:31:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.607+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb0067450 con 0x7ffbb0141a10 2023-11-24T06:31:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.608+0000 7ffba2ffd640 1 -- 172.21.15.16:0/1653923778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ffbac0147d0 con 0x7ffbb0141a10 2023-11-24T06:31:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.609+0000 7ffba2ffd640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffb9406b470 0x7ffb9406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.609+0000 7ffba2ffd640 1 -- 172.21.15.16:0/1653923778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7ffbac08a6c0 con 0x7ffbb0141a10 2023-11-24T06:31:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.609+0000 7ffbb5856640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffb9406b470 0x7ffb9406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.611+0000 7ffbb5856640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffb9406b470 0x7ffb9406d930 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffbb01421d0 tx=0x7ffba4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.613+0000 7ffba2ffd640 1 -- 172.21.15.16:0/1653923778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffbac054df0 con 0x7ffbb0141a10 2023-11-24T06:31:51.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.810+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffbb006e150 con 0x7ffbb0141a10 2023-11-24T06:31:51.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.812+0000 7ffba2ffd640 1 -- 172.21.15.16:0/1653923778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffbac058410 con 0x7ffbb0141a10 2023-11-24T06:31:51.814 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-11-24T06:31:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.814+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffb9406b470 msgr2=0x7ffb9406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.814+0000 7ffbb7059640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffb9406b470 0x7ffb9406d930 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffbb01421d0 tx=0x7ffba4008040 comp rx=0 tx=0).stop 2023-11-24T06:31:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.814+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb0141a10 msgr2=0x7ffbb00c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:51.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.814+0000 7ffbb7059640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb0141a10 0x7ffbb00c3d00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00f530 tx=0x7ffbac00f9f0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.815+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 shutdown_connections 2023-11-24T06:31:51.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.815+0000 7ffbb7059640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffb9406b470 0x7ffb9406d930 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.815+0000 7ffbb7059640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01422b0 0x7ffbb00ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.815+0000 7ffbb7059640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0141df0 0x7ffbb00abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.815+0000 7ffbb7059640 1 --2- 172.21.15.16:0/1653923778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbb0141a10 0x7ffbb00c3d00 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:51.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.815+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 >> 172.21.15.16:0/1653923778 conn(0x7ffbb006a7d0 msgr2=0x7ffbb00afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:51.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.815+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 shutdown_connections 2023-11-24T06:31:51.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:51.815+0000 7ffbb7059640 1 -- 172.21.15.16:0/1653923778 wait complete. 2023-11-24T06:31:51.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.057+0000 7f7e2359e640 1 Processor -- start 2023-11-24T06:31:52.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.057+0000 7f7e2359e640 1 -- start start 2023-11-24T06:31:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.058+0000 7f7e2359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e2415d3c0 0x7f7e240d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.058+0000 7f7e2359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e240b2c70 con 0x7f7e240b4110 2023-11-24T06:31:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.059+0000 7f7e2359e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e240cf6d0 con 0x7f7e240cfa40 2023-11-24T06:31:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.059+0000 7f7e2359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e240cf3f0 con 0x7f7e2415d3c0 2023-11-24T06:31:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.059+0000 7f7e22d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7e240b4110 0x7f7e240ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57610/0 (socket says 172.21.15.16:57610) 2023-11-24T06:31:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.059+0000 7f7e22d9d640 1 -- 172.21.15.16:0/3613618000 learned_addr learned my addr 172.21.15.16:0/3613618000 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:52.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.059+0000 7f7e2259c640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e2415d3c0 0x7f7e240d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.059+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791113535 0 0) 0x7f7e240b2c70 con 0x7f7e240b4110 2023-11-24T06:31:52.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.059+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e10003520 con 0x7f7e240b4110 2023-11-24T06:31:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364421689 0 0) 0x7f7e10003520 con 0x7f7e240b4110 2023-11-24T06:31:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e2415d3c0 msgr2=0x7f7e240d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e2415d3c0 0x7f7e240d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 >> v1:172.21.15.173:6789/0 conn(0x7f7e240cfa40 legacy=0x7f7e240cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e240b2620 con 0x7f7e240b4110 2023-11-24T06:31:52.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f7e1c0019f0 con 0x7f7e240b4110 2023-11-24T06:31:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 >> v1:172.21.15.16:6789/0 conn(0x7f7e240b4110 legacy=0x7f7e240ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e100039a0 0x7f7e10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e100047c0 0x7f7e10004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e2415d3c0 0x7f7e10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e240cf3f0 con 0x7f7e100047c0 2023-11-24T06:31:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1000ba60 con 0x7f7e100039a0 2023-11-24T06:31:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1000bc20 con 0x7f7e2415d3c0 2023-11-24T06:31:52.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.060+0000 7f7e1bfff640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e100047c0 0x7f7e10004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.061+0000 7f7e2259c640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e100039a0 0x7f7e10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.061+0000 7f7e22d9d640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e2415d3c0 0x7f7e10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.061+0000 7f7e1bfff640 1 -- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e2415d3c0 msgr2=0x7f7e10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.061+0000 7f7e1bfff640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e2415d3c0 0x7f7e10005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.061+0000 7f7e1bfff640 1 -- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e100039a0 msgr2=0x7f7e10003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.061+0000 7f7e1bfff640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e100039a0 0x7f7e10003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.061+0000 7f7e1bfff640 1 -- 172.21.15.16:0/3613618000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10003520 con 0x7f7e100047c0 2023-11-24T06:31:52.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.061+0000 7f7e2259c640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e100039a0 0x7f7e10003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.062+0000 7f7e1bfff640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e100047c0 0x7f7e10004c40 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c001720 tx=0x7f7e100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733820f7071d005 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.063+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c0052d0 con 0x7f7e100047c0 2023-11-24T06:31:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.063+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0c02e050 con 0x7f7e100047c0 2023-11-24T06:31:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.063+0000 7f7e21d9b640 1 -- 172.21.15.16:0/3613618000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c0035f0 con 0x7f7e100047c0 2023-11-24T06:31:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.063+0000 7f7e2359e640 1 -- 172.21.15.16:0/3613618000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e100047c0 msgr2=0x7f7e10004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.063+0000 7f7e2359e640 1 --2- 172.21.15.16:0/3613618000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e100047c0 0x7f7e10004c40 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c001720 tx=0x7f7e100057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.063+0000 7f7e22d9d640 1 -- 172.21.15.16:0/3613618000 reap_dead start 2023-11-24T06:31:52.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.063+0000 7f7e2359e640 1 -- 172.21.15.16:0/3613618000 shutdown_connections 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.063+0000 7f7e2359e640 1 -- 172.21.15.16:0/3613618000 >> 172.21.15.16:0/3613618000 conn(0x7f7e240d3050 msgr2=0x7f7e240d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.064+0000 7f7e2359e640 1 -- 172.21.15.16:0/3613618000 shutdown_connections 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.064+0000 7f7e2359e640 1 -- 172.21.15.16:0/3613618000 wait complete. 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.066+0000 7f7e2359e640 1 Processor -- start 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.066+0000 7f7e2359e640 1 -- start start 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.067+0000 7f7e2359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e100047c0 0x7f7e241f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.067+0000 7f7e2359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e240c8600 0x7f7e240c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e2359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e240c70c0 0x7f7e240b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e2359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e241f3420 con 0x7f7e240c70c0 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e2359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e240c7560 con 0x7f7e240c8600 2023-11-24T06:31:52.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e2359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e240c7840 con 0x7f7e100047c0 2023-11-24T06:31:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e22d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e240c70c0 0x7f7e240b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e22d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e240c70c0 0x7f7e240b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40920/0 (socket says 172.21.15.16:40920) 2023-11-24T06:31:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e22d9d640 1 -- 172.21.15.16:0/289757840 learned_addr learned my addr 172.21.15.16:0/289757840 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e2259c640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e100047c0 0x7f7e241f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e22d9d640 1 -- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e100047c0 msgr2=0x7f7e241f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e22d9d640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e100047c0 0x7f7e241f2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e22d9d640 1 -- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e240c8600 msgr2=0x7f7e240c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e22d9d640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e240c8600 0x7f7e240c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.068+0000 7f7e22d9d640 1 -- 172.21.15.16:0/289757840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e240c7b20 con 0x7f7e240c70c0 2023-11-24T06:31:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.069+0000 7f7e22d9d640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e240c70c0 0x7f7e240b7a30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f7e2408b5c0 tx=0x7f7e1c00b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.069+0000 7f7e1affd640 1 -- 172.21.15.16:0/289757840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00b8d0 con 0x7f7e240c70c0 2023-11-24T06:31:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.069+0000 7f7e1affd640 1 -- 172.21.15.16:0/289757840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c00ba30 con 0x7f7e240c70c0 2023-11-24T06:31:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.069+0000 7f7e1affd640 1 -- 172.21.15.16:0/289757840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00bde0 con 0x7f7e240c70c0 2023-11-24T06:31:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.069+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e240c7e00 con 0x7f7e240c70c0 2023-11-24T06:31:52.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.069+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e24157ca0 con 0x7f7e240c70c0 2023-11-24T06:31:52.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.070+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e240c8320 con 0x7f7e240c70c0 2023-11-24T06:31:52.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.071+0000 7f7e1affd640 1 -- 172.21.15.16:0/289757840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f7e1c007070 con 0x7f7e240c70c0 2023-11-24T06:31:52.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.072+0000 7f7e1affd640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e0406b470 0x7f7e0406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.072+0000 7f7e1affd640 1 -- 172.21.15.16:0/289757840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f7e1c08a670 con 0x7f7e240c70c0 2023-11-24T06:31:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.073+0000 7f7e2259c640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e0406b470 0x7f7e0406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.074+0000 7f7e2259c640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e0406b470 0x7f7e0406d930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7e140099c0 tx=0x7f7e14008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.077+0000 7f7e1affd640 1 -- 172.21.15.16:0/289757840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e240c8320 con 0x7f7e240c70c0 2023-11-24T06:31:52.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.311+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e240c8320 con 0x7f7e240c70c0 2023-11-24T06:31:52.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.313+0000 7f7e1affd640 1 -- 172.21.15.16:0/289757840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f7e240c8320 con 0x7f7e240c70c0 2023-11-24T06:31:52.316 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:52.316 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3235524397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":3235524397},{"type":"v1","addr":"172.21.15.173:6837","nonce":3235524397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/593524744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":593524744},{"type":"v1","addr":"172.21.15.173:6836","nonce":593524744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7173,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/825050165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":825050165},{"type":"v1","addr":"172.21.15.16:6834","nonce":825050165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:31:22.237194+0000","modified":"2023-11-24T06:31:25.583915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2773872585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2773872585},{"type":"v1","addr":"172.21.15.16:6835","nonce":2773872585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:31:52.317 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 54 2023-11-24T06:31:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.315+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e0406b470 msgr2=0x7f7e0406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.315+0000 7f7e2359e640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e0406b470 0x7f7e0406d930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7e140099c0 tx=0x7f7e14008040 comp rx=0 tx=0).stop 2023-11-24T06:31:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.315+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e240c70c0 msgr2=0x7f7e240b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.315+0000 7f7e2359e640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e240c70c0 0x7f7e240b7a30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f7e2408b5c0 tx=0x7f7e1c00b0d0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.316+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 shutdown_connections 2023-11-24T06:31:52.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.316+0000 7f7e2359e640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7e0406b470 0x7f7e0406d930 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.316+0000 7f7e2359e640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e240c70c0 0x7f7e240b7a30 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.316+0000 7f7e2359e640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e240c8600 0x7f7e240c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.316+0000 7f7e2359e640 1 --2- 172.21.15.16:0/289757840 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e100047c0 0x7f7e241f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.316+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 >> 172.21.15.16:0/289757840 conn(0x7f7e240d3050 msgr2=0x7f7e2406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:52.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.316+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 shutdown_connections 2023-11-24T06:31:52.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.316+0000 7f7e2359e640 1 -- 172.21.15.16:0/289757840 wait complete. 2023-11-24T06:31:52.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.552+0000 7fc64d735640 1 Processor -- start 2023-11-24T06:31:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.553+0000 7fc64d735640 1 -- start start 2023-11-24T06:31:52.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.554+0000 7fc64d735640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc6480ce310 0x7fc6480ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.554+0000 7fc64d735640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480d0640 0x7fc648166f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.554+0000 7fc64d735640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6480b1430 con 0x7fc6480cd820 2023-11-24T06:31:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.554+0000 7fc64d735640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480d0c40 con 0x7fc6480ce310 2023-11-24T06:31:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.554+0000 7fc64d735640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480d0f20 con 0x7fc6480d0640 2023-11-24T06:31:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.555+0000 7fc647fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc6480cd820 0x7fc6480cdc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41226/0 (socket says 172.21.15.16:41226) 2023-11-24T06:31:52.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.555+0000 7fc647fff640 1 -- 172.21.15.16:0/4038445523 learned_addr learned my addr 172.21.15.16:0/4038445523 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.555+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648567814 0 0) 0x7fc6480b1430 con 0x7fc6480cd820 2023-11-24T06:31:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.555+0000 7fc64cf34640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480d0640 0x7fc648166f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.555+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc634003520 con 0x7fc6480cd820 2023-11-24T06:31:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.555+0000 7fc6477fe640 1 --2- 172.21.15.16:0/4038445523 >> v2:172.21.15.173:3300/0 conn(0x7fc6480ce310 0x7fc6480ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 386560098 0 0) 0x7fc634003520 con 0x7fc6480cd820 2023-11-24T06:31:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480d0640 msgr2=0x7fc648166f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480d0640 0x7fc648166f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 >> v2:172.21.15.173:3300/0 conn(0x7fc6480ce310 msgr2=0x7fc6480ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 --2- 172.21.15.16:0/4038445523 >> v2:172.21.15.173:3300/0 conn(0x7fc6480ce310 0x7fc6480ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480d1200 con 0x7fc6480cd820 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fc6380019f0 con 0x7fc6480cd820 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 >> v1:172.21.15.16:6789/0 conn(0x7fc6480cd820 legacy=0x7fc6480cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634003970 0x7fc634003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6340047c0 0x7fc634004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480d0640 0x7fc634005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480d0c40 con 0x7fc6340047c0 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480d0f20 con 0x7fc634003970 2023-11-24T06:31:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63400ba30 con 0x7fc6480d0640 2023-11-24T06:31:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc6477fe640 1 --2- 172.21.15.16:0/4038445523 >> v2:172.21.15.173:3300/0 conn(0x7fc6480ce310 0x7fc6480ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:31:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc6477fe640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6340047c0 0x7fc634004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.556+0000 7fc64cf34640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480d0640 0x7fc634005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.557+0000 7fc647fff640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634003970 0x7fc634003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.557+0000 7fc6477fe640 1 -- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480d0640 msgr2=0x7fc634005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.557+0000 7fc6477fe640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480d0640 0x7fc634005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.557+0000 7fc6477fe640 1 -- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634003970 msgr2=0x7fc634003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.557+0000 7fc6477fe640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634003970 0x7fc634003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.557+0000 7fc6477fe640 1 -- 172.21.15.16:0/4038445523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc634003520 con 0x7fc6340047c0 2023-11-24T06:31:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.557+0000 7fc647fff640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634003970 0x7fc634003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:52.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.558+0000 7fc6477fe640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6340047c0 0x7fc634004c40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc630006fd0 tx=0x7fc6340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d695f997461618e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:52.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.558+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630031640 con 0x7fc6340047c0 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.558+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6300317a0 con 0x7fc6340047c0 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.558+0000 7fc646ffd640 1 -- 172.21.15.16:0/4038445523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630031b10 con 0x7fc6340047c0 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.559+0000 7fc64d735640 1 -- 172.21.15.16:0/4038445523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6340047c0 msgr2=0x7fc634004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.559+0000 7fc64d735640 1 --2- 172.21.15.16:0/4038445523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6340047c0 0x7fc634004c40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc630006fd0 tx=0x7fc6340057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.559+0000 7fc64cf34640 1 -- 172.21.15.16:0/4038445523 reap_dead start 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.559+0000 7fc64d735640 1 -- 172.21.15.16:0/4038445523 shutdown_connections 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.559+0000 7fc64d735640 1 -- 172.21.15.16:0/4038445523 >> 172.21.15.16:0/4038445523 conn(0x7fc648099a60 msgr2=0x7fc648099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.559+0000 7fc64d735640 1 -- 172.21.15.16:0/4038445523 shutdown_connections 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.559+0000 7fc64d735640 1 -- 172.21.15.16:0/4038445523 wait complete. 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.561+0000 7fc64d735640 1 Processor -- start 2023-11-24T06:31:52.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.561+0000 7fc64d735640 1 -- start start 2023-11-24T06:31:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64d735640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6340047c0 0x7fc6480cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64d735640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480ccf10 0x7fc6481601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64d735640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc648145130 0x7fc6481455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64d735640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc648145af0 con 0x7fc648145130 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64d735640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480cbe70 con 0x7fc6340047c0 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64d735640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480cc150 con 0x7fc6480ccf10 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64cf34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc648145130 0x7fc6481455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64cf34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc648145130 0x7fc6481455b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53750/0 (socket says 172.21.15.16:53750) 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64cf34640 1 -- 172.21.15.16:0/1087143427 learned_addr learned my addr 172.21.15.16:0/1087143427 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc6477fe640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480ccf10 0x7fc6481601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64cf34640 1 -- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480ccf10 msgr2=0x7fc6481601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc647fff640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6340047c0 0x7fc6480cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64cf34640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480ccf10 0x7fc6481601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64cf34640 1 -- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6340047c0 msgr2=0x7fc6480cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64cf34640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6340047c0 0x7fc6480cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.562+0000 7fc64cf34640 1 -- 172.21.15.16:0/1087143427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480cc430 con 0x7fc648145130 2023-11-24T06:31:52.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.563+0000 7fc6477fe640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480ccf10 0x7fc6481601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:52.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.563+0000 7fc64cf34640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc648145130 0x7fc6481455b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc648072290 tx=0x7fc640011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:52.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.563+0000 7fc644ff9640 1 -- 172.21.15.16:0/1087143427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640004440 con 0x7fc648145130 2023-11-24T06:31:52.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.563+0000 7fc644ff9640 1 -- 172.21.15.16:0/1087143427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc640003610 con 0x7fc648145130 2023-11-24T06:31:52.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.563+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6480cc710 con 0x7fc648145130 2023-11-24T06:31:52.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.563+0000 7fc644ff9640 1 -- 172.21.15.16:0/1087143427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640003a00 con 0x7fc648145130 2023-11-24T06:31:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.563+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc648160710 con 0x7fc648145130 2023-11-24T06:31:52.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.564+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc648067450 con 0x7fc648145130 2023-11-24T06:31:52.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.565+0000 7fc644ff9640 1 -- 172.21.15.16:0/1087143427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fc640003b60 con 0x7fc648145130 2023-11-24T06:31:52.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.566+0000 7fc644ff9640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc62806b470 0x7fc62806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:52.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.567+0000 7fc644ff9640 1 -- 172.21.15.16:0/1087143427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7fc64008a400 con 0x7fc648145130 2023-11-24T06:31:52.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.567+0000 7fc647fff640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc62806b470 0x7fc62806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:52.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.568+0000 7fc647fff640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc62806b470 0x7fc62806d930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc638002230 tx=0x7fc638009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:52.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.571+0000 7fc644ff9640 1 -- 172.21.15.16:0/1087143427 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc640054b80 con 0x7fc648145130 2023-11-24T06:31:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.765+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6480b1ea0 con 0x7fc648145130 2023-11-24T06:31:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.768+0000 7fc644ff9640 1 -- 172.21.15.16:0/1087143427 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16654 (secure 0 0 0) 0x7fc6400581a0 con 0x7fc648145130 2023-11-24T06:31:52.770 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:52.771 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:45.189909+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:31:20.522140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:31:21.257255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.770+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc62806b470 msgr2=0x7fc62806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.770+0000 7fc64d735640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc62806b470 0x7fc62806d930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc638002230 tx=0x7fc638009040 comp rx=0 tx=0).stop 2023-11-24T06:31:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc648145130 msgr2=0x7fc6481455b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc648145130 0x7fc6481455b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc648072290 tx=0x7fc640011920 comp rx=0 tx=0).stop 2023-11-24T06:31:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 shutdown_connections 2023-11-24T06:31:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fc62806b470 0x7fc62806d930 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc648145130 0x7fc6481455b0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6480ccf10 0x7fc6481601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 --2- 172.21.15.16:0/1087143427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6340047c0 0x7fc6480cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 >> 172.21.15.16:0/1087143427 conn(0x7fc648099a60 msgr2=0x7fc6480733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 shutdown_connections 2023-11-24T06:31:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:52.771+0000 7fc64d735640 1 -- 172.21.15.16:0/1087143427 wait complete. 2023-11-24T06:31:52.792 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:31:52.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.008+0000 7f0e8dd4e640 1 Processor -- start 2023-11-24T06:31:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.008+0000 7f0e8dd4e640 1 -- start start 2023-11-24T06:31:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.010+0000 7f0e8dd4e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e880cc500 0x7f0e880cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.010+0000 7f0e8dd4e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8806d160 0x7f0e8806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.010+0000 7f0e8dd4e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e88070e50 con 0x7f0e88143c40 2023-11-24T06:31:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.010+0000 7f0e8dd4e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e880723c0 con 0x7f0e880cc500 2023-11-24T06:31:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.010+0000 7f0e8dd4e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880cced0 con 0x7f0e8806d160 2023-11-24T06:31:53.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.010+0000 7f0e87fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e880cc500 0x7f0e880cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.010+0000 7f0e8cd4c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0e88143c40 0x7f0e88071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41234/0 (socket says 172.21.15.16:41234) 2023-11-24T06:31:53.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.010+0000 7f0e8cd4c640 1 -- 172.21.15.16:0/2622071917 learned_addr learned my addr 172.21.15.16:0/2622071917 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:53.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.011+0000 7f0e8d54d640 1 --2- 172.21.15.16:0/2622071917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8806d160 0x7f0e8806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.011+0000 7f0e87fff640 1 -- 172.21.15.16:0/2622071917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8806d160 msgr2=0x7f0e8806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:53.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.011+0000 7f0e87fff640 1 --2- 172.21.15.16:0/2622071917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8806d160 0x7f0e8806d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.011+0000 7f0e87fff640 1 -- 172.21.15.16:0/2622071917 >> v1:172.21.15.16:6789/0 conn(0x7f0e88143c40 legacy=0x7f0e88071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.011+0000 7f0e87fff640 1 -- 172.21.15.16:0/2622071917 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88144020 con 0x7f0e880cc500 2023-11-24T06:31:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.012+0000 7f0e87fff640 1 --2- 172.21.15.16:0/2622071917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880cc500 0x7f0e880cc990 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0e740097a0 tx=0x7f0e7402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7489eaf64351a8c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.012+0000 7f0e877fe640 1 -- 172.21.15.16:0/2622071917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74031780 con 0x7f0e880cc500 2023-11-24T06:31:53.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e877fe640 1 -- 172.21.15.16:0/2622071917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e740318e0 con 0x7f0e880cc500 2023-11-24T06:31:53.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e877fe640 1 -- 172.21.15.16:0/2622071917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74031c70 con 0x7f0e880cc500 2023-11-24T06:31:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/2622071917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880cc500 msgr2=0x7f0e880cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e8dd4e640 1 --2- 172.21.15.16:0/2622071917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880cc500 0x7f0e880cc990 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0e740097a0 tx=0x7f0e7402b820 comp rx=0 tx=0).stop 2023-11-24T06:31:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/2622071917 shutdown_connections 2023-11-24T06:31:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e8dd4e640 1 --2- 172.21.15.16:0/2622071917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e8806d160 0x7f0e8806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e8dd4e640 1 --2- 172.21.15.16:0/2622071917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e880cc500 0x7f0e880cc990 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/2622071917 >> 172.21.15.16:0/2622071917 conn(0x7f0e8806a800 msgr2=0x7f0e8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/2622071917 shutdown_connections 2023-11-24T06:31:53.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.013+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/2622071917 wait complete. 2023-11-24T06:31:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.015+0000 7f0e8dd4e640 1 Processor -- start 2023-11-24T06:31:53.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.015+0000 7f0e8dd4e640 1 -- start start 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.016+0000 7f0e8dd4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8806d160 0x7f0e88157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.016+0000 7f0e8dd4e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e880cc500 0x7f0e88133680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.016+0000 7f0e8dd4e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e88143c40 0x7f0e880b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.016+0000 7f0e8dd4e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88158160 con 0x7f0e8806d160 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.016+0000 7f0e8dd4e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88133bc0 con 0x7f0e88143c40 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.016+0000 7f0e8dd4e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88159d30 con 0x7f0e880cc500 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.016+0000 7f0e8cd4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8806d160 0x7f0e88157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e8cd4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8806d160 0x7f0e88157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53762/0 (socket says 172.21.15.16:53762) 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e8cd4c640 1 -- 172.21.15.16:0/1280049636 learned_addr learned my addr 172.21.15.16:0/1280049636 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e87fff640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e880cc500 0x7f0e88133680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e8d54d640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e88143c40 0x7f0e880b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e8cd4c640 1 -- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e880cc500 msgr2=0x7f0e88133680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e8cd4c640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e880cc500 0x7f0e88133680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e8cd4c640 1 -- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e88143c40 msgr2=0x7f0e880b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e8cd4c640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e88143c40 0x7f0e880b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e8cd4c640 1 -- 172.21.15.16:0/1280049636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e74009400 con 0x7f0e8806d160 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e87fff640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e880cc500 0x7f0e88133680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.017+0000 7f0e8cd4c640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8806d160 0x7f0e88157c20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c430 tx=0x7f0e7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.018+0000 7f0e857fa640 1 -- 172.21.15.16:0/1280049636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c005ce0 con 0x7f0e8806d160 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.018+0000 7f0e857fa640 1 -- 172.21.15.16:0/1280049636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c005e40 con 0x7f0e8806d160 2023-11-24T06:31:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.018+0000 7f0e857fa640 1 -- 172.21.15.16:0/1280049636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c007310 con 0x7f0e8806d160 2023-11-24T06:31:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.018+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e880b3b60 con 0x7f0e8806d160 2023-11-24T06:31:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.018+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e881596e0 con 0x7f0e8806d160 2023-11-24T06:31:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.019+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e88159540 con 0x7f0e8806d160 2023-11-24T06:31:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.020+0000 7f0e857fa640 1 -- 172.21.15.16:0/1280049636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f0e7c007470 con 0x7f0e8806d160 2023-11-24T06:31:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.021+0000 7f0e857fa640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0e6c06b4a0 0x7f0e6c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.021+0000 7f0e857fa640 1 -- 172.21.15.16:0/1280049636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f0e7c0896a0 con 0x7f0e8806d160 2023-11-24T06:31:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.022+0000 7f0e87fff640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0e6c06b4a0 0x7f0e6c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.023+0000 7f0e87fff640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0e6c06b4a0 0x7f0e6c06d960 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0e74036000 tx=0x7f0e7403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.025+0000 7f0e857fa640 1 -- 172.21.15.16:0/1280049636 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e88159540 con 0x7f0e8806d160 2023-11-24T06:31:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.257+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e88159540 con 0x7f0e8806d160 2023-11-24T06:31:53.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.259+0000 7f0e857fa640 1 -- 172.21.15.16:0/1280049636 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f0e88159540 con 0x7f0e8806d160 2023-11-24T06:31:53.261 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:53.261 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3235524397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":3235524397},{"type":"v1","addr":"172.21.15.173:6837","nonce":3235524397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/593524744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":593524744},{"type":"v1","addr":"172.21.15.173:6836","nonce":593524744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7173,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/825050165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":825050165},{"type":"v1","addr":"172.21.15.16:6834","nonce":825050165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:31:22.237194+0000","modified":"2023-11-24T06:31:25.583915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2773872585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2773872585},{"type":"v1","addr":"172.21.15.16:6835","nonce":2773872585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:31:53.261 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 54 2023-11-24T06:31:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0e6c06b4a0 msgr2=0x7f0e6c06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0e6c06b4a0 0x7f0e6c06d960 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0e74036000 tx=0x7f0e7403d040 comp rx=0 tx=0).stop 2023-11-24T06:31:53.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8806d160 msgr2=0x7f0e88157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8806d160 0x7f0e88157c20 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c430 tx=0x7f0e7c00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 shutdown_connections 2023-11-24T06:31:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0e6c06b4a0 0x7f0e6c06d960 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e88143c40 0x7f0e880b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e880cc500 0x7f0e88133680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 --2- 172.21.15.16:0/1280049636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8806d160 0x7f0e88157c20 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.261+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 >> 172.21.15.16:0/1280049636 conn(0x7f0e8806a800 msgr2=0x7f0e880aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.262+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 shutdown_connections 2023-11-24T06:31:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.262+0000 7f0e8dd4e640 1 -- 172.21.15.16:0/1280049636 wait complete. 2023-11-24T06:31:53.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:31:53.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.495+0000 7fd2460df640 1 Processor -- start 2023-11-24T06:31:53.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.496+0000 7fd2460df640 1 -- start start 2023-11-24T06:31:53.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.497+0000 7fd2460df640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd240146650 0x7fd2400b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.497+0000 7fd2460df640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd24015e030 0x7fd24015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.497+0000 7fd2460df640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24015dba0 con 0x7fd2400726f0 2023-11-24T06:31:53.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.497+0000 7fd2460df640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2400cd130 con 0x7fd24015e030 2023-11-24T06:31:53.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.497+0000 7fd2460df640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24006f800 con 0x7fd240146650 2023-11-24T06:31:53.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd2448dc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd2400726f0 0x7fd2400cca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41250/0 (socket says 172.21.15.16:41250) 2023-11-24T06:31:53.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd2448dc640 1 -- 172.21.15.16:0/3311674727 learned_addr learned my addr 172.21.15.16:0/3311674727 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:53.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd2458de640 1 --2- 172.21.15.16:0/3311674727 >> v2:172.21.15.16:3300/0 conn(0x7fd24015e030 0x7fd24015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd2450dd640 1 --2- 172.21.15.16:0/3311674727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd240146650 0x7fd2400b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd2458de640 1 -- 172.21.15.16:0/3311674727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd240146650 msgr2=0x7fd2400b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:53.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd2458de640 1 --2- 172.21.15.16:0/3311674727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd240146650 0x7fd2400b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd2458de640 1 -- 172.21.15.16:0/3311674727 >> v1:172.21.15.16:6789/0 conn(0x7fd2400726f0 legacy=0x7fd2400cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd2458de640 1 -- 172.21.15.16:0/3311674727 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24006f240 con 0x7fd24015e030 2023-11-24T06:31:53.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd2450dd640 1 --2- 172.21.15.16:0/3311674727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd240146650 0x7fd2400b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:53.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.498+0000 7fd237fff640 1 -- 172.21.15.16:0/3311674727 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677625383 0 0) 0x7fd24015dba0 con 0x7fd2400726f0 2023-11-24T06:31:53.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.499+0000 7fd2458de640 1 --2- 172.21.15.16:0/3311674727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 0x7fd24015e4e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fd2380097d0 tx=0x7fd23802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f6b4fd4df174b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd237fff640 1 -- 172.21.15.16:0/3311674727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23802d040 con 0x7fd24015e030 2023-11-24T06:31:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd237fff640 1 -- 172.21.15.16:0/3311674727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd238031810 con 0x7fd24015e030 2023-11-24T06:31:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd237fff640 1 -- 172.21.15.16:0/3311674727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238031ba0 con 0x7fd24015e030 2023-11-24T06:31:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd2460df640 1 -- 172.21.15.16:0/3311674727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 msgr2=0x7fd24015e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd2460df640 1 --2- 172.21.15.16:0/3311674727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 0x7fd24015e4e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fd2380097d0 tx=0x7fd23802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd2460df640 1 -- 172.21.15.16:0/3311674727 shutdown_connections 2023-11-24T06:31:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd2460df640 1 --2- 172.21.15.16:0/3311674727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 0x7fd24015e4e0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd2460df640 1 --2- 172.21.15.16:0/3311674727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd240146650 0x7fd2400b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd2460df640 1 -- 172.21.15.16:0/3311674727 >> 172.21.15.16:0/3311674727 conn(0x7fd2400b1d40 msgr2=0x7fd2400b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd2460df640 1 -- 172.21.15.16:0/3311674727 shutdown_connections 2023-11-24T06:31:53.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.500+0000 7fd2460df640 1 -- 172.21.15.16:0/3311674727 wait complete. 2023-11-24T06:31:53.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.502+0000 7fd2460df640 1 Processor -- start 2023-11-24T06:31:53.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.502+0000 7fd2460df640 1 -- start start 2023-11-24T06:31:53.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.503+0000 7fd2460df640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2400726f0 0x7fd2400b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.503+0000 7fd2460df640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd240146650 0x7fd24013ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.503+0000 7fd2460df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 0x7fd240140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.503+0000 7fd2460df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400b3d50 con 0x7fd24015e030 2023-11-24T06:31:53.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.503+0000 7fd2460df640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd240140940 con 0x7fd240146650 2023-11-24T06:31:53.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.503+0000 7fd2460df640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24013f1f0 con 0x7fd2400726f0 2023-11-24T06:31:53.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2458de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 0x7fd240140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2458de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 0x7fd240140e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53768/0 (socket says 172.21.15.16:53768) 2023-11-24T06:31:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2458de640 1 -- 172.21.15.16:0/3315565472 learned_addr learned my addr 172.21.15.16:0/3315565472 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2450dd640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2400726f0 0x7fd2400b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2458de640 1 -- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2400726f0 msgr2=0x7fd2400b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2458de640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2400726f0 0x7fd2400b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2458de640 1 -- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd240146650 msgr2=0x7fd24013ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2448dc640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd240146650 0x7fd24013ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2458de640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd240146650 0x7fd24013ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2458de640 1 -- 172.21.15.16:0/3315565472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238009400 con 0x7fd24015e030 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.504+0000 7fd2458de640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 0x7fd240140e20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd23802bec0 tx=0x7fd2380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.505+0000 7fd235ffb640 1 -- 172.21.15.16:0/3315565472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23803ace0 con 0x7fd24015e030 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.505+0000 7fd235ffb640 1 -- 172.21.15.16:0/3315565472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23803ae40 con 0x7fd24015e030 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.505+0000 7fd235ffb640 1 -- 172.21.15.16:0/3315565472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238044630 con 0x7fd24015e030 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.505+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2400b2fd0 con 0x7fd24015e030 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.505+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2400b3590 con 0x7fd24015e030 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.506+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd240067b60 con 0x7fd24015e030 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.507+0000 7fd235ffb640 1 -- 172.21.15.16:0/3315565472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fd238044790 con 0x7fd24015e030 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.508+0000 7fd235ffb640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd22406b470 0x7fd22406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.508+0000 7fd235ffb640 1 -- 172.21.15.16:0/3315565472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7fd2380ac6e0 con 0x7fd24015e030 2023-11-24T06:31:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.509+0000 7fd2450dd640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd22406b470 0x7fd22406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.510+0000 7fd2450dd640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd22406b470 0x7fd22406d930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007510 tx=0x7fd23c0074a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.513+0000 7fd235ffb640 1 -- 172.21.15.16:0/3315565472 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd238076cc0 con 0x7fd24015e030 2023-11-24T06:31:53.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.743+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd24006b0f0 con 0x7fd24015e030 2023-11-24T06:31:53.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.744+0000 7fd235ffb640 1 -- 172.21.15.16:0/3315565472 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fd23807a2e0 con 0x7fd24015e030 2023-11-24T06:31:53.746 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:53.747 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3235524397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":3235524397},{"type":"v1","addr":"172.21.15.173:6837","nonce":3235524397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/593524744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":593524744},{"type":"v1","addr":"172.21.15.173:6836","nonce":593524744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7173,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/825050165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":825050165},{"type":"v1","addr":"172.21.15.16:6834","nonce":825050165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:31:22.237194+0000","modified":"2023-11-24T06:31:25.583915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7176},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7176":{"gid":7176,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2773872585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2773872585},{"type":"v1","addr":"172.21.15.16:6835","nonce":2773872585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:31:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 54 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd22406b470 msgr2=0x7fd22406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd22406b470 0x7fd22406d930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007510 tx=0x7fd23c0074a0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 msgr2=0x7fd240140e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 0x7fd240140e20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd23802bec0 tx=0x7fd2380046c0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 shutdown_connections 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd22406b470 0x7fd22406d930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd24015e030 0x7fd240140e20 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd240146650 0x7fd24013ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 --2- 172.21.15.16:0/3315565472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2400726f0 0x7fd2400b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.747+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 >> 172.21.15.16:0/3315565472 conn(0x7fd2400b1d40 msgr2=0x7fd240145cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.748+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 shutdown_connections 2023-11-24T06:31:53.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.748+0000 7fd2460df640 1 -- 172.21.15.16:0/3315565472 wait complete. 2023-11-24T06:31:53.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:31:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.992+0000 7f2233275640 1 Processor -- start 2023-11-24T06:31:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.992+0000 7f2233275640 1 -- start start 2023-11-24T06:31:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.994+0000 7f2233275640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f222c13df80 0x7f222c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.994+0000 7f2233275640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:53.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.994+0000 7f2233275640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c06c470 con 0x7f222c13e360 2023-11-24T06:31:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.994+0000 7f2233275640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c06c750 con 0x7f222c13df80 2023-11-24T06:31:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.994+0000 7f2233275640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c06ca30 con 0x7f222c0cf9a0 2023-11-24T06:31:53.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.995+0000 7f2231812640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.995+0000 7f2231812640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:43314/0 (socket says 172.21.15.16:43314) 2023-11-24T06:31:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.995+0000 7f2231812640 1 -- 172.21.15.16:0/2523777145 learned_addr learned my addr 172.21.15.16:0/2523777145 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.995+0000 7f2231011640 1 --2- 172.21.15.16:0/2523777145 >> v2:172.21.15.173:3300/0 conn(0x7f222c13df80 0x7f222c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.996+0000 7f2231812640 1 -- 172.21.15.16:0/2523777145 >> v2:172.21.15.173:3300/0 conn(0x7f222c13df80 msgr2=0x7f222c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.996+0000 7f2231812640 1 --2- 172.21.15.16:0/2523777145 >> v2:172.21.15.173:3300/0 conn(0x7f222c13df80 0x7f222c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:53.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.996+0000 7f2231812640 1 -- 172.21.15.16:0/2523777145 >> v1:172.21.15.173:6789/0 conn(0x7f222c13e360 legacy=0x7f222c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.996+0000 7f2231812640 1 -- 172.21.15.16:0/2523777145 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c06cd10 con 0x7f222c0cf9a0 2023-11-24T06:31:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.997+0000 7f2231812640 1 --2- 172.21.15.16:0/2523777145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0cd840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2224009e50 tx=0x7f222402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=499d6c11d5cf83aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2223fff640 1 -- 172.21.15.16:0/2523777145 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222402cb00 con 0x7f222c0cf9a0 2023-11-24T06:31:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2223fff640 1 -- 172.21.15.16:0/2523777145 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222402cc60 con 0x7f222c0cf9a0 2023-11-24T06:31:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2223fff640 1 -- 172.21.15.16:0/2523777145 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22240343f0 con 0x7f222c0cf9a0 2023-11-24T06:31:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2233275640 1 -- 172.21.15.16:0/2523777145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 msgr2=0x7f222c0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2233275640 1 --2- 172.21.15.16:0/2523777145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0cd840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2224009e50 tx=0x7f222402bf60 comp rx=0 tx=0).stop 2023-11-24T06:31:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2233275640 1 -- 172.21.15.16:0/2523777145 shutdown_connections 2023-11-24T06:31:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2233275640 1 --2- 172.21.15.16:0/2523777145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0cd840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2233275640 1 --2- 172.21.15.16:0/2523777145 >> v2:172.21.15.173:3300/0 conn(0x7f222c13df80 0x7f222c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2233275640 1 -- 172.21.15.16:0/2523777145 >> 172.21.15.16:0/2523777145 conn(0x7f222c06a800 msgr2=0x7f222c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.998+0000 7f2233275640 1 -- 172.21.15.16:0/2523777145 shutdown_connections 2023-11-24T06:31:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:53.999+0000 7f2233275640 1 -- 172.21.15.16:0/2523777145 wait complete. 2023-11-24T06:31:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.000+0000 7f2233275640 1 Processor -- start 2023-11-24T06:31:54.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.000+0000 7f2233275640 1 -- start start 2023-11-24T06:31:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.001+0000 7f2233275640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.001+0000 7f2233275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222c13df80 0x7f222c1520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.001+0000 7f2233275640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222c13e360 0x7f222c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.001+0000 7f2233275640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c153320 con 0x7f222c13df80 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.001+0000 7f2233275640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c152710 con 0x7f222c13e360 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.001+0000 7f2233275640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c1529f0 con 0x7f222c0cf9a0 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2230810640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222c13df80 0x7f222c1520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2230810640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222c13df80 0x7f222c1520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53782/0 (socket says 172.21.15.16:53782) 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2230810640 1 -- 172.21.15.16:0/1136813687 learned_addr learned my addr 172.21.15.16:0/1136813687 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2231011640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2230810640 1 -- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 msgr2=0x7f222c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2230810640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2230810640 1 -- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222c13e360 msgr2=0x7f222c1502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2230810640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222c13e360 0x7f222c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2230810640 1 -- 172.21.15.16:0/1136813687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224009b30 con 0x7f222c13df80 2023-11-24T06:31:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2231011640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:54.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.002+0000 7f2230810640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222c13df80 0x7f222c1520a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c440 tx=0x7f221c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.003+0000 7f2221ffb640 1 -- 172.21.15.16:0/1136813687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c0052d0 con 0x7f222c13df80 2023-11-24T06:31:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.003+0000 7f2221ffb640 1 -- 172.21.15.16:0/1136813687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221c005430 con 0x7f222c13df80 2023-11-24T06:31:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.003+0000 7f2221ffb640 1 -- 172.21.15.16:0/1136813687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c007260 con 0x7f222c13df80 2023-11-24T06:31:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.003+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c152cd0 con 0x7f222c13df80 2023-11-24T06:31:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.003+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c150810 con 0x7f222c13df80 2023-11-24T06:31:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.004+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c153490 con 0x7f222c13df80 2023-11-24T06:31:54.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.005+0000 7f2221ffb640 1 -- 172.21.15.16:0/1136813687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f221c0073c0 con 0x7f222c13df80 2023-11-24T06:31:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.006+0000 7f2221ffb640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f221006b470 0x7f221006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.006+0000 7f2221ffb640 1 -- 172.21.15.16:0/1136813687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f221c089590 con 0x7f222c13df80 2023-11-24T06:31:54.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.007+0000 7f2231011640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f221006b470 0x7f221006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.008+0000 7f2231011640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f221006b470 0x7f221006d930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2228009ac0 tx=0x7f2228008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:54.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.013+0000 7f2221ffb640 1 -- 172.21.15.16:0/1136813687 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f221c053d10 con 0x7f222c13df80 2023-11-24T06:31:54.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:31:54.186 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:31:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.209+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f222c0675d0 con 0x7f222c13df80 2023-11-24T06:31:54.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.210+0000 7f2221ffb640 1 -- 172.21.15.16:0/1136813687 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16654 (secure 0 0 0) 0x7f221c087e60 con 0x7f222c13df80 2023-11-24T06:31:54.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:31:54.213 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:45.189909+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:31:20.522140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:31:21.257255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:31:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.212+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f221006b470 msgr2=0x7f221006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.212+0000 7f2233275640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f221006b470 0x7f221006d930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2228009ac0 tx=0x7f2228008040 comp rx=0 tx=0).stop 2023-11-24T06:31:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.212+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222c13df80 msgr2=0x7f222c1520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.212+0000 7f2233275640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222c13df80 0x7f222c1520a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c440 tx=0x7f221c001e70 comp rx=0 tx=0).stop 2023-11-24T06:31:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.213+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 shutdown_connections 2023-11-24T06:31:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.213+0000 7f2233275640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f221006b470 0x7f221006d930 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.213+0000 7f2233275640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222c13e360 0x7f222c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.213+0000 7f2233275640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222c13df80 0x7f222c1520a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.213+0000 7f2233275640 1 --2- 172.21.15.16:0/1136813687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f222c0cf9a0 0x7f222c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.213+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 >> 172.21.15.16:0/1136813687 conn(0x7f222c06a800 msgr2=0x7f222c0b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.213+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 shutdown_connections 2023-11-24T06:31:54.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.213+0000 7f2233275640 1 -- 172.21.15.16:0/1136813687 wait complete. 2023-11-24T06:31:54.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:31:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.483+0000 7f0a5ae59640 1 Processor -- start 2023-11-24T06:31:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.483+0000 7f0a5ae59640 1 -- start start 2023-11-24T06:31:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.484+0000 7f0a5ae59640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0a54160920 0x7f0a540b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.484+0000 7f0a5ae59640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a540cfea0 0x7f0a54161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.484+0000 7f0a5ae59640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a540d0a10 con 0x7f0a540cf9f0 2023-11-24T06:31:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.484+0000 7f0a5ae59640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540d1f50 con 0x7f0a540cfea0 2023-11-24T06:31:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.484+0000 7f0a5ae59640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540cf080 con 0x7f0a54160920 2023-11-24T06:31:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a5a658640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a540cfea0 0x7f0a54161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a59656640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0a540cf9f0 0x7f0a54161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41262/0 (socket says 172.21.15.16:41262) 2023-11-24T06:31:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a59656640 1 -- 172.21.15.16:0/149737171 learned_addr learned my addr 172.21.15.16:0/149737171 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a59e57640 1 --2- 172.21.15.16:0/149737171 >> v2:172.21.15.173:3300/0 conn(0x7f0a54160920 0x7f0a540b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a5a658640 1 -- 172.21.15.16:0/149737171 >> v2:172.21.15.173:3300/0 conn(0x7f0a54160920 msgr2=0x7f0a540b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:54.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a5a658640 1 --2- 172.21.15.16:0/149737171 >> v2:172.21.15.173:3300/0 conn(0x7f0a54160920 0x7f0a540b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a5a658640 1 -- 172.21.15.16:0/149737171 >> v1:172.21.15.16:6789/0 conn(0x7f0a540cf9f0 legacy=0x7f0a54161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a5a658640 1 -- 172.21.15.16:0/149737171 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a540d0310 con 0x7f0a540cfea0 2023-11-24T06:31:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a58e55640 1 -- 172.21.15.16:0/149737171 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179645215 0 0) 0x7f0a540d0a10 con 0x7f0a540cf9f0 2023-11-24T06:31:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.485+0000 7f0a59e57640 1 --2- 172.21.15.16:0/149737171 >> v2:172.21.15.173:3300/0 conn(0x7f0a54160920 0x7f0a540b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.486+0000 7f0a5a658640 1 --2- 172.21.15.16:0/149737171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cfea0 0x7f0a54161720 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0a5408c060 tx=0x7f0a4c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08cb1a3c795abd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.486+0000 7f0a58e55640 1 -- 172.21.15.16:0/149737171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c02cc70 con 0x7f0a540cfea0 2023-11-24T06:31:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a58e55640 1 -- 172.21.15.16:0/149737171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c02cdd0 con 0x7f0a540cfea0 2023-11-24T06:31:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a58e55640 1 -- 172.21.15.16:0/149737171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c034660 con 0x7f0a540cfea0 2023-11-24T06:31:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a5ae59640 1 -- 172.21.15.16:0/149737171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cfea0 msgr2=0x7f0a54161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a5ae59640 1 --2- 172.21.15.16:0/149737171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cfea0 0x7f0a54161720 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0a5408c060 tx=0x7f0a4c02bec0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a5ae59640 1 -- 172.21.15.16:0/149737171 shutdown_connections 2023-11-24T06:31:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a5ae59640 1 --2- 172.21.15.16:0/149737171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cfea0 0x7f0a54161720 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a5ae59640 1 --2- 172.21.15.16:0/149737171 >> v2:172.21.15.173:3300/0 conn(0x7f0a54160920 0x7f0a540b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a5ae59640 1 -- 172.21.15.16:0/149737171 >> 172.21.15.16:0/149737171 conn(0x7f0a540d3090 msgr2=0x7f0a540d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a5ae59640 1 -- 172.21.15.16:0/149737171 shutdown_connections 2023-11-24T06:31:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.487+0000 7f0a5ae59640 1 -- 172.21.15.16:0/149737171 wait complete. 2023-11-24T06:31:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.489+0000 7f0a5ae59640 1 Processor -- start 2023-11-24T06:31:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.489+0000 7f0a5ae59640 1 -- start start 2023-11-24T06:31:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.490+0000 7f0a5ae59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cf9f0 0x7f0a540cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.490+0000 7f0a5ae59640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a540cfea0 0x7f0a540cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.490+0000 7f0a5ae59640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a54160920 0x7f0a541594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.490+0000 7f0a5ae59640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540b5f50 con 0x7f0a540cf9f0 2023-11-24T06:31:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.490+0000 7f0a5ae59640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540cb6e0 con 0x7f0a54160920 2023-11-24T06:31:54.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.490+0000 7f0a5ae59640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540cb980 con 0x7f0a540cfea0 2023-11-24T06:31:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.490+0000 7f0a59e57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cf9f0 0x7f0a540cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.490+0000 7f0a59e57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cf9f0 0x7f0a540cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53808/0 (socket says 172.21.15.16:53808) 2023-11-24T06:31:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.490+0000 7f0a59e57640 1 -- 172.21.15.16:0/3350077028 learned_addr learned my addr 172.21.15.16:0/3350077028 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a59e57640 1 -- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a540cfea0 msgr2=0x7f0a540cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a59656640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a540cfea0 0x7f0a540cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a59e57640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a540cfea0 0x7f0a540cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a5a658640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a54160920 0x7f0a541594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a59e57640 1 -- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a54160920 msgr2=0x7f0a541594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a59e57640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a54160920 0x7f0a541594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a59e57640 1 -- 172.21.15.16:0/3350077028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c009a70 con 0x7f0a540cf9f0 2023-11-24T06:31:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a59656640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a540cfea0 0x7f0a540cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a59e57640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cf9f0 0x7f0a540cca20 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000c7f0 tx=0x7f0a5000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a4a7fc640 1 -- 172.21.15.16:0/3350077028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a50007a30 con 0x7f0a540cf9f0 2023-11-24T06:31:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a4a7fc640 1 -- 172.21.15.16:0/3350077028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a50007b90 con 0x7f0a540cf9f0 2023-11-24T06:31:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a4a7fc640 1 -- 172.21.15.16:0/3350077028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5000ce40 con 0x7f0a540cf9f0 2023-11-24T06:31:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.491+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a540cbc60 con 0x7f0a540cf9f0 2023-11-24T06:31:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.492+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a540cc220 con 0x7f0a540cf9f0 2023-11-24T06:31:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.493+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a54067450 con 0x7f0a540cf9f0 2023-11-24T06:31:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.493+0000 7f0a4a7fc640 1 -- 172.21.15.16:0/3350077028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f0a500113f0 con 0x7f0a540cf9f0 2023-11-24T06:31:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.494+0000 7f0a4a7fc640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a3806b4c0 0x7f0a3806d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.494+0000 7f0a4a7fc640 1 -- 172.21.15.16:0/3350077028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5766+0+0 (secure 0 0 0) 0x7f0a5008a2c0 con 0x7f0a540cf9f0 2023-11-24T06:31:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.495+0000 7f0a59656640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a3806b4c0 0x7f0a3806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.496+0000 7f0a59656640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a3806b4c0 0x7f0a3806d980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0a540cd160 tx=0x7f0a44009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:54.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.499+0000 7f0a4a7fc640 1 -- 172.21.15.16:0/3350077028 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a50054a70 con 0x7f0a540cf9f0 2023-11-24T06:31:54.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.731+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0a5408c7e0 con 0x7f0a540cf9f0 2023-11-24T06:31:54.733 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-11-24T06:31:54.731+0000 7f082ae01640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:31:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.735+0000 7f0a4a7fc640 1 -- 172.21.15.16:0/3350077028 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0a5001f090 con 0x7f0a540cf9f0 2023-11-24T06:31:54.737 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:31:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.738+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a3806b4c0 msgr2=0x7f0a3806d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.738+0000 7f0a5ae59640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a3806b4c0 0x7f0a3806d980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0a540cd160 tx=0x7f0a44009040 comp rx=0 tx=0).stop 2023-11-24T06:31:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.738+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cf9f0 msgr2=0x7f0a540cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.738+0000 7f0a5ae59640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cf9f0 0x7f0a540cca20 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f0a5000c7f0 tx=0x7f0a5000cbf0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.739+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 shutdown_connections 2023-11-24T06:31:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.739+0000 7f0a5ae59640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0a3806b4c0 0x7f0a3806d980 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.739+0000 7f0a5ae59640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a54160920 0x7f0a541594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.739+0000 7f0a5ae59640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a540cfea0 0x7f0a540cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.739+0000 7f0a5ae59640 1 --2- 172.21.15.16:0/3350077028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a540cf9f0 0x7f0a540cca20 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.739+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 >> 172.21.15.16:0/3350077028 conn(0x7f0a540d3090 msgr2=0x7f0a540b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.739+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 shutdown_connections 2023-11-24T06:31:54.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.739+0000 7f0a5ae59640 1 -- 172.21.15.16:0/3350077028 wait complete. 2023-11-24T06:31:54.751 INFO:tasks.ceph.mds.c.smithi016.stderr: -5344> 2023-11-24T06:31:27.731+0000 7fd9bd7d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:31:54.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:31:54.810 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-11-24T06:31:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.962+0000 7fb9d2bcb640 1 Processor -- start 2023-11-24T06:31:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.962+0000 7fb9d2bcb640 1 -- start start 2023-11-24T06:31:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.963+0000 7fb9d2bcb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc06d6d0 0x7fb9cc0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.964+0000 7fb9d2bcb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc0b1e30 con 0x7fb9cc1505b0 2023-11-24T06:31:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.964+0000 7fb9d2bcb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc0b20a0 con 0x7fb9cc15a690 2023-11-24T06:31:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.964+0000 7fb9d2bcb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0b2380 con 0x7fb9cc06d6d0 2023-11-24T06:31:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.964+0000 7fb9d1bc9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9cc1505b0 0x7fb9cc06cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41288/0 (socket says 172.21.15.16:41288) 2023-11-24T06:31:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.964+0000 7fb9d1bc9640 1 -- 172.21.15.16:0/175009138 learned_addr learned my addr 172.21.15.16:0/175009138 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:54.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.964+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927513808 0 0) 0x7fb9cc0b1e30 con 0x7fb9cc1505b0 2023-11-24T06:31:54.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.964+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9b8003440 con 0x7fb9cc1505b0 2023-11-24T06:31:54.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.964+0000 7fb9d13c8640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc06d6d0 0x7fb9cc0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2978259912 0 0) 0x7fb9b8003440 con 0x7fb9cc1505b0 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc06d6d0 msgr2=0x7fb9cc0b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc06d6d0 0x7fb9cc0b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 >> v1:172.21.15.173:6789/0 conn(0x7fb9cc15a690 legacy=0x7fb9cc0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc0b2660 con 0x7fb9cc1505b0 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fb9c8001670 con 0x7fb9cc1505b0 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 >> v1:172.21.15.16:6789/0 conn(0x7fb9cc1505b0 legacy=0x7fb9cc06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc06d6d0 0x7fb9b8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9b8004110 0x7fb9b800a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b800b0a0 0x7fb9b800d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0b2380 con 0x7fb9b800b0a0 2023-11-24T06:31:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b800dd70 con 0x7fb9b8004110 2023-11-24T06:31:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b800df30 con 0x7fb9cc06d6d0 2023-11-24T06:31:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d23ca640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b800b0a0 0x7fb9b800d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d13c8640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9b8004110 0x7fb9b800a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.965+0000 7fb9d1bc9640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc06d6d0 0x7fb9b8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.966+0000 7fb9d23ca640 1 -- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc06d6d0 msgr2=0x7fb9b8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.966+0000 7fb9d23ca640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc06d6d0 0x7fb9b8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.966+0000 7fb9d23ca640 1 -- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9b8004110 msgr2=0x7fb9b800a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.966+0000 7fb9d23ca640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9b8004110 0x7fb9b800a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.966+0000 7fb9d23ca640 1 -- 172.21.15.16:0/175009138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b8003440 con 0x7fb9b800b0a0 2023-11-24T06:31:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.967+0000 7fb9d23ca640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b800b0a0 0x7fb9b800d530 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4001e50 tx=0x7fb9b80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1aae7326fc9ae7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.967+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4003b70 con 0x7fb9b800b0a0 2023-11-24T06:31:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.967+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c4003cd0 con 0x7fb9b800b0a0 2023-11-24T06:31:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.967+0000 7fb9d0bc7640 1 -- 172.21.15.16:0/175009138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4005310 con 0x7fb9b800b0a0 2023-11-24T06:31:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.967+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/175009138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b800b0a0 msgr2=0x7fb9b800d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.967+0000 7fb9d2bcb640 1 --2- 172.21.15.16:0/175009138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9b800b0a0 0x7fb9b800d530 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4001e50 tx=0x7fb9b80036f0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.967+0000 7fb9d23ca640 1 -- 172.21.15.16:0/175009138 reap_dead start 2023-11-24T06:31:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.968+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/175009138 shutdown_connections 2023-11-24T06:31:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.968+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/175009138 >> 172.21.15.16:0/175009138 conn(0x7fb9cc06a7d0 msgr2=0x7fb9cc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.968+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/175009138 shutdown_connections 2023-11-24T06:31:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.968+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/175009138 wait complete. 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.970+0000 7fb9d2bcb640 1 Processor -- start 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.970+0000 7fb9d2bcb640 1 -- start start 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d2bcb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9b800b0a0 0x7fb9cc144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d2bcb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9cc14ab50 0x7fb9cc14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d2bcb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc15c4e0 0x7fb9cc15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d2bcb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc144d60 con 0x7fb9cc14ab50 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d2bcb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc15cea0 con 0x7fb9b800b0a0 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d2bcb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc14b8d0 con 0x7fb9cc15c4e0 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d13c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9cc14ab50 0x7fb9cc14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d13c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9cc14ab50 0x7fb9cc14afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53866/0 (socket says 172.21.15.16:53866) 2023-11-24T06:31:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d13c8640 1 -- 172.21.15.16:0/1951226439 learned_addr learned my addr 172.21.15.16:0/1951226439 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d13c8640 1 -- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc15c4e0 msgr2=0x7fb9cc15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d13c8640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc15c4e0 0x7fb9cc15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d1bc9640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9b800b0a0 0x7fb9cc144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d13c8640 1 -- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9b800b0a0 msgr2=0x7fb9cc144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d13c8640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9b800b0a0 0x7fb9cc144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.971+0000 7fb9d13c8640 1 -- 172.21.15.16:0/1951226439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc14b5f0 con 0x7fb9cc14ab50 2023-11-24T06:31:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.972+0000 7fb9d13c8640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9cc14ab50 0x7fb9cc14afd0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00f530 tx=0x7fb9bc00f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.972+0000 7fb9c27fc640 1 -- 172.21.15.16:0/1951226439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc014870 con 0x7fb9cc14ab50 2023-11-24T06:31:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.972+0000 7fb9c27fc640 1 -- 172.21.15.16:0/1951226439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc0149d0 con 0x7fb9cc14ab50 2023-11-24T06:31:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.972+0000 7fb9c27fc640 1 -- 172.21.15.16:0/1951226439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc014d80 con 0x7fb9cc14ab50 2023-11-24T06:31:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.972+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc0c8240 con 0x7fb9cc14ab50 2023-11-24T06:31:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.972+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc0c8800 con 0x7fb9cc14ab50 2023-11-24T06:31:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.973+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc067450 con 0x7fb9cc14ab50 2023-11-24T06:31:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.974+0000 7fb9c27fc640 1 -- 172.21.15.16:0/1951226439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fb9bc015410 con 0x7fb9cc14ab50 2023-11-24T06:31:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.975+0000 7fb9c27fc640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9b006b470 0x7fb9b006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.975+0000 7fb9c27fc640 1 -- 172.21.15.16:0/1951226439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fb9bc08ab40 con 0x7fb9cc14ab50 2023-11-24T06:31:54.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.976+0000 7fb9d1bc9640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9b006b470 0x7fb9b006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:54.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.977+0000 7fb9d1bc9640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9b006b470 0x7fb9b006d930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8001eb0 tx=0x7fb9c8000f50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:54.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:54.979+0000 7fb9c27fc640 1 -- 172.21.15.16:0/1951226439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9bc055270 con 0x7fb9cc14ab50 2023-11-24T06:31:55.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.211+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb9cc0b3660 con 0x7fb9cc14ab50 2023-11-24T06:31:55.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.738+0000 7fb9c27fc640 1 -- 172.21.15.16:0/1951226439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb9bc058890 con 0x7fb9cc14ab50 2023-11-24T06:31:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.741+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9b006b470 msgr2=0x7fb9b006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.741+0000 7fb9d2bcb640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9b006b470 0x7fb9b006d930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8001eb0 tx=0x7fb9c8000f50 comp rx=0 tx=0).stop 2023-11-24T06:31:55.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.741+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9cc14ab50 msgr2=0x7fb9cc14afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.741+0000 7fb9d2bcb640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9cc14ab50 0x7fb9cc14afd0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00f530 tx=0x7fb9bc00f9f0 comp rx=0 tx=0).stop 2023-11-24T06:31:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.742+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 shutdown_connections 2023-11-24T06:31:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.742+0000 7fb9d2bcb640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fb9b006b470 0x7fb9b006d930 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.742+0000 7fb9d2bcb640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc15c4e0 0x7fb9cc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:55.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.742+0000 7fb9d2bcb640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9cc14ab50 0x7fb9cc14afd0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:55.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.742+0000 7fb9d2bcb640 1 --2- 172.21.15.16:0/1951226439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9b800b0a0 0x7fb9cc144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:55.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.742+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 >> 172.21.15.16:0/1951226439 conn(0x7fb9cc06a7d0 msgr2=0x7fb9cc06ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:55.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.742+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 shutdown_connections 2023-11-24T06:31:55.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:55.742+0000 7fb9d2bcb640 1 -- 172.21.15.16:0/1951226439 wait complete. 2023-11-24T06:31:55.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:31:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.036+0000 7fefed839640 1 Processor -- start 2023-11-24T06:31:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.037+0000 7fefed839640 1 -- start start 2023-11-24T06:31:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.038+0000 7fefed839640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fefe81447f0 0x7fefe815c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.038+0000 7fefed839640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fefe806c470 0x7fefe80cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.038+0000 7fefed839640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe815c5c0 con 0x7fefe806dab0 2023-11-24T06:31:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.038+0000 7fefed839640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe806d690 con 0x7fefe81447f0 2023-11-24T06:31:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.038+0000 7fefed839640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe80cebb0 con 0x7fefe806c470 2023-11-24T06:31:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.039+0000 7fefec837640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fefe81447f0 0x7fefe815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.039+0000 7fefec837640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fefe81447f0 0x7fefe815c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53878/0 (socket says 172.21.15.16:53878) 2023-11-24T06:31:56.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.039+0000 7fefec837640 1 -- 172.21.15.16:0/737263697 learned_addr learned my addr 172.21.15.16:0/737263697 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.039+0000 7fefec837640 1 -- 172.21.15.16:0/737263697 >> v2:172.21.15.173:3300/0 conn(0x7fefe806c470 msgr2=0x7fefe80cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.039+0000 7fefec837640 1 --2- 172.21.15.16:0/737263697 >> v2:172.21.15.173:3300/0 conn(0x7fefe806c470 0x7fefe80cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.039+0000 7fefec837640 1 -- 172.21.15.16:0/737263697 >> v1:172.21.15.173:6789/0 conn(0x7fefe806dab0 legacy=0x7fefe815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.039+0000 7fefec837640 1 -- 172.21.15.16:0/737263697 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe806ca10 con 0x7fefe81447f0 2023-11-24T06:31:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.040+0000 7fefec837640 1 --2- 172.21.15.16:0/737263697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe81447f0 0x7fefe815c080 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fefe80cecf0 tx=0x7fefdc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd383f717abc3a97 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.041+0000 7fefe77fe640 1 -- 172.21.15.16:0/737263697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc0317e0 con 0x7fefe81447f0 2023-11-24T06:31:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.041+0000 7fefe77fe640 1 -- 172.21.15.16:0/737263697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc031940 con 0x7fefe81447f0 2023-11-24T06:31:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.041+0000 7fefe77fe640 1 -- 172.21.15.16:0/737263697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc031cd0 con 0x7fefe81447f0 2023-11-24T06:31:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.041+0000 7fefed839640 1 -- 172.21.15.16:0/737263697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe81447f0 msgr2=0x7fefe815c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.041+0000 7fefed839640 1 --2- 172.21.15.16:0/737263697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe81447f0 0x7fefe815c080 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fefe80cecf0 tx=0x7fefdc02b880 comp rx=0 tx=0).stop 2023-11-24T06:31:56.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.042+0000 7fefed839640 1 -- 172.21.15.16:0/737263697 shutdown_connections 2023-11-24T06:31:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.042+0000 7fefed839640 1 --2- 172.21.15.16:0/737263697 >> v2:172.21.15.173:3300/0 conn(0x7fefe806c470 0x7fefe80cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.042+0000 7fefed839640 1 --2- 172.21.15.16:0/737263697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe81447f0 0x7fefe815c080 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.042+0000 7fefed839640 1 -- 172.21.15.16:0/737263697 >> 172.21.15.16:0/737263697 conn(0x7fefe806a3e0 msgr2=0x7fefe806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.042+0000 7fefed839640 1 -- 172.21.15.16:0/737263697 shutdown_connections 2023-11-24T06:31:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.042+0000 7fefed839640 1 -- 172.21.15.16:0/737263697 wait complete. 2023-11-24T06:31:56.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.043+0000 7fefed839640 1 Processor -- start 2023-11-24T06:31:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.044+0000 7fefed839640 1 -- start start 2023-11-24T06:31:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefed839640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe806c470 0x7fefe81598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefed839640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe806dab0 0x7fefe81f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefed839640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe81447f0 0x7fefe815b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefed839640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe81f4f50 con 0x7fefe806dab0 2023-11-24T06:31:56.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefed839640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80a9190 con 0x7fefe806c470 2023-11-24T06:31:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefed839640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe813bb80 con 0x7fefe81447f0 2023-11-24T06:31:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe806dab0 0x7fefe81f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe806dab0 0x7fefe81f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53884/0 (socket says 172.21.15.16:53884) 2023-11-24T06:31:56.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefe7fff640 1 -- 172.21.15.16:0/102156639 learned_addr learned my addr 172.21.15.16:0/102156639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:56.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefec837640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe806c470 0x7fefe81598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:56.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefe7fff640 1 -- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe81447f0 msgr2=0x7fefe815b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:56.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefe7fff640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe81447f0 0x7fefe815b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:56.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefe7fff640 1 -- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe806c470 msgr2=0x7fefe81598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.045+0000 7fefe7fff640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe806c470 0x7fefe81598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.046+0000 7fefe7fff640 1 -- 172.21.15.16:0/102156639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc0094d0 con 0x7fefe806dab0 2023-11-24T06:31:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.046+0000 7fefec837640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe806c470 0x7fefe81598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.046+0000 7fefe7fff640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe806dab0 0x7fefe81f4a10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fefd400b4a0 tx=0x7fefd400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:56.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.046+0000 7fefe57fa640 1 -- 172.21.15.16:0/102156639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4010610 con 0x7fefe806dab0 2023-11-24T06:31:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.046+0000 7fefe57fa640 1 -- 172.21.15.16:0/102156639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4010770 con 0x7fefe806dab0 2023-11-24T06:31:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.046+0000 7fefe57fa640 1 -- 172.21.15.16:0/102156639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4010b20 con 0x7fefe806dab0 2023-11-24T06:31:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.046+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe813be60 con 0x7fefe806dab0 2023-11-24T06:31:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.046+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe813c420 con 0x7fefe806dab0 2023-11-24T06:31:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.047+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe8067450 con 0x7fefe806dab0 2023-11-24T06:31:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.049+0000 7fefe57fa640 1 -- 172.21.15.16:0/102156639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fefd4010c80 con 0x7fefe806dab0 2023-11-24T06:31:56.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.049+0000 7fefe57fa640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefcc06b470 0x7fefcc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.050+0000 7fefe57fa640 1 -- 172.21.15.16:0/102156639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fefd4089df0 con 0x7fefe806dab0 2023-11-24T06:31:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.050+0000 7fefec837640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefcc06b470 0x7fefcc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:56.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.051+0000 7fefec837640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefcc06b470 0x7fefcc06d930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fefdc036000 tx=0x7fefdc03d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:56.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.053+0000 7fefe57fa640 1 -- 172.21.15.16:0/102156639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fefd4054520 con 0x7fefe806dab0 2023-11-24T06:31:56.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.242+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fefe80b3fe0 con 0x7fefe806dab0 2023-11-24T06:31:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.751+0000 7fefe57fa640 1 -- 172.21.15.16:0/102156639 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 181+0+0 (secure 0 0 0) 0x7fefd401e0d0 con 0x7fefe806dab0 2023-11-24T06:31:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:31:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.754+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefcc06b470 msgr2=0x7fefcc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.754+0000 7fefed839640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefcc06b470 0x7fefcc06d930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fefdc036000 tx=0x7fefdc03d040 comp rx=0 tx=0).stop 2023-11-24T06:31:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.754+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe806dab0 msgr2=0x7fefe81f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.754+0000 7fefed839640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe806dab0 0x7fefe81f4a10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fefd400b4a0 tx=0x7fefd400b960 comp rx=0 tx=0).stop 2023-11-24T06:31:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.758+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 shutdown_connections 2023-11-24T06:31:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.758+0000 7fefed839640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fefcc06b470 0x7fefcc06d930 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.758+0000 7fefed839640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fefe81447f0 0x7fefe815b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.758+0000 7fefed839640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe806dab0 0x7fefe81f4a10 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.758+0000 7fefed839640 1 --2- 172.21.15.16:0/102156639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefe806c470 0x7fefe81598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.758+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 >> 172.21.15.16:0/102156639 conn(0x7fefe806a3e0 msgr2=0x7fefe80b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.758+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 shutdown_connections 2023-11-24T06:31:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:56.759+0000 7fefed839640 1 -- 172.21.15.16:0/102156639 wait complete. 2023-11-24T06:31:56.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:31:57.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.004+0000 7f5a688d1640 1 Processor -- start 2023-11-24T06:31:57.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.004+0000 7f5a688d1640 1 -- start start 2023-11-24T06:31:57.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.006+0000 7f5a688d1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a6413fcb0 0x7f5a64164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:57.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.006+0000 7f5a688d1640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a640cdc40 con 0x7f5a6413cfa0 2023-11-24T06:31:57.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.006+0000 7f5a688d1640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a640cdec0 con 0x7f5a6415afb0 2023-11-24T06:31:57.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.006+0000 7f5a688d1640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640ce1a0 con 0x7f5a6413fcb0 2023-11-24T06:31:57.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.006+0000 7f5a637fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a6413cfa0 0x7f5a640cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41302/0 (socket says 172.21.15.16:41302) 2023-11-24T06:31:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.006+0000 7f5a637fe640 1 -- 172.21.15.16:0/438903243 learned_addr learned my addr 172.21.15.16:0/438903243 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.006+0000 7f5a63fff640 1 --2- 172.21.15.16:0/438903243 >> v2:172.21.15.16:3300/0 conn(0x7f5a6413fcb0 0x7f5a64164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.007+0000 7f5a627fc640 1 -- 172.21.15.16:0/438903243 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051830249 0 0) 0x7f5a640cdc40 con 0x7f5a6413cfa0 2023-11-24T06:31:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.007+0000 7f5a627fc640 1 -- 172.21.15.16:0/438903243 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a50003520 con 0x7f5a6413cfa0 2023-11-24T06:31:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.007+0000 7f5a63fff640 1 -- 172.21.15.16:0/438903243 >> v1:172.21.15.173:6789/0 conn(0x7f5a6415afb0 legacy=0x7f5a6413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.007+0000 7f5a63fff640 1 -- 172.21.15.16:0/438903243 >> v1:172.21.15.16:6789/0 conn(0x7f5a6413cfa0 legacy=0x7f5a640cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.007+0000 7f5a63fff640 1 -- 172.21.15.16:0/438903243 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640ce480 con 0x7f5a6413fcb0 2023-11-24T06:31:57.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.008+0000 7f5a63fff640 1 --2- 172.21.15.16:0/438903243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 0x7f5a64164d80 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0097a0 tx=0x7f5a5c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb814e93be99f4d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.008+0000 7f5a627fc640 1 -- 172.21.15.16:0/438903243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c02e070 con 0x7f5a6413fcb0 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.008+0000 7f5a627fc640 1 -- 172.21.15.16:0/438903243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c0315a0 con 0x7f5a6413fcb0 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.008+0000 7f5a627fc640 1 -- 172.21.15.16:0/438903243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c031930 con 0x7f5a6413fcb0 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.009+0000 7f5a688d1640 1 -- 172.21.15.16:0/438903243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 msgr2=0x7f5a64164d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.009+0000 7f5a688d1640 1 --2- 172.21.15.16:0/438903243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 0x7f5a64164d80 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0097a0 tx=0x7f5a5c02b840 comp rx=0 tx=0).stop 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.009+0000 7f5a688d1640 1 -- 172.21.15.16:0/438903243 shutdown_connections 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.009+0000 7f5a688d1640 1 --2- 172.21.15.16:0/438903243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 0x7f5a64164d80 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.009+0000 7f5a688d1640 1 -- 172.21.15.16:0/438903243 >> 172.21.15.16:0/438903243 conn(0x7f5a6406a800 msgr2=0x7f5a6406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.009+0000 7f5a688d1640 1 -- 172.21.15.16:0/438903243 shutdown_connections 2023-11-24T06:31:57.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.009+0000 7f5a688d1640 1 -- 172.21.15.16:0/438903243 wait complete. 2023-11-24T06:31:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.011+0000 7f5a688d1640 1 Processor -- start 2023-11-24T06:31:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.011+0000 7f5a688d1640 1 -- start start 2023-11-24T06:31:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.012+0000 7f5a688d1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a6413cfa0 0x7f5a6413a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.012+0000 7f5a688d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 0x7f5a64153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.012+0000 7f5a688d1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a6415afb0 0x7f5a6413de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.012+0000 7f5a688d1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6413eeb0 con 0x7f5a6413fcb0 2023-11-24T06:31:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.012+0000 7f5a688d1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64154080 con 0x7f5a6415afb0 2023-11-24T06:31:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.012+0000 7f5a688d1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6413e390 con 0x7f5a6413cfa0 2023-11-24T06:31:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.012+0000 7f5a62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 0x7f5a64153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.012+0000 7f5a62ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 0x7f5a64153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53888/0 (socket says 172.21.15.16:53888) 2023-11-24T06:31:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.012+0000 7f5a62ffd640 1 -- 172.21.15.16:0/1682353166 learned_addr learned my addr 172.21.15.16:0/1682353166 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a63fff640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a6415afb0 0x7f5a6413de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a637fe640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a6413cfa0 0x7f5a6413a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a62ffd640 1 -- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a6413cfa0 msgr2=0x7f5a6413a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a62ffd640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a6413cfa0 0x7f5a6413a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a62ffd640 1 -- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a6415afb0 msgr2=0x7f5a6413de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a62ffd640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a6415afb0 0x7f5a6413de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a62ffd640 1 -- 172.21.15.16:0/1682353166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c009400 con 0x7f5a6413fcb0 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a63fff640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a6415afb0 0x7f5a6413de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a62ffd640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 0x7f5a64153b40 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00b3f0 tx=0x7f5a4c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a5bfff640 1 -- 172.21.15.16:0/1682353166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c011830 con 0x7f5a6413fcb0 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a5bfff640 1 -- 172.21.15.16:0/1682353166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c011990 con 0x7f5a6413fcb0 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6413e610 con 0x7f5a6413fcb0 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a5bfff640 1 -- 172.21.15.16:0/1682353166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c011d40 con 0x7f5a6413fcb0 2023-11-24T06:31:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.013+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6413ebd0 con 0x7f5a6413fcb0 2023-11-24T06:31:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.014+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a64067450 con 0x7f5a6413fcb0 2023-11-24T06:31:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.016+0000 7f5a5bfff640 1 -- 172.21.15.16:0/1682353166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f5a4c01a400 con 0x7f5a6413fcb0 2023-11-24T06:31:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.016+0000 7f5a5bfff640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5a4406b470 0x7f5a4406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.017+0000 7f5a5bfff640 1 -- 172.21.15.16:0/1682353166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5414+0+0 (secure 0 0 0) 0x7f5a4c089e60 con 0x7f5a6413fcb0 2023-11-24T06:31:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.017+0000 7f5a637fe640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5a4406b470 0x7f5a4406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.018+0000 7f5a637fe640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5a4406b470 0x7f5a4406d930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5a54001560 tx=0x7f5a54009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.021+0000 7f5a5bfff640 1 -- 172.21.15.16:0/1682353166 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a4c054740 con 0x7f5a6413fcb0 2023-11-24T06:31:57.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:57.216+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a6415a910 con 0x7f5a6413fcb0 2023-11-24T06:31:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.053+0000 7f5a5bfff640 1 -- 172.21.15.16:0/1682353166 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5a4c0150d0 con 0x7f5a6413fcb0 2023-11-24T06:31:58.056 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-11-24T06:31:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5a4406b470 msgr2=0x7f5a4406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5a4406b470 0x7f5a4406d930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5a54001560 tx=0x7f5a54009040 comp rx=0 tx=0).stop 2023-11-24T06:31:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 msgr2=0x7f5a64153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 0x7f5a64153b40 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00b3f0 tx=0x7f5a4c00b8b0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 shutdown_connections 2023-11-24T06:31:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f5a4406b470 0x7f5a4406d930 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a6415afb0 0x7f5a6413de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6413fcb0 0x7f5a64153b40 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 --2- 172.21.15.16:0/1682353166 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a6413cfa0 0x7f5a6413a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.057+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 >> 172.21.15.16:0/1682353166 conn(0x7f5a6406a800 msgr2=0x7f5a6413c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.058+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 shutdown_connections 2023-11-24T06:31:58.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.058+0000 7f5a688d1640 1 -- 172.21.15.16:0/1682353166 wait complete. 2023-11-24T06:31:58.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:31:58.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.283+0000 7f499359e640 1 Processor -- start 2023-11-24T06:31:58.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.284+0000 7f499359e640 1 -- start start 2023-11-24T06:31:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.285+0000 7f499359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49940adf20 0x7f4994070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.285+0000 7f499359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499415f4e0 con 0x7f499415bf10 2023-11-24T06:31:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.285+0000 7f499359e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4994071130 con 0x7f49940ccb50 2023-11-24T06:31:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.285+0000 7f499359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499415f970 con 0x7f49940adf20 2023-11-24T06:31:58.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.285+0000 7f498bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f499415bf10 0x7f49940cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41316/0 (socket says 172.21.15.16:41316) 2023-11-24T06:31:58.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.285+0000 7f498bfff640 1 -- 172.21.15.16:0/3994061210 learned_addr learned my addr 172.21.15.16:0/3994061210 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f499259c640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49940adf20 0x7f4994070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798955260 0 0) 0x7f499415f4e0 con 0x7f499415bf10 2023-11-24T06:31:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4980003520 con 0x7f499415bf10 2023-11-24T06:31:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2554497532 0 0) 0x7f4980003520 con 0x7f499415bf10 2023-11-24T06:31:58.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49940adf20 msgr2=0x7f4994070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49940adf20 0x7f4994070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 >> v1:172.21.15.173:6789/0 conn(0x7f49940ccb50 legacy=0x7f4994143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499415c390 con 0x7f499415bf10 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f497c001670 con 0x7f499415bf10 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 >> v1:172.21.15.16:6789/0 conn(0x7f499415bf10 legacy=0x7f49940cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49800039a0 0x7f4980003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49940adf20 0x7f49800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4980004d00 0x7f4980005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499415f970 con 0x7f49800039a0 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498000bab0 con 0x7f4980004d00 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.286+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498000bc70 con 0x7f49940adf20 2023-11-24T06:31:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.287+0000 7f499259c640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49800039a0 0x7f4980003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.287+0000 7f498bfff640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49940adf20 0x7f49800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.287+0000 7f499259c640 1 -- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49940adf20 msgr2=0x7f49800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:58.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.287+0000 7f499259c640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49940adf20 0x7f49800047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.287+0000 7f499259c640 1 -- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4980004d00 msgr2=0x7f4980005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:58.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.287+0000 7f499259c640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4980004d00 0x7f4980005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.287+0000 7f499259c640 1 -- 172.21.15.16:0/3994061210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980003520 con 0x7f49800039a0 2023-11-24T06:31:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.288+0000 7f499259c640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49800039a0 0x7f4980003dc0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4984009ac0 tx=0x7f49800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71b18740b5bdaaa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.288+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498402c600 con 0x7f49800039a0 2023-11-24T06:31:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.289+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498402c760 con 0x7f49800039a0 2023-11-24T06:31:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.289+0000 7f4991d9b640 1 -- 172.21.15.16:0/3994061210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498402caf0 con 0x7f49800039a0 2023-11-24T06:31:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.289+0000 7f499359e640 1 -- 172.21.15.16:0/3994061210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49800039a0 msgr2=0x7f4980003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.289+0000 7f499359e640 1 --2- 172.21.15.16:0/3994061210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49800039a0 0x7f4980003dc0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4984009ac0 tx=0x7f49800057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.289+0000 7f4992d9d640 1 -- 172.21.15.16:0/3994061210 reap_dead start 2023-11-24T06:31:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.289+0000 7f499359e640 1 -- 172.21.15.16:0/3994061210 shutdown_connections 2023-11-24T06:31:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.289+0000 7f499359e640 1 -- 172.21.15.16:0/3994061210 >> 172.21.15.16:0/3994061210 conn(0x7f499406a7d0 msgr2=0x7f499406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.289+0000 7f499359e640 1 -- 172.21.15.16:0/3994061210 shutdown_connections 2023-11-24T06:31:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.289+0000 7f499359e640 1 -- 172.21.15.16:0/3994061210 wait complete. 2023-11-24T06:31:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.291+0000 7f499359e640 1 Processor -- start 2023-11-24T06:31:58.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.291+0000 7f499359e640 1 -- start start 2023-11-24T06:31:58.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.292+0000 7f499359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49800039a0 0x7f49941ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.292+0000 7f499359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49941ef860 0x7f49941edf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.292+0000 7f499359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f499413bd20 0x7f499413fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.292+0000 7f499359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49941f0be0 con 0x7f49941ef860 2023-11-24T06:31:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.292+0000 7f499359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49941effd0 con 0x7f49800039a0 2023-11-24T06:31:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.292+0000 7f499359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49941f02b0 con 0x7f499413bd20 2023-11-24T06:31:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f498bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49941ef860 0x7f49941edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f498bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49941ef860 0x7f49941edf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53902/0 (socket says 172.21.15.16:53902) 2023-11-24T06:31:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f498bfff640 1 -- 172.21.15.16:0/2918535751 learned_addr learned my addr 172.21.15.16:0/2918535751 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f499259c640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49800039a0 0x7f49941ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f4992d9d640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f499413bd20 0x7f499413fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f498bfff640 1 -- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f499413bd20 msgr2=0x7f499413fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f498bfff640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f499413bd20 0x7f499413fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f498bfff640 1 -- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49800039a0 msgr2=0x7f49941ef1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f498bfff640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49800039a0 0x7f49941ef1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f498bfff640 1 -- 172.21.15.16:0/2918535751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49941f0590 con 0x7f49941ef860 2023-11-24T06:31:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.293+0000 7f498bfff640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49941ef860 0x7f49941edf60 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d800 tx=0x7f497c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.294+0000 7f498affd640 1 -- 172.21.15.16:0/2918535751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c0074e0 con 0x7f49941ef860 2023-11-24T06:31:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.294+0000 7f498affd640 1 -- 172.21.15.16:0/2918535751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c007640 con 0x7f49941ef860 2023-11-24T06:31:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.294+0000 7f498affd640 1 -- 172.21.15.16:0/2918535751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c004a00 con 0x7f49941ef860 2023-11-24T06:31:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.294+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49941f0870 con 0x7f49941ef860 2023-11-24T06:31:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.294+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49941ee4a0 con 0x7f49941ef860 2023-11-24T06:31:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.295+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49941f0a00 con 0x7f49941ef860 2023-11-24T06:31:58.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.296+0000 7f498affd640 1 -- 172.21.15.16:0/2918535751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f497c004b60 con 0x7f49941ef860 2023-11-24T06:31:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.297+0000 7f498affd640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497406b430 0x7f497406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.297+0000 7f498affd640 1 -- 172.21.15.16:0/2918535751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f497c089950 con 0x7f49941ef860 2023-11-24T06:31:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.298+0000 7f499259c640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497406b430 0x7f497406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.299+0000 7f499259c640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497406b430 0x7f497406d8f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4984036000 tx=0x7f4984030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.302+0000 7f498affd640 1 -- 172.21.15.16:0/2918535751 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f497c054250 con 0x7f49941ef860 2023-11-24T06:31:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.495+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4994067450 con 0x7f49941ef860 2023-11-24T06:31:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.502+0000 7f498affd640 1 -- 172.21.15.16:0/2918535751 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f497c057870 con 0x7f49941ef860 2023-11-24T06:31:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497406b430 msgr2=0x7f497406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497406b430 0x7f497406d8f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4984036000 tx=0x7f4984030000 comp rx=0 tx=0).stop 2023-11-24T06:31:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49941ef860 msgr2=0x7f49941edf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49941ef860 0x7f49941edf60 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d800 tx=0x7f497c00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 shutdown_connections 2023-11-24T06:31:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f497406b430 0x7f497406d8f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f499413bd20 0x7f499413fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49941ef860 0x7f49941edf60 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 --2- 172.21.15.16:0/2918535751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49800039a0 0x7f49941ef1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.505+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 >> 172.21.15.16:0/2918535751 conn(0x7f499406a7d0 msgr2=0x7f49940cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:58.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.506+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 shutdown_connections 2023-11-24T06:31:58.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.506+0000 7f499359e640 1 -- 172.21.15.16:0/2918535751 wait complete. 2023-11-24T06:31:58.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T06:31:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.756+0000 7f8845101640 1 Processor -- start 2023-11-24T06:31:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.756+0000 7f8845101640 1 -- start start 2023-11-24T06:31:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.757+0000 7f8845101640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f88400cf9a0 0x7f884015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.757+0000 7f8845101640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88400b1ee0 con 0x7f884015e1e0 2023-11-24T06:31:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.757+0000 7f8845101640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88400b2780 con 0x7f8840143ae0 2023-11-24T06:31:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.757+0000 7f8845101640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884015e990 con 0x7f88400cf9a0 2023-11-24T06:31:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.758+0000 7f883f59e640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f884015e1e0 0x7f88400702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41324/0 (socket says 172.21.15.16:41324) 2023-11-24T06:31:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.758+0000 7f883f59e640 1 -- 172.21.15.16:0/707013429 learned_addr learned my addr 172.21.15.16:0/707013429 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:31:58.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.758+0000 7f883659c640 1 --2- 172.21.15.16:0/707013429 >> v2:172.21.15.173:3300/0 conn(0x7f88400cf9a0 0x7f884015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.758+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613690391 0 0) 0x7f88400b1ee0 con 0x7f884015e1e0 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.758+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f882c003520 con 0x7f884015e1e0 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 715211894 0 0) 0x7f882c003520 con 0x7f884015e1e0 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 >> v2:172.21.15.173:3300/0 conn(0x7f88400cf9a0 msgr2=0x7f884015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 --2- 172.21.15.16:0/707013429 >> v2:172.21.15.173:3300/0 conn(0x7f88400cf9a0 0x7f884015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 >> v1:172.21.15.173:6789/0 conn(0x7f8840143ae0 legacy=0x7f88400cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400b2130 con 0x7f884015e1e0 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f8838001670 con 0x7f884015e1e0 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 >> v1:172.21.15.16:6789/0 conn(0x7f884015e1e0 legacy=0x7f88400702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f882c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f882c0047c0 0x7f882c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f882c005180 0x7f882c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884015e990 con 0x7f882c003970 2023-11-24T06:31:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c00c020 con 0x7f882c005180 2023-11-24T06:31:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c00c1e0 con 0x7f882c0047c0 2023-11-24T06:31:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883ed9d640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f882c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883f59e640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f882c005180 0x7f882c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.759+0000 7f883659c640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f882c0047c0 0x7f882c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.760+0000 7f883ed9d640 1 -- 172.21.15.16:0/707013429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f882c0047c0 msgr2=0x7f882c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.760+0000 7f883ed9d640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f882c0047c0 0x7f882c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.760+0000 7f883ed9d640 1 -- 172.21.15.16:0/707013429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f882c005180 msgr2=0x7f882c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:31:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.760+0000 7f883ed9d640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f882c005180 0x7f882c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.760+0000 7f883f59e640 1 -- 172.21.15.16:0/707013429 reap_dead start 2023-11-24T06:31:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.760+0000 7f883ed9d640 1 -- 172.21.15.16:0/707013429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c003520 con 0x7f882c003970 2023-11-24T06:31:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.761+0000 7f883ed9d640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f882c003d90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8830001a30 tx=0x7f882c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dca9c5c0a965184 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830003b70 con 0x7f882c003970 2023-11-24T06:31:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8830003cd0 con 0x7f882c003970 2023-11-24T06:31:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f883e59c640 1 -- 172.21.15.16:0/707013429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830030a20 con 0x7f882c003970 2023-11-24T06:31:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f8845101640 1 -- 172.21.15.16:0/707013429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 msgr2=0x7f882c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f8845101640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f882c003d90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8830001a30 tx=0x7f882c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f8845101640 1 -- 172.21.15.16:0/707013429 shutdown_connections 2023-11-24T06:31:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f8845101640 1 --2- 172.21.15.16:0/707013429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f882c003d90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f8845101640 1 -- 172.21.15.16:0/707013429 >> 172.21.15.16:0/707013429 conn(0x7f884006a800 msgr2=0x7f884006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f8845101640 1 -- 172.21.15.16:0/707013429 shutdown_connections 2023-11-24T06:31:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.762+0000 7f8845101640 1 -- 172.21.15.16:0/707013429 wait complete. 2023-11-24T06:31:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.764+0000 7f8845101640 1 Processor -- start 2023-11-24T06:31:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.764+0000 7f8845101640 1 -- start start 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.765+0000 7f8845101640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f88400cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.765+0000 7f8845101640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f884015c240 0x7f88400a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.765+0000 7f8845101640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f884015a4e0 0x7f884015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.765+0000 7f8845101640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884015d350 con 0x7f882c003970 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.765+0000 7f8845101640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884015c740 con 0x7f884015c240 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.765+0000 7f8845101640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884015ca20 con 0x7f884015a4e0 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.765+0000 7f883ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f88400cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f88400cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53932/0 (socket says 172.21.15.16:53932) 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883ed9d640 1 -- 172.21.15.16:0/3181884477 learned_addr learned my addr 172.21.15.16:0/3181884477 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:31:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883659c640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f884015c240 0x7f88400a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883f59e640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f884015a4e0 0x7f884015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883ed9d640 1 -- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f884015a4e0 msgr2=0x7f884015a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883ed9d640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f884015a4e0 0x7f884015a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883ed9d640 1 -- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f884015c240 msgr2=0x7f88400a8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883ed9d640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f884015c240 0x7f88400a8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883ed9d640 1 -- 172.21.15.16:0/3181884477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884015cd00 con 0x7f882c003970 2023-11-24T06:31:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.766+0000 7f883ed9d640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f88400cdad0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8830030450 tx=0x7f88300014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.767+0000 7f8837fff640 1 -- 172.21.15.16:0/3181884477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830031e20 con 0x7f882c003970 2023-11-24T06:31:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.767+0000 7f8837fff640 1 -- 172.21.15.16:0/3181884477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8830007db0 con 0x7f882c003970 2023-11-24T06:31:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.767+0000 7f8837fff640 1 -- 172.21.15.16:0/3181884477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300052d0 con 0x7f882c003970 2023-11-24T06:31:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.767+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884015cfe0 con 0x7f882c003970 2023-11-24T06:31:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.767+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88400ce1f0 con 0x7f882c003970 2023-11-24T06:31:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.768+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88400675d0 con 0x7f882c003970 2023-11-24T06:31:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.769+0000 7f8837fff640 1 -- 172.21.15.16:0/3181884477 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8830005430 con 0x7f882c003970 2023-11-24T06:31:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.770+0000 7f8837fff640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f882006b4a0 0x7f882006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:31:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.770+0000 7f8837fff640 1 -- 172.21.15.16:0/3181884477 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f88300af9c0 con 0x7f882c003970 2023-11-24T06:31:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.770+0000 7f883659c640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f882006b4a0 0x7f882006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:31:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.771+0000 7f883659c640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f882006b4a0 0x7f882006d960 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f88400cf510 tx=0x7f8828009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:31:58.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:58.774+0000 7f8837fff640 1 -- 172.21.15.16:0/3181884477 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f883007a410 con 0x7f882c003970 2023-11-24T06:31:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.037+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7f88400ce1f0 con 0x7f882c003970 2023-11-24T06:31:59.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.066+0000 7f8837fff640 1 -- 172.21.15.16:0/3181884477 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v300) v1 ==== 164+0+0 (secure 0 0 0) 0x7f88400ce1f0 con 0x7f882c003970 2023-11-24T06:31:59.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f882006b4a0 msgr2=0x7f882006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f882006b4a0 0x7f882006d960 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f88400cf510 tx=0x7f8828009190 comp rx=0 tx=0).stop 2023-11-24T06:31:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 msgr2=0x7f88400cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:31:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f88400cdad0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8830030450 tx=0x7f88300014a0 comp rx=0 tx=0).stop 2023-11-24T06:31:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 shutdown_connections 2023-11-24T06:31:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f882006b4a0 0x7f882006d960 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f884015a4e0 0x7f884015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f884015c240 0x7f88400a8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 --2- 172.21.15.16:0/3181884477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882c003970 0x7f88400cdad0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:31:59.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 >> 172.21.15.16:0/3181884477 conn(0x7f884006a800 msgr2=0x7f88400af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:31:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 shutdown_connections 2023-11-24T06:31:59.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:31:59.069+0000 7f8845101640 1 -- 172.21.15.16:0/3181884477 wait complete. 2023-11-24T06:31:59.094 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ERROR 2023-11-24T06:31:59.094 INFO:tasks.cephfs_test_runner: 2023-11-24T06:31:59.094 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 4537, in test_subvolume_inherited_snapshot_ls 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner: 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner:Ran 10 tests in 260.236s 2023-11-24T06:31:59.095 INFO:tasks.cephfs_test_runner: 2023-11-24T06:31:59.097 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=5) 2023-11-24T06:31:59.097 INFO:tasks.cephfs_test_runner: 2023-11-24T06:31:59.097 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:31:59.097 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:31:59.097 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:31:59.097 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:31:59.097 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 4537, in test_subvolume_inherited_snapshot_ls 2023-11-24T06:31:59.098 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T06:31:59.098 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T06:31:59.098 INFO:tasks.cephfs_test_runner: 2023-11-24T06:31:59.098 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:31:59.203 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1f75226033be47bca1a68d69c0bad254 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:31:59.204 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T06:31:59.218 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-24T06:31:59.226 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-24T06:31:59.226 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:31:59.227 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:59.292 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:31:59.292 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-11-24T06:31:59.292 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:59.350 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:31:59.350 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:31:59.350 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:59.412 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:31:59.412 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-11-24T06:31:59.412 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:31:59.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054303,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498832k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:31:59.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:59.466 DEBUG:teuthology.orchestra.run.smithi173:> ip netns list 2023-11-24T06:31:59.521 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:31:59.521 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:31:59.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:59.522 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:31:59.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:59.620 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:31:59.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:59.708 DEBUG:teuthology.orchestra.run.smithi173:> sudo ip link delete ceph-brx 2023-11-24T06:31:59.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:59.832 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-11-24T06:31:59.896 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:31:59.897 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:31:59.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:31:59.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:32:00.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:32:00.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:32:00.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:32:00.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2023-11-24T06:32:00.248 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T06:32:00.259 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:32:00.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:32:00.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.460+0000 7f44b56a6640 1 Processor -- start 2023-11-24T06:32:00.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.460+0000 7f44b56a6640 1 -- start start 2023-11-24T06:32:00.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.462+0000 7f44b56a6640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f44b014b2c0 0x7f44b00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.462+0000 7f44b56a6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b00cfbf0 con 0x7f44b00cf7d0 2023-11-24T06:32:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.462+0000 7f44b56a6640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b0071690 con 0x7f44b0071070 2023-11-24T06:32:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.462+0000 7f44b56a6640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b014b6a0 con 0x7f44b014b2c0 2023-11-24T06:32:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.462+0000 7f44ae7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f44b00cf7d0 0x7f44b0070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41334/0 (socket says 172.21.15.16:41334) 2023-11-24T06:32:00.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.462+0000 7f44ae7fc640 1 -- 172.21.15.16:0/800508812 learned_addr learned my addr 172.21.15.16:0/800508812 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:00.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144001179 0 0) 0x7f44b00cfbf0 con 0x7f44b00cf7d0 2023-11-24T06:32:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f449c003520 con 0x7f44b00cf7d0 2023-11-24T06:32:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44aeffd640 1 --2- 172.21.15.16:0/800508812 >> v2:172.21.15.173:3300/0 conn(0x7f44b014b2c0 0x7f44b00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1899663114 0 0) 0x7f449c003520 con 0x7f44b00cf7d0 2023-11-24T06:32:00.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 >> v2:172.21.15.173:3300/0 conn(0x7f44b014b2c0 msgr2=0x7f44b00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:00.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 --2- 172.21.15.16:0/800508812 >> v2:172.21.15.173:3300/0 conn(0x7f44b014b2c0 0x7f44b00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 >> v1:172.21.15.173:6789/0 conn(0x7f44b0071070 legacy=0x7f44b00cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0071970 con 0x7f44b00cf7d0 2023-11-24T06:32:00.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f4498001670 con 0x7f44b00cf7d0 2023-11-24T06:32:00.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 >> v1:172.21.15.16:6789/0 conn(0x7f44b00cf7d0 legacy=0x7f44b0070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c003970 0x7f449c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0047c0 0x7f449c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c005180 0x7f449c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b014b6a0 con 0x7f449c003970 2023-11-24T06:32:00.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c00c0d0 con 0x7f449c005180 2023-11-24T06:32:00.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c00c290 con 0x7f449c0047c0 2023-11-24T06:32:00.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.463+0000 7f44aeffd640 1 --2- 172.21.15.16:0/800508812 >> v2:172.21.15.173:3300/0 conn(0x7f44b014b2c0 0x7f44b00cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:32:00.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.464+0000 7f44aeffd640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c003970 0x7f449c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.464+0000 7f44ae7fc640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0047c0 0x7f449c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.464+0000 7f44af7fe640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c005180 0x7f449c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.464+0000 7f44aeffd640 1 -- 172.21.15.16:0/800508812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0047c0 msgr2=0x7f449c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:00.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.464+0000 7f44aeffd640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0047c0 0x7f449c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.464+0000 7f44aeffd640 1 -- 172.21.15.16:0/800508812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c005180 msgr2=0x7f449c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:00.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.464+0000 7f44aeffd640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c005180 0x7f449c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.464+0000 7f44aeffd640 1 -- 172.21.15.16:0/800508812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c003520 con 0x7f449c003970 2023-11-24T06:32:00.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.464+0000 7f44af7fe640 1 -- 172.21.15.16:0/800508812 reap_dead start 2023-11-24T06:32:00.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.465+0000 7f44aeffd640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c003970 0x7f449c003d90 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f44a00073e0 tx=0x7f449c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b2fab89c0baa66 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:00.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.465+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a002cd30 con 0x7f449c003970 2023-11-24T06:32:00.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.465+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a002ce90 con 0x7f449c003970 2023-11-24T06:32:00.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.465+0000 7f44adffb640 1 -- 172.21.15.16:0/800508812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0034640 con 0x7f449c003970 2023-11-24T06:32:00.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.466+0000 7f44b56a6640 1 -- 172.21.15.16:0/800508812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c003970 msgr2=0x7f449c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.466+0000 7f44b56a6640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c003970 0x7f449c003d90 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f44a00073e0 tx=0x7f449c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.466+0000 7f44b56a6640 1 -- 172.21.15.16:0/800508812 shutdown_connections 2023-11-24T06:32:00.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.466+0000 7f44b56a6640 1 --2- 172.21.15.16:0/800508812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c003970 0x7f449c003d90 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.466+0000 7f44b56a6640 1 -- 172.21.15.16:0/800508812 >> 172.21.15.16:0/800508812 conn(0x7f44b006a800 msgr2=0x7f44b006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.466+0000 7f44b56a6640 1 -- 172.21.15.16:0/800508812 shutdown_connections 2023-11-24T06:32:00.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.466+0000 7f44b56a6640 1 -- 172.21.15.16:0/800508812 wait complete. 2023-11-24T06:32:00.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.468+0000 7f44b56a6640 1 Processor -- start 2023-11-24T06:32:00.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.468+0000 7f44b56a6640 1 -- start start 2023-11-24T06:32:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.469+0000 7f44b56a6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c003970 0x7f44b0161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.469+0000 7f44b56a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44b0143d20 0x7f44b01640c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.469+0000 7f44b56a6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44b0164600 0x7f44b0142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.469+0000 7f44b56a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b0164a80 con 0x7f44b0143d20 2023-11-24T06:32:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.469+0000 7f44b56a6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b01617a0 con 0x7f449c003970 2023-11-24T06:32:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.469+0000 7f44b56a6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b0162500 con 0x7f44b0164600 2023-11-24T06:32:00.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.470+0000 7f44ae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44b0143d20 0x7f44b01640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.470+0000 7f44ae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44b0143d20 0x7f44b01640c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53954/0 (socket says 172.21.15.16:53954) 2023-11-24T06:32:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.470+0000 7f44ae7fc640 1 -- 172.21.15.16:0/2647307836 learned_addr learned my addr 172.21.15.16:0/2647307836 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.470+0000 7f44ae7fc640 1 -- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44b0164600 msgr2=0x7f44b0142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.470+0000 7f44ae7fc640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44b0164600 0x7f44b0142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.470+0000 7f44ae7fc640 1 -- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c003970 msgr2=0x7f44b0161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.470+0000 7f44aeffd640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c003970 0x7f44b0161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.470+0000 7f44ae7fc640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c003970 0x7f44b0161e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.470+0000 7f44ae7fc640 1 -- 172.21.15.16:0/2647307836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b01627e0 con 0x7f44b0143d20 2023-11-24T06:32:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.471+0000 7f44ae7fc640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44b0143d20 0x7f44b01640c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f449800e790 tx=0x7f449800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.471+0000 7f44a77fe640 1 -- 172.21.15.16:0/2647307836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449800eea0 con 0x7f44b0143d20 2023-11-24T06:32:00.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.471+0000 7f44a77fe640 1 -- 172.21.15.16:0/2647307836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449800c8d0 con 0x7f44b0143d20 2023-11-24T06:32:00.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.471+0000 7f44a77fe640 1 -- 172.21.15.16:0/2647307836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449800cc60 con 0x7f44b0143d20 2023-11-24T06:32:00.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.471+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b0162ac0 con 0x7f44b0143d20 2023-11-24T06:32:00.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.471+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b0163080 con 0x7f44b0143d20 2023-11-24T06:32:00.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.472+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b00675d0 con 0x7f44b0143d20 2023-11-24T06:32:00.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.473+0000 7f44a77fe640 1 -- 172.21.15.16:0/2647307836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f449801d460 con 0x7f44b0143d20 2023-11-24T06:32:00.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.474+0000 7f44a77fe640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f449006b470 0x7f449006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.474+0000 7f44a77fe640 1 -- 172.21.15.16:0/2647307836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f4498089eb0 con 0x7f44b0143d20 2023-11-24T06:32:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.475+0000 7f44aeffd640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f449006b470 0x7f449006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.476+0000 7f44aeffd640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f449006b470 0x7f449006d930 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f44a0036000 tx=0x7f44a0030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:00.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.479+0000 7f44a77fe640 1 -- 172.21.15.16:0/2647307836 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4498058390 con 0x7f44b0143d20 2023-11-24T06:32:00.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.671+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44b00cbc40 con 0x7f449006b470 2023-11-24T06:32:00.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.677+0000 7f44a77fe640 1 -- 172.21.15.16:0/2647307836 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48205 (secure 0 0 0) 0x7f44a0032340 con 0x7f449006b470 2023-11-24T06:32:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:32:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:32:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f449006b470 msgr2=0x7f449006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f449006b470 0x7f449006d930 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f44a0036000 tx=0x7f44a0030000 comp rx=0 tx=0).stop 2023-11-24T06:32:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44b0143d20 msgr2=0x7f44b01640c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44b0143d20 0x7f44b01640c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f449800e790 tx=0x7f449800ec50 comp rx=0 tx=0).stop 2023-11-24T06:32:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 shutdown_connections 2023-11-24T06:32:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f449006b470 0x7f449006d930 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44b0164600 0x7f44b0142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44b0143d20 0x7f44b01640c0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 --2- 172.21.15.16:0/2647307836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c003970 0x7f44b0161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 >> 172.21.15.16:0/2647307836 conn(0x7f44b006a800 msgr2=0x7f44b00aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.680+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 shutdown_connections 2023-11-24T06:32:00.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.681+0000 7f44b56a6640 1 -- 172.21.15.16:0/2647307836 wait complete. 2023-11-24T06:32:00.702 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2023-11-24T06:32:00.208358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259404,"kb_used_data":9952,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715316,"statfs":{"total":773094113280,"available":772828483584,"internally_reserved":0,"allocated":10190848,"data_stored":5053494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36065,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-69,"num_write_kb":-60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002016"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-24T06:31:56.759975+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:31:56.759975+0000","last_peered":"2023-11-24T06:31:56.759975+0000","last_clean":"2023-11-24T06:31:56.759975+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:31:56.759975+0000","last_undegraded":"2023-11-24T06:31:56.759975+0000","last_fullsized":"2023-11-24T06:31:56.759975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1720320,"data_stored":1072896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.149,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5209999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.216,"5min":1.2589999999999999,"15min":1.2889999999999999},"min":{"1min":0.72099999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.704,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0700000000000001,"15min":1.0269999999999999},"min":{"1min":0.64900000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.827,"5min":2.456,"15min":2.456},"last":1.827},{"interface":"front","average":{"1min":1.258,"5min":1.2969999999999999,"15min":1.3620000000000001},"min":{"1min":0.83399999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.2769999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.204,"15min":1.2649999999999999},"min":{"1min":0.73099999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5900000000000001,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.179,"15min":1.1950000000000001},"min":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.4119999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.4119999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3,"15min":1.3819999999999999},"min":{"1min":0.78600000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6100000000000001,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.256,"5min":1.2330000000000001,"15min":1.2130000000000001},"min":{"1min":0.89500000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.653,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.653}]},{"osd":4,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3560000000000001,"15min":1.5229999999999999},"min":{"1min":0.78800000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7190000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.47,"15min":1.556},"min":{"1min":1.0449999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7509999999999999,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.4770000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3080000000000001,"15min":1.3819999999999999},"min":{"1min":0.82999999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.919,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.919},{"interface":"front","average":{"1min":1.46,"5min":1.7,"15min":1.752},"min":{"1min":0.88500000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.871,"5min":17.582999999999998,"15min":17.582999999999998},"last":1.871}]},{"osd":6,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6040000000000001,"15min":1.804},"min":{"1min":1.085,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.988,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.988},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5049999999999999,"15min":1.655},"min":{"1min":1.101,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.78,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.78}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0920000000000001,"15min":1.1000000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.73,"5min":1.829,"15min":1.829},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.129,"5min":1.109,"15min":1.111},"min":{"1min":0.65500000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.79,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2350000000000001,"15min":1.218},"min":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.595},{"interface":"front","average":{"1min":1.252,"5min":1.2110000000000001,"15min":1.21},"min":{"1min":0.875,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.645,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.401}]},{"osd":2,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.137,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.754,"5min":3.597,"15min":3.597},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.284,"15min":1.3109999999999999},"min":{"1min":0.89500000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0470000000000002,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.0069999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2909999999999999,"15min":1.3109999999999999},"min":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.8049999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.26,"15min":1.3120000000000001},"min":{"1min":0.85399999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.546,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3680000000000001,"15min":1.4279999999999999},"min":{"1min":0.98999999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.742,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3600000000000001,"15min":1.389},"min":{"1min":0.91000000000000003,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7390000000000001,"5min":3.04,"15min":3.04},"last":1.516}]},{"osd":5,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":1.0660000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7969999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.526,"15min":1.593},"min":{"1min":1.246,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9019999999999999,"5min":3.774,"15min":3.774},"last":1.671}]},{"osd":7,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5189999999999999,"15min":1.5509999999999999},"min":{"1min":1.0640000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.012,"5min":3.403,"15min":3.403},"last":1.639},{"interface":"front","average":{"1min":1.492,"5min":1.482,"15min":1.506},"min":{"1min":0.96699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.9610000000000001,"5min":3.649,"15min":3.649},"last":1.5529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":506012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.155,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7629999999999999,"5min":1.903,"15min":1.903},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.248,"15min":1.3},"min":{"1min":0.91300000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.883,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":2.569}]},{"osd":1,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.133,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.1799999999999999,"15min":1.125},"min":{"1min":0.74299999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8560000000000001,"5min":1.96,"15min":1.96},"last":0.89200000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2070000000000001,"15min":1.2050000000000001},"min":{"1min":0.877,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8300000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":2.0880000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3,"15min":1.294},"min":{"1min":0.70599999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.927}]},{"osd":3,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.302,"15min":1.3640000000000001},"min":{"1min":0.748,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.502,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":2.7639999999999998},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.385},"min":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.954,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":2.2530000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.304,"15min":1.329},"min":{"1min":0.77600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.6919999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4670000000000001,"15min":1.5249999999999999},"min":{"1min":1.1439999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.77,"5min":3.085,"15min":3.085},"last":2.4289999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.482,"15min":1.4490000000000001},"min":{"1min":1.028,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.032,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.671},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.4810000000000001,"15min":1.47},"min":{"1min":1.0429999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.9039999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.163}]},{"osd":7,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.593,"15min":1.617},"min":{"1min":1.0669999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8919999999999999,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":2.6629999999999998},{"interface":"front","average":{"1min":1.6499999999999999,"5min":1.6539999999999999,"15min":1.704},"min":{"1min":1.3520000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.089,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":2.0249999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":477053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1020000000000001,"15min":1.0820000000000001},"min":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6479999999999999,"5min":1.982,"15min":1.982},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.079,"5min":1.0900000000000001,"15min":1.1120000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.177,"15min":1.123},"min":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7370000000000001,"5min":1.909,"15min":1.909},"last":1.296},{"interface":"front","average":{"1min":1.165,"5min":1.2090000000000001,"15min":1.288},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.9339999999999999,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.349}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.216,"15min":1.2809999999999999},"min":{"1min":0.84899999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.74,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2929999999999999,"15min":1.325},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.1419999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.25,"15min":1.3180000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":5.0300000000000002,"5min":5.0300000000000002,"15min":5.0300000000000002},"last":1.768},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":0.93100000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.8220000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.8220000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.456,"15min":1.506},"min":{"1min":0.875,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.452,"15min":1.538},"min":{"1min":0.79600000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":2.0390000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4419999999999999,"15min":1.4710000000000001},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":1.9830000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.5109999999999999,"15min":1.5629999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.593}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4790000000000001,"15min":1.617},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":1.913},{"interface":"front","average":{"1min":1.635,"5min":1.5920000000000001,"15min":1.605},"min":{"1min":0.83199999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.032,"5min":4.032,"15min":4.032},"last":2.2229999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.83899999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.96799999999999997,"15min":0.96999999999999997},"min":{"1min":0.65800000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0940000000000001,"15min":1.1140000000000001},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.857,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.58999999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6679999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.60399999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.488,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.792}]},{"osd":4,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.327,"15min":1.383},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7190000000000001,"5min":3.081,"15min":3.081},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3440000000000001,"15min":1.3420000000000001},"min":{"1min":0.81000000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7909999999999999,"5min":2.758,"15min":2.758},"last":1.256}]},{"osd":5,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3540000000000001,"15min":1.3959999999999999},"min":{"1min":0.88900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6899999999999999,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":2.7000000000000002},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.544,"15min":1.7230000000000001},"min":{"1min":0.91000000000000003,"5min":0.745,"15min":0.745},"max":{"1min":1.861,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":2.6280000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.80000000000000004,"5min":0.753,"15min":0.753},"max":{"1min":2.1480000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":2.0859999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.51,"15min":1.5369999999999999},"min":{"1min":1.024,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.847,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.516}]},{"osd":7,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.552,"15min":1.601},"min":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":4.641,"15min":4.641},"last":1.603},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.528,"15min":1.5680000000000001},"min":{"1min":0.92500000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0299999999999998,"5min":3.798,"15min":3.798},"last":1.8819999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.88100000000000001},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.99399999999999999},"min":{"1min":0.47899999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.2769999999999999,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.92700000000000005,"15min":0.95799999999999996},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.659,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.59799999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.524,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.84299999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.077,"15min":1.1020000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.069,"15min":1.1080000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.575,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.397,"15min":1.4890000000000001},"min":{"1min":0.876,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6830000000000001,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.262,"5min":1.4059999999999999,"15min":1.4690000000000001},"min":{"1min":0.995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6850000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.96399999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.45,"15min":1.4630000000000001},"min":{"1min":0.97299999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0539999999999998,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.095},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4359999999999999,"15min":1.49},"min":{"1min":1.01,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.821,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.236}]},{"osd":6,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3819999999999999,"15min":1.4219999999999999},"min":{"1min":0.875,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.681,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.484,"15min":1.528},"min":{"1min":0.71399999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.804,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.611}]},{"osd":7,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.4550000000000001,"15min":1.5289999999999999},"min":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.873,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.454},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.5169999999999999,"15min":1.5349999999999999},"min":{"1min":0.94699999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.9359999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.8069999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92200000000000004,"15min":1.006},"min":{"1min":0.55800000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.3700000000000001,"5min":2.593,"15min":2.593},"last":1.073},{"interface":"front","average":{"1min":0.997,"5min":1.0149999999999999,"15min":1.0309999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4830000000000001,"5min":1.74,"15min":1.74},"last":1.0049999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.05,"5min":1.03,"15min":1.014},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.796,"5min":1.829,"15min":1.829},"last":0.93700000000000006}]},{"osd":3,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1120000000000001,"15min":1.123},"min":{"1min":0.58599999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.26},{"interface":"front","average":{"1min":1.026,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.53,"5min":1.798,"15min":1.798},"last":1.335}]},{"osd":4,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2450000000000001,"15min":1.2529999999999999},"min":{"1min":0.66300000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9410000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3999999999999999,"15min":1.4399999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1269999999999998,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.534}]},{"osd":5,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.345,"15min":1.319},"min":{"1min":0.98299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.504},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.3919999999999999,"15min":1.3999999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.0939999999999999,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.377}]},{"osd":6,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4470000000000001,"15min":1.411},"min":{"1min":1.0329999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.446,"15min":1.454},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.998,"5min":2.371,"15min":2.371},"last":1.1919999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.458,"15min":1.4530000000000001},"min":{"1min":0.83399999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0659999999999998,"5min":2.246,"15min":2.246},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.514,"15min":1.526},"min":{"1min":1.087,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.2410000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.4410000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1708032,"data_stored":1067421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84099999999999997,"15min":0.84699999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.355,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.88800000000000001,"15min":0.94699999999999995},"min":{"1min":0.504,"5min":0.31,"15min":0.31},"max":{"1min":1.5329999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.284}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.014,"15min":1.0029999999999999},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.206,"5min":1.0840000000000001,"15min":1.125},"min":{"1min":0.58599999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.373}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90700000000000003,"15min":0.94099999999999995},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5860000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.056,"15min":1.0800000000000001},"min":{"1min":0.53300000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.48,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2430000000000001,"15min":1.2310000000000001},"min":{"1min":0.84799999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.788,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.232},{"interface":"front","average":{"1min":1.399,"5min":1.3420000000000001,"15min":1.3180000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8109999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.294,"15min":1.3240000000000001},"min":{"1min":0.88300000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7390000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.2589999999999999,"15min":1.274},"min":{"1min":0.84499999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.671,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4359999999999999,"15min":1.476},"min":{"1min":1.0289999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0209999999999999,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.117},{"interface":"front","average":{"1min":1.407,"5min":1.4079999999999999,"15min":1.49},"min":{"1min":1.089,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9339999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.0509999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4299999999999999,"15min":1.45},"min":{"1min":0.86899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0750000000000002,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.405,"15min":1.421},"min":{"1min":1.0269999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0150000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.1970000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:32:00.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:32:00.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.928+0000 7f49f359e640 1 Processor -- start 2023-11-24T06:32:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.928+0000 7f49f359e640 1 -- start start 2023-11-24T06:32:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f359e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f49f4071410 0x7f49f4146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f406c7c0 0x7f49f40717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49f4071d30 0x7f49f40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f406cc20 con 0x7f49f4071d30 2023-11-24T06:32:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f359e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f406cec0 con 0x7f49f4071410 2023-11-24T06:32:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f406d1a0 con 0x7f49f406c7c0 2023-11-24T06:32:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49f4071d30 0x7f49f40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f2d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49f4071d30 0x7f49f40b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53956/0 (socket says 172.21.15.16:53956) 2023-11-24T06:32:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f2d9d640 1 -- 172.21.15.16:0/1539400641 learned_addr learned my addr 172.21.15.16:0/1539400641 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.930+0000 7f49f259c640 1 --2- 172.21.15.16:0/1539400641 >> v2:172.21.15.173:3300/0 conn(0x7f49f4071410 0x7f49f4146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.931+0000 7f49ebfff640 1 --2- 172.21.15.16:0/1539400641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f406c7c0 0x7f49f40717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.931+0000 7f49f2d9d640 1 -- 172.21.15.16:0/1539400641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f406c7c0 msgr2=0x7f49f40717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.931+0000 7f49f2d9d640 1 --2- 172.21.15.16:0/1539400641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f406c7c0 0x7f49f40717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.931+0000 7f49f2d9d640 1 -- 172.21.15.16:0/1539400641 >> v2:172.21.15.173:3300/0 conn(0x7f49f4071410 msgr2=0x7f49f4146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.931+0000 7f49f2d9d640 1 --2- 172.21.15.16:0/1539400641 >> v2:172.21.15.173:3300/0 conn(0x7f49f4071410 0x7f49f4146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.931+0000 7f49f2d9d640 1 -- 172.21.15.16:0/1539400641 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f406d480 con 0x7f49f4071d30 2023-11-24T06:32:00.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.932+0000 7f49f2d9d640 1 --2- 172.21.15.16:0/1539400641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 0x7f49f40b1e00 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f49f40ad5f0 tx=0x7f49ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aced887589805109 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.932+0000 7f49f1d9b640 1 -- 172.21.15.16:0/1539400641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec02ca70 con 0x7f49f4071d30 2023-11-24T06:32:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.932+0000 7f49f1d9b640 1 -- 172.21.15.16:0/1539400641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec02cbd0 con 0x7f49f4071d30 2023-11-24T06:32:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.932+0000 7f49f1d9b640 1 -- 172.21.15.16:0/1539400641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0343f0 con 0x7f49f4071d30 2023-11-24T06:32:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.933+0000 7f49f359e640 1 -- 172.21.15.16:0/1539400641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 msgr2=0x7f49f40b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.933+0000 7f49f359e640 1 --2- 172.21.15.16:0/1539400641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 0x7f49f40b1e00 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f49f40ad5f0 tx=0x7f49ec02bed0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.933+0000 7f49f359e640 1 -- 172.21.15.16:0/1539400641 shutdown_connections 2023-11-24T06:32:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.933+0000 7f49f359e640 1 --2- 172.21.15.16:0/1539400641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 0x7f49f40b1e00 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.933+0000 7f49f359e640 1 --2- 172.21.15.16:0/1539400641 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f406c7c0 0x7f49f40717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.933+0000 7f49f359e640 1 --2- 172.21.15.16:0/1539400641 >> v2:172.21.15.173:3300/0 conn(0x7f49f4071410 0x7f49f4146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.933+0000 7f49f359e640 1 -- 172.21.15.16:0/1539400641 >> 172.21.15.16:0/1539400641 conn(0x7f49f406a800 msgr2=0x7f49f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.933+0000 7f49f359e640 1 -- 172.21.15.16:0/1539400641 shutdown_connections 2023-11-24T06:32:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.933+0000 7f49f359e640 1 -- 172.21.15.16:0/1539400641 wait complete. 2023-11-24T06:32:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.935+0000 7f49f359e640 1 Processor -- start 2023-11-24T06:32:00.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.935+0000 7f49f359e640 1 -- start start 2023-11-24T06:32:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.936+0000 7f49f359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49f406c7c0 0x7f49f413b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.936+0000 7f49f359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f4071410 0x7f49f413d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.936+0000 7f49f359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 0x7f49f413a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.936+0000 7f49f359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f413de20 con 0x7f49f4071d30 2023-11-24T06:32:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.936+0000 7f49f359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f415eb70 con 0x7f49f406c7c0 2023-11-24T06:32:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.936+0000 7f49f359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f413d5a0 con 0x7f49f4071410 2023-11-24T06:32:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.936+0000 7f49f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 0x7f49f413a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.936+0000 7f49f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 0x7f49f413a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53976/0 (socket says 172.21.15.16:53976) 2023-11-24T06:32:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.936+0000 7f49f2d9d640 1 -- 172.21.15.16:0/2283268420 learned_addr learned my addr 172.21.15.16:0/2283268420 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.937+0000 7f49ebfff640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f4071410 0x7f49f413d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.937+0000 7f49f259c640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49f406c7c0 0x7f49f413b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.937+0000 7f49f2d9d640 1 -- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f4071410 msgr2=0x7f49f413d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.937+0000 7f49f2d9d640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f4071410 0x7f49f413d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.937+0000 7f49f2d9d640 1 -- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49f406c7c0 msgr2=0x7f49f413b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.937+0000 7f49f2d9d640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49f406c7c0 0x7f49f413b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.937+0000 7f49f2d9d640 1 -- 172.21.15.16:0/2283268420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec009b30 con 0x7f49f4071d30 2023-11-24T06:32:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.937+0000 7f49f2d9d640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 0x7f49f413a040 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009f40 tx=0x7f49ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.938+0000 7f49eaffd640 1 -- 172.21.15.16:0/2283268420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec03e450 con 0x7f49f4071d30 2023-11-24T06:32:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.938+0000 7f49eaffd640 1 -- 172.21.15.16:0/2283268420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec03e5b0 con 0x7f49f4071d30 2023-11-24T06:32:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.938+0000 7f49eaffd640 1 -- 172.21.15.16:0/2283268420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec03e960 con 0x7f49f4071d30 2023-11-24T06:32:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.938+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f413d860 con 0x7f49f4071d30 2023-11-24T06:32:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.938+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f415e800 con 0x7f49f4071d30 2023-11-24T06:32:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.939+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f415ed00 con 0x7f49f4071d30 2023-11-24T06:32:00.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.940+0000 7f49eaffd640 1 -- 172.21.15.16:0/2283268420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f49ec03eac0 con 0x7f49f4071d30 2023-11-24T06:32:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.941+0000 7f49eaffd640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49d406b470 0x7f49d406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.941+0000 7f49eaffd640 1 -- 172.21.15.16:0/2283268420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f49ec0acf10 con 0x7f49f4071d30 2023-11-24T06:32:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.942+0000 7f49f259c640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49d406b470 0x7f49d406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.943+0000 7f49f259c640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49d406b470 0x7f49d406d930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:00.945+0000 7f49eaffd640 1 -- 172.21.15.16:0/2283268420 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49ec0779b0 con 0x7f49f4071d30 2023-11-24T06:32:01.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.142+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49f4072150 con 0x7f49d406b470 2023-11-24T06:32:01.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.148+0000 7f49eaffd640 1 -- 172.21.15.16:0/2283268420 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48205 (secure 0 0 0) 0x7f49e400f830 con 0x7f49d406b470 2023-11-24T06:32:01.151 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:32:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:32:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49d406b470 msgr2=0x7f49d406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49d406b470 0x7f49d406d930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4008040 comp rx=0 tx=0).stop 2023-11-24T06:32:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 msgr2=0x7f49f413a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 0x7f49f413a040 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009f40 tx=0x7f49ec004a40 comp rx=0 tx=0).stop 2023-11-24T06:32:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 shutdown_connections 2023-11-24T06:32:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f49d406b470 0x7f49d406d930 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49f4071d30 0x7f49f413a040 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49f4071410 0x7f49f413d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 --2- 172.21.15.16:0/2283268420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49f406c7c0 0x7f49f413b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 >> 172.21.15.16:0/2283268420 conn(0x7f49f406a800 msgr2=0x7f49f41465a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:01.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 shutdown_connections 2023-11-24T06:32:01.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.151+0000 7f49f359e640 1 -- 172.21.15.16:0/2283268420 wait complete. 2023-11-24T06:32:01.174 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2023-11-24T06:32:00.208358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259404,"kb_used_data":9952,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715316,"statfs":{"total":773094113280,"available":772828483584,"internally_reserved":0,"allocated":10190848,"data_stored":5053494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36065,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-69,"num_write_kb":-60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002016"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-24T06:31:56.759975+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:31:56.759975+0000","last_peered":"2023-11-24T06:31:56.759975+0000","last_clean":"2023-11-24T06:31:56.759975+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:31:56.759975+0000","last_undegraded":"2023-11-24T06:31:56.759975+0000","last_fullsized":"2023-11-24T06:31:56.759975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1720320,"data_stored":1072896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.149,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5209999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.216,"5min":1.2589999999999999,"15min":1.2889999999999999},"min":{"1min":0.72099999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.704,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0700000000000001,"15min":1.0269999999999999},"min":{"1min":0.64900000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.827,"5min":2.456,"15min":2.456},"last":1.827},{"interface":"front","average":{"1min":1.258,"5min":1.2969999999999999,"15min":1.3620000000000001},"min":{"1min":0.83399999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.2769999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.204,"15min":1.2649999999999999},"min":{"1min":0.73099999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5900000000000001,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.179,"15min":1.1950000000000001},"min":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.4119999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.4119999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3,"15min":1.3819999999999999},"min":{"1min":0.78600000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6100000000000001,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.256,"5min":1.2330000000000001,"15min":1.2130000000000001},"min":{"1min":0.89500000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.653,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.653}]},{"osd":4,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3560000000000001,"15min":1.5229999999999999},"min":{"1min":0.78800000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7190000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.47,"15min":1.556},"min":{"1min":1.0449999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7509999999999999,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.4770000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3080000000000001,"15min":1.3819999999999999},"min":{"1min":0.82999999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.919,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.919},{"interface":"front","average":{"1min":1.46,"5min":1.7,"15min":1.752},"min":{"1min":0.88500000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.871,"5min":17.582999999999998,"15min":17.582999999999998},"last":1.871}]},{"osd":6,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6040000000000001,"15min":1.804},"min":{"1min":1.085,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.988,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.988},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5049999999999999,"15min":1.655},"min":{"1min":1.101,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.78,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.78}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0920000000000001,"15min":1.1000000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.73,"5min":1.829,"15min":1.829},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.129,"5min":1.109,"15min":1.111},"min":{"1min":0.65500000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.79,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2350000000000001,"15min":1.218},"min":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.595},{"interface":"front","average":{"1min":1.252,"5min":1.2110000000000001,"15min":1.21},"min":{"1min":0.875,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.645,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.401}]},{"osd":2,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.137,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.754,"5min":3.597,"15min":3.597},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.284,"15min":1.3109999999999999},"min":{"1min":0.89500000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0470000000000002,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.0069999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2909999999999999,"15min":1.3109999999999999},"min":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.8049999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.26,"15min":1.3120000000000001},"min":{"1min":0.85399999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.546,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3680000000000001,"15min":1.4279999999999999},"min":{"1min":0.98999999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.742,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3600000000000001,"15min":1.389},"min":{"1min":0.91000000000000003,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7390000000000001,"5min":3.04,"15min":3.04},"last":1.516}]},{"osd":5,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":1.0660000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7969999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.526,"15min":1.593},"min":{"1min":1.246,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9019999999999999,"5min":3.774,"15min":3.774},"last":1.671}]},{"osd":7,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5189999999999999,"15min":1.5509999999999999},"min":{"1min":1.0640000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.012,"5min":3.403,"15min":3.403},"last":1.639},{"interface":"front","average":{"1min":1.492,"5min":1.482,"15min":1.506},"min":{"1min":0.96699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.9610000000000001,"5min":3.649,"15min":3.649},"last":1.5529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":506012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.155,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7629999999999999,"5min":1.903,"15min":1.903},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.248,"15min":1.3},"min":{"1min":0.91300000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.883,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":2.569}]},{"osd":1,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.133,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.1799999999999999,"15min":1.125},"min":{"1min":0.74299999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8560000000000001,"5min":1.96,"15min":1.96},"last":0.89200000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2070000000000001,"15min":1.2050000000000001},"min":{"1min":0.877,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8300000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":2.0880000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3,"15min":1.294},"min":{"1min":0.70599999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.927}]},{"osd":3,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.302,"15min":1.3640000000000001},"min":{"1min":0.748,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.502,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":2.7639999999999998},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.385},"min":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.954,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":2.2530000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.304,"15min":1.329},"min":{"1min":0.77600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.6919999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4670000000000001,"15min":1.5249999999999999},"min":{"1min":1.1439999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.77,"5min":3.085,"15min":3.085},"last":2.4289999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.482,"15min":1.4490000000000001},"min":{"1min":1.028,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.032,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.671},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.4810000000000001,"15min":1.47},"min":{"1min":1.0429999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.9039999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.163}]},{"osd":7,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.593,"15min":1.617},"min":{"1min":1.0669999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8919999999999999,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":2.6629999999999998},{"interface":"front","average":{"1min":1.6499999999999999,"5min":1.6539999999999999,"15min":1.704},"min":{"1min":1.3520000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.089,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":2.0249999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":477053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1020000000000001,"15min":1.0820000000000001},"min":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6479999999999999,"5min":1.982,"15min":1.982},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.079,"5min":1.0900000000000001,"15min":1.1120000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.177,"15min":1.123},"min":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7370000000000001,"5min":1.909,"15min":1.909},"last":1.296},{"interface":"front","average":{"1min":1.165,"5min":1.2090000000000001,"15min":1.288},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.9339999999999999,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.349}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.216,"15min":1.2809999999999999},"min":{"1min":0.84899999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.74,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2929999999999999,"15min":1.325},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.1419999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.25,"15min":1.3180000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":5.0300000000000002,"5min":5.0300000000000002,"15min":5.0300000000000002},"last":1.768},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":0.93100000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.8220000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.8220000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.456,"15min":1.506},"min":{"1min":0.875,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.452,"15min":1.538},"min":{"1min":0.79600000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":2.0390000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4419999999999999,"15min":1.4710000000000001},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":1.9830000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.5109999999999999,"15min":1.5629999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.593}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4790000000000001,"15min":1.617},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":1.913},{"interface":"front","average":{"1min":1.635,"5min":1.5920000000000001,"15min":1.605},"min":{"1min":0.83199999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.032,"5min":4.032,"15min":4.032},"last":2.2229999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.83899999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.96799999999999997,"15min":0.96999999999999997},"min":{"1min":0.65800000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0940000000000001,"15min":1.1140000000000001},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.857,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.58999999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6679999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.60399999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.488,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.792}]},{"osd":4,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.327,"15min":1.383},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7190000000000001,"5min":3.081,"15min":3.081},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3440000000000001,"15min":1.3420000000000001},"min":{"1min":0.81000000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7909999999999999,"5min":2.758,"15min":2.758},"last":1.256}]},{"osd":5,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3540000000000001,"15min":1.3959999999999999},"min":{"1min":0.88900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6899999999999999,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":2.7000000000000002},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.544,"15min":1.7230000000000001},"min":{"1min":0.91000000000000003,"5min":0.745,"15min":0.745},"max":{"1min":1.861,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":2.6280000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.80000000000000004,"5min":0.753,"15min":0.753},"max":{"1min":2.1480000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":2.0859999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.51,"15min":1.5369999999999999},"min":{"1min":1.024,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.847,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.516}]},{"osd":7,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.552,"15min":1.601},"min":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":4.641,"15min":4.641},"last":1.603},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.528,"15min":1.5680000000000001},"min":{"1min":0.92500000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0299999999999998,"5min":3.798,"15min":3.798},"last":1.8819999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.88100000000000001},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.99399999999999999},"min":{"1min":0.47899999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.2769999999999999,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.92700000000000005,"15min":0.95799999999999996},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.659,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.59799999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.524,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.84299999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.077,"15min":1.1020000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.069,"15min":1.1080000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.575,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.397,"15min":1.4890000000000001},"min":{"1min":0.876,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6830000000000001,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.262,"5min":1.4059999999999999,"15min":1.4690000000000001},"min":{"1min":0.995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6850000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.96399999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.45,"15min":1.4630000000000001},"min":{"1min":0.97299999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0539999999999998,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.095},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4359999999999999,"15min":1.49},"min":{"1min":1.01,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.821,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.236}]},{"osd":6,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3819999999999999,"15min":1.4219999999999999},"min":{"1min":0.875,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.681,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.484,"15min":1.528},"min":{"1min":0.71399999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.804,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.611}]},{"osd":7,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.4550000000000001,"15min":1.5289999999999999},"min":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.873,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.454},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.5169999999999999,"15min":1.5349999999999999},"min":{"1min":0.94699999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.9359999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.8069999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92200000000000004,"15min":1.006},"min":{"1min":0.55800000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.3700000000000001,"5min":2.593,"15min":2.593},"last":1.073},{"interface":"front","average":{"1min":0.997,"5min":1.0149999999999999,"15min":1.0309999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4830000000000001,"5min":1.74,"15min":1.74},"last":1.0049999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.05,"5min":1.03,"15min":1.014},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.796,"5min":1.829,"15min":1.829},"last":0.93700000000000006}]},{"osd":3,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1120000000000001,"15min":1.123},"min":{"1min":0.58599999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.26},{"interface":"front","average":{"1min":1.026,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.53,"5min":1.798,"15min":1.798},"last":1.335}]},{"osd":4,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2450000000000001,"15min":1.2529999999999999},"min":{"1min":0.66300000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9410000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3999999999999999,"15min":1.4399999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1269999999999998,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.534}]},{"osd":5,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.345,"15min":1.319},"min":{"1min":0.98299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.504},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.3919999999999999,"15min":1.3999999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.0939999999999999,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.377}]},{"osd":6,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4470000000000001,"15min":1.411},"min":{"1min":1.0329999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.446,"15min":1.454},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.998,"5min":2.371,"15min":2.371},"last":1.1919999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.458,"15min":1.4530000000000001},"min":{"1min":0.83399999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0659999999999998,"5min":2.246,"15min":2.246},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.514,"15min":1.526},"min":{"1min":1.087,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.2410000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.4410000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1708032,"data_stored":1067421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84099999999999997,"15min":0.84699999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.355,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.88800000000000001,"15min":0.94699999999999995},"min":{"1min":0.504,"5min":0.31,"15min":0.31},"max":{"1min":1.5329999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.284}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.014,"15min":1.0029999999999999},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.206,"5min":1.0840000000000001,"15min":1.125},"min":{"1min":0.58599999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.373}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90700000000000003,"15min":0.94099999999999995},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5860000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.056,"15min":1.0800000000000001},"min":{"1min":0.53300000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.48,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2430000000000001,"15min":1.2310000000000001},"min":{"1min":0.84799999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.788,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.232},{"interface":"front","average":{"1min":1.399,"5min":1.3420000000000001,"15min":1.3180000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8109999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.294,"15min":1.3240000000000001},"min":{"1min":0.88300000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7390000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.2589999999999999,"15min":1.274},"min":{"1min":0.84499999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.671,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4359999999999999,"15min":1.476},"min":{"1min":1.0289999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0209999999999999,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.117},{"interface":"front","average":{"1min":1.407,"5min":1.4079999999999999,"15min":1.49},"min":{"1min":1.089,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9339999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.0509999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4299999999999999,"15min":1.45},"min":{"1min":0.86899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0750000000000002,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.405,"15min":1.421},"min":{"1min":1.0269999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0150000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.1970000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:32:01.177 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:32:01.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:32:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.400+0000 7f29698ea640 1 Processor -- start 2023-11-24T06:32:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.400+0000 7f29698ea640 1 -- start start 2023-11-24T06:32:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.401+0000 7f29698ea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415e570 0x7f29640cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.401+0000 7f29698ea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f296406b3f0 0x7f296406b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.401+0000 7f29698ea640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f296415eae0 0x7f296415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.401+0000 7f29698ea640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296406bd70 con 0x7f296406b3f0 2023-11-24T06:32:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.401+0000 7f29698ea640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640b26f0 con 0x7f296415eae0 2023-11-24T06:32:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.401+0000 7f29698ea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29640b29b0 con 0x7f296415e570 2023-11-24T06:32:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f2963fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f296406b3f0 0x7f296406b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f2963fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f296406b3f0 0x7f296406b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53982/0 (socket says 172.21.15.16:53982) 2023-11-24T06:32:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f2963fff640 1 -- 172.21.15.16:0/2271937409 learned_addr learned my addr 172.21.15.16:0/2271937409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f29688e8640 1 --2- 172.21.15.16:0/2271937409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415e570 0x7f29640cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f29690e9640 1 --2- 172.21.15.16:0/2271937409 >> v2:172.21.15.173:3300/0 conn(0x7f296415eae0 0x7f296415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f2963fff640 1 -- 172.21.15.16:0/2271937409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415e570 msgr2=0x7f29640cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f2963fff640 1 --2- 172.21.15.16:0/2271937409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415e570 0x7f29640cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f2963fff640 1 -- 172.21.15.16:0/2271937409 >> v2:172.21.15.173:3300/0 conn(0x7f296415eae0 msgr2=0x7f296415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f2963fff640 1 --2- 172.21.15.16:0/2271937409 >> v2:172.21.15.173:3300/0 conn(0x7f296415eae0 0x7f296415ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.402+0000 7f2963fff640 1 -- 172.21.15.16:0/2271937409 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640b2c90 con 0x7f296406b3f0 2023-11-24T06:32:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.403+0000 7f2963fff640 1 --2- 172.21.15.16:0/2271937409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 0x7f296406b830 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f2964066300 tx=0x7f295802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355f758746c0e28 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29637fe640 1 -- 172.21.15.16:0/2271937409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958031870 con 0x7f296406b3f0 2023-11-24T06:32:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29637fe640 1 -- 172.21.15.16:0/2271937409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29580319d0 con 0x7f296406b3f0 2023-11-24T06:32:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29637fe640 1 -- 172.21.15.16:0/2271937409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958031d60 con 0x7f296406b3f0 2023-11-24T06:32:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29698ea640 1 -- 172.21.15.16:0/2271937409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 msgr2=0x7f296406b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29698ea640 1 --2- 172.21.15.16:0/2271937409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 0x7f296406b830 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f2964066300 tx=0x7f295802b910 comp rx=0 tx=0).stop 2023-11-24T06:32:01.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29698ea640 1 -- 172.21.15.16:0/2271937409 shutdown_connections 2023-11-24T06:32:01.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29698ea640 1 --2- 172.21.15.16:0/2271937409 >> v2:172.21.15.173:3300/0 conn(0x7f296415eae0 0x7f296415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29698ea640 1 --2- 172.21.15.16:0/2271937409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 0x7f296406b830 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29698ea640 1 --2- 172.21.15.16:0/2271937409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415e570 0x7f29640cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29698ea640 1 -- 172.21.15.16:0/2271937409 >> 172.21.15.16:0/2271937409 conn(0x7f2964070970 msgr2=0x7f2964070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29698ea640 1 -- 172.21.15.16:0/2271937409 shutdown_connections 2023-11-24T06:32:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.404+0000 7f29698ea640 1 -- 172.21.15.16:0/2271937409 wait complete. 2023-11-24T06:32:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.406+0000 7f29698ea640 1 Processor -- start 2023-11-24T06:32:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.406+0000 7f29698ea640 1 -- start start 2023-11-24T06:32:01.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.407+0000 7f29698ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 0x7f29640b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.407+0000 7f29698ea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f296415e570 0x7f296415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.407+0000 7f29698ea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415eae0 0x7f296414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.407+0000 7f29698ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29641f1070 con 0x7f296406b3f0 2023-11-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.407+0000 7f29698ea640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29641f1350 con 0x7f296415e570 2023-11-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.407+0000 7f29698ea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29641f1630 con 0x7f296415eae0 2023-11-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.407+0000 7f29688e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 0x7f29640b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29688e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 0x7f29640b3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53992/0 (socket says 172.21.15.16:53992) 2023-11-24T06:32:01.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29688e8640 1 -- 172.21.15.16:0/1289164830 learned_addr learned my addr 172.21.15.16:0/1289164830 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f2963fff640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f296415e570 0x7f296415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29690e9640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415eae0 0x7f296414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29688e8640 1 -- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415eae0 msgr2=0x7f296414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29688e8640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415eae0 0x7f296414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29688e8640 1 -- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f296415e570 msgr2=0x7f296415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29688e8640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f296415e570 0x7f296415c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29688e8640 1 -- 172.21.15.16:0/1289164830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29580094d0 con 0x7f296406b3f0 2023-11-24T06:32:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29690e9640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415eae0 0x7f296414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.408+0000 7f29688e8640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 0x7f29640b3700 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2954007e60 tx=0x7f295400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.409+0000 7f29617fa640 1 -- 172.21.15.16:0/1289164830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954014620 con 0x7f296406b3f0 2023-11-24T06:32:01.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.409+0000 7f29617fa640 1 -- 172.21.15.16:0/1289164830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295400ee90 con 0x7f296406b3f0 2023-11-24T06:32:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.409+0000 7f29617fa640 1 -- 172.21.15.16:0/1289164830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540149c0 con 0x7f296406b3f0 2023-11-24T06:32:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.409+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29641f1910 con 0x7f296406b3f0 2023-11-24T06:32:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.409+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29640b3c40 con 0x7f296406b3f0 2023-11-24T06:32:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.410+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29640b3e30 con 0x7f296406b3f0 2023-11-24T06:32:01.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.411+0000 7f29617fa640 1 -- 172.21.15.16:0/1289164830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2954014b20 con 0x7f296406b3f0 2023-11-24T06:32:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.412+0000 7f29617fa640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f295006b470 0x7f295006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.412+0000 7f29617fa640 1 -- 172.21.15.16:0/1289164830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f295408a7c0 con 0x7f296406b3f0 2023-11-24T06:32:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.413+0000 7f2963fff640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f295006b470 0x7f295006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.414+0000 7f2963fff640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f295006b470 0x7f295006d930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2958036000 tx=0x7f295803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.416+0000 7f29617fa640 1 -- 172.21.15.16:0/1289164830 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2954055260 con 0x7f296406b3f0 2023-11-24T06:32:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.607+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29640b3250 con 0x7f295006b470 2023-11-24T06:32:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.613+0000 7f29617fa640 1 -- 172.21.15.16:0/1289164830 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48205 (secure 0 0 0) 0x7f2958044400 con 0x7f295006b470 2023-11-24T06:32:01.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f295006b470 msgr2=0x7f295006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f295006b470 0x7f295006d930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2958036000 tx=0x7f295803c040 comp rx=0 tx=0).stop 2023-11-24T06:32:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 msgr2=0x7f29640b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 0x7f29640b3700 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2954007e60 tx=0x7f295400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 shutdown_connections 2023-11-24T06:32:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f295006b470 0x7f295006d930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f296415eae0 0x7f296414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f296415e570 0x7f296415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 --2- 172.21.15.16:0/1289164830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296406b3f0 0x7f29640b3700 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 >> 172.21.15.16:0/1289164830 conn(0x7f2964070970 msgr2=0x7f2964158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 shutdown_connections 2023-11-24T06:32:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.616+0000 7f29698ea640 1 -- 172.21.15.16:0/1289164830 wait complete. 2023-11-24T06:32:01.638 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2023-11-24T06:32:00.208358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259404,"kb_used_data":9952,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715316,"statfs":{"total":773094113280,"available":772828483584,"internally_reserved":0,"allocated":10190848,"data_stored":5053494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36065,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-69,"num_write_kb":-60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002016"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-24T06:31:56.759975+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:31:56.759975+0000","last_peered":"2023-11-24T06:31:56.759975+0000","last_clean":"2023-11-24T06:31:56.759975+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:31:56.759975+0000","last_undegraded":"2023-11-24T06:31:56.759975+0000","last_fullsized":"2023-11-24T06:31:56.759975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1720320,"data_stored":1072896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.149,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5209999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.216,"5min":1.2589999999999999,"15min":1.2889999999999999},"min":{"1min":0.72099999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.704,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0700000000000001,"15min":1.0269999999999999},"min":{"1min":0.64900000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.827,"5min":2.456,"15min":2.456},"last":1.827},{"interface":"front","average":{"1min":1.258,"5min":1.2969999999999999,"15min":1.3620000000000001},"min":{"1min":0.83399999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.2769999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.204,"15min":1.2649999999999999},"min":{"1min":0.73099999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5900000000000001,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.179,"15min":1.1950000000000001},"min":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.4119999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.4119999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3,"15min":1.3819999999999999},"min":{"1min":0.78600000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6100000000000001,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.256,"5min":1.2330000000000001,"15min":1.2130000000000001},"min":{"1min":0.89500000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.653,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.653}]},{"osd":4,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3560000000000001,"15min":1.5229999999999999},"min":{"1min":0.78800000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7190000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.47,"15min":1.556},"min":{"1min":1.0449999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7509999999999999,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.4770000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3080000000000001,"15min":1.3819999999999999},"min":{"1min":0.82999999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.919,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.919},{"interface":"front","average":{"1min":1.46,"5min":1.7,"15min":1.752},"min":{"1min":0.88500000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.871,"5min":17.582999999999998,"15min":17.582999999999998},"last":1.871}]},{"osd":6,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6040000000000001,"15min":1.804},"min":{"1min":1.085,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.988,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.988},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5049999999999999,"15min":1.655},"min":{"1min":1.101,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.78,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.78}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0920000000000001,"15min":1.1000000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.73,"5min":1.829,"15min":1.829},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.129,"5min":1.109,"15min":1.111},"min":{"1min":0.65500000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.79,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2350000000000001,"15min":1.218},"min":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.595},{"interface":"front","average":{"1min":1.252,"5min":1.2110000000000001,"15min":1.21},"min":{"1min":0.875,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.645,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.401}]},{"osd":2,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.137,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.754,"5min":3.597,"15min":3.597},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.284,"15min":1.3109999999999999},"min":{"1min":0.89500000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0470000000000002,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.0069999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2909999999999999,"15min":1.3109999999999999},"min":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.8049999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.26,"15min":1.3120000000000001},"min":{"1min":0.85399999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.546,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3680000000000001,"15min":1.4279999999999999},"min":{"1min":0.98999999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.742,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3600000000000001,"15min":1.389},"min":{"1min":0.91000000000000003,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7390000000000001,"5min":3.04,"15min":3.04},"last":1.516}]},{"osd":5,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":1.0660000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7969999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.526,"15min":1.593},"min":{"1min":1.246,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9019999999999999,"5min":3.774,"15min":3.774},"last":1.671}]},{"osd":7,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5189999999999999,"15min":1.5509999999999999},"min":{"1min":1.0640000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.012,"5min":3.403,"15min":3.403},"last":1.639},{"interface":"front","average":{"1min":1.492,"5min":1.482,"15min":1.506},"min":{"1min":0.96699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.9610000000000001,"5min":3.649,"15min":3.649},"last":1.5529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":506012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.155,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7629999999999999,"5min":1.903,"15min":1.903},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.248,"15min":1.3},"min":{"1min":0.91300000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.883,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":2.569}]},{"osd":1,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.133,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.1799999999999999,"15min":1.125},"min":{"1min":0.74299999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8560000000000001,"5min":1.96,"15min":1.96},"last":0.89200000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2070000000000001,"15min":1.2050000000000001},"min":{"1min":0.877,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8300000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":2.0880000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3,"15min":1.294},"min":{"1min":0.70599999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.927}]},{"osd":3,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.302,"15min":1.3640000000000001},"min":{"1min":0.748,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.502,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":2.7639999999999998},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.385},"min":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.954,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":2.2530000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.304,"15min":1.329},"min":{"1min":0.77600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.6919999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4670000000000001,"15min":1.5249999999999999},"min":{"1min":1.1439999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.77,"5min":3.085,"15min":3.085},"last":2.4289999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.482,"15min":1.4490000000000001},"min":{"1min":1.028,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.032,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.671},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.4810000000000001,"15min":1.47},"min":{"1min":1.0429999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.9039999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.163}]},{"osd":7,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.593,"15min":1.617},"min":{"1min":1.0669999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8919999999999999,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":2.6629999999999998},{"interface":"front","average":{"1min":1.6499999999999999,"5min":1.6539999999999999,"15min":1.704},"min":{"1min":1.3520000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.089,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":2.0249999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":477053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1020000000000001,"15min":1.0820000000000001},"min":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6479999999999999,"5min":1.982,"15min":1.982},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.079,"5min":1.0900000000000001,"15min":1.1120000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.177,"15min":1.123},"min":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7370000000000001,"5min":1.909,"15min":1.909},"last":1.296},{"interface":"front","average":{"1min":1.165,"5min":1.2090000000000001,"15min":1.288},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.9339999999999999,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.349}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.216,"15min":1.2809999999999999},"min":{"1min":0.84899999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.74,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2929999999999999,"15min":1.325},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.1419999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.25,"15min":1.3180000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":5.0300000000000002,"5min":5.0300000000000002,"15min":5.0300000000000002},"last":1.768},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":0.93100000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.8220000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.8220000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.456,"15min":1.506},"min":{"1min":0.875,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.452,"15min":1.538},"min":{"1min":0.79600000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":2.0390000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4419999999999999,"15min":1.4710000000000001},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":1.9830000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.5109999999999999,"15min":1.5629999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.593}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4790000000000001,"15min":1.617},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":1.913},{"interface":"front","average":{"1min":1.635,"5min":1.5920000000000001,"15min":1.605},"min":{"1min":0.83199999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.032,"5min":4.032,"15min":4.032},"last":2.2229999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.83899999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.96799999999999997,"15min":0.96999999999999997},"min":{"1min":0.65800000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0940000000000001,"15min":1.1140000000000001},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.857,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.58999999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6679999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.60399999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.488,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.792}]},{"osd":4,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.327,"15min":1.383},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7190000000000001,"5min":3.081,"15min":3.081},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3440000000000001,"15min":1.3420000000000001},"min":{"1min":0.81000000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7909999999999999,"5min":2.758,"15min":2.758},"last":1.256}]},{"osd":5,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3540000000000001,"15min":1.3959999999999999},"min":{"1min":0.88900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6899999999999999,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":2.7000000000000002},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.544,"15min":1.7230000000000001},"min":{"1min":0.91000000000000003,"5min":0.745,"15min":0.745},"max":{"1min":1.861,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":2.6280000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.80000000000000004,"5min":0.753,"15min":0.753},"max":{"1min":2.1480000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":2.0859999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.51,"15min":1.5369999999999999},"min":{"1min":1.024,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.847,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.516}]},{"osd":7,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.552,"15min":1.601},"min":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":4.641,"15min":4.641},"last":1.603},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.528,"15min":1.5680000000000001},"min":{"1min":0.92500000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0299999999999998,"5min":3.798,"15min":3.798},"last":1.8819999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83799999999999997,"15min":0.88100000000000001},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.92900000000000005,"15min":0.99399999999999999},"min":{"1min":0.47899999999999998,"5min":0.31,"15min":0.31},"max":{"1min":1.2769999999999999,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.92700000000000005,"15min":0.95799999999999996},"min":{"1min":0.49199999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.659,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.59799999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.524,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.84299999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.077,"15min":1.1020000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7270000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.069,"15min":1.1080000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.575,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.397,"15min":1.4890000000000001},"min":{"1min":0.876,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6830000000000001,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.262,"5min":1.4059999999999999,"15min":1.4690000000000001},"min":{"1min":0.995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6850000000000001,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.96399999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.45,"15min":1.4630000000000001},"min":{"1min":0.97299999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0539999999999998,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.095},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4359999999999999,"15min":1.49},"min":{"1min":1.01,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.821,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.236}]},{"osd":6,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3819999999999999,"15min":1.4219999999999999},"min":{"1min":0.875,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.681,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.484,"15min":1.528},"min":{"1min":0.71399999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.804,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.611}]},{"osd":7,"last update":"Fri Nov 24 06:30:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.4550000000000001,"15min":1.5289999999999999},"min":{"1min":0.73399999999999999,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.873,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.454},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.5169999999999999,"15min":1.5349999999999999},"min":{"1min":0.94699999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.9359999999999999,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.8069999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92200000000000004,"15min":1.006},"min":{"1min":0.55800000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.3700000000000001,"5min":2.593,"15min":2.593},"last":1.073},{"interface":"front","average":{"1min":0.997,"5min":1.0149999999999999,"15min":1.0309999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4830000000000001,"5min":1.74,"15min":1.74},"last":1.0049999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.05,"5min":1.03,"15min":1.014},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.796,"5min":1.829,"15min":1.829},"last":0.93700000000000006}]},{"osd":3,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1120000000000001,"15min":1.123},"min":{"1min":0.58599999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.26},{"interface":"front","average":{"1min":1.026,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.53,"5min":1.798,"15min":1.798},"last":1.335}]},{"osd":4,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2450000000000001,"15min":1.2529999999999999},"min":{"1min":0.66300000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9410000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3999999999999999,"15min":1.4399999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1269999999999998,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.534}]},{"osd":5,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.345,"15min":1.319},"min":{"1min":0.98299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.504},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.3919999999999999,"15min":1.3999999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.0939999999999999,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.377}]},{"osd":6,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4470000000000001,"15min":1.411},"min":{"1min":1.0329999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.446,"15min":1.454},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.998,"5min":2.371,"15min":2.371},"last":1.1919999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.458,"15min":1.4530000000000001},"min":{"1min":0.83399999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0659999999999998,"5min":2.246,"15min":2.246},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.514,"15min":1.526},"min":{"1min":1.087,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.2410000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.4410000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1708032,"data_stored":1067421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84099999999999997,"15min":0.84699999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.355,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.88800000000000001,"15min":0.94699999999999995},"min":{"1min":0.504,"5min":0.31,"15min":0.31},"max":{"1min":1.5329999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.284}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.014,"15min":1.0029999999999999},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.206,"5min":1.0840000000000001,"15min":1.125},"min":{"1min":0.58599999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.373}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90700000000000003,"15min":0.94099999999999995},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5860000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.056,"15min":1.0800000000000001},"min":{"1min":0.53300000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.48,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2430000000000001,"15min":1.2310000000000001},"min":{"1min":0.84799999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.788,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.232},{"interface":"front","average":{"1min":1.399,"5min":1.3420000000000001,"15min":1.3180000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8109999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.3300000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.294,"15min":1.3240000000000001},"min":{"1min":0.88300000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7390000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.2589999999999999,"15min":1.274},"min":{"1min":0.84499999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.671,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4359999999999999,"15min":1.476},"min":{"1min":1.0289999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0209999999999999,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.117},{"interface":"front","average":{"1min":1.407,"5min":1.4079999999999999,"15min":1.49},"min":{"1min":1.089,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9339999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.0509999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4299999999999999,"15min":1.45},"min":{"1min":0.86899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0750000000000002,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.405,"15min":1.421},"min":{"1min":1.0269999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0150000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.1970000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:32:01.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:32:01.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.862+0000 7ffa234ea640 1 Processor -- start 2023-11-24T06:32:01.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.862+0000 7ffa234ea640 1 -- start start 2023-11-24T06:32:01.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.863+0000 7ffa234ea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa1c160920 0x7ffa1c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.863+0000 7ffa234ea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cfea0 0x7ffa1c161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.863+0000 7ffa234ea640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1c0d0a10 con 0x7ffa1c0cf9f0 2023-11-24T06:32:01.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.864+0000 7ffa234ea640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1c0d0330 con 0x7ffa1c160920 2023-11-24T06:32:01.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.864+0000 7ffa234ea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c0d1f50 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.864+0000 7ffa21ce7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffa1c0cf9f0 0x7ffa1c161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41336/0 (socket says 172.21.15.16:41336) 2023-11-24T06:32:01.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.864+0000 7ffa21ce7640 1 -- 172.21.15.16:0/1655491214 learned_addr learned my addr 172.21.15.16:0/1655491214 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:01.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.864+0000 7ffa224e8640 1 --2- 172.21.15.16:0/1655491214 >> v2:172.21.15.16:3300/0 conn(0x7ffa1c160920 0x7ffa1c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.864+0000 7ffa22ce9640 1 --2- 172.21.15.16:0/1655491214 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cfea0 0x7ffa1c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.864+0000 7ffa224e8640 1 -- 172.21.15.16:0/1655491214 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cfea0 msgr2=0x7ffa1c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:01.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.864+0000 7ffa224e8640 1 --2- 172.21.15.16:0/1655491214 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cfea0 0x7ffa1c161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.864+0000 7ffa224e8640 1 -- 172.21.15.16:0/1655491214 >> v1:172.21.15.16:6789/0 conn(0x7ffa1c0cf9f0 legacy=0x7ffa1c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.865+0000 7ffa224e8640 1 -- 172.21.15.16:0/1655491214 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c0cf080 con 0x7ffa1c160920 2023-11-24T06:32:01.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.865+0000 7ffa214e6640 1 -- 172.21.15.16:0/1655491214 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368527844 0 0) 0x7ffa1c0d0a10 con 0x7ffa1c0cf9f0 2023-11-24T06:32:01.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.866+0000 7ffa224e8640 1 --2- 172.21.15.16:0/1655491214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c160920 0x7ffa1c0b3f30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097a0 tx=0x7ffa1802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df565bfbc00c006c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:01.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.866+0000 7ffa214e6640 1 -- 172.21.15.16:0/1655491214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1802d040 con 0x7ffa1c160920 2023-11-24T06:32:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.866+0000 7ffa214e6640 1 -- 172.21.15.16:0/1655491214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa18031810 con 0x7ffa1c160920 2023-11-24T06:32:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.866+0000 7ffa214e6640 1 -- 172.21.15.16:0/1655491214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18031ba0 con 0x7ffa1c160920 2023-11-24T06:32:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.867+0000 7ffa234ea640 1 -- 172.21.15.16:0/1655491214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c160920 msgr2=0x7ffa1c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.867+0000 7ffa234ea640 1 --2- 172.21.15.16:0/1655491214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c160920 0x7ffa1c0b3f30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097a0 tx=0x7ffa1802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.867+0000 7ffa234ea640 1 -- 172.21.15.16:0/1655491214 shutdown_connections 2023-11-24T06:32:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.867+0000 7ffa234ea640 1 --2- 172.21.15.16:0/1655491214 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cfea0 0x7ffa1c161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.867+0000 7ffa234ea640 1 --2- 172.21.15.16:0/1655491214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c160920 0x7ffa1c0b3f30 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.867+0000 7ffa234ea640 1 -- 172.21.15.16:0/1655491214 >> 172.21.15.16:0/1655491214 conn(0x7ffa1c0d3090 msgr2=0x7ffa1c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.867+0000 7ffa234ea640 1 -- 172.21.15.16:0/1655491214 shutdown_connections 2023-11-24T06:32:01.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.867+0000 7ffa234ea640 1 -- 172.21.15.16:0/1655491214 wait complete. 2023-11-24T06:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.869+0000 7ffa234ea640 1 Processor -- start 2023-11-24T06:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.869+0000 7ffa234ea640 1 -- start start 2023-11-24T06:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.870+0000 7ffa234ea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cf9f0 0x7ffa1c1f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.870+0000 7ffa234ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c0cfea0 0x7ffa1c0b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.870+0000 7ffa234ea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa1c160920 0x7ffa1c0cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.870+0000 7ffa234ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c15f5e0 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.870+0000 7ffa234ea640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c0ca160 con 0x7ffa1c160920 2023-11-24T06:32:01.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.870+0000 7ffa234ea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c0ca440 con 0x7ffa1c0cf9f0 2023-11-24T06:32:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.870+0000 7ffa21ce7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c0cfea0 0x7ffa1c0b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa21ce7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c0cfea0 0x7ffa1c0b7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54012/0 (socket says 172.21.15.16:54012) 2023-11-24T06:32:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa21ce7640 1 -- 172.21.15.16:0/1362161888 learned_addr learned my addr 172.21.15.16:0/1362161888 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa224e8640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cf9f0 0x7ffa1c1f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa22ce9640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa1c160920 0x7ffa1c0cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa21ce7640 1 -- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cf9f0 msgr2=0x7ffa1c1f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa21ce7640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cf9f0 0x7ffa1c1f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa21ce7640 1 -- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa1c160920 msgr2=0x7ffa1c0cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa21ce7640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa1c160920 0x7ffa1c0cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa21ce7640 1 -- 172.21.15.16:0/1362161888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa18009400 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.871+0000 7ffa21ce7640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c0cfea0 0x7ffa1c0b7350 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c430 tx=0x7ffa0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.872+0000 7ffa12ffd640 1 -- 172.21.15.16:0/1362161888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0074e0 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.872+0000 7ffa12ffd640 1 -- 172.21.15.16:0/1362161888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c007660 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.872+0000 7ffa12ffd640 1 -- 172.21.15.16:0/1362161888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c004a20 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.872+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c0ca720 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.872+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c0cace0 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.873+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c067450 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.874+0000 7ffa12ffd640 1 -- 172.21.15.16:0/1362161888 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ffa0c004b80 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.875+0000 7ffa12ffd640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0006b470 0x7ffa0006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.875+0000 7ffa12ffd640 1 -- 172.21.15.16:0/1362161888 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7ffa0c089ca0 con 0x7ffa1c0cfea0 2023-11-24T06:32:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.876+0000 7ffa224e8640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0006b470 0x7ffa0006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.877+0000 7ffa224e8640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0006b470 0x7ffa0006d930 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffa18036000 tx=0x7ffa1803c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:01.880+0000 7ffa12ffd640 1 -- 172.21.15.16:0/1362161888 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa0c0546f0 con 0x7ffa1c0cfea0 2023-11-24T06:32:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.076+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa1c08c7e0 con 0x7ffa1c0cfea0 2023-11-24T06:32:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.079+0000 7ffa12ffd640 1 -- 172.21.15.16:0/1362161888 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+13031 (secure 0 0 0) 0x7ffa0c057d10 con 0x7ffa1c0cfea0 2023-11-24T06:32:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:32:02.082 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"fsid":"6dc1668b-58b0-47dd-b8dc-4164e5a228ab","created":"2023-11-24T06:26:36.947380+0000","modified":"2023-11-24T06:31:58.049660+0000","last_up_change":"2023-11-24T06:26:48.927403+0000","last_in_change":"2023-11-24T06:26:40.447025+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:26:50.170210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"147be2d1-6231-4031-8f98-d0b50bcd068f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6801","nonce":4232398165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6803","nonce":4232398165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6807","nonce":4232398165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":4232398165},{"type":"v1","addr":"172.21.15.16:6805","nonce":4232398165}]},"public_addr":"172.21.15.16:6801/4232398165","cluster_addr":"172.21.15.16:6803/4232398165","heartbeat_back_addr":"172.21.15.16:6807/4232398165","heartbeat_front_addr":"172.21.15.16:6805/4232398165","state":["exists","up"]},{"osd":1,"uuid":"55e17526-cd19-4a3d-a36b-c75b35e02906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6812","nonce":3842150677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6815","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6817","nonce":3842150677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6825","nonce":3842150677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3842150677},{"type":"v1","addr":"172.21.15.16:6821","nonce":3842150677}]},"public_addr":"172.21.15.16:6812/3842150677","cluster_addr":"172.21.15.16:6817/3842150677","heartbeat_back_addr":"172.21.15.16:6825/3842150677","heartbeat_front_addr":"172.21.15.16:6821/3842150677","state":["exists","up"]},{"osd":2,"uuid":"b4036fde-c18b-4aab-8fbb-5e735b8e5371","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6809","nonce":620212774}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6811","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6813","nonce":620212774}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6819","nonce":620212774}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":620212774},{"type":"v1","addr":"172.21.15.16:6816","nonce":620212774}]},"public_addr":"172.21.15.16:6809/620212774","cluster_addr":"172.21.15.16:6813/620212774","heartbeat_back_addr":"172.21.15.16:6819/620212774","heartbeat_front_addr":"172.21.15.16:6816/620212774","state":["exists","up"]},{"osd":3,"uuid":"e4695946-b031-4353-b5b2-de60bbf183e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6824","nonce":3227335215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6827","nonce":3227335215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6831","nonce":3227335215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3227335215},{"type":"v1","addr":"172.21.15.16:6829","nonce":3227335215}]},"public_addr":"172.21.15.16:6824/3227335215","cluster_addr":"172.21.15.16:6827/3227335215","heartbeat_back_addr":"172.21.15.16:6831/3227335215","heartbeat_front_addr":"172.21.15.16:6829/3227335215","state":["exists","up"]},{"osd":4,"uuid":"0132f098-065d-4b24-9340-025b067ab6e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6801","nonce":3033116473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6803","nonce":3033116473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6807","nonce":3033116473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3033116473},{"type":"v1","addr":"172.21.15.173:6805","nonce":3033116473}]},"public_addr":"172.21.15.173:6801/3033116473","cluster_addr":"172.21.15.173:6803/3033116473","heartbeat_back_addr":"172.21.15.173:6807/3033116473","heartbeat_front_addr":"172.21.15.173:6805/3033116473","state":["exists","up"]},{"osd":5,"uuid":"36be1f62-f8bb-4c5d-a94c-1b59a67cbead","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6817","nonce":2053607116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6819","nonce":2053607116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6829","nonce":2053607116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6821","nonce":2053607116},{"type":"v1","addr":"172.21.15.173:6824","nonce":2053607116}]},"public_addr":"172.21.15.173:6817/2053607116","cluster_addr":"172.21.15.173:6819/2053607116","heartbeat_back_addr":"172.21.15.173:6829/2053607116","heartbeat_front_addr":"172.21.15.173:6824/2053607116","state":["exists","up"]},{"osd":6,"uuid":"0fb31be6-68df-42ac-9002-e21325dd058e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6809","nonce":2309591130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6811","nonce":2309591130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6815","nonce":2309591130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2309591130},{"type":"v1","addr":"172.21.15.173:6813","nonce":2309591130}]},"public_addr":"172.21.15.173:6809/2309591130","cluster_addr":"172.21.15.173:6811/2309591130","heartbeat_back_addr":"172.21.15.173:6815/2309591130","heartbeat_front_addr":"172.21.15.173:6813/2309591130","state":["exists","up"]},{"osd":7,"uuid":"b8fc590d-7dcb-4691-9d07-7ec1b870d761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6822","nonce":2972943157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6823","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6825","nonce":2972943157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6831","nonce":2972943157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2972943157},{"type":"v1","addr":"172.21.15.173:6828","nonce":2972943157}]},"public_addr":"172.21.15.173:6822/2972943157","cluster_addr":"172.21.15.173:6825/2972943157","heartbeat_back_addr":"172.21.15.173:6831/2972943157","heartbeat_front_addr":"172.21.15.173:6828/2972943157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.760035+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.818945+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.793072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.815218+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.750230+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.792728+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.813432+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:26:45.559510+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2773872585":"2023-11-25T06:31:54.731955+0000","172.21.15.16:6833/2773872585":"2023-11-25T06:31:54.731955+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:32:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.081+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0006b470 msgr2=0x7ffa0006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.081+0000 7ffa234ea640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0006b470 0x7ffa0006d930 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffa18036000 tx=0x7ffa1803c040 comp rx=0 tx=0).stop 2023-11-24T06:32:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.081+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c0cfea0 msgr2=0x7ffa1c0b7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.081+0000 7ffa234ea640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c0cfea0 0x7ffa1c0b7350 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c430 tx=0x7ffa0c00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:32:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.082+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 shutdown_connections 2023-11-24T06:32:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.082+0000 7ffa234ea640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ffa0006b470 0x7ffa0006d930 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.082+0000 7ffa234ea640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa1c160920 0x7ffa1c0cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.082+0000 7ffa234ea640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1c0cfea0 0x7ffa1c0b7350 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.082+0000 7ffa234ea640 1 --2- 172.21.15.16:0/1362161888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa1c0cf9f0 0x7ffa1c1f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.082+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 >> 172.21.15.16:0/1362161888 conn(0x7ffa1c0d3090 msgr2=0x7ffa1c15faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.082+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 shutdown_connections 2023-11-24T06:32:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:02.082+0000 7ffa234ea640 1 -- 172.21.15.16:0/1362161888 wait complete. 2023-11-24T06:32:03.110 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T06:32:03.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:32:03.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.345+0000 7fe84759e640 1 Processor -- start 2023-11-24T06:32:03.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.345+0000 7fe84759e640 1 -- start start 2023-11-24T06:32:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.347+0000 7fe84759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe848159590 0x7fe848070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.347+0000 7fe84759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8480b3c20 0x7fe8480ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.347+0000 7fe84759e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe848071790 con 0x7fe8480cd220 2023-11-24T06:32:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.347+0000 7fe84759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe848071100 con 0x7fe8480b3c20 2023-11-24T06:32:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.347+0000 7fe84759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe848070860 con 0x7fe848159590 2023-11-24T06:32:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.347+0000 7fe83ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8480b3c20 0x7fe8480ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.347+0000 7fe83ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe8480b3c20 0x7fe8480ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40040/0 (socket says 172.21.15.16:40040) 2023-11-24T06:32:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.347+0000 7fe83ffff640 1 -- 172.21.15.16:0/1370323164 learned_addr learned my addr 172.21.15.16:0/1370323164 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.348+0000 7fe84659c640 1 --2- 172.21.15.16:0/1370323164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe848159590 0x7fe848070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.348+0000 7fe83ffff640 1 -- 172.21.15.16:0/1370323164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe848159590 msgr2=0x7fe848070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:03.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.348+0000 7fe83ffff640 1 --2- 172.21.15.16:0/1370323164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe848159590 0x7fe848070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.348+0000 7fe83ffff640 1 -- 172.21.15.16:0/1370323164 >> v1:172.21.15.173:6789/0 conn(0x7fe8480cd220 legacy=0x7fe84806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.348+0000 7fe83ffff640 1 -- 172.21.15.16:0/1370323164 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe848070b40 con 0x7fe8480b3c20 2023-11-24T06:32:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.349+0000 7fe83ffff640 1 --2- 172.21.15.16:0/1370323164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 0x7fe8480ccce0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe83000aa10 tx=0x7fe83002cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52e8ea89279336d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.350+0000 7fe845d9b640 1 -- 172.21.15.16:0/1370323164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83002db80 con 0x7fe8480b3c20 2023-11-24T06:32:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.350+0000 7fe845d9b640 1 -- 172.21.15.16:0/1370323164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83002dce0 con 0x7fe8480b3c20 2023-11-24T06:32:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.350+0000 7fe845d9b640 1 -- 172.21.15.16:0/1370323164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830035660 con 0x7fe8480b3c20 2023-11-24T06:32:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.351+0000 7fe84759e640 1 -- 172.21.15.16:0/1370323164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 msgr2=0x7fe8480ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.351+0000 7fe84759e640 1 --2- 172.21.15.16:0/1370323164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 0x7fe8480ccce0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe83000aa10 tx=0x7fe83002cdd0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.351+0000 7fe84759e640 1 -- 172.21.15.16:0/1370323164 shutdown_connections 2023-11-24T06:32:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.351+0000 7fe84759e640 1 --2- 172.21.15.16:0/1370323164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 0x7fe8480ccce0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.351+0000 7fe84759e640 1 --2- 172.21.15.16:0/1370323164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe848159590 0x7fe848070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.351+0000 7fe84759e640 1 -- 172.21.15.16:0/1370323164 >> 172.21.15.16:0/1370323164 conn(0x7fe84806a800 msgr2=0x7fe84806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.351+0000 7fe84759e640 1 -- 172.21.15.16:0/1370323164 shutdown_connections 2023-11-24T06:32:03.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.351+0000 7fe84759e640 1 -- 172.21.15.16:0/1370323164 wait complete. 2023-11-24T06:32:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.353+0000 7fe84759e640 1 Processor -- start 2023-11-24T06:32:03.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.353+0000 7fe84759e640 1 -- start start 2023-11-24T06:32:03.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe84759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 0x7fe8481f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe84759e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe8480cd220 0x7fe8481f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe84759e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe848159590 0x7fe84814b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe84759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8481506a0 con 0x7fe8480b3c20 2023-11-24T06:32:03.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe84759e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe848150370 con 0x7fe848159590 2023-11-24T06:32:03.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe84759e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8481f6cf0 con 0x7fe8480cd220 2023-11-24T06:32:03.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe84659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 0x7fe8481f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe84659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 0x7fe8481f3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40054/0 (socket says 172.21.15.16:40054) 2023-11-24T06:32:03.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe84659c640 1 -- 172.21.15.16:0/717961158 learned_addr learned my addr 172.21.15.16:0/717961158 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:03.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.354+0000 7fe83ffff640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe8480cd220 0x7fe8481f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.355+0000 7fe846d9d640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe848159590 0x7fe84814b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.355+0000 7fe84659c640 1 -- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe8480cd220 msgr2=0x7fe8481f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.355+0000 7fe84659c640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe8480cd220 0x7fe8481f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.355+0000 7fe84659c640 1 -- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe848159590 msgr2=0x7fe84814b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.355+0000 7fe84659c640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe848159590 0x7fe84814b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.355+0000 7fe84659c640 1 -- 172.21.15.16:0/717961158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83000a670 con 0x7fe8480b3c20 2023-11-24T06:32:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.355+0000 7fe84659c640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 0x7fe8481f3cd0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe83800e870 tx=0x7fe83800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.355+0000 7fe83effd640 1 -- 172.21.15.16:0/717961158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8380094f0 con 0x7fe8480b3c20 2023-11-24T06:32:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.356+0000 7fe83effd640 1 -- 172.21.15.16:0/717961158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83800c940 con 0x7fe8480b3c20 2023-11-24T06:32:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.356+0000 7fe83effd640 1 -- 172.21.15.16:0/717961158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83800ccf0 con 0x7fe8480b3c20 2023-11-24T06:32:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.356+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84814bc40 con 0x7fe8480b3c20 2023-11-24T06:32:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.356+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8481f7270 con 0x7fe8480b3c20 2023-11-24T06:32:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.357+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8480b1f30 con 0x7fe8480b3c20 2023-11-24T06:32:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.358+0000 7fe83effd640 1 -- 172.21.15.16:0/717961158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe838013470 con 0x7fe8480b3c20 2023-11-24T06:32:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.359+0000 7fe83effd640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe82806b470 0x7fe82806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.359+0000 7fe83effd640 1 -- 172.21.15.16:0/717961158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fe83808a240 con 0x7fe8480b3c20 2023-11-24T06:32:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.359+0000 7fe83effd640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7fe8280712d0 0x7fe8280736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.359+0000 7fe83effd640 1 -- 172.21.15.16:0/717961158 --> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe828073dc0 con 0x7fe8280712d0 2023-11-24T06:32:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.359+0000 7fe83effd640 1 -- 172.21.15.16:0/717961158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe83808a520 con 0x7fe8480b3c20 2023-11-24T06:32:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.359+0000 7fe83ffff640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe82806b470 0x7fe82806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.359+0000 7fe846d9d640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7fe8280712d0 0x7fe8280736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.360+0000 7fe846d9d640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7fe8280712d0 0x7fe8280736f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.360+0000 7fe83ffff640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe82806b470 0x7fe82806d930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe830037000 tx=0x7fe830008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.361+0000 7fe83effd640 1 -- 172.21.15.16:0/717961158 <== osd.0 v2:172.21.15.16:6800/4232398165 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe828073dc0 con 0x7fe8280712d0 2023-11-24T06:32:03.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.376+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 --> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe84806d760 con 0x7fe8280712d0 2023-11-24T06:32:03.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.376+0000 7fe83effd640 1 -- 172.21.15.16:0/717961158 <== osd.0 v2:172.21.15.16:6800/4232398165 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe840001e90 con 0x7fe8280712d0 2023-11-24T06:32:03.379 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:32:03.379 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:32:03.379 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:32:03.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.376+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7fe8280712d0 msgr2=0x7fe8280736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.376+0000 7fe84759e640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.16:6800/4232398165,v1:172.21.15.16:6801/4232398165] conn(0x7fe8280712d0 0x7fe8280736f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.377+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe82806b470 msgr2=0x7fe82806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.377+0000 7fe84759e640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe82806b470 0x7fe82806d930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe830037000 tx=0x7fe830008040 comp rx=0 tx=0).stop 2023-11-24T06:32:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.377+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 msgr2=0x7fe8481f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.377+0000 7fe84759e640 1 --2- 172.21.15.16:0/717961158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8480b3c20 0x7fe8481f3cd0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fe83800e870 tx=0x7fe83800ed30 comp rx=0 tx=0).stop 2023-11-24T06:32:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.377+0000 7fe846d9d640 1 -- 172.21.15.16:0/717961158 reap_dead start 2023-11-24T06:32:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.377+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 shutdown_connections 2023-11-24T06:32:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.377+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 >> 172.21.15.16:0/717961158 conn(0x7fe84806a800 msgr2=0x7fe84814c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.377+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 shutdown_connections 2023-11-24T06:32:03.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.377+0000 7fe84759e640 1 -- 172.21.15.16:0/717961158 wait complete. 2023-11-24T06:32:03.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T06:32:03.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.609+0000 7f992bfff640 1 Processor -- start 2023-11-24T06:32:03.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.609+0000 7f992bfff640 1 -- start start 2023-11-24T06:32:03.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.610+0000 7f992bfff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f992c1447f0 0x7f992c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c06c470 0x7f992c0cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f992c06dab0 0x7f992c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992bfff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c06d690 con 0x7f992c06dab0 2023-11-24T06:32:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992bfff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c0cebb0 con 0x7f992c1447f0 2023-11-24T06:32:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c15c5c0 con 0x7f992c06c470 2023-11-24T06:32:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f992c06dab0 0x7f992c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f992c06dab0 0x7f992c0ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40064/0 (socket says 172.21.15.16:40064) 2023-11-24T06:32:03.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992b7fe640 1 -- 172.21.15.16:0/1384460831 learned_addr learned my addr 172.21.15.16:0/1384460831 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f99227fc640 1 --2- 172.21.15.16:0/1384460831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c06c470 0x7f992c0cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992b7fe640 1 -- 172.21.15.16:0/1384460831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c06c470 msgr2=0x7f992c0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992affd640 1 --2- 172.21.15.16:0/1384460831 >> v2:172.21.15.173:3300/0 conn(0x7f992c1447f0 0x7f992c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.611+0000 7f992b7fe640 1 --2- 172.21.15.16:0/1384460831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c06c470 0x7f992c0cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.612+0000 7f992b7fe640 1 -- 172.21.15.16:0/1384460831 >> v2:172.21.15.173:3300/0 conn(0x7f992c1447f0 msgr2=0x7f992c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.612+0000 7f992b7fe640 1 --2- 172.21.15.16:0/1384460831 >> v2:172.21.15.173:3300/0 conn(0x7f992c1447f0 0x7f992c15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.612+0000 7f992b7fe640 1 -- 172.21.15.16:0/1384460831 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c06ca10 con 0x7f992c06dab0 2023-11-24T06:32:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.612+0000 7f99227fc640 1 --2- 172.21.15.16:0/1384460831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c06c470 0x7f992c0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.612+0000 7f992affd640 1 --2- 172.21.15.16:0/1384460831 >> v2:172.21.15.173:3300/0 conn(0x7f992c1447f0 0x7f992c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.613+0000 7f992b7fe640 1 --2- 172.21.15.16:0/1384460831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 0x7f992c0ccb20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f992c072520 tx=0x7f992402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7757617cb626e7cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.613+0000 7f992a7fc640 1 -- 172.21.15.16:0/1384460831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992402c9f0 con 0x7f992c06dab0 2023-11-24T06:32:03.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.613+0000 7f992a7fc640 1 -- 172.21.15.16:0/1384460831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992402cb50 con 0x7f992c06dab0 2023-11-24T06:32:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.613+0000 7f992a7fc640 1 -- 172.21.15.16:0/1384460831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240343f0 con 0x7f992c06dab0 2023-11-24T06:32:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.614+0000 7f992bfff640 1 -- 172.21.15.16:0/1384460831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 msgr2=0x7f992c0ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.614+0000 7f992bfff640 1 --2- 172.21.15.16:0/1384460831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 0x7f992c0ccb20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f992c072520 tx=0x7f992402be50 comp rx=0 tx=0).stop 2023-11-24T06:32:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.614+0000 7f992bfff640 1 -- 172.21.15.16:0/1384460831 shutdown_connections 2023-11-24T06:32:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.614+0000 7f992bfff640 1 --2- 172.21.15.16:0/1384460831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 0x7f992c0ccb20 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.614+0000 7f992bfff640 1 --2- 172.21.15.16:0/1384460831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c06c470 0x7f992c0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.614+0000 7f992bfff640 1 --2- 172.21.15.16:0/1384460831 >> v2:172.21.15.173:3300/0 conn(0x7f992c1447f0 0x7f992c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.614+0000 7f992bfff640 1 -- 172.21.15.16:0/1384460831 >> 172.21.15.16:0/1384460831 conn(0x7f992c06a3e0 msgr2=0x7f992c06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.614+0000 7f992bfff640 1 -- 172.21.15.16:0/1384460831 shutdown_connections 2023-11-24T06:32:03.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.614+0000 7f992bfff640 1 -- 172.21.15.16:0/1384460831 wait complete. 2023-11-24T06:32:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.616+0000 7f992bfff640 1 Processor -- start 2023-11-24T06:32:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.616+0000 7f992bfff640 1 -- start start 2023-11-24T06:32:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.617+0000 7f992bfff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f992c06c470 0x7f992c1f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.617+0000 7f992bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 0x7f992c13e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.617+0000 7f992bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c1447f0 0x7f992c1399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.617+0000 7f992bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c15a1e0 con 0x7f992c06dab0 2023-11-24T06:32:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.617+0000 7f992bfff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c13a1e0 con 0x7f992c06c470 2023-11-24T06:32:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.617+0000 7f992bfff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c13a4c0 con 0x7f992c1447f0 2023-11-24T06:32:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.617+0000 7f99227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 0x7f992c13e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.617+0000 7f99227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 0x7f992c13e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40072/0 (socket says 172.21.15.16:40072) 2023-11-24T06:32:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.617+0000 7f99227fc640 1 -- 172.21.15.16:0/1065435003 learned_addr learned my addr 172.21.15.16:0/1065435003 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f992b7fe640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c1447f0 0x7f992c1399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f992affd640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f992c06c470 0x7f992c1f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f99227fc640 1 -- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c1447f0 msgr2=0x7f992c1399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f99227fc640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c1447f0 0x7f992c1399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f99227fc640 1 -- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f992c06c470 msgr2=0x7f992c1f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f99227fc640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f992c06c470 0x7f992c1f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f99227fc640 1 -- 172.21.15.16:0/1065435003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924009b30 con 0x7f992c06dab0 2023-11-24T06:32:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f992affd640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f992c06c470 0x7f992c1f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f99227fc640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 0x7f992c13e670 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f991400c7f0 tx=0x7f991400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:03.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.618+0000 7f9928ff9640 1 -- 172.21.15.16:0/1065435003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914007820 con 0x7f992c06dab0 2023-11-24T06:32:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.619+0000 7f9928ff9640 1 -- 172.21.15.16:0/1065435003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9914007980 con 0x7f992c06dab0 2023-11-24T06:32:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.619+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c13a7a0 con 0x7f992c06dab0 2023-11-24T06:32:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.619+0000 7f9928ff9640 1 -- 172.21.15.16:0/1065435003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914007d30 con 0x7f992c06dab0 2023-11-24T06:32:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.619+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c139ef0 con 0x7f992c06dab0 2023-11-24T06:32:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.620+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c067450 con 0x7f992c06dab0 2023-11-24T06:32:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.621+0000 7f9928ff9640 1 -- 172.21.15.16:0/1065435003 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f99140113f0 con 0x7f992c06dab0 2023-11-24T06:32:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.622+0000 7f9928ff9640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f990c06b470 0x7f990c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.622+0000 7f9928ff9640 1 -- 172.21.15.16:0/1065435003 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f991408a2f0 con 0x7f992c06dab0 2023-11-24T06:32:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.622+0000 7f992affd640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f990c06b470 0x7f990c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:03.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.624+0000 7f992affd640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f990c06b470 0x7f990c06d930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f991c006fd0 tx=0x7f991c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:03.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.627+0000 7f9928ff9640 1 -- 172.21.15.16:0/1065435003 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9914054b70 con 0x7f992c06dab0 2023-11-24T06:32:03.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.820+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f992c13a630 con 0x7f990c06b470 2023-11-24T06:32:03.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.821+0000 7f9928ff9640 1 -- 172.21.15.16:0/1065435003 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f992c13a630 con 0x7f990c06b470 2023-11-24T06:32:03.823 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T06:32:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f990c06b470 msgr2=0x7f990c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f990c06b470 0x7f990c06d930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f991c006fd0 tx=0x7f991c008040 comp rx=0 tx=0).stop 2023-11-24T06:32:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 msgr2=0x7f992c13e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 0x7f992c13e670 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f991400c7f0 tx=0x7f991400ccb0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 shutdown_connections 2023-11-24T06:32:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f990c06b470 0x7f990c06d930 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f992c1447f0 0x7f992c1399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f992c06dab0 0x7f992c13e670 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 --2- 172.21.15.16:0/1065435003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f992c06c470 0x7f992c1f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 >> 172.21.15.16:0/1065435003 conn(0x7f992c06a3e0 msgr2=0x7f992c142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.823+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 shutdown_connections 2023-11-24T06:32:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:03.824+0000 7f992bfff640 1 -- 172.21.15.16:0/1065435003 wait complete. 2023-11-24T06:32:03.844 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T06:32:03.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:32:04.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.066+0000 7f813b59e640 1 Processor -- start 2023-11-24T06:32:04.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.066+0000 7f813b59e640 1 -- start start 2023-11-24T06:32:04.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.068+0000 7f813b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c143c40 0x7f813c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.068+0000 7f813b59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f813c0717d0 0x7f813c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.068+0000 7f813b59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813c070e50 con 0x7f813c0cca40 2023-11-24T06:32:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.068+0000 7f813b59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813c0723c0 con 0x7f813c0717d0 2023-11-24T06:32:04.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.068+0000 7f813b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0cce70 con 0x7f813c143c40 2023-11-24T06:32:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.069+0000 7f813a59c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c143c40 0x7f813c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.069+0000 7f813a59c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c143c40 0x7f813c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.16:34064/0 (socket says 172.21.15.16:34064) 2023-11-24T06:32:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.069+0000 7f813a59c640 1 -- 172.21.15.16:0/3134694799 learned_addr learned my addr 172.21.15.16:0/3134694799 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:04.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.069+0000 7f8139d9b640 1 --2- 172.21.15.16:0/3134694799 >> v2:172.21.15.173:3300/0 conn(0x7f813c0717d0 0x7f813c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.070+0000 7f813a59c640 1 -- 172.21.15.16:0/3134694799 >> v2:172.21.15.173:3300/0 conn(0x7f813c0717d0 msgr2=0x7f813c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.070+0000 7f813a59c640 1 --2- 172.21.15.16:0/3134694799 >> v2:172.21.15.173:3300/0 conn(0x7f813c0717d0 0x7f813c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.070+0000 7f813a59c640 1 -- 172.21.15.16:0/3134694799 >> v1:172.21.15.173:6789/0 conn(0x7f813c0cca40 legacy=0x7f813c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.070+0000 7f813a59c640 1 -- 172.21.15.16:0/3134694799 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c06e170 con 0x7f813c143c40 2023-11-24T06:32:04.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.072+0000 7f813a59c640 1 --2- 172.21.15.16:0/3134694799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c143c40 0x7f813c071290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f812c009870 tx=0x7f812c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42999547d7ca0814 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.072+0000 7f813959a640 1 -- 172.21.15.16:0/3134694799 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c031640 con 0x7f813c143c40 2023-11-24T06:32:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.072+0000 7f813959a640 1 -- 172.21.15.16:0/3134694799 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812c0317a0 con 0x7f813c143c40 2023-11-24T06:32:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.072+0000 7f813959a640 1 -- 172.21.15.16:0/3134694799 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812c031b30 con 0x7f813c143c40 2023-11-24T06:32:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.073+0000 7f813b59e640 1 -- 172.21.15.16:0/3134694799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c143c40 msgr2=0x7f813c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.073+0000 7f813b59e640 1 --2- 172.21.15.16:0/3134694799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c143c40 0x7f813c071290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f812c009870 tx=0x7f812c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.073+0000 7f813b59e640 1 -- 172.21.15.16:0/3134694799 shutdown_connections 2023-11-24T06:32:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.073+0000 7f813b59e640 1 --2- 172.21.15.16:0/3134694799 >> v2:172.21.15.173:3300/0 conn(0x7f813c0717d0 0x7f813c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.073+0000 7f813b59e640 1 --2- 172.21.15.16:0/3134694799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c143c40 0x7f813c071290 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.073+0000 7f813b59e640 1 -- 172.21.15.16:0/3134694799 >> 172.21.15.16:0/3134694799 conn(0x7f813c06a800 msgr2=0x7f813c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.074+0000 7f813b59e640 1 -- 172.21.15.16:0/3134694799 shutdown_connections 2023-11-24T06:32:04.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.074+0000 7f813b59e640 1 -- 172.21.15.16:0/3134694799 wait complete. 2023-11-24T06:32:04.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.076+0000 7f813b59e640 1 Processor -- start 2023-11-24T06:32:04.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.076+0000 7f813b59e640 1 -- start start 2023-11-24T06:32:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f813b59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c0717d0 0x7f813c136cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f813b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c0cca40 0x7f813c15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f813b59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f813c143c40 0x7f813c13dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f813b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0a8530 con 0x7f813c0cca40 2023-11-24T06:32:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f813b59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c135c50 con 0x7f813c143c40 2023-11-24T06:32:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f813b59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c135f30 con 0x7f813c0717d0 2023-11-24T06:32:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f8139d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c0cca40 0x7f813c15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f8139d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c0cca40 0x7f813c15a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40088/0 (socket says 172.21.15.16:40088) 2023-11-24T06:32:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f8139d9b640 1 -- 172.21.15.16:0/579458480 learned_addr learned my addr 172.21.15.16:0/579458480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.078+0000 7f813a59c640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c0717d0 0x7f813c136cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f813ad9d640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f813c143c40 0x7f813c13dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f8139d9b640 1 -- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c0717d0 msgr2=0x7f813c136cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f8139d9b640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f813c0717d0 0x7f813c136cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f8139d9b640 1 -- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f813c143c40 msgr2=0x7f813c13dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f8139d9b640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f813c143c40 0x7f813c13dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f8139d9b640 1 -- 172.21.15.16:0/579458480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c0094d0 con 0x7f813c0cca40 2023-11-24T06:32:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f813ad9d640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f813c143c40 0x7f813c13dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f8139d9b640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c0cca40 0x7f813c15a6c0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f813000e9b0 tx=0x7f813000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f812affd640 1 -- 172.21.15.16:0/579458480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130009670 con 0x7f813c0cca40 2023-11-24T06:32:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f812affd640 1 -- 172.21.15.16:0/579458480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813000c9a0 con 0x7f813c0cca40 2023-11-24T06:32:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f812affd640 1 -- 172.21.15.16:0/579458480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813000cd50 con 0x7f813c0cca40 2023-11-24T06:32:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.079+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813c136210 con 0x7f813c0cca40 2023-11-24T06:32:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.080+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c1367d0 con 0x7f813c0cca40 2023-11-24T06:32:04.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.080+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f813c0720e0 con 0x7f813c0cca40 2023-11-24T06:32:04.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.082+0000 7f812affd640 1 -- 172.21.15.16:0/579458480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8130013470 con 0x7f813c0cca40 2023-11-24T06:32:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.084+0000 7f812affd640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f811c06b470 0x7f811c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.084+0000 7f812affd640 1 -- 172.21.15.16:0/579458480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f813008a110 con 0x7f813c0cca40 2023-11-24T06:32:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.084+0000 7f812affd640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f811c0712d0 0x7f811c0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.084+0000 7f812affd640 1 -- 172.21.15.16:0/579458480 --> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f811c073dc0 con 0x7f811c0712d0 2023-11-24T06:32:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.084+0000 7f812affd640 1 -- 172.21.15.16:0/579458480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f813008c570 con 0x7f813c0cca40 2023-11-24T06:32:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.084+0000 7f813a59c640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f811c06b470 0x7f811c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.085+0000 7f813ad9d640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f811c0712d0 0x7f811c0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.085+0000 7f813ad9d640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f811c0712d0 0x7f811c0736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.086+0000 7f813a59c640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f811c06b470 0x7f811c06d930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f812c036000 tx=0x7f812c03c040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.086+0000 7f812affd640 1 -- 172.21.15.16:0/579458480 <== osd.1 v2:172.21.15.16:6810/3842150677 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f811c073dc0 con 0x7f811c0712d0 2023-11-24T06:32:04.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.101+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 --> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f813c071e00 con 0x7f811c0712d0 2023-11-24T06:32:04.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.102+0000 7f812affd640 1 -- 172.21.15.16:0/579458480 <== osd.1 v2:172.21.15.16:6810/3842150677 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8134001ea0 con 0x7f811c0712d0 2023-11-24T06:32:04.104 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:32:04.104 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:32:04.105 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:32:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.102+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f811c0712d0 msgr2=0x7f811c0736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.102+0000 7f813b59e640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.16:6810/3842150677,v1:172.21.15.16:6812/3842150677] conn(0x7f811c0712d0 0x7f811c0736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.103+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f811c06b470 msgr2=0x7f811c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.103+0000 7f813b59e640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f811c06b470 0x7f811c06d930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f812c036000 tx=0x7f812c03c040 comp rx=0 tx=0).stop 2023-11-24T06:32:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.103+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c0cca40 msgr2=0x7f813c15a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.103+0000 7f813b59e640 1 --2- 172.21.15.16:0/579458480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c0cca40 0x7f813c15a6c0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f813000e9b0 tx=0x7f813000ee70 comp rx=0 tx=0).stop 2023-11-24T06:32:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.103+0000 7f813ad9d640 1 -- 172.21.15.16:0/579458480 reap_dead start 2023-11-24T06:32:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.103+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 shutdown_connections 2023-11-24T06:32:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.103+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 >> 172.21.15.16:0/579458480 conn(0x7f813c06a800 msgr2=0x7f813c13f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.103+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 shutdown_connections 2023-11-24T06:32:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.103+0000 7f813b59e640 1 -- 172.21.15.16:0/579458480 wait complete. 2023-11-24T06:32:04.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T06:32:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.348+0000 7ff4c29a0640 1 Processor -- start 2023-11-24T06:32:04.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.348+0000 7ff4c29a0640 1 -- start start 2023-11-24T06:32:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.350+0000 7ff4c29a0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4bc146850 0x7ff4bc0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.350+0000 7ff4c29a0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4bc0adcd0 0x7ff4bc06c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.350+0000 7ff4c29a0640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4bc071f70 con 0x7ff4bc071b10 2023-11-24T06:32:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.350+0000 7ff4c29a0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4bc0ae110 con 0x7ff4bc0adcd0 2023-11-24T06:32:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.350+0000 7ff4c29a0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc0b3490 con 0x7ff4bc146850 2023-11-24T06:32:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.350+0000 7ff4c219f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4bc0adcd0 0x7ff4bc06c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.350+0000 7ff4c219f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4bc0adcd0 0x7ff4bc06c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40098/0 (socket says 172.21.15.16:40098) 2023-11-24T06:32:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.350+0000 7ff4c219f640 1 -- 172.21.15.16:0/4278146014 learned_addr learned my addr 172.21.15.16:0/4278146014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.351+0000 7ff4c199e640 1 --2- 172.21.15.16:0/4278146014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4bc146850 0x7ff4bc0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.351+0000 7ff4c219f640 1 -- 172.21.15.16:0/4278146014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4bc146850 msgr2=0x7ff4bc0715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.351+0000 7ff4c219f640 1 --2- 172.21.15.16:0/4278146014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4bc146850 0x7ff4bc0715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.351+0000 7ff4c219f640 1 -- 172.21.15.16:0/4278146014 >> v1:172.21.15.173:6789/0 conn(0x7ff4bc071b10 legacy=0x7ff4bc0ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:04.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.351+0000 7ff4c219f640 1 -- 172.21.15.16:0/4278146014 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc0b3770 con 0x7ff4bc0adcd0 2023-11-24T06:32:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.352+0000 7ff4c219f640 1 --2- 172.21.15.16:0/4278146014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc0adcd0 0x7ff4bc06c470 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009e50 tx=0x7ff4b402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f3ff5aea36cc3b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c099c640 1 -- 172.21.15.16:0/4278146014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b402cb00 con 0x7ff4bc0adcd0 2023-11-24T06:32:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c099c640 1 -- 172.21.15.16:0/4278146014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b402cc60 con 0x7ff4bc0adcd0 2023-11-24T06:32:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c099c640 1 -- 172.21.15.16:0/4278146014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40343f0 con 0x7ff4bc0adcd0 2023-11-24T06:32:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c29a0640 1 -- 172.21.15.16:0/4278146014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc0adcd0 msgr2=0x7ff4bc06c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c29a0640 1 --2- 172.21.15.16:0/4278146014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc0adcd0 0x7ff4bc06c470 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009e50 tx=0x7ff4b402bf60 comp rx=0 tx=0).stop 2023-11-24T06:32:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c29a0640 1 -- 172.21.15.16:0/4278146014 shutdown_connections 2023-11-24T06:32:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c29a0640 1 --2- 172.21.15.16:0/4278146014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc0adcd0 0x7ff4bc06c470 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c29a0640 1 --2- 172.21.15.16:0/4278146014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4bc146850 0x7ff4bc0715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c29a0640 1 -- 172.21.15.16:0/4278146014 >> 172.21.15.16:0/4278146014 conn(0x7ff4bc06a3e0 msgr2=0x7ff4bc06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.353+0000 7ff4c29a0640 1 -- 172.21.15.16:0/4278146014 shutdown_connections 2023-11-24T06:32:04.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.354+0000 7ff4c29a0640 1 -- 172.21.15.16:0/4278146014 wait complete. 2023-11-24T06:32:04.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.355+0000 7ff4c29a0640 1 Processor -- start 2023-11-24T06:32:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.355+0000 7ff4c29a0640 1 -- start start 2023-11-24T06:32:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.356+0000 7ff4c29a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc071b10 0x7ff4bc139fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.356+0000 7ff4c29a0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4bc0adcd0 0x7ff4bc0c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.356+0000 7ff4c29a0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4bc146850 0x7ff4bc0c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.356+0000 7ff4c29a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc13a520 con 0x7ff4bc071b10 2023-11-24T06:32:04.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.356+0000 7ff4c29a0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc15a5f0 con 0x7ff4bc146850 2023-11-24T06:32:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.356+0000 7ff4c29a0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4bc15a8d0 con 0x7ff4bc0adcd0 2023-11-24T06:32:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.357+0000 7ff4c199e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc071b10 0x7ff4bc139fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.357+0000 7ff4c199e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc071b10 0x7ff4bc139fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40104/0 (socket says 172.21.15.16:40104) 2023-11-24T06:32:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.357+0000 7ff4c199e640 1 -- 172.21.15.16:0/2086897346 learned_addr learned my addr 172.21.15.16:0/2086897346 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.357+0000 7ff4c199e640 1 -- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4bc0adcd0 msgr2=0x7ff4bc0c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.357+0000 7ff4c199e640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4bc0adcd0 0x7ff4bc0c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.357+0000 7ff4c199e640 1 -- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4bc146850 msgr2=0x7ff4bc0c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.357+0000 7ff4c199e640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4bc146850 0x7ff4bc0c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.357+0000 7ff4c199e640 1 -- 172.21.15.16:0/2086897346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4009b30 con 0x7ff4bc071b10 2023-11-24T06:32:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.357+0000 7ff4c199e640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc071b10 0x7ff4bc139fe0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d5f0 tx=0x7ff4b800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.358+0000 7ff4b27fc640 1 -- 172.21.15.16:0/2086897346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80125f0 con 0x7ff4bc071b10 2023-11-24T06:32:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.358+0000 7ff4b27fc640 1 -- 172.21.15.16:0/2086897346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b8012750 con 0x7ff4bc071b10 2023-11-24T06:32:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.358+0000 7ff4b27fc640 1 -- 172.21.15.16:0/2086897346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8012b00 con 0x7ff4bc071b10 2023-11-24T06:32:04.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.358+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4bc15b310 con 0x7ff4bc071b10 2023-11-24T06:32:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.358+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4bc15b8d0 con 0x7ff4bc071b10 2023-11-24T06:32:04.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.359+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4bc15ba90 con 0x7ff4bc071b10 2023-11-24T06:32:04.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.360+0000 7ff4b27fc640 1 -- 172.21.15.16:0/2086897346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff4b8012c60 con 0x7ff4bc071b10 2023-11-24T06:32:04.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.362+0000 7ff4b27fc640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff4a006b470 0x7ff4a006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.362+0000 7ff4b27fc640 1 -- 172.21.15.16:0/2086897346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7ff4b808a2d0 con 0x7ff4bc071b10 2023-11-24T06:32:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.362+0000 7ff4c119d640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff4a006b470 0x7ff4a006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.364+0000 7ff4c119d640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff4a006b470 0x7ff4a006d930 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac000f60 tx=0x7ff4ac0079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.366+0000 7ff4b27fc640 1 -- 172.21.15.16:0/2086897346 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4b8054d70 con 0x7ff4bc071b10 2023-11-24T06:32:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.561+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4bc146c30 con 0x7ff4a006b470 2023-11-24T06:32:04.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.562+0000 7ff4b27fc640 1 -- 172.21.15.16:0/2086897346 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff4ac005080 con 0x7ff4a006b470 2023-11-24T06:32:04.564 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T06:32:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff4a006b470 msgr2=0x7ff4a006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff4a006b470 0x7ff4a006d930 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac000f60 tx=0x7ff4ac0079b0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc071b10 msgr2=0x7ff4bc139fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc071b10 0x7ff4bc139fe0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d5f0 tx=0x7ff4b800dab0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 shutdown_connections 2023-11-24T06:32:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff4a006b470 0x7ff4a006d930 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4bc146850 0x7ff4bc0c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4bc0adcd0 0x7ff4bc0c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 --2- 172.21.15.16:0/2086897346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4bc071b10 0x7ff4bc139fe0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 >> 172.21.15.16:0/2086897346 conn(0x7ff4bc06a3e0 msgr2=0x7ff4bc0affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 shutdown_connections 2023-11-24T06:32:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.564+0000 7ff4c29a0640 1 -- 172.21.15.16:0/2086897346 wait complete. 2023-11-24T06:32:04.587 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T06:32:04.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:32:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.820+0000 7fda152ff640 1 Processor -- start 2023-11-24T06:32:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.821+0000 7fda152ff640 1 -- start start 2023-11-24T06:32:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.822+0000 7fda152ff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10071290 0x7fda101539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.822+0000 7fda152ff640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1006aea0 con 0x7fda10161890 2023-11-24T06:32:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.822+0000 7fda152ff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1006b180 con 0x7fda100cef70 2023-11-24T06:32:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.822+0000 7fda152ff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda100cd390 con 0x7fda10071290 2023-11-24T06:32:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.823+0000 7fda0f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fda10161890 0x7fda10070b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38564/0 (socket says 172.21.15.16:38564) 2023-11-24T06:32:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.823+0000 7fda0f7fe640 1 -- 172.21.15.16:0/2330448853 learned_addr learned my addr 172.21.15.16:0/2330448853 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.823+0000 7fda14afe640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10071290 0x7fda101539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.823+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002510380 0 0) 0x7fda1006aea0 con 0x7fda10161890 2023-11-24T06:32:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.823+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f8003520 con 0x7fda10161890 2023-11-24T06:32:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28371798 0 0) 0x7fd9f8003520 con 0x7fda10161890 2023-11-24T06:32:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10071290 msgr2=0x7fda101539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10071290 0x7fda101539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 >> v1:172.21.15.173:6789/0 conn(0x7fda100cef70 legacy=0x7fda1006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda10160db0 con 0x7fda10161890 2023-11-24T06:32:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7fda04001810 con 0x7fda10161890 2023-11-24T06:32:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 >> v1:172.21.15.16:6789/0 conn(0x7fda10161890 legacy=0x7fda10070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9f80039a0 0x7fd9f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10071290 0x7fd9f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 0x7fd9f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f800bb10 con 0x7fd9f8004d00 2023-11-24T06:32:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f800bc80 con 0x7fd9f80039a0 2023-11-24T06:32:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f800be40 con 0x7fda10071290 2023-11-24T06:32:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.824+0000 7fda14afe640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 0x7fd9f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.825+0000 7fda0f7fe640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10071290 0x7fd9f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.825+0000 7fda0ffff640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9f80039a0 0x7fd9f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.825+0000 7fda14afe640 1 -- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10071290 msgr2=0x7fd9f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.825+0000 7fda14afe640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10071290 0x7fd9f80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.825+0000 7fda14afe640 1 -- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9f80039a0 msgr2=0x7fd9f8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.825+0000 7fda14afe640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9f80039a0 0x7fd9f8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.825+0000 7fda14afe640 1 -- 172.21.15.16:0/2330448853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8003520 con 0x7fd9f8004d00 2023-11-24T06:32:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.826+0000 7fda14afe640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 0x7fd9f8005180 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fda08009ac0 tx=0x7fd9f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d37536861ae4dfb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.826+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0802c6b0 con 0x7fd9f8004d00 2023-11-24T06:32:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.826+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0802c810 con 0x7fd9f8004d00 2023-11-24T06:32:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.826+0000 7fda0effd640 1 -- 172.21.15.16:0/2330448853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0802cba0 con 0x7fd9f8004d00 2023-11-24T06:32:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.827+0000 7fda152ff640 1 -- 172.21.15.16:0/2330448853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 msgr2=0x7fd9f8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.827+0000 7fda152ff640 1 --2- 172.21.15.16:0/2330448853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 0x7fd9f8005180 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fda08009ac0 tx=0x7fd9f80057f0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.827+0000 7fda14afe640 1 -- 172.21.15.16:0/2330448853 reap_dead start 2023-11-24T06:32:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.827+0000 7fda152ff640 1 -- 172.21.15.16:0/2330448853 shutdown_connections 2023-11-24T06:32:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.827+0000 7fda152ff640 1 -- 172.21.15.16:0/2330448853 >> 172.21.15.16:0/2330448853 conn(0x7fda100cc1d0 msgr2=0x7fda100cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.827+0000 7fda152ff640 1 -- 172.21.15.16:0/2330448853 shutdown_connections 2023-11-24T06:32:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.827+0000 7fda152ff640 1 -- 172.21.15.16:0/2330448853 wait complete. 2023-11-24T06:32:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.829+0000 7fda152ff640 1 Processor -- start 2023-11-24T06:32:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.829+0000 7fda152ff640 1 -- start start 2023-11-24T06:32:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.830+0000 7fda152ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 0x7fda100ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.830+0000 7fda152ff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10159f40 0x7fda1006dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.830+0000 7fda152ff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda101f6820 0x7fda101f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.830+0000 7fda152ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1015a3d0 con 0x7fd9f8004d00 2023-11-24T06:32:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.830+0000 7fda152ff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1015a6b0 con 0x7fda101f6820 2023-11-24T06:32:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.830+0000 7fda152ff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1015a990 con 0x7fda10159f40 2023-11-24T06:32:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.830+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 0x7fda100ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.830+0000 7fda0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 0x7fda100ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40108/0 (socket says 172.21.15.16:40108) 2023-11-24T06:32:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.830+0000 7fda0ffff640 1 -- 172.21.15.16:0/350672626 learned_addr learned my addr 172.21.15.16:0/350672626 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0f7fe640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10159f40 0x7fda1006dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda14afe640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda101f6820 0x7fda101f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0ffff640 1 -- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10159f40 msgr2=0x7fda1006dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0ffff640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda10159f40 0x7fda1006dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0ffff640 1 -- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda101f6820 msgr2=0x7fda101f6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0ffff640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda101f6820 0x7fda101f6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0ffff640 1 -- 172.21.15.16:0/350672626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1015ac70 con 0x7fd9f8004d00 2023-11-24T06:32:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0ffff640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 0x7fda100ae800 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fda0000e8c0 tx=0x7fda0000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0cff9640 1 -- 172.21.15.16:0/350672626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00003b80 con 0x7fd9f8004d00 2023-11-24T06:32:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0cff9640 1 -- 172.21.15.16:0/350672626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda00003ce0 con 0x7fd9f8004d00 2023-11-24T06:32:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda101f6300 con 0x7fd9f8004d00 2023-11-24T06:32:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.831+0000 7fda0cff9640 1 -- 172.21.15.16:0/350672626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda000052f0 con 0x7fd9f8004d00 2023-11-24T06:32:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.832+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1006e2d0 con 0x7fd9f8004d00 2023-11-24T06:32:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.832+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda1015d1e0 con 0x7fd9f8004d00 2023-11-24T06:32:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.834+0000 7fda0cff9640 1 -- 172.21.15.16:0/350672626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fda00005450 con 0x7fd9f8004d00 2023-11-24T06:32:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.834+0000 7fda0cff9640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd9fc06b470 0x7fd9fc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.835+0000 7fda0cff9640 1 -- 172.21.15.16:0/350672626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fda0008a050 con 0x7fd9f8004d00 2023-11-24T06:32:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.835+0000 7fda0cff9640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7fd9fc0712d0 0x7fd9fc0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.835+0000 7fda0cff9640 1 -- 172.21.15.16:0/350672626 --> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9fc073dc0 con 0x7fd9fc0712d0 2023-11-24T06:32:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.835+0000 7fda0cff9640 1 -- 172.21.15.16:0/350672626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda0008c510 con 0x7fd9f8004d00 2023-11-24T06:32:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.835+0000 7fda14afe640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7fd9fc0712d0 0x7fd9fc0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.835+0000 7fda0f7fe640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd9fc06b470 0x7fd9fc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.836+0000 7fda14afe640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7fd9fc0712d0 0x7fd9fc0736f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.836+0000 7fda0f7fe640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd9fc06b470 0x7fd9fc06d930 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fda101f60c0 tx=0x7fda04001050 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.837+0000 7fda0cff9640 1 -- 172.21.15.16:0/350672626 <== osd.2 v2:172.21.15.16:6808/620212774 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd9fc073dc0 con 0x7fd9fc0712d0 2023-11-24T06:32:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.851+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 --> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fda1006bb50 con 0x7fd9fc0712d0 2023-11-24T06:32:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda0cff9640 1 -- 172.21.15.16:0/350672626 <== osd.2 v2:172.21.15.16:6808/620212774 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fda0800f0a0 con 0x7fd9fc0712d0 2023-11-24T06:32:04.855 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:32:04.856 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:32:04.856 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:32:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7fd9fc0712d0 msgr2=0x7fd9fc0736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda152ff640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.16:6808/620212774,v1:172.21.15.16:6809/620212774] conn(0x7fd9fc0712d0 0x7fd9fc0736f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd9fc06b470 msgr2=0x7fd9fc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda152ff640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fd9fc06b470 0x7fd9fc06d930 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fda101f60c0 tx=0x7fda04001050 comp rx=0 tx=0).stop 2023-11-24T06:32:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 msgr2=0x7fda100ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda152ff640 1 --2- 172.21.15.16:0/350672626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9f8004d00 0x7fda100ae800 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fda0000e8c0 tx=0x7fda0000ed80 comp rx=0 tx=0).stop 2023-11-24T06:32:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda14afe640 1 -- 172.21.15.16:0/350672626 reap_dead start 2023-11-24T06:32:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 shutdown_connections 2023-11-24T06:32:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 >> 172.21.15.16:0/350672626 conn(0x7fda100cc1d0 msgr2=0x7fda100b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.852+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 shutdown_connections 2023-11-24T06:32:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:04.853+0000 7fda152ff640 1 -- 172.21.15.16:0/350672626 wait complete. 2023-11-24T06:32:04.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T06:32:05.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.088+0000 7f8e7b5c5640 1 Processor -- start 2023-11-24T06:32:05.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.088+0000 7f8e7b5c5640 1 -- start start 2023-11-24T06:32:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.089+0000 7f8e7b5c5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e740cf9a0 0x7f8e7415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.089+0000 7f8e7b5c5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e74070c40 con 0x7f8e740ae5a0 2023-11-24T06:32:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.089+0000 7f8e7b5c5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7406c470 con 0x7f8e740b3ae0 2023-11-24T06:32:05.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.089+0000 7f8e7b5c5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e74071450 con 0x7f8e740cf9a0 2023-11-24T06:32:05.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.090+0000 7f8e79b62640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e740cf9a0 0x7f8e7415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.090+0000 7f8e79361640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8e740ae5a0 0x7f8e7406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38576/0 (socket says 172.21.15.16:38576) 2023-11-24T06:32:05.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.090+0000 7f8e79361640 1 -- 172.21.15.16:0/3145231468 learned_addr learned my addr 172.21.15.16:0/3145231468 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:05.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.090+0000 7f8e79b62640 1 -- 172.21.15.16:0/3145231468 >> v1:172.21.15.173:6789/0 conn(0x7f8e740b3ae0 legacy=0x7f8e740cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:05.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.091+0000 7f8e79b62640 1 -- 172.21.15.16:0/3145231468 >> v1:172.21.15.16:6789/0 conn(0x7f8e740ae5a0 legacy=0x7f8e7406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.091+0000 7f8e79b62640 1 -- 172.21.15.16:0/3145231468 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7406c750 con 0x7f8e740cf9a0 2023-11-24T06:32:05.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.091+0000 7f8e6bfff640 1 -- 172.21.15.16:0/3145231468 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2193948507 0 0) 0x7f8e74070c40 con 0x7f8e740ae5a0 2023-11-24T06:32:05.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.092+0000 7f8e79b62640 1 --2- 172.21.15.16:0/3145231468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740cf9a0 0x7f8e7415e430 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009d90 tx=0x7f8e6c02bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c9ff81e825b96f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.092+0000 7f8e6bfff640 1 -- 172.21.15.16:0/3145231468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c02cd80 con 0x7f8e740cf9a0 2023-11-24T06:32:05.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.092+0000 7f8e6bfff640 1 -- 172.21.15.16:0/3145231468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c034440 con 0x7f8e740cf9a0 2023-11-24T06:32:05.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.092+0000 7f8e6bfff640 1 -- 172.21.15.16:0/3145231468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c0347d0 con 0x7f8e740cf9a0 2023-11-24T06:32:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.093+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/3145231468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740cf9a0 msgr2=0x7f8e7415e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.093+0000 7f8e7b5c5640 1 --2- 172.21.15.16:0/3145231468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740cf9a0 0x7f8e7415e430 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009d90 tx=0x7f8e6c02bfd0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.093+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/3145231468 shutdown_connections 2023-11-24T06:32:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.093+0000 7f8e7b5c5640 1 --2- 172.21.15.16:0/3145231468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740cf9a0 0x7f8e7415e430 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.093+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/3145231468 >> 172.21.15.16:0/3145231468 conn(0x7f8e7406a800 msgr2=0x7f8e7406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.093+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/3145231468 shutdown_connections 2023-11-24T06:32:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.093+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/3145231468 wait complete. 2023-11-24T06:32:05.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.095+0000 7f8e7b5c5640 1 Processor -- start 2023-11-24T06:32:05.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.095+0000 7f8e7b5c5640 1 -- start start 2023-11-24T06:32:05.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e7b5c5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e740ae5a0 0x7f8e74147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e7b5c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740b3ae0 0x7f8e74147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e7b5c5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e740cf9a0 0x7f8e7415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e7b5c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7415f560 con 0x7f8e740b3ae0 2023-11-24T06:32:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e7b5c5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740c3e50 con 0x7f8e740cf9a0 2023-11-24T06:32:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e7b5c5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e740cbec0 con 0x7f8e740ae5a0 2023-11-24T06:32:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e78b60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740b3ae0 0x7f8e74147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e78b60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740b3ae0 0x7f8e74147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40134/0 (socket says 172.21.15.16:40134) 2023-11-24T06:32:05.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e78b60640 1 -- 172.21.15.16:0/789719734 learned_addr learned my addr 172.21.15.16:0/789719734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:05.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.096+0000 7f8e79361640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e740ae5a0 0x7f8e74147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e78b60640 1 -- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e740ae5a0 msgr2=0x7f8e74147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e78b60640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e740ae5a0 0x7f8e74147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e78b60640 1 -- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e740cf9a0 msgr2=0x7f8e7415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e78b60640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e740cf9a0 0x7f8e7415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e78b60640 1 -- 172.21.15.16:0/789719734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c009a70 con 0x7f8e740b3ae0 2023-11-24T06:32:05.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e79361640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e740ae5a0 0x7f8e74147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:05.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e78b60640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740b3ae0 0x7f8e74147010 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400c450 tx=0x7f8e6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e69ffb640 1 -- 172.21.15.16:0/789719734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e640052d0 con 0x7f8e740b3ae0 2023-11-24T06:32:05.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e69ffb640 1 -- 172.21.15.16:0/789719734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e64005430 con 0x7f8e740b3ae0 2023-11-24T06:32:05.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e69ffb640 1 -- 172.21.15.16:0/789719734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64012870 con 0x7f8e740b3ae0 2023-11-24T06:32:05.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e740c40f0 con 0x7f8e740b3ae0 2023-11-24T06:32:05.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.097+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7414dca0 con 0x7f8e740b3ae0 2023-11-24T06:32:05.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.099+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e740c4610 con 0x7f8e740b3ae0 2023-11-24T06:32:05.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.100+0000 7f8e69ffb640 1 -- 172.21.15.16:0/789719734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8e640129d0 con 0x7f8e740b3ae0 2023-11-24T06:32:05.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.100+0000 7f8e69ffb640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8e5806b470 0x7f8e5806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.101+0000 7f8e69ffb640 1 -- 172.21.15.16:0/789719734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f8e64089910 con 0x7f8e740b3ae0 2023-11-24T06:32:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.101+0000 7f8e79361640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8e5806b470 0x7f8e5806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.102+0000 7f8e79361640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8e5806b470 0x7f8e5806d930 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e70000f60 tx=0x7f8e70009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.105+0000 7f8e69ffb640 1 -- 172.21.15.16:0/789719734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e640543f0 con 0x7f8e740b3ae0 2023-11-24T06:32:05.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.301+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e74070550 con 0x7f8e5806b470 2023-11-24T06:32:05.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.302+0000 7f8e69ffb640 1 -- 172.21.15.16:0/789719734 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8e70001d20 con 0x7f8e5806b470 2023-11-24T06:32:05.305 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T06:32:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.304+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8e5806b470 msgr2=0x7f8e5806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.304+0000 7f8e7b5c5640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8e5806b470 0x7f8e5806d930 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e70000f60 tx=0x7f8e70009040 comp rx=0 tx=0).stop 2023-11-24T06:32:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.304+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740b3ae0 msgr2=0x7f8e74147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.304+0000 7f8e7b5c5640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740b3ae0 0x7f8e74147010 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400c450 tx=0x7f8e6400c910 comp rx=0 tx=0).stop 2023-11-24T06:32:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.304+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 shutdown_connections 2023-11-24T06:32:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.304+0000 7f8e7b5c5640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f8e5806b470 0x7f8e5806d930 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.304+0000 7f8e7b5c5640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e740cf9a0 0x7f8e7415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.304+0000 7f8e7b5c5640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e740b3ae0 0x7f8e74147010 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.304+0000 7f8e7b5c5640 1 --2- 172.21.15.16:0/789719734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e740ae5a0 0x7f8e74147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.305+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 >> 172.21.15.16:0/789719734 conn(0x7f8e7406a800 msgr2=0x7f8e7406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.305+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 shutdown_connections 2023-11-24T06:32:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.305+0000 7f8e7b5c5640 1 -- 172.21.15.16:0/789719734 wait complete. 2023-11-24T06:32:05.327 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T06:32:05.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:32:05.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.551+0000 7f6c17305640 1 Processor -- start 2023-11-24T06:32:05.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.551+0000 7f6c17305640 1 -- start start 2023-11-24T06:32:05.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.552+0000 7f6c17305640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c1013e360 0x7f6c100cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.552+0000 7f6c17305640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c1015f9c0 0x7f6c100cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.553+0000 7f6c17305640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1015c5f0 con 0x7f6c1013df80 2023-11-24T06:32:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.553+0000 7f6c17305640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1006c470 con 0x7f6c1015f9c0 2023-11-24T06:32:05.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.553+0000 7f6c17305640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1006c750 con 0x7f6c1013e360 2023-11-24T06:32:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.553+0000 7f6c158a2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c1015f9c0 0x7f6c100cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.553+0000 7f6c150a1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c1013df80 0x7f6c1015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38588/0 (socket says 172.21.15.16:38588) 2023-11-24T06:32:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.553+0000 7f6c158a2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c1015f9c0 0x7f6c100cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40144/0 (socket says 172.21.15.16:40144) 2023-11-24T06:32:05.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.553+0000 7f6c150a1640 1 -- 172.21.15.16:0/3237527577 learned_addr learned my addr 172.21.15.16:0/3237527577 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.553+0000 7f6c148a0640 1 --2- 172.21.15.16:0/3237527577 >> v2:172.21.15.173:3300/0 conn(0x7f6c1013e360 0x7f6c100cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.553+0000 7f6c158a2640 1 -- 172.21.15.16:0/3237527577 >> v2:172.21.15.173:3300/0 conn(0x7f6c1013e360 msgr2=0x7f6c100cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.554+0000 7f6c158a2640 1 --2- 172.21.15.16:0/3237527577 >> v2:172.21.15.173:3300/0 conn(0x7f6c1013e360 0x7f6c100cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.554+0000 7f6c158a2640 1 -- 172.21.15.16:0/3237527577 >> v1:172.21.15.16:6789/0 conn(0x7f6c1013df80 legacy=0x7f6c1015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.554+0000 7f6c158a2640 1 -- 172.21.15.16:0/3237527577 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1006ca30 con 0x7f6c1015f9c0 2023-11-24T06:32:05.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.555+0000 7f6c158a2640 1 --2- 172.21.15.16:0/3237527577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1015f9c0 0x7f6c100cf7d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6c1006ded0 tx=0x7f6c0802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe1df37ac854514 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.555+0000 7f6c07fff640 1 -- 172.21.15.16:0/3237527577 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0802cc70 con 0x7f6c1015f9c0 2023-11-24T06:32:05.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.555+0000 7f6c07fff640 1 -- 172.21.15.16:0/3237527577 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0802cdd0 con 0x7f6c1015f9c0 2023-11-24T06:32:05.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.555+0000 7f6c07fff640 1 -- 172.21.15.16:0/3237527577 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08034660 con 0x7f6c1015f9c0 2023-11-24T06:32:05.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.556+0000 7f6c17305640 1 -- 172.21.15.16:0/3237527577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1015f9c0 msgr2=0x7f6c100cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.556+0000 7f6c17305640 1 --2- 172.21.15.16:0/3237527577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1015f9c0 0x7f6c100cf7d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6c1006ded0 tx=0x7f6c0802bec0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.556+0000 7f6c17305640 1 -- 172.21.15.16:0/3237527577 shutdown_connections 2023-11-24T06:32:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.556+0000 7f6c17305640 1 --2- 172.21.15.16:0/3237527577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1015f9c0 0x7f6c100cf7d0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.556+0000 7f6c17305640 1 --2- 172.21.15.16:0/3237527577 >> v2:172.21.15.173:3300/0 conn(0x7f6c1013e360 0x7f6c100cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.556+0000 7f6c17305640 1 -- 172.21.15.16:0/3237527577 >> 172.21.15.16:0/3237527577 conn(0x7f6c1006a800 msgr2=0x7f6c1006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.556+0000 7f6c17305640 1 -- 172.21.15.16:0/3237527577 shutdown_connections 2023-11-24T06:32:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.556+0000 7f6c17305640 1 -- 172.21.15.16:0/3237527577 wait complete. 2023-11-24T06:32:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.558+0000 7f6c17305640 1 Processor -- start 2023-11-24T06:32:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.558+0000 7f6c17305640 1 -- start start 2023-11-24T06:32:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.559+0000 7f6c17305640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c1013df80 0x7f6c100a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.559+0000 7f6c17305640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1013e360 0x7f6c101f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.559+0000 7f6c17305640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c1015f9c0 0x7f6c1015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.559+0000 7f6c17305640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10152850 con 0x7f6c1013e360 2023-11-24T06:32:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.559+0000 7f6c17305640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100a9410 con 0x7f6c1013df80 2023-11-24T06:32:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.559+0000 7f6c17305640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10151bd0 con 0x7f6c1015f9c0 2023-11-24T06:32:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.559+0000 7f6c148a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1013e360 0x7f6c101f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.559+0000 7f6c148a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1013e360 0x7f6c101f3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40148/0 (socket says 172.21.15.16:40148) 2023-11-24T06:32:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.559+0000 7f6c148a0640 1 -- 172.21.15.16:0/2971181831 learned_addr learned my addr 172.21.15.16:0/2971181831 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:05.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c158a2640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c1015f9c0 0x7f6c1015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c150a1640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c1013df80 0x7f6c100a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c148a0640 1 -- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c1015f9c0 msgr2=0x7f6c1015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c148a0640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c1015f9c0 0x7f6c1015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c148a0640 1 -- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c1013df80 msgr2=0x7f6c100a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c148a0640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c1013df80 0x7f6c100a8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c148a0640 1 -- 172.21.15.16:0/2971181831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c08009a70 con 0x7f6c1013e360 2023-11-24T06:32:05.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c150a1640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c1013df80 0x7f6c100a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:05.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c148a0640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1013e360 0x7f6c101f3ef0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000d7d0 tx=0x7f6c0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c05ffb640 1 -- 172.21.15.16:0/2971181831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00018070 con 0x7f6c1013e360 2023-11-24T06:32:05.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c05ffb640 1 -- 172.21.15.16:0/2971181831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c00004020 con 0x7f6c1013e360 2023-11-24T06:32:05.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c10151eb0 con 0x7f6c1013e360 2023-11-24T06:32:05.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c05ffb640 1 -- 172.21.15.16:0/2971181831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000041b0 con 0x7f6c1013e360 2023-11-24T06:32:05.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.560+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c10152470 con 0x7f6c1013e360 2023-11-24T06:32:05.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.561+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c1006cff0 con 0x7f6c1013e360 2023-11-24T06:32:05.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.563+0000 7f6c05ffb640 1 -- 172.21.15.16:0/2971181831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6c00004310 con 0x7f6c1013e360 2023-11-24T06:32:05.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.564+0000 7f6c05ffb640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bf406b470 0x7f6bf406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.564+0000 7f6c05ffb640 1 -- 172.21.15.16:0/2971181831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f6c0008a750 con 0x7f6c1013e360 2023-11-24T06:32:05.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.564+0000 7f6c05ffb640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f6bf40712d0 0x7f6bf40736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.564+0000 7f6c05ffb640 1 -- 172.21.15.16:0/2971181831 --> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6bf4073dc0 con 0x7f6bf40712d0 2023-11-24T06:32:05.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.564+0000 7f6c05ffb640 1 -- 172.21.15.16:0/2971181831 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c0008a9b0 con 0x7f6c1013e360 2023-11-24T06:32:05.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.564+0000 7f6c150a1640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bf406b470 0x7f6bf406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.564+0000 7f6c158a2640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f6bf40712d0 0x7f6bf40736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.565+0000 7f6c158a2640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f6bf40712d0 0x7f6bf40736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.565+0000 7f6c150a1640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bf406b470 0x7f6bf406d930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c000f60 tx=0x7f6c0c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.566+0000 7f6c05ffb640 1 -- 172.21.15.16:0/2971181831 <== osd.3 v2:172.21.15.16:6822/3227335215 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6bf4073dc0 con 0x7f6bf40712d0 2023-11-24T06:32:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.581+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 --> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6c10152ac0 con 0x7f6bf40712d0 2023-11-24T06:32:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.581+0000 7f6c05ffb640 1 -- 172.21.15.16:0/2971181831 <== osd.3 v2:172.21.15.16:6822/3227335215 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6c10152ac0 con 0x7f6bf40712d0 2023-11-24T06:32:05.594 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:32:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:32:05.595 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:32:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.581+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f6bf40712d0 msgr2=0x7f6bf40736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c17305640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.16:6822/3227335215,v1:172.21.15.16:6824/3227335215] conn(0x7f6bf40712d0 0x7f6bf40736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bf406b470 msgr2=0x7f6bf406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c17305640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f6bf406b470 0x7f6bf406d930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c000f60 tx=0x7f6c0c009040 comp rx=0 tx=0).stop 2023-11-24T06:32:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1013e360 msgr2=0x7f6c101f3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c17305640 1 --2- 172.21.15.16:0/2971181831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c1013e360 0x7f6c101f3ef0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000d7d0 tx=0x7f6c0000dc90 comp rx=0 tx=0).stop 2023-11-24T06:32:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c158a2640 1 -- 172.21.15.16:0/2971181831 reap_dead start 2023-11-24T06:32:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 shutdown_connections 2023-11-24T06:32:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 >> 172.21.15.16:0/2971181831 conn(0x7f6c1006a800 msgr2=0x7f6c10070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 shutdown_connections 2023-11-24T06:32:05.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.582+0000 7f6c17305640 1 -- 172.21.15.16:0/2971181831 wait complete. 2023-11-24T06:32:05.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T06:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.826+0000 7febe6eea640 1 Processor -- start 2023-11-24T06:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.826+0000 7febe6eea640 1 -- start start 2023-11-24T06:32:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.828+0000 7febe6eea640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7febe015d2b0 0x7febe006f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.828+0000 7febe6eea640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00b33d0 con 0x7febe0071da0 2023-11-24T06:32:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.828+0000 7febe6eea640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00b30f0 con 0x7febe006aaf0 2023-11-24T06:32:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.828+0000 7febe6eea640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe00712c0 con 0x7febe015d2b0 2023-11-24T06:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.828+0000 7febe56e7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7febe0071da0 0x7febe006a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38602/0 (socket says 172.21.15.16:38602) 2023-11-24T06:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.828+0000 7febe56e7640 1 -- 172.21.15.16:0/4127139685 learned_addr learned my addr 172.21.15.16:0/4127139685 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe5ee8640 1 --2- 172.21.15.16:0/4127139685 >> v2:172.21.15.173:3300/0 conn(0x7febe015d2b0 0x7febe006f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371174031 0 0) 0x7febe00b33d0 con 0x7febe0071da0 2023-11-24T06:32:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febcc003520 con 0x7febe0071da0 2023-11-24T06:32:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676966669 0 0) 0x7febcc003520 con 0x7febe0071da0 2023-11-24T06:32:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 >> v2:172.21.15.173:3300/0 conn(0x7febe015d2b0 msgr2=0x7febe006f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 --2- 172.21.15.16:0/4127139685 >> v2:172.21.15.173:3300/0 conn(0x7febe015d2b0 0x7febe006f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 >> v1:172.21.15.173:6789/0 conn(0x7febe006aaf0 legacy=0x7febe00b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00715a0 con 0x7febe0071da0 2023-11-24T06:32:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7febd0001670 con 0x7febe0071da0 2023-11-24T06:32:05.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 >> v1:172.21.15.16:6789/0 conn(0x7febe0071da0 legacy=0x7febe006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febcc0039c0 0x7febcc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0047c0 0x7febcc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc0052a0 0x7febcc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00712c0 con 0x7febcc0052a0 2023-11-24T06:32:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc00c1c0 con 0x7febcc0039c0 2023-11-24T06:32:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.829+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febcc00c380 con 0x7febcc0047c0 2023-11-24T06:32:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.830+0000 7febe66e9640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc0052a0 0x7febcc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.830+0000 7febe5ee8640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febcc0039c0 0x7febcc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.830+0000 7febe56e7640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0047c0 0x7febcc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.830+0000 7febe66e9640 1 -- 172.21.15.16:0/4127139685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0047c0 msgr2=0x7febcc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:05.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.830+0000 7febe66e9640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0047c0 0x7febcc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.830+0000 7febe66e9640 1 -- 172.21.15.16:0/4127139685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febcc0039c0 msgr2=0x7febcc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:05.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.830+0000 7febe66e9640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febcc0039c0 0x7febcc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.830+0000 7febe66e9640 1 -- 172.21.15.16:0/4127139685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc003520 con 0x7febcc0052a0 2023-11-24T06:32:05.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.830+0000 7febe56e7640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0047c0 0x7febcc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:05.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.831+0000 7febe66e9640 1 -- 172.21.15.16:0/4127139685 reap_dead start 2023-11-24T06:32:05.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.831+0000 7febe66e9640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc0052a0 0x7febcc00b980 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7febd8001720 tx=0x7febcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49532cbac222b75 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd80052d0 con 0x7febcc0052a0 2023-11-24T06:32:05.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd802e050 con 0x7febcc0052a0 2023-11-24T06:32:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe4ee6640 1 -- 172.21.15.16:0/4127139685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd80035f0 con 0x7febcc0052a0 2023-11-24T06:32:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe6eea640 1 -- 172.21.15.16:0/4127139685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc0052a0 msgr2=0x7febcc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe6eea640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc0052a0 0x7febcc00b980 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7febd8001720 tx=0x7febcc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe6eea640 1 -- 172.21.15.16:0/4127139685 shutdown_connections 2023-11-24T06:32:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe6eea640 1 --2- 172.21.15.16:0/4127139685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc0052a0 0x7febcc00b980 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe6eea640 1 -- 172.21.15.16:0/4127139685 >> 172.21.15.16:0/4127139685 conn(0x7febe00b1a50 msgr2=0x7febe00b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe6eea640 1 -- 172.21.15.16:0/4127139685 shutdown_connections 2023-11-24T06:32:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.832+0000 7febe6eea640 1 -- 172.21.15.16:0/4127139685 wait complete. 2023-11-24T06:32:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.834+0000 7febe6eea640 1 Processor -- start 2023-11-24T06:32:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.834+0000 7febe6eea640 1 -- start start 2023-11-24T06:32:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.835+0000 7febe6eea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0052a0 0x7febe014acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.835+0000 7febe6eea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe015b000 0x7febe015b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.835+0000 7febe6eea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe01532c0 0x7febe0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.835+0000 7febe6eea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe015b9c0 con 0x7febe01532c0 2023-11-24T06:32:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.835+0000 7febe6eea640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe014b200 con 0x7febe015b000 2023-11-24T06:32:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.835+0000 7febe6eea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe0154030 con 0x7febcc0052a0 2023-11-24T06:32:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe66e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe01532c0 0x7febe0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe66e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe01532c0 0x7febe0153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40172/0 (socket says 172.21.15.16:40172) 2023-11-24T06:32:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe66e9640 1 -- 172.21.15.16:0/298705714 learned_addr learned my addr 172.21.15.16:0/298705714 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe5ee8640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0052a0 0x7febe014acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe56e7640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe015b000 0x7febe015b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe66e9640 1 -- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0052a0 msgr2=0x7febe014acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe66e9640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0052a0 0x7febe014acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe66e9640 1 -- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe015b000 msgr2=0x7febe015b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe66e9640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe015b000 0x7febe015b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe66e9640 1 -- 172.21.15.16:0/298705714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe015df40 con 0x7febe01532c0 2023-11-24T06:32:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.836+0000 7febe66e9640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe01532c0 0x7febe0153740 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7febd8001840 tx=0x7febd8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.837+0000 7febd67fc640 1 -- 172.21.15.16:0/298705714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd80054c0 con 0x7febe01532c0 2023-11-24T06:32:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.837+0000 7febd67fc640 1 -- 172.21.15.16:0/298705714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd8038040 con 0x7febe01532c0 2023-11-24T06:32:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.837+0000 7febd67fc640 1 -- 172.21.15.16:0/298705714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8007250 con 0x7febe01532c0 2023-11-24T06:32:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.837+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe015e220 con 0x7febe01532c0 2023-11-24T06:32:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.837+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe0153c80 con 0x7febe01532c0 2023-11-24T06:32:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.838+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe0074a00 con 0x7febe01532c0 2023-11-24T06:32:05.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.839+0000 7febd67fc640 1 -- 172.21.15.16:0/298705714 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7febd80073b0 con 0x7febe01532c0 2023-11-24T06:32:05.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.840+0000 7febd67fc640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febc406b470 0x7febc406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:05.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.840+0000 7febd67fc640 1 -- 172.21.15.16:0/298705714 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7febd80aaab0 con 0x7febe01532c0 2023-11-24T06:32:05.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.841+0000 7febe5ee8640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febc406b470 0x7febc406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:05.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.842+0000 7febe5ee8640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febc406b470 0x7febc406d930 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7febdc0042b0 tx=0x7febdc00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:05.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:05.844+0000 7febd67fc640 1 -- 172.21.15.16:0/298705714 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febd8075570 con 0x7febe01532c0 2023-11-24T06:32:06.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.038+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7febe00721e0 con 0x7febc406b470 2023-11-24T06:32:06.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.040+0000 7febd67fc640 1 -- 172.21.15.16:0/298705714 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7febdc00f050 con 0x7febc406b470 2023-11-24T06:32:06.043 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T06:32:06.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.042+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febc406b470 msgr2=0x7febc406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.042+0000 7febe6eea640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febc406b470 0x7febc406d930 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7febdc0042b0 tx=0x7febdc00b040 comp rx=0 tx=0).stop 2023-11-24T06:32:06.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.042+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe01532c0 msgr2=0x7febe0153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.042+0000 7febe6eea640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe01532c0 0x7febe0153740 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7febd8001840 tx=0x7febd8001120 comp rx=0 tx=0).stop 2023-11-24T06:32:06.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.043+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 shutdown_connections 2023-11-24T06:32:06.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.043+0000 7febe6eea640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7febc406b470 0x7febc406d930 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.043+0000 7febe6eea640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febe01532c0 0x7febe0153740 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.043+0000 7febe6eea640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febe015b000 0x7febe015b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.043+0000 7febe6eea640 1 --2- 172.21.15.16:0/298705714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febcc0052a0 0x7febe014acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.043+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 >> 172.21.15.16:0/298705714 conn(0x7febe00b1a50 msgr2=0x7febe01615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:06.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.043+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 shutdown_connections 2023-11-24T06:32:06.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.043+0000 7febe6eea640 1 -- 172.21.15.16:0/298705714 wait complete. 2023-11-24T06:32:06.066 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T06:32:06.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:32:06.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.290+0000 7f243f341640 1 Processor -- start 2023-11-24T06:32:06.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.291+0000 7f243f341640 1 -- start start 2023-11-24T06:32:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.292+0000 7f243f341640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f243813e360 0x7f24380cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.292+0000 7f243f341640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f243815f9c0 0x7f24380cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.292+0000 7f243f341640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243815c5f0 con 0x7f243813df80 2023-11-24T06:32:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.292+0000 7f243f341640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243806c470 con 0x7f243813e360 2023-11-24T06:32:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.292+0000 7f243f341640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243806c750 con 0x7f243815f9c0 2023-11-24T06:32:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f243c8dc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f243813e360 0x7f24380cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f243d0dd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f243813df80 0x7f243815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38604/0 (socket says 172.21.15.16:38604) 2023-11-24T06:32:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f243d0dd640 1 -- 172.21.15.16:0/2875757964 learned_addr learned my addr 172.21.15.16:0/2875757964 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:06.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f243c8dc640 1 -- 172.21.15.16:0/2875757964 >> v2:172.21.15.173:3300/0 conn(0x7f243815f9c0 msgr2=0x7f24380cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f243d8de640 1 --2- 172.21.15.16:0/2875757964 >> v2:172.21.15.173:3300/0 conn(0x7f243815f9c0 0x7f24380cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f243c8dc640 1 --2- 172.21.15.16:0/2875757964 >> v2:172.21.15.173:3300/0 conn(0x7f243815f9c0 0x7f24380cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f242ffff640 1 -- 172.21.15.16:0/2875757964 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192049050 0 0) 0x7f243815c5f0 con 0x7f243813df80 2023-11-24T06:32:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f243c8dc640 1 -- 172.21.15.16:0/2875757964 >> v1:172.21.15.16:6789/0 conn(0x7f243813df80 legacy=0x7f243815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f243c8dc640 1 -- 172.21.15.16:0/2875757964 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243806ca30 con 0x7f243813e360 2023-11-24T06:32:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.293+0000 7f243d8de640 1 --2- 172.21.15.16:0/2875757964 >> v2:172.21.15.173:3300/0 conn(0x7f243815f9c0 0x7f24380cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.294+0000 7f243c8dc640 1 --2- 172.21.15.16:0/2875757964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f243813e360 0x7f24380cf290 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f24380ccae0 tx=0x7f242802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e428a922c49c04 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f242ffff640 1 -- 172.21.15.16:0/2875757964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428031830 con 0x7f243813e360 2023-11-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f242ffff640 1 -- 172.21.15.16:0/2875757964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2428031990 con 0x7f243813e360 2023-11-24T06:32:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f242ffff640 1 -- 172.21.15.16:0/2875757964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428031d20 con 0x7f243813e360 2023-11-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f243f341640 1 -- 172.21.15.16:0/2875757964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f243813e360 msgr2=0x7f24380cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f243f341640 1 --2- 172.21.15.16:0/2875757964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f243813e360 0x7f24380cf290 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f24380ccae0 tx=0x7f242802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f243f341640 1 -- 172.21.15.16:0/2875757964 shutdown_connections 2023-11-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f243f341640 1 --2- 172.21.15.16:0/2875757964 >> v2:172.21.15.173:3300/0 conn(0x7f243815f9c0 0x7f24380cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f243f341640 1 --2- 172.21.15.16:0/2875757964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f243813e360 0x7f24380cf290 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f243f341640 1 -- 172.21.15.16:0/2875757964 >> 172.21.15.16:0/2875757964 conn(0x7f243806a800 msgr2=0x7f243806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:06.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.295+0000 7f243f341640 1 -- 172.21.15.16:0/2875757964 shutdown_connections 2023-11-24T06:32:06.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.296+0000 7f243f341640 1 -- 172.21.15.16:0/2875757964 wait complete. 2023-11-24T06:32:06.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.297+0000 7f243f341640 1 Processor -- start 2023-11-24T06:32:06.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.298+0000 7f243f341640 1 -- start start 2023-11-24T06:32:06.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243f341640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f243813df80 0x7f243813d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243f341640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f243815f9c0 0x7f24380b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243f341640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24380b22b0 0x7f2438150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243f341640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24380b3200 con 0x7f24380b22b0 2023-11-24T06:32:06.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243f341640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24380b34e0 con 0x7f243813df80 2023-11-24T06:32:06.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243f341640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24380b37c0 con 0x7f243815f9c0 2023-11-24T06:32:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243d8de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24380b22b0 0x7f2438150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243d8de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24380b22b0 0x7f2438150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40188/0 (socket says 172.21.15.16:40188) 2023-11-24T06:32:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243d8de640 1 -- 172.21.15.16:0/1452422215 learned_addr learned my addr 172.21.15.16:0/1452422215 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243c8dc640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f243815f9c0 0x7f24380b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.299+0000 7f243d0dd640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f243813df80 0x7f243813d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f243d8de640 1 -- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f243815f9c0 msgr2=0x7f24380b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f243d8de640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f243815f9c0 0x7f24380b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f243d8de640 1 -- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f243813df80 msgr2=0x7f243813d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f243d8de640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f243813df80 0x7f243813d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f243d8de640 1 -- 172.21.15.16:0/1452422215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428009400 con 0x7f24380b22b0 2023-11-24T06:32:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f243d8de640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24380b22b0 0x7f2438150600 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f243000c7f0 tx=0x7f243000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f242dffb640 1 -- 172.21.15.16:0/1452422215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004220 con 0x7f24380b22b0 2023-11-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f242dffb640 1 -- 172.21.15.16:0/1452422215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300043a0 con 0x7f24380b22b0 2023-11-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f242dffb640 1 -- 172.21.15.16:0/1452422215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430007860 con 0x7f24380b22b0 2023-11-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.300+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24380b3aa0 con 0x7f24380b22b0 2023-11-24T06:32:06.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.301+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243813dca0 con 0x7f24380b22b0 2023-11-24T06:32:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.301+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24380b3650 con 0x7f24380b22b0 2023-11-24T06:32:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.303+0000 7f242dffb640 1 -- 172.21.15.16:0/1452422215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2430007bd0 con 0x7f24380b22b0 2023-11-24T06:32:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.304+0000 7f242dffb640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241c06b470 0x7f241c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.304+0000 7f242dffb640 1 -- 172.21.15.16:0/1452422215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f2430089ed0 con 0x7f24380b22b0 2023-11-24T06:32:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.304+0000 7f242dffb640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f241c0712d0 0x7f241c0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.304+0000 7f242dffb640 1 -- 172.21.15.16:0/1452422215 --> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f241c073dc0 con 0x7f241c0712d0 2023-11-24T06:32:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.304+0000 7f242dffb640 1 -- 172.21.15.16:0/1452422215 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f243008c390 con 0x7f24380b22b0 2023-11-24T06:32:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.304+0000 7f243d0dd640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241c06b470 0x7f241c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.305+0000 7f243c8dc640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f241c0712d0 0x7f241c0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.306+0000 7f243d0dd640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241c06b470 0x7f241c06d930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f24340012e0 tx=0x7f2434009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.306+0000 7f243c8dc640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f241c0712d0 0x7f241c0736f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.308+0000 7f242dffb640 1 -- 172.21.15.16:0/1452422215 <== osd.4 v2:172.21.15.173:6800/3033116473 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f241c073dc0 con 0x7f241c0712d0 2023-11-24T06:32:06.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.323+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 --> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f24380b3930 con 0x7f241c0712d0 2023-11-24T06:32:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.324+0000 7f242dffb640 1 -- 172.21.15.16:0/1452422215 <== osd.4 v2:172.21.15.173:6800/3033116473 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f24380b3930 con 0x7f241c0712d0 2023-11-24T06:32:06.327 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:32:06.328 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:32:06.328 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:32:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.324+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f241c0712d0 msgr2=0x7f241c0736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.324+0000 7f243f341640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6800/3033116473,v1:172.21.15.173:6801/3033116473] conn(0x7f241c0712d0 0x7f241c0736f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.324+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241c06b470 msgr2=0x7f241c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.324+0000 7f243f341640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f241c06b470 0x7f241c06d930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f24340012e0 tx=0x7f2434009040 comp rx=0 tx=0).stop 2023-11-24T06:32:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.324+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24380b22b0 msgr2=0x7f2438150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.325+0000 7f243f341640 1 --2- 172.21.15.16:0/1452422215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24380b22b0 0x7f2438150600 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f243000c7f0 tx=0x7f243000ccb0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.325+0000 7f243d8de640 1 -- 172.21.15.16:0/1452422215 reap_dead start 2023-11-24T06:32:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.325+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 shutdown_connections 2023-11-24T06:32:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.325+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 >> 172.21.15.16:0/1452422215 conn(0x7f243806a800 msgr2=0x7f243815a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.325+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 shutdown_connections 2023-11-24T06:32:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.325+0000 7f243f341640 1 -- 172.21.15.16:0/1452422215 wait complete. 2023-11-24T06:32:06.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T06:32:06.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.569+0000 7f55e4893640 1 Processor -- start 2023-11-24T06:32:06.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.569+0000 7f55e4893640 1 -- start start 2023-11-24T06:32:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.571+0000 7f55e4893640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f55e006c470 0x7f55e00cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.571+0000 7f55e4893640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55e006dab0 0x7f55e00ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.571+0000 7f55e4893640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e006ca10 con 0x7f55e01447f0 2023-11-24T06:32:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.571+0000 7f55e4893640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e006d690 con 0x7f55e006c470 2023-11-24T06:32:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.571+0000 7f55e4893640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00cebb0 con 0x7f55e006dab0 2023-11-24T06:32:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.571+0000 7f55ded9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f55e01447f0 0x7f55e015c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38612/0 (socket says 172.21.15.16:38612) 2023-11-24T06:32:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.571+0000 7f55ded9d640 1 -- 172.21.15.16:0/1280830458 learned_addr learned my addr 172.21.15.16:0/1280830458 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:06.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952858492 0 0) 0x7f55e006ca10 con 0x7f55e01447f0 2023-11-24T06:32:06.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55cc003520 con 0x7f55e01447f0 2023-11-24T06:32:06.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55df59e640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55e006dab0 0x7f55e00ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55d659c640 1 --2- 172.21.15.16:0/1280830458 >> v2:172.21.15.173:3300/0 conn(0x7f55e006c470 0x7f55e00cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2887178101 0 0) 0x7f55cc003520 con 0x7f55e01447f0 2023-11-24T06:32:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55e006dab0 msgr2=0x7f55e00ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55e006dab0 0x7f55e00ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 >> v2:172.21.15.173:3300/0 conn(0x7f55e006c470 msgr2=0x7f55e00cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 --2- 172.21.15.16:0/1280830458 >> v2:172.21.15.173:3300/0 conn(0x7f55e006c470 0x7f55e00cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e006ccf0 con 0x7f55e01447f0 2023-11-24T06:32:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f55d0001670 con 0x7f55e01447f0 2023-11-24T06:32:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 >> v1:172.21.15.16:6789/0 conn(0x7f55e01447f0 legacy=0x7f55e015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55cc003970 0x7f55cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55cc0047c0 0x7f55cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55e006dab0 0x7f55cc005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e006d690 con 0x7f55cc003970 2023-11-24T06:32:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00cebb0 con 0x7f55cc0047c0 2023-11-24T06:32:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc00bbc0 con 0x7f55e006dab0 2023-11-24T06:32:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.572+0000 7f55ded9d640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55cc003970 0x7f55cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.573+0000 7f55d659c640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55cc0047c0 0x7f55cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.573+0000 7f55ded9d640 1 -- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55e006dab0 msgr2=0x7f55cc005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.573+0000 7f55df59e640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55e006dab0 0x7f55cc005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.573+0000 7f55ded9d640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55e006dab0 0x7f55cc005250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.573+0000 7f55ded9d640 1 -- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55cc0047c0 msgr2=0x7f55cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.573+0000 7f55ded9d640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55cc0047c0 0x7f55cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.573+0000 7f55ded9d640 1 -- 172.21.15.16:0/1280830458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc003520 con 0x7f55cc003970 2023-11-24T06:32:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.573+0000 7f55d659c640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55cc0047c0 0x7f55cc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.574+0000 7f55ded9d640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55cc003970 0x7f55cc003d90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c110 tx=0x7f55cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66b68faf1c835fc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.574+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0005ad0 con 0x7f55cc003970 2023-11-24T06:32:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.575+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d0005c30 con 0x7f55cc003970 2023-11-24T06:32:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.575+0000 7f55de59c640 1 -- 172.21.15.16:0/1280830458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d00072b0 con 0x7f55cc003970 2023-11-24T06:32:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.575+0000 7f55e4893640 1 -- 172.21.15.16:0/1280830458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55cc003970 msgr2=0x7f55cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.575+0000 7f55e4893640 1 --2- 172.21.15.16:0/1280830458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55cc003970 0x7f55cc003d90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c110 tx=0x7f55cc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.575+0000 7f55df59e640 1 -- 172.21.15.16:0/1280830458 reap_dead start 2023-11-24T06:32:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.575+0000 7f55e4893640 1 -- 172.21.15.16:0/1280830458 shutdown_connections 2023-11-24T06:32:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.575+0000 7f55e4893640 1 -- 172.21.15.16:0/1280830458 >> 172.21.15.16:0/1280830458 conn(0x7f55e006a3e0 msgr2=0x7f55e006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.575+0000 7f55e4893640 1 -- 172.21.15.16:0/1280830458 shutdown_connections 2023-11-24T06:32:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.575+0000 7f55e4893640 1 -- 172.21.15.16:0/1280830458 wait complete. 2023-11-24T06:32:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.577+0000 7f55e4893640 1 Processor -- start 2023-11-24T06:32:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.577+0000 7f55e4893640 1 -- start start 2023-11-24T06:32:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.578+0000 7f55e4893640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55cc003970 0x7f55e0139d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.578+0000 7f55e4893640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e00ac8d0 0x7f55e013b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.578+0000 7f55e4893640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e013bca0 0x7f55e00715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.578+0000 7f55e4893640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e013cba0 con 0x7f55e013bca0 2023-11-24T06:32:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.578+0000 7f55e4893640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e013a270 con 0x7f55e00ac8d0 2023-11-24T06:32:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.578+0000 7f55e4893640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e013c300 con 0x7f55cc003970 2023-11-24T06:32:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55df59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e013bca0 0x7f55e00715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55df59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e013bca0 0x7f55e00715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40218/0 (socket says 172.21.15.16:40218) 2023-11-24T06:32:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55df59e640 1 -- 172.21.15.16:0/822682366 learned_addr learned my addr 172.21.15.16:0/822682366 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55ded9d640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55cc003970 0x7f55e0139d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55df59e640 1 -- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55cc003970 msgr2=0x7f55e0139d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55df59e640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55cc003970 0x7f55e0139d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55df59e640 1 -- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e00ac8d0 msgr2=0x7f55e013b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55df59e640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e00ac8d0 0x7f55e013b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55df59e640 1 -- 172.21.15.16:0/822682366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e013c5e0 con 0x7f55e013bca0 2023-11-24T06:32:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.579+0000 7f55df59e640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e013bca0 0x7f55e00715d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f55e015eb50 tx=0x7f55d800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.580+0000 7f55d7fff640 1 -- 172.21.15.16:0/822682366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d800c070 con 0x7f55e013bca0 2023-11-24T06:32:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.580+0000 7f55d7fff640 1 -- 172.21.15.16:0/822682366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d8013610 con 0x7f55e013bca0 2023-11-24T06:32:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.580+0000 7f55d7fff640 1 -- 172.21.15.16:0/822682366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80139c0 con 0x7f55e013bca0 2023-11-24T06:32:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.580+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e013c8c0 con 0x7f55e013bca0 2023-11-24T06:32:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.580+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e00acd50 con 0x7f55e013bca0 2023-11-24T06:32:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.581+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e00b0680 con 0x7f55e013bca0 2023-11-24T06:32:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.582+0000 7f55d7fff640 1 -- 172.21.15.16:0/822682366 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f55d8013b20 con 0x7f55e013bca0 2023-11-24T06:32:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.583+0000 7f55d7fff640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55c006b470 0x7f55c006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.583+0000 7f55d7fff640 1 -- 172.21.15.16:0/822682366 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f55d80897e0 con 0x7f55e013bca0 2023-11-24T06:32:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.584+0000 7f55ded9d640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55c006b470 0x7f55c006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:06.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.585+0000 7f55ded9d640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55c006b470 0x7f55c006d930 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f55d0034000 tx=0x7f55d0007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:06.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.588+0000 7f55d7fff640 1 -- 172.21.15.16:0/822682366 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55e00b0680 con 0x7f55e013bca0 2023-11-24T06:32:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.781+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f55e0144bd0 con 0x7f55c006b470 2023-11-24T06:32:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.783+0000 7f55d7fff640 1 -- 172.21.15.16:0/822682366 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f55d003a070 con 0x7f55c006b470 2023-11-24T06:32:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T06:32:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55c006b470 msgr2=0x7f55c006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55c006b470 0x7f55c006d930 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f55d0034000 tx=0x7f55d0007000 comp rx=0 tx=0).stop 2023-11-24T06:32:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e013bca0 msgr2=0x7f55e00715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e013bca0 0x7f55e00715d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f55e015eb50 tx=0x7f55d800e830 comp rx=0 tx=0).stop 2023-11-24T06:32:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 shutdown_connections 2023-11-24T06:32:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f55c006b470 0x7f55c006d930 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e013bca0 0x7f55e00715d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55e00ac8d0 0x7f55e013b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 --2- 172.21.15.16:0/822682366 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55cc003970 0x7f55e0139d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.785+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 >> 172.21.15.16:0/822682366 conn(0x7f55e006a3e0 msgr2=0x7f55e015a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.786+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 shutdown_connections 2023-11-24T06:32:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:06.786+0000 7f55e4893640 1 -- 172.21.15.16:0/822682366 wait complete. 2023-11-24T06:32:06.806 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T06:32:06.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:32:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.035+0000 7f0c8aaea640 1 Processor -- start 2023-11-24T06:32:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.035+0000 7f0c8aaea640 1 -- start start 2023-11-24T06:32:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c8aaea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c84143c40 0x7f0c84071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c8aaea640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0c840ccc10 0x7f0c8406d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c8aaea640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c84070e50 con 0x7f0c840717d0 2023-11-24T06:32:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c8aaea640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c840723c0 con 0x7f0c840ccc10 2023-11-24T06:32:07.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c8aaea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84144020 con 0x7f0c84143c40 2023-11-24T06:32:07.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c892e7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0c840717d0 0x7f0c840cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38628/0 (socket says 172.21.15.16:38628) 2023-11-24T06:32:07.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c892e7640 1 -- 172.21.15.16:0/3047291714 learned_addr learned my addr 172.21.15.16:0/3047291714 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c89ae8640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c84143c40 0x7f0c84071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c8a2e9640 1 --2- 172.21.15.16:0/3047291714 >> v2:172.21.15.173:3300/0 conn(0x7f0c840ccc10 0x7f0c8406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.037+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465919407 0 0) 0x7f0c84070e50 con 0x7f0c840717d0 2023-11-24T06:32:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c70003520 con 0x7f0c840717d0 2023-11-24T06:32:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4141079962 0 0) 0x7f0c70003520 con 0x7f0c840717d0 2023-11-24T06:32:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c84143c40 msgr2=0x7f0c84071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c84143c40 0x7f0c84071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 >> v2:172.21.15.173:3300/0 conn(0x7f0c840ccc10 msgr2=0x7f0c8406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 --2- 172.21.15.16:0/3047291714 >> v2:172.21.15.173:3300/0 conn(0x7f0c840ccc10 0x7f0c8406d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c84071e00 con 0x7f0c840717d0 2023-11-24T06:32:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f0c74001670 con 0x7f0c840717d0 2023-11-24T06:32:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 >> v1:172.21.15.16:6789/0 conn(0x7f0c840717d0 legacy=0x7f0c840cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c84143c40 0x7f0c70003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c70003f40 0x7f0c700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c70004d00 0x7f0c70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840723c0 con 0x7f0c70004d00 2023-11-24T06:32:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84144020 con 0x7f0c70003f40 2023-11-24T06:32:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7000bae0 con 0x7f0c84143c40 2023-11-24T06:32:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c8a2e9640 1 --2- 172.21.15.16:0/3047291714 >> v2:172.21.15.173:3300/0 conn(0x7f0c840ccc10 0x7f0c8406d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:32:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.038+0000 7f0c8a2e9640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c70004d00 0x7f0c70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.039+0000 7f0c89ae8640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c84143c40 0x7f0c70003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.039+0000 7f0c8a2e9640 1 -- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c84143c40 msgr2=0x7f0c70003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.039+0000 7f0c8a2e9640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c84143c40 0x7f0c70003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.039+0000 7f0c8a2e9640 1 -- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c70003f40 msgr2=0x7f0c700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.039+0000 7f0c8a2e9640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c70003f40 0x7f0c700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.039+0000 7f0c8a2e9640 1 -- 172.21.15.16:0/3047291714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70003520 con 0x7f0c70004d00 2023-11-24T06:32:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.040+0000 7f0c8a2e9640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c70004d00 0x7f0c70005180 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0073e0 tx=0x7f0c700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0839767bc866c41 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.040+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c02cdc0 con 0x7f0c70004d00 2023-11-24T06:32:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.040+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c034440 con 0x7f0c70004d00 2023-11-24T06:32:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.040+0000 7f0c88ae6640 1 -- 172.21.15.16:0/3047291714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0347b0 con 0x7f0c70004d00 2023-11-24T06:32:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.041+0000 7f0c8aaea640 1 -- 172.21.15.16:0/3047291714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c70004d00 msgr2=0x7f0c70005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.041+0000 7f0c8aaea640 1 --2- 172.21.15.16:0/3047291714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c70004d00 0x7f0c70005180 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0073e0 tx=0x7f0c700057f0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.041+0000 7f0c8a2e9640 1 -- 172.21.15.16:0/3047291714 reap_dead start 2023-11-24T06:32:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.041+0000 7f0c8aaea640 1 -- 172.21.15.16:0/3047291714 shutdown_connections 2023-11-24T06:32:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.041+0000 7f0c8aaea640 1 -- 172.21.15.16:0/3047291714 >> 172.21.15.16:0/3047291714 conn(0x7f0c8406a800 msgr2=0x7f0c8406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.041+0000 7f0c8aaea640 1 -- 172.21.15.16:0/3047291714 shutdown_connections 2023-11-24T06:32:07.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.041+0000 7f0c8aaea640 1 -- 172.21.15.16:0/3047291714 wait complete. 2023-11-24T06:32:07.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.043+0000 7f0c8aaea640 1 Processor -- start 2023-11-24T06:32:07.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.043+0000 7f0c8aaea640 1 -- start start 2023-11-24T06:32:07.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.044+0000 7f0c8aaea640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c70004d00 0x7f0c84134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.044+0000 7f0c8aaea640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c840af890 0x7f0c840afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.044+0000 7f0c8aaea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8413a8b0 0x7f0c8413ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.044+0000 7f0c8aaea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84137670 con 0x7f0c8413a8b0 2023-11-24T06:32:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.044+0000 7f0c8aaea640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c840b0230 con 0x7f0c70004d00 2023-11-24T06:32:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.044+0000 7f0c8aaea640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c841371e0 con 0x7f0c840af890 2023-11-24T06:32:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.044+0000 7f0c8a2e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8413a8b0 0x7f0c8413ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.044+0000 7f0c8a2e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8413a8b0 0x7f0c8413ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40240/0 (socket says 172.21.15.16:40240) 2023-11-24T06:32:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.044+0000 7f0c8a2e9640 1 -- 172.21.15.16:0/2340730956 learned_addr learned my addr 172.21.15.16:0/2340730956 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c89ae8640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c70004d00 0x7f0c84134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c892e7640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c840af890 0x7f0c840afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c8a2e9640 1 -- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c840af890 msgr2=0x7f0c840afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c8a2e9640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c840af890 0x7f0c840afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c8a2e9640 1 -- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c70004d00 msgr2=0x7f0c84134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c8a2e9640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c70004d00 0x7f0c84134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c8a2e9640 1 -- 172.21.15.16:0/2340730956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c841351f0 con 0x7f0c8413a8b0 2023-11-24T06:32:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c892e7640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c840af890 0x7f0c840afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:07.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c8a2e9640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8413a8b0 0x7f0c8413ad30 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c007500 tx=0x7f0c7c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.045+0000 7f0c7a7fc640 1 -- 172.21.15.16:0/2340730956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c03e440 con 0x7f0c8413a8b0 2023-11-24T06:32:07.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.046+0000 7f0c7a7fc640 1 -- 172.21.15.16:0/2340730956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c03e5a0 con 0x7f0c8413a8b0 2023-11-24T06:32:07.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.046+0000 7f0c7a7fc640 1 -- 172.21.15.16:0/2340730956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c03e950 con 0x7f0c8413a8b0 2023-11-24T06:32:07.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.046+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c84136610 con 0x7f0c8413a8b0 2023-11-24T06:32:07.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.046+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c84136b90 con 0x7f0c8413a8b0 2023-11-24T06:32:07.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.047+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c840cd030 con 0x7f0c8413a8b0 2023-11-24T06:32:07.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.048+0000 7f0c7a7fc640 1 -- 172.21.15.16:0/2340730956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f0c7c03eab0 con 0x7f0c8413a8b0 2023-11-24T06:32:07.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.049+0000 7f0c7a7fc640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0c6806b4a0 0x7f0c6806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.049+0000 7f0c7a7fc640 1 -- 172.21.15.16:0/2340730956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f0c7c0accd0 con 0x7f0c8413a8b0 2023-11-24T06:32:07.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.049+0000 7f0c7a7fc640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f0c68071300 0x7f0c68073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.049+0000 7f0c7a7fc640 1 -- 172.21.15.16:0/2340730956 --> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c68073df0 con 0x7f0c68071300 2023-11-24T06:32:07.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.049+0000 7f0c7a7fc640 1 -- 172.21.15.16:0/2340730956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c7c0acf30 con 0x7f0c8413a8b0 2023-11-24T06:32:07.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.050+0000 7f0c89ae8640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0c6806b4a0 0x7f0c6806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.050+0000 7f0c892e7640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f0c68071300 0x7f0c68073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.051+0000 7f0c892e7640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f0c68071300 0x7f0c68073720 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.051+0000 7f0c89ae8640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0c6806b4a0 0x7f0c6806d960 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009a10 tx=0x7f0c80008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.053+0000 7f0c7a7fc640 1 -- 172.21.15.16:0/2340730956 <== osd.5 v2:172.21.15.173:6816/2053607116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0c68073df0 con 0x7f0c68071300 2023-11-24T06:32:07.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.068+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 --> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0c8406cc40 con 0x7f0c68071300 2023-11-24T06:32:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.069+0000 7f0c7a7fc640 1 -- 172.21.15.16:0/2340730956 <== osd.5 v2:172.21.15.173:6816/2053607116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0c74007be0 con 0x7f0c68071300 2023-11-24T06:32:07.072 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:32:07.072 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:32:07.072 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:32:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.069+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f0c68071300 msgr2=0x7f0c68073720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.069+0000 7f0c8aaea640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6816/2053607116,v1:172.21.15.173:6817/2053607116] conn(0x7f0c68071300 0x7f0c68073720 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.070+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0c6806b4a0 msgr2=0x7f0c6806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.070+0000 7f0c8aaea640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f0c6806b4a0 0x7f0c6806d960 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009a10 tx=0x7f0c80008040 comp rx=0 tx=0).stop 2023-11-24T06:32:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.070+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8413a8b0 msgr2=0x7f0c8413ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.070+0000 7f0c8aaea640 1 --2- 172.21.15.16:0/2340730956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c8413a8b0 0x7f0c8413ad30 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c007500 tx=0x7f0c7c004ab0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.070+0000 7f0c8a2e9640 1 -- 172.21.15.16:0/2340730956 reap_dead start 2023-11-24T06:32:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.070+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 shutdown_connections 2023-11-24T06:32:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.070+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 >> 172.21.15.16:0/2340730956 conn(0x7f0c8406a800 msgr2=0x7f0c840ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.070+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 shutdown_connections 2023-11-24T06:32:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.070+0000 7f0c8aaea640 1 -- 172.21.15.16:0/2340730956 wait complete. 2023-11-24T06:32:07.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T06:32:07.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.313+0000 7ff093fff640 1 Processor -- start 2023-11-24T06:32:07.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.313+0000 7ff093fff640 1 -- start start 2023-11-24T06:32:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.315+0000 7ff093fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff0941447f0 0x7ff09415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.315+0000 7ff093fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0940cccf0 0x7ff09415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff093fff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09415c5c0 con 0x7ff09406c470 2023-11-24T06:32:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff093fff640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09406d690 con 0x7ff0940cccf0 2023-11-24T06:32:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff093fff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0940cebb0 con 0x7ff0941447f0 2023-11-24T06:32:07.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff0937fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0940cccf0 0x7ff09415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff0937fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0940cccf0 0x7ff09415c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40242/0 (socket says 172.21.15.16:40242) 2023-11-24T06:32:07.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff0937fe640 1 -- 172.21.15.16:0/1198053428 learned_addr learned my addr 172.21.15.16:0/1198053428 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff0937fe640 1 -- 172.21.15.16:0/1198053428 >> v2:172.21.15.173:3300/0 conn(0x7ff0941447f0 msgr2=0x7ff09415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff0937fe640 1 --2- 172.21.15.16:0/1198053428 >> v2:172.21.15.173:3300/0 conn(0x7ff0941447f0 0x7ff09415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff0937fe640 1 -- 172.21.15.16:0/1198053428 >> v1:172.21.15.173:6789/0 conn(0x7ff09406c470 legacy=0x7ff0940cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.316+0000 7ff0937fe640 1 -- 172.21.15.16:0/1198053428 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09406ca10 con 0x7ff0940cccf0 2023-11-24T06:32:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.317+0000 7ff0937fe640 1 --2- 172.21.15.16:0/1198053428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0940cccf0 0x7ff09415c7c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff094072520 tx=0x7ff08c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6496c4523a4594c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.318+0000 7ff091ffb640 1 -- 172.21.15.16:0/1198053428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0317e0 con 0x7ff0940cccf0 2023-11-24T06:32:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.318+0000 7ff091ffb640 1 -- 172.21.15.16:0/1198053428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c031940 con 0x7ff0940cccf0 2023-11-24T06:32:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.318+0000 7ff091ffb640 1 -- 172.21.15.16:0/1198053428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c031cd0 con 0x7ff0940cccf0 2023-11-24T06:32:07.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.318+0000 7ff093fff640 1 -- 172.21.15.16:0/1198053428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0940cccf0 msgr2=0x7ff09415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.318+0000 7ff093fff640 1 --2- 172.21.15.16:0/1198053428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0940cccf0 0x7ff09415c7c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff094072520 tx=0x7ff08c02b880 comp rx=0 tx=0).stop 2023-11-24T06:32:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.319+0000 7ff093fff640 1 -- 172.21.15.16:0/1198053428 shutdown_connections 2023-11-24T06:32:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.319+0000 7ff093fff640 1 --2- 172.21.15.16:0/1198053428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0940cccf0 0x7ff09415c7c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.319+0000 7ff093fff640 1 --2- 172.21.15.16:0/1198053428 >> v2:172.21.15.173:3300/0 conn(0x7ff0941447f0 0x7ff09415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.319+0000 7ff093fff640 1 -- 172.21.15.16:0/1198053428 >> 172.21.15.16:0/1198053428 conn(0x7ff09406a3e0 msgr2=0x7ff09406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.319+0000 7ff093fff640 1 -- 172.21.15.16:0/1198053428 shutdown_connections 2023-11-24T06:32:07.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.319+0000 7ff093fff640 1 -- 172.21.15.16:0/1198053428 wait complete. 2023-11-24T06:32:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.321+0000 7ff093fff640 1 Processor -- start 2023-11-24T06:32:07.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.321+0000 7ff093fff640 1 -- start start 2023-11-24T06:32:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.322+0000 7ff093fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09406c470 0x7ff094158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff093fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0940cccf0 0x7ff09413c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff093fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0941447f0 0x7ff09413e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff093fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0940acdb0 con 0x7ff0941447f0 2023-11-24T06:32:07.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff093fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0940ad090 con 0x7ff09406c470 2023-11-24T06:32:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff093fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0940ad370 con 0x7ff0940cccf0 2023-11-24T06:32:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0941447f0 0x7ff09413e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0941447f0 0x7ff09413e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40244/0 (socket says 172.21.15.16:40244) 2023-11-24T06:32:07.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0937fe640 1 -- 172.21.15.16:0/812540226 learned_addr learned my addr 172.21.15.16:0/812540226 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0927fc640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0940cccf0 0x7ff09413c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff092ffd640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09406c470 0x7ff094158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0937fe640 1 -- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0940cccf0 msgr2=0x7ff09413c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0937fe640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0940cccf0 0x7ff09413c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0937fe640 1 -- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09406c470 msgr2=0x7ff094158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0937fe640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09406c470 0x7ff094158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0937fe640 1 -- 172.21.15.16:0/812540226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c0094d0 con 0x7ff0941447f0 2023-11-24T06:32:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff0927fc640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0940cccf0 0x7ff09413c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.323+0000 7ff092ffd640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09406c470 0x7ff094158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.324+0000 7ff0937fe640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0941447f0 0x7ff09413e270 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0090b0 tx=0x7ff08c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.324+0000 7ff08bfff640 1 -- 172.21.15.16:0/812540226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c032880 con 0x7ff0941447f0 2023-11-24T06:32:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.324+0000 7ff08bfff640 1 -- 172.21.15.16:0/812540226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c032a00 con 0x7ff0941447f0 2023-11-24T06:32:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.324+0000 7ff08bfff640 1 -- 172.21.15.16:0/812540226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c032db0 con 0x7ff0941447f0 2023-11-24T06:32:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.324+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0940ad650 con 0x7ff0941447f0 2023-11-24T06:32:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.324+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09413cc20 con 0x7ff0941447f0 2023-11-24T06:32:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.326+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff094067450 con 0x7ff0941447f0 2023-11-24T06:32:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.326+0000 7ff08bfff640 1 -- 172.21.15.16:0/812540226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff08c04d3f0 con 0x7ff0941447f0 2023-11-24T06:32:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.328+0000 7ff08bfff640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff07406b470 0x7ff07406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.328+0000 7ff08bfff640 1 -- 172.21.15.16:0/812540226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7ff08c0abe20 con 0x7ff0941447f0 2023-11-24T06:32:07.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.329+0000 7ff092ffd640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff07406b470 0x7ff07406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.330+0000 7ff092ffd640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff07406b470 0x7ff07406d930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff08400a870 tx=0x7ff084008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.333+0000 7ff08bfff640 1 -- 172.21.15.16:0/812540226 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff08c076730 con 0x7ff0941447f0 2023-11-24T06:32:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.524+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7ff09413ce70 con 0x7ff07406b470 2023-11-24T06:32:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.525+0000 7ff08bfff640 1 -- 172.21.15.16:0/812540226 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff08400c7d0 con 0x7ff07406b470 2023-11-24T06:32:07.528 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T06:32:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.527+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff07406b470 msgr2=0x7ff07406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.527+0000 7ff093fff640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff07406b470 0x7ff07406d930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff08400a870 tx=0x7ff084008040 comp rx=0 tx=0).stop 2023-11-24T06:32:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.527+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0941447f0 msgr2=0x7ff09413e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.527+0000 7ff093fff640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0941447f0 0x7ff09413e270 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0090b0 tx=0x7ff08c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.528+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 shutdown_connections 2023-11-24T06:32:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.528+0000 7ff093fff640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7ff07406b470 0x7ff07406d930 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.528+0000 7ff093fff640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0941447f0 0x7ff09413e270 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.528+0000 7ff093fff640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff0940cccf0 0x7ff09413c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.528+0000 7ff093fff640 1 --2- 172.21.15.16:0/812540226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff09406c470 0x7ff094158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.528+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 >> 172.21.15.16:0/812540226 conn(0x7ff09406a3e0 msgr2=0x7ff094159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.528+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 shutdown_connections 2023-11-24T06:32:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.528+0000 7ff093fff640 1 -- 172.21.15.16:0/812540226 wait complete. 2023-11-24T06:32:07.549 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T06:32:07.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:32:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.771+0000 7f85a1048640 1 Processor -- start 2023-11-24T06:32:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.772+0000 7f85a1048640 1 -- start start 2023-11-24T06:32:07.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.773+0000 7f85a1048640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f859c0cd240 0x7f859c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.773+0000 7f85a1048640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859c06cea0 con 0x7f859c06c6b0 2023-11-24T06:32:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.773+0000 7f85a1048640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859c06cbc0 con 0x7f859c09a170 2023-11-24T06:32:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.773+0000 7f85a1048640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c06f350 con 0x7f859c0cd240 2023-11-24T06:32:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.774+0000 7f859a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f859c06c6b0 0x7f859c099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38638/0 (socket says 172.21.15.16:38638) 2023-11-24T06:32:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.774+0000 7f859a59c640 1 -- 172.21.15.16:0/1813261204 learned_addr learned my addr 172.21.15.16:0/1813261204 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.774+0000 7f859ad9d640 1 --2- 172.21.15.16:0/1813261204 >> v2:172.21.15.16:3300/0 conn(0x7f859c0cd240 0x7f859c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.774+0000 7f8599d9b640 1 -- 172.21.15.16:0/1813261204 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980638657 0 0) 0x7f859c06cea0 con 0x7f859c06c6b0 2023-11-24T06:32:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.774+0000 7f8599d9b640 1 -- 172.21.15.16:0/1813261204 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8584003520 con 0x7f859c06c6b0 2023-11-24T06:32:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.774+0000 7f859ad9d640 1 -- 172.21.15.16:0/1813261204 >> v1:172.21.15.173:6789/0 conn(0x7f859c09a170 legacy=0x7f859c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.774+0000 7f859ad9d640 1 -- 172.21.15.16:0/1813261204 >> v1:172.21.15.16:6789/0 conn(0x7f859c06c6b0 legacy=0x7f859c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.774+0000 7f859ad9d640 1 -- 172.21.15.16:0/1813261204 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c06f630 con 0x7f859c0cd240 2023-11-24T06:32:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.775+0000 7f859ad9d640 1 --2- 172.21.15.16:0/1813261204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c0cd240 0x7f859c0b3820 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f858c0097a0 tx=0x7f858c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf429c835f5b59e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.776+0000 7f8599d9b640 1 -- 172.21.15.16:0/1813261204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c030610 con 0x7f859c0cd240 2023-11-24T06:32:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.776+0000 7f8599d9b640 1 -- 172.21.15.16:0/1813261204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c030770 con 0x7f859c0cd240 2023-11-24T06:32:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.776+0000 7f8599d9b640 1 -- 172.21.15.16:0/1813261204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c030b00 con 0x7f859c0cd240 2023-11-24T06:32:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.776+0000 7f85a1048640 1 -- 172.21.15.16:0/1813261204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c0cd240 msgr2=0x7f859c0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.776+0000 7f85a1048640 1 --2- 172.21.15.16:0/1813261204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c0cd240 0x7f859c0b3820 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f858c0097a0 tx=0x7f858c02b960 comp rx=0 tx=0).stop 2023-11-24T06:32:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.776+0000 7f85a1048640 1 -- 172.21.15.16:0/1813261204 shutdown_connections 2023-11-24T06:32:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.776+0000 7f85a1048640 1 --2- 172.21.15.16:0/1813261204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c0cd240 0x7f859c0b3820 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.776+0000 7f85a1048640 1 -- 172.21.15.16:0/1813261204 >> 172.21.15.16:0/1813261204 conn(0x7f859c0b2d00 msgr2=0x7f859c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.777+0000 7f85a1048640 1 -- 172.21.15.16:0/1813261204 shutdown_connections 2023-11-24T06:32:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.777+0000 7f85a1048640 1 -- 172.21.15.16:0/1813261204 wait complete. 2023-11-24T06:32:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.778+0000 7f85a1048640 1 Processor -- start 2023-11-24T06:32:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.778+0000 7f85a1048640 1 -- start start 2023-11-24T06:32:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.779+0000 7f85a1048640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c06c6b0 0x7f859c0c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.779+0000 7f85a1048640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f859c09a170 0x7f859c0c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.779+0000 7f85a1048640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f859c0cd240 0x7f859c1f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.779+0000 7f85a1048640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c0c8030 con 0x7f859c06c6b0 2023-11-24T06:32:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.779+0000 7f85a1048640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c0c7a70 con 0x7f859c09a170 2023-11-24T06:32:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.779+0000 7f85a1048640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c0c7d50 con 0x7f859c0cd240 2023-11-24T06:32:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c06c6b0 0x7f859c0c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c06c6b0 0x7f859c0c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40264/0 (socket says 172.21.15.16:40264) 2023-11-24T06:32:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859ad9d640 1 -- 172.21.15.16:0/3733490387 learned_addr learned my addr 172.21.15.16:0/3733490387 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859a59c640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f859c09a170 0x7f859c0c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859ad9d640 1 -- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f859c0cd240 msgr2=0x7f859c1f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859ad9d640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f859c0cd240 0x7f859c1f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859ad9d640 1 -- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f859c09a170 msgr2=0x7f859c0c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859ad9d640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f859c09a170 0x7f859c0c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859ad9d640 1 -- 172.21.15.16:0/3733490387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c009400 con 0x7f859c06c6b0 2023-11-24T06:32:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.780+0000 7f859ad9d640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c06c6b0 0x7f859c0c4ec0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f858c0098c0 tx=0x7f858c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.781+0000 7f85837fe640 1 -- 172.21.15.16:0/3733490387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c03e070 con 0x7f859c06c6b0 2023-11-24T06:32:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.781+0000 7f85837fe640 1 -- 172.21.15.16:0/3733490387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c030e10 con 0x7f859c06c6b0 2023-11-24T06:32:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.781+0000 7f85837fe640 1 -- 172.21.15.16:0/3733490387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c039c30 con 0x7f859c06c6b0 2023-11-24T06:32:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.781+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859c0c93d0 con 0x7f859c06c6b0 2023-11-24T06:32:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.781+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859c0c7760 con 0x7f859c06c6b0 2023-11-24T06:32:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.782+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f859c0cd620 con 0x7f859c06c6b0 2023-11-24T06:32:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.783+0000 7f85837fe640 1 -- 172.21.15.16:0/3733490387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f858c055430 con 0x7f859c06c6b0 2023-11-24T06:32:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.784+0000 7f85837fe640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f858806b470 0x7f858806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.784+0000 7f85837fe640 1 -- 172.21.15.16:0/3733490387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f858c0accc0 con 0x7f859c06c6b0 2023-11-24T06:32:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.784+0000 7f85837fe640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7f85880712d0 0x7f85880736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.784+0000 7f85837fe640 1 -- 172.21.15.16:0/3733490387 --> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8588073dc0 con 0x7f85880712d0 2023-11-24T06:32:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.784+0000 7f85837fe640 1 -- 172.21.15.16:0/3733490387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f858c0acfa0 con 0x7f859c06c6b0 2023-11-24T06:32:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.785+0000 7f859a59c640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f858806b470 0x7f858806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.785+0000 7f859b59e640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7f85880712d0 0x7f85880736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.786+0000 7f859a59c640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f858806b470 0x7f858806d930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f859c0d06c0 tx=0x7f8590009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.786+0000 7f859b59e640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7f85880712d0 0x7f85880736f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:07.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.788+0000 7f85837fe640 1 -- 172.21.15.16:0/3733490387 <== osd.6 v2:172.21.15.173:6808/2309591130 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8588073dc0 con 0x7f85880712d0 2023-11-24T06:32:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.802+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 --> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f859c0675d0 con 0x7f85880712d0 2023-11-24T06:32:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.803+0000 7f85837fe640 1 -- 172.21.15.16:0/3733490387 <== osd.6 v2:172.21.15.173:6808/2309591130 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f859c0675d0 con 0x7f85880712d0 2023-11-24T06:32:07.807 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:32:07.807 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:32:07.807 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:32:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7f85880712d0 msgr2=0x7f85880736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f85a1048640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6808/2309591130,v1:172.21.15.173:6809/2309591130] conn(0x7f85880712d0 0x7f85880736f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f858806b470 msgr2=0x7f858806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f85a1048640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f858806b470 0x7f858806d930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f859c0d06c0 tx=0x7f8590009040 comp rx=0 tx=0).stop 2023-11-24T06:32:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c06c6b0 msgr2=0x7f859c0c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f85a1048640 1 --2- 172.21.15.16:0/3733490387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f859c06c6b0 0x7f859c0c4ec0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f858c0098c0 tx=0x7f858c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:32:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f859b59e640 1 -- 172.21.15.16:0/3733490387 reap_dead start 2023-11-24T06:32:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 shutdown_connections 2023-11-24T06:32:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 >> 172.21.15.16:0/3733490387 conn(0x7f859c0b2d00 msgr2=0x7f859c099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.804+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 shutdown_connections 2023-11-24T06:32:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:07.805+0000 7f85a1048640 1 -- 172.21.15.16:0/3733490387 wait complete. 2023-11-24T06:32:07.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T06:32:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.052+0000 7f9aee706640 1 Processor -- start 2023-11-24T06:32:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.053+0000 7f9aee706640 1 -- start start 2023-11-24T06:32:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.054+0000 7f9aee706640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ae806c470 0x7f9ae80cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.054+0000 7f9aee706640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9ae806dab0 0x7f9ae80ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.054+0000 7f9aee706640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae806ca10 con 0x7f9ae81447f0 2023-11-24T06:32:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.054+0000 7f9aee706640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae806d690 con 0x7f9ae806c470 2023-11-24T06:32:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.054+0000 7f9aee706640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae80cebb0 con 0x7f9ae806dab0 2023-11-24T06:32:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.055+0000 7f9aecf03640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ae806c470 0x7f9ae80cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.055+0000 7f9aecf03640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ae806c470 0x7f9ae80cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40274/0 (socket says 172.21.15.16:40274) 2023-11-24T06:32:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.055+0000 7f9aecf03640 1 -- 172.21.15.16:0/1612985553 learned_addr learned my addr 172.21.15.16:0/1612985553 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.055+0000 7f9aecf03640 1 -- 172.21.15.16:0/1612985553 >> v2:172.21.15.173:3300/0 conn(0x7f9ae806dab0 msgr2=0x7f9ae80ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.055+0000 7f9aecf03640 1 --2- 172.21.15.16:0/1612985553 >> v2:172.21.15.173:3300/0 conn(0x7f9ae806dab0 0x7f9ae80ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.055+0000 7f9aecf03640 1 -- 172.21.15.16:0/1612985553 >> v1:172.21.15.173:6789/0 conn(0x7f9ae81447f0 legacy=0x7f9ae815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.055+0000 7f9aecf03640 1 -- 172.21.15.16:0/1612985553 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae806ccf0 con 0x7f9ae806c470 2023-11-24T06:32:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.056+0000 7f9aecf03640 1 --2- 172.21.15.16:0/1612985553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 0x7f9ae80cc5e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80aea60 tx=0x7f9ad802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf865fcdbfe13f42 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9adffff640 1 -- 172.21.15.16:0/1612985553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad80317e0 con 0x7f9ae806c470 2023-11-24T06:32:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9adffff640 1 -- 172.21.15.16:0/1612985553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad8031940 con 0x7f9ae806c470 2023-11-24T06:32:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9adffff640 1 -- 172.21.15.16:0/1612985553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8031cd0 con 0x7f9ae806c470 2023-11-24T06:32:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9aee706640 1 -- 172.21.15.16:0/1612985553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 msgr2=0x7f9ae80cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9aee706640 1 --2- 172.21.15.16:0/1612985553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 0x7f9ae80cc5e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80aea60 tx=0x7f9ad802b880 comp rx=0 tx=0).stop 2023-11-24T06:32:08.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9aee706640 1 -- 172.21.15.16:0/1612985553 shutdown_connections 2023-11-24T06:32:08.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9aee706640 1 --2- 172.21.15.16:0/1612985553 >> v2:172.21.15.173:3300/0 conn(0x7f9ae806dab0 0x7f9ae80ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9aee706640 1 --2- 172.21.15.16:0/1612985553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 0x7f9ae80cc5e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9aee706640 1 -- 172.21.15.16:0/1612985553 >> 172.21.15.16:0/1612985553 conn(0x7f9ae806a3e0 msgr2=0x7f9ae806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:08.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.057+0000 7f9aee706640 1 -- 172.21.15.16:0/1612985553 shutdown_connections 2023-11-24T06:32:08.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.058+0000 7f9aee706640 1 -- 172.21.15.16:0/1612985553 wait complete. 2023-11-24T06:32:08.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.059+0000 7f9aee706640 1 Processor -- start 2023-11-24T06:32:08.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.059+0000 7f9aee706640 1 -- start start 2023-11-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.060+0000 7f9aee706640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 0x7f9ae81580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.060+0000 7f9aee706640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ae806dab0 0x7f9ae80ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.060+0000 7f9aee706640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ae81447f0 0x7f9ae814b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.060+0000 7f9aee706640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8158600 con 0x7f9ae806c470 2023-11-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.060+0000 7f9aee706640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80ae380 con 0x7f9ae81447f0 2023-11-24T06:32:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.060+0000 7f9aee706640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80b0b70 con 0x7f9ae806dab0 2023-11-24T06:32:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aed704640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 0x7f9ae81580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aed704640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 0x7f9ae81580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40288/0 (socket says 172.21.15.16:40288) 2023-11-24T06:32:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aed704640 1 -- 172.21.15.16:0/4159390021 learned_addr learned my addr 172.21.15.16:0/4159390021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aecf03640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ae806dab0 0x7f9ae80ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aed704640 1 -- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ae806dab0 msgr2=0x7f9ae80ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aed704640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ae806dab0 0x7f9ae80ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aed704640 1 -- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ae81447f0 msgr2=0x7f9ae814b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aed704640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ae81447f0 0x7f9ae814b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aed704640 1 -- 172.21.15.16:0/4159390021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80094d0 con 0x7f9ae806c470 2023-11-24T06:32:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aecf03640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ae806dab0 0x7f9ae80ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.061+0000 7f9aed704640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 0x7f9ae81580c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400b3f0 tx=0x7f9ae400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.062+0000 7f9addffb640 1 -- 172.21.15.16:0/4159390021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4011830 con 0x7f9ae806c470 2023-11-24T06:32:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.062+0000 7f9addffb640 1 -- 172.21.15.16:0/4159390021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae4011990 con 0x7f9ae806c470 2023-11-24T06:32:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.062+0000 7f9addffb640 1 -- 172.21.15.16:0/4159390021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4011d40 con 0x7f9ae806c470 2023-11-24T06:32:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.062+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae80b0240 con 0x7f9ae806c470 2023-11-24T06:32:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.062+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae80b0800 con 0x7f9ae806c470 2023-11-24T06:32:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.063+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae8067450 con 0x7f9ae806c470 2023-11-24T06:32:08.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.064+0000 7f9addffb640 1 -- 172.21.15.16:0/4159390021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f9ae401a400 con 0x7f9ae806c470 2023-11-24T06:32:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.066+0000 7f9addffb640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9acc06b470 0x7f9acc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.066+0000 7f9addffb640 1 -- 172.21.15.16:0/4159390021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f9ae4089d20 con 0x7f9ae806c470 2023-11-24T06:32:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.066+0000 7f9aecf03640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9acc06b470 0x7f9acc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.067+0000 7f9aecf03640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9acc06b470 0x7f9acc06d930 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8036000 tx=0x7f9ad803d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.070+0000 7f9addffb640 1 -- 172.21.15.16:0/4159390021 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ae4054770 con 0x7f9ae806c470 2023-11-24T06:32:08.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.262+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ae8144bd0 con 0x7f9acc06b470 2023-11-24T06:32:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.263+0000 7f9addffb640 1 -- 172.21.15.16:0/4159390021 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9ad8031c30 con 0x7f9acc06b470 2023-11-24T06:32:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T06:32:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.265+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9acc06b470 msgr2=0x7f9acc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.265+0000 7f9aee706640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9acc06b470 0x7f9acc06d930 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8036000 tx=0x7f9ad803d040 comp rx=0 tx=0).stop 2023-11-24T06:32:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.265+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 msgr2=0x7f9ae81580c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.265+0000 7f9aee706640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 0x7f9ae81580c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400b3f0 tx=0x7f9ae400b8b0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.266+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 shutdown_connections 2023-11-24T06:32:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.266+0000 7f9aee706640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f9acc06b470 0x7f9acc06d930 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.266+0000 7f9aee706640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ae81447f0 0x7f9ae814b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.266+0000 7f9aee706640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ae806dab0 0x7f9ae80ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.266+0000 7f9aee706640 1 --2- 172.21.15.16:0/4159390021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae806c470 0x7f9ae81580c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.266+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 >> 172.21.15.16:0/4159390021 conn(0x7f9ae806a3e0 msgr2=0x7f9ae8142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.266+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 shutdown_connections 2023-11-24T06:32:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.266+0000 7f9aee706640 1 -- 172.21.15.16:0/4159390021 wait complete. 2023-11-24T06:32:08.288 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T06:32:08.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:32:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.497+0000 7f3667f3e640 1 Processor -- start 2023-11-24T06:32:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.498+0000 7f3667f3e640 1 -- start start 2023-11-24T06:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.499+0000 7f3667f3e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600d1c40 0x7f36600b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.499+0000 7f3667f3e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36600b3400 con 0x7f366015d610 2023-11-24T06:32:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.499+0000 7f3667f3e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36600b27f0 con 0x7f36600d17e0 2023-11-24T06:32:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.499+0000 7f3667f3e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36600d1390 con 0x7f36600d1c40 2023-11-24T06:32:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3665cda640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f366015d610 0x7f36600ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38642/0 (socket says 172.21.15.16:38642) 2023-11-24T06:32:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3665cda640 1 -- 172.21.15.16:0/110581560 learned_addr learned my addr 172.21.15.16:0/110581560 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f36664db640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600d1c40 0x7f36600b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632635261 0 0) 0x7f36600b3400 con 0x7f366015d610 2023-11-24T06:32:08.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f364c003520 con 0x7f366015d610 2023-11-24T06:32:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2735081744 0 0) 0x7f364c003520 con 0x7f366015d610 2023-11-24T06:32:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600d1c40 msgr2=0x7f36600b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3664cd8640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600d1c40 0x7f36600b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 >> v1:172.21.15.173:6789/0 conn(0x7f36600d17e0 legacy=0x7f36600b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36600b2ad0 con 0x7f366015d610 2023-11-24T06:32:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.500+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f365c001670 con 0x7f366015d610 2023-11-24T06:32:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 >> v1:172.21.15.16:6789/0 conn(0x7f366015d610 legacy=0x7f36600ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f3664cd8640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f364c0039a0 0x7f364c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f3664cd8640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 0x7f364c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f3664cd8640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600d1c40 0x7f364c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36600d1390 con 0x7f364c0047c0 2023-11-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c00ba60 con 0x7f364c0039a0 2023-11-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c00bc20 con 0x7f36600d1c40 2023-11-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f36654d9640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 0x7f364c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f36664db640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600d1c40 0x7f364c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f3665cda640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f364c0039a0 0x7f364c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f36654d9640 1 -- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600d1c40 msgr2=0x7f364c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f36654d9640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600d1c40 0x7f364c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f36654d9640 1 -- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f364c0039a0 msgr2=0x7f364c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f36654d9640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f364c0039a0 0x7f364c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f36654d9640 1 -- 172.21.15.16:0/110581560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c003520 con 0x7f364c0047c0 2023-11-24T06:32:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.501+0000 7f3665cda640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f364c0039a0 0x7f364c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.503+0000 7f36654d9640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 0x7f364c004c40 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3650001720 tx=0x7f364c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42445720a0581b77 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.503+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500052d0 con 0x7f364c0047c0 2023-11-24T06:32:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.503+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365002e050 con 0x7f364c0047c0 2023-11-24T06:32:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.503+0000 7f3664cd8640 1 -- 172.21.15.16:0/110581560 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500035f0 con 0x7f364c0047c0 2023-11-24T06:32:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.503+0000 7f3667f3e640 1 -- 172.21.15.16:0/110581560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 msgr2=0x7f364c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.503+0000 7f3667f3e640 1 --2- 172.21.15.16:0/110581560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 0x7f364c004c40 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f3650001720 tx=0x7f364c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.503+0000 7f36664db640 1 -- 172.21.15.16:0/110581560 reap_dead start 2023-11-24T06:32:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.503+0000 7f3667f3e640 1 -- 172.21.15.16:0/110581560 shutdown_connections 2023-11-24T06:32:08.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.503+0000 7f3667f3e640 1 -- 172.21.15.16:0/110581560 >> 172.21.15.16:0/110581560 conn(0x7f366009c540 msgr2=0x7f36600d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.504+0000 7f3667f3e640 1 -- 172.21.15.16:0/110581560 shutdown_connections 2023-11-24T06:32:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.504+0000 7f3667f3e640 1 -- 172.21.15.16:0/110581560 wait complete. 2023-11-24T06:32:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.505+0000 7f3667f3e640 1 Processor -- start 2023-11-24T06:32:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.505+0000 7f3667f3e640 1 -- start start 2023-11-24T06:32:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.506+0000 7f3667f3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 0x7f3660159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.506+0000 7f3667f3e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600b6a60 0x7f36600cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.506+0000 7f3667f3e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36600b8980 0x7f36600cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.506+0000 7f3667f3e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36600ca2d0 con 0x7f364c0047c0 2023-11-24T06:32:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.506+0000 7f3667f3e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36600b6f60 con 0x7f36600b8980 2023-11-24T06:32:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.506+0000 7f3667f3e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36600b7240 con 0x7f36600b6a60 2023-11-24T06:32:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.507+0000 7f3665cda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 0x7f3660159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.507+0000 7f3665cda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 0x7f3660159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40314/0 (socket says 172.21.15.16:40314) 2023-11-24T06:32:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.507+0000 7f3665cda640 1 -- 172.21.15.16:0/1586984983 learned_addr learned my addr 172.21.15.16:0/1586984983 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.507+0000 7f3665cda640 1 -- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600b6a60 msgr2=0x7f36600cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.507+0000 7f3665cda640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36600b6a60 0x7f36600cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.507+0000 7f3665cda640 1 -- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36600b8980 msgr2=0x7f36600cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.507+0000 7f3665cda640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36600b8980 0x7f36600cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.507+0000 7f3665cda640 1 -- 172.21.15.16:0/1586984983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36600b7520 con 0x7f364c0047c0 2023-11-24T06:32:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.507+0000 7f3665cda640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 0x7f3660159a20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f365c00d7a0 tx=0x7f365c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.508+0000 7f36567fc640 1 -- 172.21.15.16:0/1586984983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0074e0 con 0x7f364c0047c0 2023-11-24T06:32:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.508+0000 7f36567fc640 1 -- 172.21.15.16:0/1586984983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c007640 con 0x7f364c0047c0 2023-11-24T06:32:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.508+0000 7f36567fc640 1 -- 172.21.15.16:0/1586984983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c004a20 con 0x7f364c0047c0 2023-11-24T06:32:08.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.508+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36600b7800 con 0x7f364c0047c0 2023-11-24T06:32:08.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.508+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36600a0a00 con 0x7f364c0047c0 2023-11-24T06:32:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.509+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36600d3910 con 0x7f364c0047c0 2023-11-24T06:32:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.510+0000 7f36567fc640 1 -- 172.21.15.16:0/1586984983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f365c004b80 con 0x7f364c0047c0 2023-11-24T06:32:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.511+0000 7f36567fc640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f364406b470 0x7f364406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.511+0000 7f36567fc640 1 -- 172.21.15.16:0/1586984983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f365c089870 con 0x7f364c0047c0 2023-11-24T06:32:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.511+0000 7f36567fc640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7f36440712d0 0x7f36440736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.511+0000 7f36567fc640 1 -- 172.21.15.16:0/1586984983 --> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3644073dc0 con 0x7f36440712d0 2023-11-24T06:32:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.511+0000 7f36567fc640 1 -- 172.21.15.16:0/1586984983 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f365c08bcd0 con 0x7f364c0047c0 2023-11-24T06:32:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.511+0000 7f36654d9640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f364406b470 0x7f364406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.512+0000 7f36664db640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7f36440712d0 0x7f36440736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.513+0000 7f36654d9640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f364406b470 0x7f364406d930 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3650033000 tx=0x7f365002f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.513+0000 7f36664db640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7f36440712d0 0x7f36440736f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.515+0000 7f36567fc640 1 -- 172.21.15.16:0/1586984983 <== osd.7 v2:172.21.15.173:6820/2972943157 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3644073dc0 con 0x7f36440712d0 2023-11-24T06:32:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.529+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 --> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f36600d20d0 con 0x7f36440712d0 2023-11-24T06:32:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.531+0000 7f36567fc640 1 -- 172.21.15.16:0/1586984983 <== osd.7 v2:172.21.15.173:6820/2972943157 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f365800f040 con 0x7f36440712d0 2023-11-24T06:32:08.534 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-11-24T06:32:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:32:08.534 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-11-24T06:32:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.531+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7f36440712d0 msgr2=0x7f36440736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.531+0000 7f3667f3e640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6820/2972943157,v1:172.21.15.173:6822/2972943157] conn(0x7f36440712d0 0x7f36440736f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.532+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f364406b470 msgr2=0x7f364406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.532+0000 7f3667f3e640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f364406b470 0x7f364406d930 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3650033000 tx=0x7f365002f000 comp rx=0 tx=0).stop 2023-11-24T06:32:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.532+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 msgr2=0x7f3660159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.532+0000 7f3667f3e640 1 --2- 172.21.15.16:0/1586984983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c0047c0 0x7f3660159a20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f365c00d7a0 tx=0x7f365c00dc60 comp rx=0 tx=0).stop 2023-11-24T06:32:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.532+0000 7f36664db640 1 -- 172.21.15.16:0/1586984983 reap_dead start 2023-11-24T06:32:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.532+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 shutdown_connections 2023-11-24T06:32:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.532+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 >> 172.21.15.16:0/1586984983 conn(0x7f366009c540 msgr2=0x7f36600d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.532+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 shutdown_connections 2023-11-24T06:32:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.532+0000 7f3667f3e640 1 -- 172.21.15.16:0/1586984983 wait complete. 2023-11-24T06:32:08.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T06:32:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.772+0000 7fce849d5640 1 Processor -- start 2023-11-24T06:32:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.772+0000 7fce849d5640 1 -- start start 2023-11-24T06:32:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.774+0000 7fce849d5640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fce80146730 0x7fce800ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.774+0000 7fce849d5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce800d1520 0x7fce80161770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.774+0000 7fce849d5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce80146b10 con 0x7fce800d1090 2023-11-24T06:32:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.774+0000 7fce849d5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce800cb6c0 con 0x7fce800d1520 2023-11-24T06:32:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.774+0000 7fce849d5640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce800c6860 con 0x7fce80146730 2023-11-24T06:32:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.774+0000 7fce7ed9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce800d1520 0x7fce80161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.775+0000 7fce7dd9b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce800d1090 0x7fce80161060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38648/0 (socket says 172.21.15.16:38648) 2023-11-24T06:32:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.775+0000 7fce7dd9b640 1 -- 172.21.15.16:0/2912900412 learned_addr learned my addr 172.21.15.16:0/2912900412 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.775+0000 7fce7e59c640 1 --2- 172.21.15.16:0/2912900412 >> v2:172.21.15.173:3300/0 conn(0x7fce80146730 0x7fce800ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.775+0000 7fce7ed9d640 1 -- 172.21.15.16:0/2912900412 >> v2:172.21.15.173:3300/0 conn(0x7fce80146730 msgr2=0x7fce800ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.775+0000 7fce7ed9d640 1 --2- 172.21.15.16:0/2912900412 >> v2:172.21.15.173:3300/0 conn(0x7fce80146730 0x7fce800ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.775+0000 7fce7ed9d640 1 -- 172.21.15.16:0/2912900412 >> v1:172.21.15.16:6789/0 conn(0x7fce800d1090 legacy=0x7fce80161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.775+0000 7fce7ed9d640 1 -- 172.21.15.16:0/2912900412 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800d1990 con 0x7fce800d1520 2023-11-24T06:32:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.775+0000 7fce7d59a640 1 -- 172.21.15.16:0/2912900412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186081435 0 0) 0x7fce80146b10 con 0x7fce800d1090 2023-11-24T06:32:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.776+0000 7fce7ed9d640 1 --2- 172.21.15.16:0/2912900412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce800d1520 0x7fce80161770 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fce800d1060 tx=0x7fce7802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59b864394ac217c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce7d59a640 1 -- 172.21.15.16:0/2912900412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78031830 con 0x7fce800d1520 2023-11-24T06:32:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce7d59a640 1 -- 172.21.15.16:0/2912900412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce78031990 con 0x7fce800d1520 2023-11-24T06:32:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce7d59a640 1 -- 172.21.15.16:0/2912900412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78031d20 con 0x7fce800d1520 2023-11-24T06:32:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce849d5640 1 -- 172.21.15.16:0/2912900412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce800d1520 msgr2=0x7fce80161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce849d5640 1 --2- 172.21.15.16:0/2912900412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce800d1520 0x7fce80161770 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fce800d1060 tx=0x7fce7802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce849d5640 1 -- 172.21.15.16:0/2912900412 shutdown_connections 2023-11-24T06:32:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce849d5640 1 --2- 172.21.15.16:0/2912900412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce800d1520 0x7fce80161770 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce849d5640 1 --2- 172.21.15.16:0/2912900412 >> v2:172.21.15.173:3300/0 conn(0x7fce80146730 0x7fce800ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce849d5640 1 -- 172.21.15.16:0/2912900412 >> 172.21.15.16:0/2912900412 conn(0x7fce80066690 msgr2=0x7fce8006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce849d5640 1 -- 172.21.15.16:0/2912900412 shutdown_connections 2023-11-24T06:32:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.777+0000 7fce849d5640 1 -- 172.21.15.16:0/2912900412 wait complete. 2023-11-24T06:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.779+0000 7fce849d5640 1 Processor -- start 2023-11-24T06:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.779+0000 7fce849d5640 1 -- start start 2023-11-24T06:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.780+0000 7fce849d5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce800d1090 0x7fce801339c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.780+0000 7fce849d5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce800d1520 0x7fce80140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.780+0000 7fce849d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce80146730 0x7fce80140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.780+0000 7fce849d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce80136460 con 0x7fce80146730 2023-11-24T06:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.780+0000 7fce849d5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce80143910 con 0x7fce800d1090 2023-11-24T06:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.780+0000 7fce849d5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce80143350 con 0x7fce800d1520 2023-11-24T06:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce80146730 0x7fce80140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce80146730 0x7fce80140b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40336/0 (socket says 172.21.15.16:40336) 2023-11-24T06:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7ed9d640 1 -- 172.21.15.16:0/1761139300 learned_addr learned my addr 172.21.15.16:0/1761139300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7dd9b640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce800d1520 0x7fce80140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7e59c640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce800d1090 0x7fce801339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7ed9d640 1 -- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce800d1520 msgr2=0x7fce80140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7ed9d640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce800d1520 0x7fce80140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7ed9d640 1 -- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce800d1090 msgr2=0x7fce801339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7ed9d640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce800d1090 0x7fce801339c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.781+0000 7fce7ed9d640 1 -- 172.21.15.16:0/1761139300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78009400 con 0x7fce80146730 2023-11-24T06:32:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.782+0000 7fce7ed9d640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce80146730 0x7fce80140b50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fce78009890 tx=0x7fce780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.782+0000 7fce76ffd640 1 -- 172.21.15.16:0/1761139300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78004230 con 0x7fce80146730 2023-11-24T06:32:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.782+0000 7fce76ffd640 1 -- 172.21.15.16:0/1761139300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce78004390 con 0x7fce80146730 2023-11-24T06:32:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.782+0000 7fce76ffd640 1 -- 172.21.15.16:0/1761139300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7803a8a0 con 0x7fce80146730 2023-11-24T06:32:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.782+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce80143630 con 0x7fce80146730 2023-11-24T06:32:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.782+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce800c95a0 con 0x7fce80146730 2023-11-24T06:32:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.783+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce80074a50 con 0x7fce80146730 2023-11-24T06:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.784+0000 7fce76ffd640 1 -- 172.21.15.16:0/1761139300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fce7803aa00 con 0x7fce80146730 2023-11-24T06:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.785+0000 7fce76ffd640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce6006b470 0x7fce6006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.785+0000 7fce76ffd640 1 -- 172.21.15.16:0/1761139300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fce780ac7d0 con 0x7fce80146730 2023-11-24T06:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.786+0000 7fce7e59c640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce6006b470 0x7fce6006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.787+0000 7fce7e59c640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce6006b470 0x7fce6006d930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fce801f6680 tx=0x7fce700039b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.790+0000 7fce76ffd640 1 -- 172.21.15.16:0/1761139300 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce78077280 con 0x7fce80146730 2023-11-24T06:32:08.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.987+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fce800c6580 con 0x7fce6006b470 2023-11-24T06:32:08.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.988+0000 7fce76ffd640 1 -- 172.21.15.16:0/1761139300 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fce700096f0 con 0x7fce6006b470 2023-11-24T06:32:08.993 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T06:32:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce6006b470 msgr2=0x7fce6006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce6006b470 0x7fce6006d930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fce801f6680 tx=0x7fce700039b0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce80146730 msgr2=0x7fce80140b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce80146730 0x7fce80140b50 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fce78009890 tx=0x7fce780046c0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 shutdown_connections 2023-11-24T06:32:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fce6006b470 0x7fce6006d930 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce80146730 0x7fce80140b50 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fce800d1520 0x7fce80140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 --2- 172.21.15.16:0/1761139300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce800d1090 0x7fce801339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 >> 172.21.15.16:0/1761139300 conn(0x7fce80066690 msgr2=0x7fce800cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 shutdown_connections 2023-11-24T06:32:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:08.991+0000 7fce849d5640 1 -- 172.21.15.16:0/1761139300 wait complete. 2023-11-24T06:32:09.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:32:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.235+0000 7fe7123ac640 1 Processor -- start 2023-11-24T06:32:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.236+0000 7fe7123ac640 1 -- start start 2023-11-24T06:32:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.237+0000 7fe7123ac640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c15e670 0x7fe70c0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.237+0000 7fe7123ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe70c0ce6a0 0x7fe70c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.237+0000 7fe7123ac640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c0cca90 con 0x7fe70c15ea50 2023-11-24T06:32:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.237+0000 7fe7123ac640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c0cf170 con 0x7fe70c0ce6a0 2023-11-24T06:32:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.237+0000 7fe7123ac640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0ccd70 con 0x7fe70c15e670 2023-11-24T06:32:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.237+0000 7fe710ba9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe70c0ce6a0 0x7fe70c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.237+0000 7fe710ba9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe70c0ce6a0 0x7fe70c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40344/0 (socket says 172.21.15.16:40344) 2023-11-24T06:32:09.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.237+0000 7fe710ba9640 1 -- 172.21.15.16:0/3851949794 learned_addr learned my addr 172.21.15.16:0/3851949794 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.238+0000 7fe7113aa640 1 --2- 172.21.15.16:0/3851949794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c15e670 0x7fe70c0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:09.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.238+0000 7fe710ba9640 1 -- 172.21.15.16:0/3851949794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c15e670 msgr2=0x7fe70c0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.238+0000 7fe710ba9640 1 --2- 172.21.15.16:0/3851949794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c15e670 0x7fe70c0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.238+0000 7fe710ba9640 1 -- 172.21.15.16:0/3851949794 >> v1:172.21.15.173:6789/0 conn(0x7fe70c15ea50 legacy=0x7fe70c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.238+0000 7fe710ba9640 1 -- 172.21.15.16:0/3851949794 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c0ceab0 con 0x7fe70c0ce6a0 2023-11-24T06:32:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.239+0000 7fe710ba9640 1 --2- 172.21.15.16:0/3851949794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ce6a0 0x7fe70c0cf590 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe700009870 tx=0x7fe70002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b68ba5863dcb14 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.239+0000 7fe70bfff640 1 -- 172.21.15.16:0/3851949794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700031640 con 0x7fe70c0ce6a0 2023-11-24T06:32:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.239+0000 7fe70bfff640 1 -- 172.21.15.16:0/3851949794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7000317a0 con 0x7fe70c0ce6a0 2023-11-24T06:32:09.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.239+0000 7fe70bfff640 1 -- 172.21.15.16:0/3851949794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700031b30 con 0x7fe70c0ce6a0 2023-11-24T06:32:09.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.240+0000 7fe7123ac640 1 -- 172.21.15.16:0/3851949794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ce6a0 msgr2=0x7fe70c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:09.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.240+0000 7fe7123ac640 1 --2- 172.21.15.16:0/3851949794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ce6a0 0x7fe70c0cf590 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe700009870 tx=0x7fe70002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.240+0000 7fe7123ac640 1 -- 172.21.15.16:0/3851949794 shutdown_connections 2023-11-24T06:32:09.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.240+0000 7fe7123ac640 1 --2- 172.21.15.16:0/3851949794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ce6a0 0x7fe70c0cf590 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.240+0000 7fe7123ac640 1 --2- 172.21.15.16:0/3851949794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c15e670 0x7fe70c0b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.240+0000 7fe7123ac640 1 -- 172.21.15.16:0/3851949794 >> 172.21.15.16:0/3851949794 conn(0x7fe70c0b2a30 msgr2=0x7fe70c0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:09.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.240+0000 7fe7123ac640 1 -- 172.21.15.16:0/3851949794 shutdown_connections 2023-11-24T06:32:09.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.240+0000 7fe7123ac640 1 -- 172.21.15.16:0/3851949794 wait complete. 2023-11-24T06:32:09.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.242+0000 7fe7123ac640 1 Processor -- start 2023-11-24T06:32:09.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.242+0000 7fe7123ac640 1 -- start start 2023-11-24T06:32:09.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.243+0000 7fe7123ac640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70c15e670 0x7fe70c153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:09.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.243+0000 7fe7123ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c15ea50 0x7fe70c0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:09.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.243+0000 7fe7123ac640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c0c9df0 0x7fe70c154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:09.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.243+0000 7fe7123ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0c47b0 con 0x7fe70c15ea50 2023-11-24T06:32:09.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.243+0000 7fe7123ac640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c1539b0 con 0x7fe70c15e670 2023-11-24T06:32:09.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.243+0000 7fe7123ac640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0c88c0 con 0x7fe70c0c9df0 2023-11-24T06:32:09.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.243+0000 7fe710ba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c15ea50 0x7fe70c0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:09.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.243+0000 7fe710ba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c15ea50 0x7fe70c0c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40354/0 (socket says 172.21.15.16:40354) 2023-11-24T06:32:09.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.243+0000 7fe710ba9640 1 -- 172.21.15.16:0/230205956 learned_addr learned my addr 172.21.15.16:0/230205956 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:09.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe7113aa640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70c15e670 0x7fe70c153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:09.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe711bab640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c0c9df0 0x7fe70c154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:09.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe710ba9640 1 -- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c0c9df0 msgr2=0x7fe70c154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe710ba9640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c0c9df0 0x7fe70c154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe710ba9640 1 -- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70c15e670 msgr2=0x7fe70c153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe710ba9640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70c15e670 0x7fe70c153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe710ba9640 1 -- 172.21.15.16:0/230205956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000094d0 con 0x7fe70c15ea50 2023-11-24T06:32:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe711bab640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c0c9df0 0x7fe70c154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe7113aa640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70c15e670 0x7fe70c153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe710ba9640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c15ea50 0x7fe70c0c7fb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fe700009870 tx=0x7fe7000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe709ffb640 1 -- 172.21.15.16:0/230205956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70003ac60 con 0x7fe70c15ea50 2023-11-24T06:32:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.244+0000 7fe709ffb640 1 -- 172.21.15.16:0/230205956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70003adc0 con 0x7fe70c15ea50 2023-11-24T06:32:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.245+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c0c8ba0 con 0x7fe70c15ea50 2023-11-24T06:32:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.245+0000 7fe709ffb640 1 -- 172.21.15.16:0/230205956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700044630 con 0x7fe70c15ea50 2023-11-24T06:32:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.245+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c0c4a00 con 0x7fe70c15ea50 2023-11-24T06:32:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.246+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70c0c4bf0 con 0x7fe70c15ea50 2023-11-24T06:32:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.247+0000 7fe709ffb640 1 -- 172.21.15.16:0/230205956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe70002f050 con 0x7fe70c15ea50 2023-11-24T06:32:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.248+0000 7fe709ffb640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6f806b4a0 0x7fe6f806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.248+0000 7fe709ffb640 1 -- 172.21.15.16:0/230205956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fe7000abdb0 con 0x7fe70c15ea50 2023-11-24T06:32:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.248+0000 7fe7113aa640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6f806b4a0 0x7fe6f806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.249+0000 7fe7113aa640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6f806b4a0 0x7fe6f806d960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.252+0000 7fe709ffb640 1 -- 172.21.15.16:0/230205956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe700076740 con 0x7fe70c15ea50 2023-11-24T06:32:09.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.445+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe70c0b4270 con 0x7fe6f806b4a0 2023-11-24T06:32:09.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.451+0000 7fe709ffb640 1 -- 172.21.15.16:0/230205956 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48086 (secure 0 0 0) 0x7fe6fc009e10 con 0x7fe6f806b4a0 2023-11-24T06:32:09.455 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:32:09.455 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:32:09.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6f806b4a0 msgr2=0x7fe6f806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:09.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6f806b4a0 0x7fe6f806d960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc008040 comp rx=0 tx=0).stop 2023-11-24T06:32:09.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c15ea50 msgr2=0x7fe70c0c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:09.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c15ea50 0x7fe70c0c7fb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fe700009870 tx=0x7fe7000046c0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 shutdown_connections 2023-11-24T06:32:09.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7fe6f806b4a0 0x7fe6f806d960 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe70c0c9df0 0x7fe70c154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c15ea50 0x7fe70c0c7fb0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 --2- 172.21.15.16:0/230205956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe70c15e670 0x7fe70c153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:09.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 >> 172.21.15.16:0/230205956 conn(0x7fe70c0b2a30 msgr2=0x7fe70c0b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:09.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 shutdown_connections 2023-11-24T06:32:09.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:09.454+0000 7fe7123ac640 1 -- 172.21.15.16:0/230205956 wait complete. 2023-11-24T06:32:09.478 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2023-11-24T06:32:08.210420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259424,"kb_used_data":9960,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715296,"statfs":{"total":773094113280,"available":772828463104,"internally_reserved":0,"allocated":10199040,"data_stored":5040960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.153968"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-24T06:31:56.759975+0000","last_change":"2023-11-24T06:26:51.784978+0000","last_active":"2023-11-24T06:31:56.759975+0000","last_peered":"2023-11-24T06:31:56.759975+0000","last_clean":"2023-11-24T06:31:56.759975+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:31:56.759975+0000","last_undegraded":"2023-11-24T06:31:56.759975+0000","last_fullsized":"2023-11-24T06:31:56.759975+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:26:50.351558+0000","last_clean_scrub_stamp":"2023-11-24T06:26:50.351558+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.383957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1720320,"data_stored":1072896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.149,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5209999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.071},{"interface":"front","average":{"1min":1.216,"5min":1.2589999999999999,"15min":1.2889999999999999},"min":{"1min":0.72099999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.704,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":0.88600000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0700000000000001,"15min":1.0269999999999999},"min":{"1min":0.64900000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.827,"5min":2.456,"15min":2.456},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.258,"5min":1.2969999999999999,"15min":1.3620000000000001},"min":{"1min":0.83399999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.129}]},{"osd":2,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.204,"15min":1.2649999999999999},"min":{"1min":0.73099999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5900000000000001,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.258},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.179,"15min":1.1950000000000001},"min":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.4119999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.093}]},{"osd":3,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3,"15min":1.3819999999999999},"min":{"1min":0.78600000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6100000000000001,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":1.111},{"interface":"front","average":{"1min":1.256,"5min":1.2330000000000001,"15min":1.2130000000000001},"min":{"1min":0.89500000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.653,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.0329999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3560000000000001,"15min":1.5229999999999999},"min":{"1min":0.78800000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7190000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.454,"5min":1.47,"15min":1.556},"min":{"1min":1.0449999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7509999999999999,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.92500000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3080000000000001,"15min":1.3819999999999999},"min":{"1min":0.82999999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.919,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.46,"5min":1.7,"15min":1.752},"min":{"1min":0.88500000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.871,"5min":17.582999999999998,"15min":17.582999999999998},"last":0.76900000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6040000000000001,"15min":1.804},"min":{"1min":1.085,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.988,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5049999999999999,"15min":1.655},"min":{"1min":1.101,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.78,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.052}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0920000000000001,"15min":1.1000000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.73,"5min":1.829,"15min":1.829},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.129,"5min":1.109,"15min":1.111},"min":{"1min":0.65500000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.79,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.2549999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2350000000000001,"15min":1.218},"min":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.595},{"interface":"front","average":{"1min":1.252,"5min":1.2110000000000001,"15min":1.21},"min":{"1min":0.875,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.645,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.401}]},{"osd":2,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.137,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.754,"5min":3.597,"15min":3.597},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.284,"15min":1.3109999999999999},"min":{"1min":0.89500000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0470000000000002,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.0069999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2909999999999999,"15min":1.3109999999999999},"min":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.8049999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.26,"15min":1.3120000000000001},"min":{"1min":0.85399999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.546,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3680000000000001,"15min":1.4279999999999999},"min":{"1min":0.98999999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.742,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3600000000000001,"15min":1.389},"min":{"1min":0.91000000000000003,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7390000000000001,"5min":3.04,"15min":3.04},"last":1.516}]},{"osd":5,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":1.0660000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7969999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.526,"15min":1.593},"min":{"1min":1.246,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9019999999999999,"5min":3.774,"15min":3.774},"last":1.671}]},{"osd":7,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5189999999999999,"15min":1.5509999999999999},"min":{"1min":1.0640000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.012,"5min":3.403,"15min":3.403},"last":1.639},{"interface":"front","average":{"1min":1.492,"5min":1.482,"15min":1.506},"min":{"1min":0.96699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.9610000000000001,"5min":3.649,"15min":3.649},"last":1.5529999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.155,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7629999999999999,"5min":1.903,"15min":1.903},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.248,"15min":1.3},"min":{"1min":0.91300000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.883,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.278}]},{"osd":1,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.133,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.1799999999999999,"15min":1.125},"min":{"1min":0.74299999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8560000000000001,"5min":1.96,"15min":1.96},"last":0.92900000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2070000000000001,"15min":1.2050000000000001},"min":{"1min":0.877,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8300000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.3,"15min":1.294},"min":{"1min":0.70599999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.321}]},{"osd":3,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.302,"15min":1.3640000000000001},"min":{"1min":0.748,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.502,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.385},"min":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.954,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.415}]},{"osd":4,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.304,"15min":1.329},"min":{"1min":0.77600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.6919999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4670000000000001,"15min":1.5249999999999999},"min":{"1min":1.1439999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.77,"5min":3.085,"15min":3.085},"last":1.5109999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.482,"15min":1.4490000000000001},"min":{"1min":1.028,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.032,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.581},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.4810000000000001,"15min":1.47},"min":{"1min":1.0429999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.9039999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.806}]},{"osd":7,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.593,"15min":1.617},"min":{"1min":1.0669999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8919999999999999,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.613},{"interface":"front","average":{"1min":1.6499999999999999,"5min":1.6539999999999999,"15min":1.704},"min":{"1min":1.3520000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.089,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.758}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1020000000000001,"15min":1.0820000000000001},"min":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6479999999999999,"5min":1.982,"15min":1.982},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.079,"5min":1.0900000000000001,"15min":1.1120000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.4139999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.177,"15min":1.123},"min":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7370000000000001,"5min":1.909,"15min":1.909},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.2090000000000001,"15min":1.288},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.9339999999999999,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.2909999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.216,"15min":1.2809999999999999},"min":{"1min":0.84899999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.74,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2929999999999999,"15min":1.325},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.0109999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.25,"15min":1.3180000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":5.0300000000000002,"5min":5.0300000000000002,"15min":5.0300000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":0.93100000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.8220000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.6200000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.456,"15min":1.506},"min":{"1min":0.875,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":1.462},{"interface":"front","average":{"1min":1.5,"5min":1.452,"15min":1.538},"min":{"1min":0.79600000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":1.222}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4419999999999999,"15min":1.4710000000000001},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.54,"5min":1.5109999999999999,"15min":1.5629999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.127}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4790000000000001,"15min":1.617},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.5920000000000001,"15min":1.605},"min":{"1min":0.83199999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.032,"5min":4.032,"15min":4.032},"last":1.675}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.45600000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94599999999999995,"15min":0.95999999999999996},"min":{"1min":0.50800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":1.51,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.78400000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.96799999999999997,"15min":0.96999999999999997},"min":{"1min":0.65800000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0940000000000001,"15min":1.1140000000000001},"min":{"1min":0.54000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.857,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.3029999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.054,"15min":1.0720000000000001},"min":{"1min":0.58999999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6679999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0940000000000001,"15min":1.1080000000000001},"min":{"1min":0.60399999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.488,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":1.264}]},{"osd":4,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.327,"15min":1.383},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7190000000000001,"5min":3.081,"15min":3.081},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3440000000000001,"15min":1.3420000000000001},"min":{"1min":0.81000000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.7909999999999999,"5min":2.758,"15min":2.758},"last":1.087}]},{"osd":5,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3540000000000001,"15min":1.3959999999999999},"min":{"1min":0.88900000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6899999999999999,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.544,"15min":1.7230000000000001},"min":{"1min":0.91000000000000003,"5min":0.745,"15min":0.745},"max":{"1min":1.861,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.476}]},{"osd":6,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.80000000000000004,"5min":0.753,"15min":0.753},"max":{"1min":2.1480000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.51,"15min":1.5369999999999999},"min":{"1min":1.024,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.847,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.194}]},{"osd":7,"last update":"Fri Nov 24 06:31:03 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.552,"15min":1.601},"min":{"1min":0.76800000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9810000000000001,"5min":4.641,"15min":4.641},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.528,"15min":1.5680000000000001},"min":{"1min":0.92500000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0299999999999998,"5min":3.798,"15min":3.798},"last":1.617}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.83399999999999996,"15min":0.88},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":1.5289999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.92400000000000004,"15min":0.99199999999999999},"min":{"1min":0.46600000000000003,"5min":0.31,"15min":0.31},"max":{"1min":1.6859999999999999,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.64600000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92400000000000004,"15min":0.95699999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3700000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7310000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.44500000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.079,"15min":1.103},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7010000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.044,"15min":1.1000000000000001},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.76,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3520000000000001,"15min":1.474},"min":{"1min":0.89300000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.897,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.379,"15min":1.46},"min":{"1min":0.78400000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.802,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.83699999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4350000000000001,"15min":1.458},"min":{"1min":0.93200000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0219999999999998,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.4079999999999999,"15min":1.48},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.9550000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.1140000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3580000000000001,"15min":1.4139999999999999},"min":{"1min":0.877,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.718,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.754},{"interface":"front","average":{"1min":1.419,"5min":1.458,"15min":1.5189999999999999},"min":{"1min":0.78400000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.0299999999999998,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.371}]},{"osd":7,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.421,"15min":1.518},"min":{"1min":0.82499999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8540000000000001,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.522,"5min":1.5129999999999999,"15min":1.5329999999999999},"min":{"1min":0.80600000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.0790000000000002,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.423}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92200000000000004,"15min":1.006},"min":{"1min":0.55800000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.3700000000000001,"5min":2.593,"15min":2.593},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.997,"5min":1.0149999999999999,"15min":1.0309999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4830000000000001,"5min":1.74,"15min":1.74},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.51600000000000001},{"interface":"front","average":{"1min":1.05,"5min":1.03,"15min":1.014},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.796,"5min":1.829,"15min":1.829},"last":0.67600000000000005}]},{"osd":3,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1120000000000001,"15min":1.123},"min":{"1min":0.58599999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.026,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.53,"5min":1.798,"15min":1.798},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2450000000000001,"15min":1.2529999999999999},"min":{"1min":0.66300000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9410000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3999999999999999,"15min":1.4399999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1269999999999998,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.0700000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.345,"15min":1.319},"min":{"1min":0.98299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.3919999999999999,"15min":1.3999999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.0939999999999999,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.1200000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4470000000000001,"15min":1.411},"min":{"1min":1.0329999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.446,"15min":1.454},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.998,"5min":2.371,"15min":2.371},"last":0.93400000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.458,"15min":1.4530000000000001},"min":{"1min":0.83399999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0659999999999998,"5min":2.246,"15min":2.246},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.514,"15min":1.526},"min":{"1min":1.087,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.2410000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.0529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1720320,"data_stored":1072896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84099999999999997,"15min":0.84699999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.355,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.88800000000000001,"15min":0.94699999999999995},"min":{"1min":0.504,"5min":0.31,"15min":0.31},"max":{"1min":1.5329999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":1.0600000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.014,"15min":1.0029999999999999},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.109},{"interface":"front","average":{"1min":1.206,"5min":1.0840000000000001,"15min":1.125},"min":{"1min":0.58599999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.518}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90700000000000003,"15min":0.94099999999999995},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5860000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.056,"15min":1.0800000000000001},"min":{"1min":0.53300000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.48,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.244}]},{"osd":4,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2430000000000001,"15min":1.2310000000000001},"min":{"1min":0.84799999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.788,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.3420000000000001,"15min":1.3180000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8109999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.425}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.294,"15min":1.3240000000000001},"min":{"1min":0.88300000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7390000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.732},{"interface":"front","average":{"1min":1.298,"5min":1.2589999999999999,"15min":1.274},"min":{"1min":0.84499999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.671,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1579999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4359999999999999,"15min":1.476},"min":{"1min":1.0289999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0209999999999999,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.575},{"interface":"front","average":{"1min":1.407,"5min":1.4079999999999999,"15min":1.49},"min":{"1min":1.089,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9339999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.8560000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4299999999999999,"15min":1.45},"min":{"1min":0.86899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0750000000000002,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.962},{"interface":"front","average":{"1min":1.472,"5min":1.405,"15min":1.421},"min":{"1min":1.0269999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0150000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.3839999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:32:09.482 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T06:26:50.351558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=26, tm_sec=50, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=32, tm_sec=2, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T06:32:09.482 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T06:32:24.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:32:24.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:32:29.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:32:29.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.714+0000 7f7f6f59e640 1 Processor -- start 2023-11-24T06:32:29.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.714+0000 7f7f6f59e640 1 -- start start 2023-11-24T06:32:29.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.716+0000 7f7f6f59e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7f7006d8d0 0x7f7f7016c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:29.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.716+0000 7f7f6f59e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f700702e0 con 0x7f7f70162430 2023-11-24T06:32:29.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.716+0000 7f7f6f59e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f700705c0 con 0x7f7f70162810 2023-11-24T06:32:29.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.716+0000 7f7f6f59e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f700708a0 con 0x7f7f7006d8d0 2023-11-24T06:32:29.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.716+0000 7f7f6e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7f70162430 0x7f7f700b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57008/0 (socket says 172.21.15.16:57008) 2023-11-24T06:32:29.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.716+0000 7f7f6e59c640 1 -- 172.21.15.16:0/3580763450 learned_addr learned my addr 172.21.15.16:0/3580763450 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436102500 0 0) 0x7f7f700702e0 con 0x7f7f70162430 2023-11-24T06:32:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f58003520 con 0x7f7f70162430 2023-11-24T06:32:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6ed9d640 1 --2- 172.21.15.16:0/3580763450 >> v2:172.21.15.173:3300/0 conn(0x7f7f7006d8d0 0x7f7f7016c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1259801264 0 0) 0x7f7f58003520 con 0x7f7f70162430 2023-11-24T06:32:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 >> v2:172.21.15.173:3300/0 conn(0x7f7f7006d8d0 msgr2=0x7f7f7016c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 --2- 172.21.15.16:0/3580763450 >> v2:172.21.15.173:3300/0 conn(0x7f7f7006d8d0 0x7f7f7016c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 >> v1:172.21.15.173:6789/0 conn(0x7f7f70162810 legacy=0x7f7f70071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f70070b80 con 0x7f7f70162430 2023-11-24T06:32:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f7f60001670 con 0x7f7f70162430 2023-11-24T06:32:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 >> v1:172.21.15.16:6789/0 conn(0x7f7f70162430 legacy=0x7f7f700b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f580039c0 0x7f7f58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f580047c0 0x7f7f58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f580051d0 0x7f7f5800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700708a0 con 0x7f7f580047c0 2023-11-24T06:32:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5800c120 con 0x7f7f580039c0 2023-11-24T06:32:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.717+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5800c2e0 con 0x7f7f580051d0 2023-11-24T06:32:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.718+0000 7f7f6dd9b640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f580047c0 0x7f7f58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.718+0000 7f7f6e59c640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f580039c0 0x7f7f58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.718+0000 7f7f6dd9b640 1 -- 172.21.15.16:0/3580763450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f580051d0 msgr2=0x7f7f5800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.718+0000 7f7f6dd9b640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f580051d0 0x7f7f5800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.718+0000 7f7f6dd9b640 1 -- 172.21.15.16:0/3580763450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f580039c0 msgr2=0x7f7f58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.718+0000 7f7f6dd9b640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f580039c0 0x7f7f58003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.718+0000 7f7f6dd9b640 1 -- 172.21.15.16:0/3580763450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f58003520 con 0x7f7f580047c0 2023-11-24T06:32:29.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.718+0000 7f7f6ed9d640 1 -- 172.21.15.16:0/3580763450 reap_dead start 2023-11-24T06:32:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.719+0000 7f7f6dd9b640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f580047c0 0x7f7f58004c40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7f64001720 tx=0x7f7f580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd2711e6eb8009e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.720+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f640052d0 con 0x7f7f580047c0 2023-11-24T06:32:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.720+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6402e050 con 0x7f7f580047c0 2023-11-24T06:32:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.720+0000 7f7f6d59a640 1 -- 172.21.15.16:0/3580763450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64003830 con 0x7f7f580047c0 2023-11-24T06:32:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.720+0000 7f7f6f59e640 1 -- 172.21.15.16:0/3580763450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f580047c0 msgr2=0x7f7f58004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.720+0000 7f7f6f59e640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f580047c0 0x7f7f58004c40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7f64001720 tx=0x7f7f580057f0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.720+0000 7f7f6f59e640 1 -- 172.21.15.16:0/3580763450 shutdown_connections 2023-11-24T06:32:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.720+0000 7f7f6f59e640 1 --2- 172.21.15.16:0/3580763450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f580047c0 0x7f7f58004c40 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.720+0000 7f7f6f59e640 1 -- 172.21.15.16:0/3580763450 >> 172.21.15.16:0/3580763450 conn(0x7f7f7006a800 msgr2=0x7f7f7006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.721+0000 7f7f6f59e640 1 -- 172.21.15.16:0/3580763450 shutdown_connections 2023-11-24T06:32:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.721+0000 7f7f6f59e640 1 -- 172.21.15.16:0/3580763450 wait complete. 2023-11-24T06:32:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.722+0000 7f7f6f59e640 1 Processor -- start 2023-11-24T06:32:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.722+0000 7f7f6f59e640 1 -- start start 2023-11-24T06:32:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.723+0000 7f7f6f59e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f580047c0 0x7f7f70156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.723+0000 7f7f6f59e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f701f6fa0 0x7f7f700ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.723+0000 7f7f6f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f700ade70 0x7f7f70149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.723+0000 7f7f6f59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700ae330 con 0x7f7f700ade70 2023-11-24T06:32:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.723+0000 7f7f6f59e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700a85f0 con 0x7f7f580047c0 2023-11-24T06:32:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.723+0000 7f7f6f59e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700a8890 con 0x7f7f701f6fa0 2023-11-24T06:32:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f700ade70 0x7f7f70149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f700ade70 0x7f7f70149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36162/0 (socket says 172.21.15.16:36162) 2023-11-24T06:32:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6ed9d640 1 -- 172.21.15.16:0/196407942 learned_addr learned my addr 172.21.15.16:0/196407942 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6dd9b640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f701f6fa0 0x7f7f700ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6e59c640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f580047c0 0x7f7f70156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6ed9d640 1 -- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f701f6fa0 msgr2=0x7f7f700ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6ed9d640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f701f6fa0 0x7f7f700ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6ed9d640 1 -- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f580047c0 msgr2=0x7f7f70156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6ed9d640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f580047c0 0x7f7f70156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.724+0000 7f7f6ed9d640 1 -- 172.21.15.16:0/196407942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700a8b70 con 0x7f7f700ade70 2023-11-24T06:32:29.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.725+0000 7f7f6ed9d640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f700ade70 0x7f7f70149940 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7f700b2bf0 tx=0x7f7f6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:29.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.725+0000 7f7f5effd640 1 -- 172.21.15.16:0/196407942 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68007e40 con 0x7f7f700ade70 2023-11-24T06:32:29.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.725+0000 7f7f5effd640 1 -- 172.21.15.16:0/196407942 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f68014420 con 0x7f7f700ade70 2023-11-24T06:32:29.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.725+0000 7f7f5effd640 1 -- 172.21.15.16:0/196407942 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680147d0 con 0x7f7f700ade70 2023-11-24T06:32:29.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.725+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f700a8e50 con 0x7f7f700ade70 2023-11-24T06:32:29.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.725+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f70156860 con 0x7f7f700ade70 2023-11-24T06:32:29.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.726+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f700a9040 con 0x7f7f700ade70 2023-11-24T06:32:29.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.727+0000 7f7f5effd640 1 -- 172.21.15.16:0/196407942 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f7f68014930 con 0x7f7f700ade70 2023-11-24T06:32:29.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.728+0000 7f7f5effd640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7f5006b470 0x7f7f5006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:29.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.728+0000 7f7f5effd640 1 -- 172.21.15.16:0/196407942 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f7f6808a1d0 con 0x7f7f700ade70 2023-11-24T06:32:29.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.729+0000 7f7f6e59c640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7f5006b470 0x7f7f5006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:29.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.730+0000 7f7f6e59c640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7f5006b470 0x7f7f5006d930 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f70067b00 tx=0x7f7f60000f50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:29.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.733+0000 7f7f5effd640 1 -- 172.21.15.16:0/196407942 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f68054cb0 con 0x7f7f700ade70 2023-11-24T06:32:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.925+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 --> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f70156860 con 0x7f7f5006b470 2023-11-24T06:32:29.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.932+0000 7f7f5effd640 1 -- 172.21.15.16:0/196407942 <== mgr.4109 v2:172.21.15.173:6832/2030233038 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47954 (secure 0 0 0) 0x7f7f70156860 con 0x7f7f5006b470 2023-11-24T06:32:29.936 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:32:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-11-24T06:32:29.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7f5006b470 msgr2=0x7f7f5006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7f5006b470 0x7f7f5006d930 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f70067b00 tx=0x7f7f60000f50 comp rx=0 tx=0).stop 2023-11-24T06:32:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f700ade70 msgr2=0x7f7f70149940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f700ade70 0x7f7f70149940 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f7f700b2bf0 tx=0x7f7f6800e4b0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 shutdown_connections 2023-11-24T06:32:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f7f5006b470 0x7f7f5006d930 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f700ade70 0x7f7f70149940 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f701f6fa0 0x7f7f700ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 --2- 172.21.15.16:0/196407942 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f580047c0 0x7f7f70156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 >> 172.21.15.16:0/196407942 conn(0x7f7f7006a800 msgr2=0x7f7f700afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 shutdown_connections 2023-11-24T06:32:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:29.935+0000 7f7f6f59e640 1 -- 172.21.15.16:0/196407942 wait complete. 2023-11-24T06:32:29.959 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-11-24T06:32:28.216836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259408,"kb_used_data":9960,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715312,"statfs":{"total":773094113280,"available":772828479488,"internally_reserved":0,"allocated":10199040,"data_stored":5040960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"1.0","version":"91'119","reported_seq":249,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-24T06:32:09.436778+0000","last_change":"2023-11-24T06:32:09.436841+0000","last_active":"2023-11-24T06:32:09.436778+0000","last_peered":"2023-11-24T06:32:09.436778+0000","last_clean":"2023-11-24T06:32:09.436778+0000","last_became_active":"2023-11-24T06:26:51.782708+0000","last_became_peered":"2023-11-24T06:26:51.782708+0000","last_unstale":"2023-11-24T06:32:09.436778+0000","last_undegraded":"2023-11-24T06:32:09.436778+0000","last_fullsized":"2023-11-24T06:32:09.436778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'119","last_scrub_stamp":"2023-11-24T06:32:09.436588+0000","last_deep_scrub":"91'119","last_deep_scrub_stamp":"2023-11-24T06:32:09.436588+0000","last_clean_scrub_stamp":"2023-11-24T06:32:09.436588+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:04:56.009339+0000","scrub_duration":0.012490296999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1720320,"data_stored":1072896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.149,"15min":1.2809999999999999},"min":{"1min":0.82099999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.5209999999999999,"5min":4.7359999999999998,"15min":4.7359999999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.2589999999999999,"15min":1.2889999999999999},"min":{"1min":0.72099999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.704,"5min":4.8179999999999996,"15min":4.8179999999999996},"last":1.6870000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0700000000000001,"15min":1.0269999999999999},"min":{"1min":0.64900000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.827,"5min":2.456,"15min":2.456},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2969999999999999,"15min":1.3620000000000001},"min":{"1min":0.83399999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.6220000000000001,"5min":6.3319999999999999,"15min":6.3319999999999999},"last":1.603}]},{"osd":2,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.204,"15min":1.2649999999999999},"min":{"1min":0.73099999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5900000000000001,"5min":5.4589999999999996,"15min":5.4589999999999996},"last":1.123},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.179,"15min":1.1950000000000001},"min":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.4119999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":1.319}]},{"osd":3,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3,"15min":1.3819999999999999},"min":{"1min":0.78600000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6100000000000001,"5min":8.3539999999999992,"15min":8.3539999999999992},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.2330000000000001,"15min":1.2130000000000001},"min":{"1min":0.89500000000000002,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.653,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.4339999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3560000000000001,"15min":1.5229999999999999},"min":{"1min":0.78800000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7190000000000001,"5min":6.2649999999999997,"15min":6.2649999999999997},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.47,"15min":1.556},"min":{"1min":1.0449999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7509999999999999,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":1.47}]},{"osd":5,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3080000000000001,"15min":1.3819999999999999},"min":{"1min":0.82999999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.919,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.7,"15min":1.752},"min":{"1min":0.88500000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.871,"5min":17.582999999999998,"15min":17.582999999999998},"last":1.052}]},{"osd":6,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.6040000000000001,"15min":1.804},"min":{"1min":1.085,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.988,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":1.375},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5049999999999999,"15min":1.655},"min":{"1min":1.101,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.78,"5min":6.2210000000000001,"15min":6.2210000000000001},"last":1.3460000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.0920000000000001,"15min":1.1000000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.73,"5min":1.829,"15min":1.829},"last":1.319},{"interface":"front","average":{"1min":1.129,"5min":1.109,"15min":1.111},"min":{"1min":0.65500000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.79,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.89500000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.2350000000000001,"15min":1.218},"min":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.2110000000000001,"15min":1.21},"min":{"1min":0.875,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.645,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.056}]},{"osd":2,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.137,"15min":1.1559999999999999},"min":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.754,"5min":3.597,"15min":3.597},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.284,"15min":1.3109999999999999},"min":{"1min":0.89500000000000002,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0470000000000002,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.5449999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2909999999999999,"15min":1.3109999999999999},"min":{"1min":0.75900000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.8049999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.26,"15min":1.3120000000000001},"min":{"1min":0.85399999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.546,"5min":3.4670000000000001,"15min":3.4670000000000001},"last":1.4099999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3680000000000001,"15min":1.4279999999999999},"min":{"1min":0.98999999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.742,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3600000000000001,"15min":1.389},"min":{"1min":0.91000000000000003,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7390000000000001,"5min":3.04,"15min":3.04},"last":1.248}]},{"osd":5,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":1.0660000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7969999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.526,"15min":1.593},"min":{"1min":1.246,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9019999999999999,"5min":3.774,"15min":3.774},"last":1.4370000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5189999999999999,"15min":1.5509999999999999},"min":{"1min":1.0640000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.012,"5min":3.403,"15min":3.403},"last":1.508},{"interface":"front","average":{"1min":1.492,"5min":1.482,"15min":1.506},"min":{"1min":0.96699999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.9610000000000001,"5min":3.649,"15min":3.649},"last":1.4810000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.155,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7629999999999999,"5min":1.903,"15min":1.903},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.248,"15min":1.3},"min":{"1min":0.91300000000000003,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.883,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.369}]},{"osd":1,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.133,"15min":1.1359999999999999},"min":{"1min":0.754,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.1799999999999999,"15min":1.125},"min":{"1min":0.74299999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8560000000000001,"5min":1.96,"15min":1.96},"last":1.252}]},{"osd":2,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2070000000000001,"15min":1.2050000000000001},"min":{"1min":0.877,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.8300000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.169},{"interface":"front","average":{"1min":1.258,"5min":1.3,"15min":1.294},"min":{"1min":0.70599999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.3360000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.302,"15min":1.3640000000000001},"min":{"1min":0.748,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.502,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.385},"min":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.954,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.4399999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.304,"15min":1.329},"min":{"1min":0.77600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.6919999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.49},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4670000000000001,"15min":1.5249999999999999},"min":{"1min":1.1439999999999999,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.77,"5min":3.085,"15min":3.085},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.482,"15min":1.4490000000000001},"min":{"1min":1.028,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.032,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.536},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.4810000000000001,"15min":1.47},"min":{"1min":1.0429999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.9039999999999999,"5min":2.4790000000000001,"15min":2.4790000000000001},"last":1.5980000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:31:55 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.593,"15min":1.617},"min":{"1min":1.0669999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8919999999999999,"5min":2.6720000000000002,"15min":2.6720000000000002},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.6499999999999999,"5min":1.6539999999999999,"15min":1.704},"min":{"1min":1.3520000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.089,"5min":4.6929999999999996,"15min":4.6929999999999996},"last":1.645}]}]},{"osd":4,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1020000000000001,"15min":1.0820000000000001},"min":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6479999999999999,"5min":1.982,"15min":1.982},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.0900000000000001,"15min":1.1120000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.169}]},{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.177,"15min":1.123},"min":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7370000000000001,"5min":1.909,"15min":1.909},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.165,"5min":1.2090000000000001,"15min":1.288},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.9339999999999999,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.3999999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.216,"15min":1.2809999999999999},"min":{"1min":0.84899999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.74,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2929999999999999,"15min":1.325},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.4550000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.25,"15min":1.3180000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":5.0300000000000002,"5min":5.0300000000000002,"15min":5.0300000000000002},"last":1.359},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3480000000000001,"15min":1.3620000000000001},"min":{"1min":0.93100000000000005,"5min":0.746,"15min":0.746},"max":{"1min":1.8220000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4830000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.456,"15min":1.506},"min":{"1min":0.875,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.7040000000000002,"5min":2.7040000000000002,"15min":2.7040000000000002},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.5,"5min":1.452,"15min":1.538},"min":{"1min":0.79600000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":1.421}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4419999999999999,"15min":1.4710000000000001},"min":{"1min":0.75900000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.222,"5min":3.222,"15min":3.222},"last":1.022},{"interface":"front","average":{"1min":1.54,"5min":1.5109999999999999,"15min":1.5629999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.0589999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4790000000000001,"15min":1.617},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":5.1909999999999998,"5min":5.1909999999999998,"15min":5.1909999999999998},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.5920000000000001,"15min":1.605},"min":{"1min":0.83199999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.032,"5min":4.032,"15min":4.032},"last":1.2210000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77900000000000003,"15min":0.80900000000000005},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.304,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.92600000000000005,"15min":0.95399999999999996},"min":{"1min":0.38,"5min":0.372,"15min":0.372},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.024}]},{"osd":1,"last update":"Fri Nov 24 06:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.96399999999999997,"15min":0.96899999999999997},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.633,"5min":1.637,"15min":1.637},"last":0.755},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0740000000000001,"15min":1.1080000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.7450000000000001,"5min":1.9510000000000001,"15min":1.9510000000000001},"last":1.575}]},{"osd":2,"last update":"Fri Nov 24 06:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.0289999999999999,"15min":1.0640000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.663,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.216},{"interface":"front","average":{"1min":0.997,"5min":1.0700000000000001,"15min":1.1000000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.792,"5min":3.5910000000000002,"15min":3.5910000000000002},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2849999999999999,"15min":1.369},"min":{"1min":0.82199999999999995,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.982,"5min":3.081,"15min":3.081},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.337,"15min":1.3400000000000001},"min":{"1min":0.72399999999999998,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.1120000000000001,"5min":2.758,"15min":2.758},"last":1.4550000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3460000000000001,"15min":1.3939999999999999},"min":{"1min":0.82999999999999996,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.7000000000000002,"5min":3.6960000000000002,"15min":3.6960000000000002},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.4710000000000001,"15min":1.6990000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.6280000000000001,"5min":8.8879999999999999,"15min":8.8879999999999999},"last":1.6579999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0859999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.351},{"interface":"front","average":{"1min":1.329,"5min":1.466,"15min":1.522},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0049999999999999,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":1.615}]},{"osd":7,"last update":"Fri Nov 24 06:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.504,"15min":1.585},"min":{"1min":0.79800000000000004,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.073,"5min":4.641,"15min":4.641},"last":1.399},{"interface":"front","average":{"1min":1.47,"5min":1.504,"15min":1.5600000000000001},"min":{"1min":0.91700000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1640000000000001,"5min":3.798,"15min":3.798},"last":1.512}]}]},{"osd":2,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.83399999999999996,"15min":0.88},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":1.5289999999999999,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.999,"5min":0.92400000000000004,"15min":0.99199999999999999},"min":{"1min":0.46600000000000003,"5min":0.31,"15min":0.31},"max":{"1min":1.6859999999999999,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.5509999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.92400000000000004,"15min":0.95699999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3700000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0329999999999999,"15min":1.0469999999999999},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7310000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.012}]},{"osd":3,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.079,"15min":1.103},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.7010000000000001,"5min":2.5049999999999999,"15min":2.5049999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.044,"15min":1.1000000000000001},"min":{"1min":0.47999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.76,"5min":3.2170000000000001,"15min":3.2170000000000001},"last":1.3109999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3520000000000001,"15min":1.474},"min":{"1min":0.89300000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.897,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.379,"15min":1.46},"min":{"1min":0.78400000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.802,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.621}]},{"osd":5,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4350000000000001,"15min":1.458},"min":{"1min":0.93200000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.0219999999999998,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.4079999999999999,"15min":1.48},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.9550000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":1.5880000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3580000000000001,"15min":1.4139999999999999},"min":{"1min":0.877,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.718,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":1.206},{"interface":"front","average":{"1min":1.419,"5min":1.458,"15min":1.5189999999999999},"min":{"1min":0.78400000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.0299999999999998,"5min":4.1210000000000004,"15min":4.1210000000000004},"last":1.4570000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.421,"15min":1.518},"min":{"1min":0.82499999999999996,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8540000000000001,"5min":3.8119999999999998,"15min":3.8119999999999998},"last":1.419},{"interface":"front","average":{"1min":1.522,"5min":1.5129999999999999,"15min":1.5329999999999999},"min":{"1min":0.80600000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.0790000000000002,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.5209999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":482528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.92200000000000004,"15min":1.006},"min":{"1min":0.55800000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.3700000000000001,"5min":2.593,"15min":2.593},"last":1.03},{"interface":"front","average":{"1min":0.997,"5min":1.0149999999999999,"15min":1.0309999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4830000000000001,"5min":1.74,"15min":1.74},"last":1.347}]},{"osd":2,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.89200000000000002,"15min":0.88200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.05,"5min":1.03,"15min":1.014},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.796,"5min":1.829,"15min":1.829},"last":1.4119999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1120000000000001,"15min":1.123},"min":{"1min":0.58599999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.133},{"interface":"front","average":{"1min":1.026,"5min":1.044,"15min":1.0740000000000001},"min":{"1min":0.48199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.53,"5min":1.798,"15min":1.798},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.2450000000000001,"15min":1.2529999999999999},"min":{"1min":0.66300000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9410000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.466},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.3999999999999999,"15min":1.4399999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.1269999999999998,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.54}]},{"osd":5,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.345,"15min":1.319},"min":{"1min":0.98299999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.22},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.3919999999999999,"15min":1.3999999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.0939999999999999,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3009999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4470000000000001,"15min":1.411},"min":{"1min":1.0329999999999999,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.446,"15min":1.454},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.998,"5min":2.371,"15min":2.371},"last":1.1819999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.458,"15min":1.4530000000000001},"min":{"1min":0.83399999999999996,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.0659999999999998,"5min":2.246,"15min":2.246},"last":1.262},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.514,"15min":1.526},"min":{"1min":1.087,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":2.2410000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.506}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1720320,"data_stored":1072896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.84099999999999997,"15min":0.84699999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.355,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.88800000000000001,"15min":0.94699999999999995},"min":{"1min":0.504,"5min":0.31,"15min":0.31},"max":{"1min":1.5329999999999999,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.81599999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.014,"15min":1.0029999999999999},"min":{"1min":0.56799999999999995,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":1.143},{"interface":"front","average":{"1min":1.206,"5min":1.0840000000000001,"15min":1.125},"min":{"1min":0.58599999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.5680000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90700000000000003,"15min":0.94099999999999995},"min":{"1min":0.47699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.5860000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.056,"15min":1.0800000000000001},"min":{"1min":0.53300000000000003,"5min":0.499,"15min":0.499},"max":{"1min":1.48,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":1.3169999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2430000000000001,"15min":1.2310000000000001},"min":{"1min":0.84799999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.788,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.097},{"interface":"front","average":{"1min":1.399,"5min":1.3420000000000001,"15min":1.3180000000000001},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8109999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.5209999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.294,"15min":1.3240000000000001},"min":{"1min":0.88300000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7390000000000001,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":1.377},{"interface":"front","average":{"1min":1.298,"5min":1.2589999999999999,"15min":1.274},"min":{"1min":0.84499999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.671,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.4550000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4359999999999999,"15min":1.476},"min":{"1min":1.0289999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.0209999999999999,"5min":2.2440000000000002,"15min":2.2440000000000002},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.4079999999999999,"15min":1.49},"min":{"1min":1.089,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9339999999999999,"5min":2.4950000000000001,"15min":2.4950000000000001},"last":1.728}]},{"osd":7,"last update":"Fri Nov 24 06:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4299999999999999,"15min":1.45},"min":{"1min":0.86899999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.0750000000000002,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.659},{"interface":"front","average":{"1min":1.472,"5min":1.405,"15min":1.421},"min":{"1min":1.0269999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.0150000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.23}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:32:29.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T06:32:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.179+0000 7f20b359e640 1 Processor -- start 2023-11-24T06:32:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.180+0000 7f20b359e640 1 -- start start 2023-11-24T06:32:30.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20b359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20b406c730 0x7f20b40b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20b359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b4071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20b359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b40708d0 con 0x7f20b406cb10 2023-11-24T06:32:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20b359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b4070bb0 con 0x7f20b406c730 2023-11-24T06:32:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20b359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4070e90 con 0x7f20b4070160 2023-11-24T06:32:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20b259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20b406c730 0x7f20b40b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20b2d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20b406cb10 0x7f20b406cfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57024/0 (socket says 172.21.15.16:57024) 2023-11-24T06:32:30.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20b2d9d640 1 -- 172.21.15.16:0/1797920829 learned_addr learned my addr 172.21.15.16:0/1797920829 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20abfff640 1 --2- 172.21.15.16:0/1797920829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b4071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.182+0000 7f20b259c640 1 -- 172.21.15.16:0/1797920829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 msgr2=0x7f20b4071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.183+0000 7f20b259c640 1 --2- 172.21.15.16:0/1797920829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b4071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.183+0000 7f20b259c640 1 -- 172.21.15.16:0/1797920829 >> v1:172.21.15.16:6789/0 conn(0x7f20b406cb10 legacy=0x7f20b406cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.183+0000 7f20b259c640 1 -- 172.21.15.16:0/1797920829 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b4071170 con 0x7f20b406c730 2023-11-24T06:32:30.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.183+0000 7f20abfff640 1 --2- 172.21.15.16:0/1797920829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b4071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:30.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.184+0000 7f20b259c640 1 --2- 172.21.15.16:0/1797920829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406c730 0x7f20b40b2e00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f20a40097a0 tx=0x7f20a402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47483a19109ce5f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:30.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.184+0000 7f20b1d9b640 1 -- 172.21.15.16:0/1797920829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4031780 con 0x7f20b406c730 2023-11-24T06:32:30.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.184+0000 7f20b1d9b640 1 -- 172.21.15.16:0/1797920829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a40318e0 con 0x7f20b406c730 2023-11-24T06:32:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.184+0000 7f20b1d9b640 1 -- 172.21.15.16:0/1797920829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4031c70 con 0x7f20b406c730 2023-11-24T06:32:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.185+0000 7f20b359e640 1 -- 172.21.15.16:0/1797920829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406c730 msgr2=0x7f20b40b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.185+0000 7f20b359e640 1 --2- 172.21.15.16:0/1797920829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406c730 0x7f20b40b2e00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f20a40097a0 tx=0x7f20a402b820 comp rx=0 tx=0).stop 2023-11-24T06:32:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.185+0000 7f20b359e640 1 -- 172.21.15.16:0/1797920829 shutdown_connections 2023-11-24T06:32:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.185+0000 7f20b359e640 1 --2- 172.21.15.16:0/1797920829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b4071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.185+0000 7f20b359e640 1 --2- 172.21.15.16:0/1797920829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406c730 0x7f20b40b2e00 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.185+0000 7f20b359e640 1 -- 172.21.15.16:0/1797920829 >> 172.21.15.16:0/1797920829 conn(0x7f20b406a7d0 msgr2=0x7f20b406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.185+0000 7f20b359e640 1 -- 172.21.15.16:0/1797920829 shutdown_connections 2023-11-24T06:32:30.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.185+0000 7f20b359e640 1 -- 172.21.15.16:0/1797920829 wait complete. 2023-11-24T06:32:30.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.187+0000 7f20b359e640 1 Processor -- start 2023-11-24T06:32:30.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.187+0000 7f20b359e640 1 -- start start 2023-11-24T06:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.188+0000 7f20b359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b406c730 0x7f20b40ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.188+0000 7f20b359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406cb10 0x7f20b4160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.188+0000 7f20b359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b41fa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.188+0000 7f20b359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40ad850 con 0x7f20b406cb10 2023-11-24T06:32:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.188+0000 7f20b359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40a9510 con 0x7f20b406c730 2023-11-24T06:32:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.188+0000 7f20b359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40c3ea0 con 0x7f20b4070160 2023-11-24T06:32:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.188+0000 7f20abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406cb10 0x7f20b4160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.188+0000 7f20abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406cb10 0x7f20b4160ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36178/0 (socket says 172.21.15.16:36178) 2023-11-24T06:32:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.188+0000 7f20abfff640 1 -- 172.21.15.16:0/1597887415 learned_addr learned my addr 172.21.15.16:0/1597887415 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.189+0000 7f20b259c640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b406c730 0x7f20b40ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.189+0000 7f20b2d9d640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b41fa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.189+0000 7f20abfff640 1 -- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 msgr2=0x7f20b41fa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.189+0000 7f20abfff640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b41fa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.189+0000 7f20abfff640 1 -- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b406c730 msgr2=0x7f20b40ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.189+0000 7f20abfff640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b406c730 0x7f20b40ad310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.189+0000 7f20abfff640 1 -- 172.21.15.16:0/1597887415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a4009400 con 0x7f20b406cb10 2023-11-24T06:32:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.189+0000 7f20b2d9d640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b41fa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.189+0000 7f20abfff640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406cb10 0x7f20b4160ea0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f209c00c7f0 tx=0x7f209c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.190+0000 7f20aaffd640 1 -- 172.21.15.16:0/1597887415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c007820 con 0x7f20b406cb10 2023-11-24T06:32:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.190+0000 7f20aaffd640 1 -- 172.21.15.16:0/1597887415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c007980 con 0x7f20b406cb10 2023-11-24T06:32:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.190+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b40c4180 con 0x7f20b406cb10 2023-11-24T06:32:30.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.190+0000 7f20aaffd640 1 -- 172.21.15.16:0/1597887415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c007cf0 con 0x7f20b406cb10 2023-11-24T06:32:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.190+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b40a9a40 con 0x7f20b406cb10 2023-11-24T06:32:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.191+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b40c43c0 con 0x7f20b406cb10 2023-11-24T06:32:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.192+0000 7f20aaffd640 1 -- 172.21.15.16:0/1597887415 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f209c0113f0 con 0x7f20b406cb10 2023-11-24T06:32:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.193+0000 7f20aaffd640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209406b470 0x7f209406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.193+0000 7f20aaffd640 1 -- 172.21.15.16:0/1597887415 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f209c08a1f0 con 0x7f20b406cb10 2023-11-24T06:32:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.193+0000 7f20b259c640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209406b470 0x7f209406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.195+0000 7f20b259c640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209406b470 0x7f209406d930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f20a4036000 tx=0x7f20a403d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.197+0000 7f20aaffd640 1 -- 172.21.15.16:0/1597887415 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f209c054b80 con 0x7f20b406cb10 2023-11-24T06:32:30.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.393+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f20b4067450 con 0x7f20b406cb10 2023-11-24T06:32:30.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.398+0000 7f20aaffd640 1 -- 172.21.15.16:0/1597887415 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v20)=0 v20) v1 ==== 123+0+0 (secure 0 0 0) 0x7f209c0581a0 con 0x7f20b406cb10 2023-11-24T06:32:30.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.399+0000 7f20aaffd640 1 -- 172.21.15.16:0/1597887415 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f209c087ed0 con 0x7f20b406cb10 2023-11-24T06:32:30.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.401+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209406b470 msgr2=0x7f209406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.401+0000 7f20b359e640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209406b470 0x7f209406d930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f20a4036000 tx=0x7f20a403d040 comp rx=0 tx=0).stop 2023-11-24T06:32:30.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.401+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406cb10 msgr2=0x7f20b4160ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.401+0000 7f20b359e640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406cb10 0x7f20b4160ea0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f209c00c7f0 tx=0x7f209c00ccb0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.402+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 shutdown_connections 2023-11-24T06:32:30.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.402+0000 7f20b359e640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f209406b470 0x7f209406d930 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.402+0000 7f20b359e640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20b4070160 0x7f20b41fa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.402+0000 7f20b359e640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20b406cb10 0x7f20b4160ea0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.402+0000 7f20b359e640 1 --2- 172.21.15.16:0/1597887415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20b406c730 0x7f20b40ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.402+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 >> 172.21.15.16:0/1597887415 conn(0x7f20b406a7d0 msgr2=0x7f20b40afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:30.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.402+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 shutdown_connections 2023-11-24T06:32:30.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.402+0000 7f20b359e640 1 -- 172.21.15.16:0/1597887415 wait complete. 2023-11-24T06:32:30.422 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:32:30.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:32:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.636+0000 7f651359e640 1 Processor -- start 2023-11-24T06:32:30.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.636+0000 7f651359e640 1 -- start start 2023-11-24T06:32:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.637+0000 7f651359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65141616d0 0x7f65140aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.637+0000 7f651359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65140af5a0 con 0x7f651415d2c0 2023-11-24T06:32:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.637+0000 7f651359e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65140b2cc0 con 0x7f65140af140 2023-11-24T06:32:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.637+0000 7f651359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6514160c80 con 0x7f65141616d0 2023-11-24T06:32:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.638+0000 7f651259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f651415d2c0 0x7f65140b4890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57030/0 (socket says 172.21.15.16:57030) 2023-11-24T06:32:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.638+0000 7f651259c640 1 -- 172.21.15.16:0/3062618313 learned_addr learned my addr 172.21.15.16:0/3062618313 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-11-24T06:32:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.638+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169048573 0 0) 0x7f65140af5a0 con 0x7f651415d2c0 2023-11-24T06:32:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.638+0000 7f6511d9b640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65141616d0 0x7f65140aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.638+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6500003520 con 0x7f651415d2c0 2023-11-24T06:32:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.638+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2668759101 0 0) 0x7f6500003520 con 0x7f651415d2c0 2023-11-24T06:32:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65141616d0 msgr2=0x7f65140aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65141616d0 0x7f65140aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 >> v1:172.21.15.173:6789/0 conn(0x7f65140af140 legacy=0x7f651406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6514161240 con 0x7f651415d2c0 2023-11-24T06:32:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1507029606 0 0) 0x7f65040019f0 con 0x7f651415d2c0 2023-11-24T06:32:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 >> v1:172.21.15.16:6789/0 conn(0x7f651415d2c0 legacy=0x7f65140b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65000039a0 0x7f6500003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f6500004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65141616d0 0x7f6500005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6514160c80 con 0x7f65000047c0 2023-11-24T06:32:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650000ba30 con 0x7f65000039a0 2023-11-24T06:32:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650000bbf0 con 0x7f65141616d0 2023-11-24T06:32:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f6511d9b640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f6500004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f651259c640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65000039a0 0x7f6500003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.639+0000 7f6512d9d640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65141616d0 0x7f6500005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.640+0000 7f6511d9b640 1 -- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65141616d0 msgr2=0x7f6500005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.640+0000 7f6511d9b640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65141616d0 0x7f6500005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.640+0000 7f6511d9b640 1 -- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65000039a0 msgr2=0x7f6500003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:32:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.640+0000 7f6511d9b640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65000039a0 0x7f6500003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.640+0000 7f6511d9b640 1 -- 172.21.15.16:0/3062618313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500003520 con 0x7f65000047c0 2023-11-24T06:32:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.640+0000 7f651259c640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65000039a0 0x7f6500003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.640+0000 7f6512d9d640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f65141616d0 0x7f6500005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:32:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.641+0000 7f6511d9b640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f6500004c40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009a90 tx=0x7f65000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae69a1b2784a9a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.641+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc02c660 con 0x7f65000047c0 2023-11-24T06:32:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.641+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f64fc02c7c0 con 0x7f65000047c0 2023-11-24T06:32:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.641+0000 7f651159a640 1 -- 172.21.15.16:0/3062618313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc02cb70 con 0x7f65000047c0 2023-11-24T06:32:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.642+0000 7f651359e640 1 -- 172.21.15.16:0/3062618313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 msgr2=0x7f6500004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.642+0000 7f651359e640 1 --2- 172.21.15.16:0/3062618313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f6500004c40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009a90 tx=0x7f65000057f0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.642+0000 7f6512d9d640 1 -- 172.21.15.16:0/3062618313 reap_dead start 2023-11-24T06:32:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.642+0000 7f651359e640 1 -- 172.21.15.16:0/3062618313 shutdown_connections 2023-11-24T06:32:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.642+0000 7f651359e640 1 -- 172.21.15.16:0/3062618313 >> 172.21.15.16:0/3062618313 conn(0x7f65140b19c0 msgr2=0x7f65140b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.642+0000 7f651359e640 1 -- 172.21.15.16:0/3062618313 shutdown_connections 2023-11-24T06:32:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.642+0000 7f651359e640 1 -- 172.21.15.16:0/3062618313 wait complete. 2023-11-24T06:32:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.644+0000 7f651359e640 1 Processor -- start 2023-11-24T06:32:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.644+0000 7f651359e640 1 -- start start 2023-11-24T06:32:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f65141e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f651414f6e0 0x7f65141f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65141f2e50 0x7f651415f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651414f400 con 0x7f65000047c0 2023-11-24T06:32:30.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65141e7a40 con 0x7f65141f2e50 2023-11-24T06:32:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65141e7760 con 0x7f651414f6e0 2023-11-24T06:32:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f65141e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f65141e6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36184/0 (socket says 172.21.15.16:36184) 2023-11-24T06:32:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651259c640 1 -- 172.21.15.16:0/4006209910 learned_addr learned my addr 172.21.15.16:0/4006209910 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-11-24T06:32:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651259c640 1 -- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f651414f6e0 msgr2=0x7f65141f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651259c640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f651414f6e0 0x7f65141f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651259c640 1 -- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65141f2e50 msgr2=0x7f651415f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651259c640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65141f2e50 0x7f651415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.645+0000 7f651259c640 1 -- 172.21.15.16:0/4006209910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651415ed50 con 0x7f65000047c0 2023-11-24T06:32:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.646+0000 7f651259c640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f65141e6a30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f650400dfc0 tx=0x7f650400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.646+0000 7f650affd640 1 -- 172.21.15.16:0/4006209910 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504004d60 con 0x7f65000047c0 2023-11-24T06:32:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.646+0000 7f650affd640 1 -- 172.21.15.16:0/4006209910 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f65040070d0 con 0x7f65000047c0 2023-11-24T06:32:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.646+0000 7f650affd640 1 -- 172.21.15.16:0/4006209910 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504007480 con 0x7f65000047c0 2023-11-24T06:32:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.646+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651415f030 con 0x7f65000047c0 2023-11-24T06:32:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.646+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65141e7380 con 0x7f65000047c0 2023-11-24T06:32:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.648+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6514072440 con 0x7f65000047c0 2023-11-24T06:32:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.648+0000 7f650affd640 1 -- 172.21.15.16:0/4006209910 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f65040075e0 con 0x7f65000047c0 2023-11-24T06:32:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.649+0000 7f650affd640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64f406b470 0x7f64f406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:32:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.649+0000 7f650affd640 1 -- 172.21.15.16:0/4006209910 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f650408a000 con 0x7f65000047c0 2023-11-24T06:32:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.650+0000 7f6511d9b640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64f406b470 0x7f64f406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:32:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.651+0000 7f6511d9b640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64f406b470 0x7f64f406d930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f64fc036000 tx=0x7f64fc02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:32:30.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.653+0000 7f650affd640 1 -- 172.21.15.16:0/4006209910 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65040549f0 con 0x7f65000047c0 2023-11-24T06:32:30.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.884+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f651415ebb0 con 0x7f65000047c0 2023-11-24T06:32:30.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.885+0000 7f650affd640 1 -- 172.21.15.16:0/4006209910 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f651415ebb0 con 0x7f65000047c0 2023-11-24T06:32:30.890 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-11-24T06:32:30.890 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3235524397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":3235524397},{"type":"v1","addr":"172.21.15.173:6837","nonce":3235524397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6836/593524744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":593524744},{"type":"v1","addr":"172.21.15.173:6836","nonce":593524744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7173,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/825050165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":825050165},{"type":"v1","addr":"172.21.15.16:6834","nonce":825050165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7521,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3582059562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":3582059562},{"type":"v1","addr":"172.21.15.16:6835","nonce":3582059562}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2023-11-24T06:32:30.891 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 56 2023-11-24T06:32:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64f406b470 msgr2=0x7f64f406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64f406b470 0x7f64f406d930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f64fc036000 tx=0x7f64fc02e000 comp rx=0 tx=0).stop 2023-11-24T06:32:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 msgr2=0x7f65141e6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:32:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f65141e6a30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f650400dfc0 tx=0x7f650400b560 comp rx=0 tx=0).stop 2023-11-24T06:32:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 shutdown_connections 2023-11-24T06:32:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:6832/2030233038,v1:172.21.15.173:6833/2030233038] conn(0x7f64f406b470 0x7f64f406d930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f65141f2e50 0x7f651415f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f651414f6e0 0x7f65141f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 --2- 172.21.15.16:0/4006209910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65000047c0 0x7f65141e6a30 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:32:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.888+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 >> 172.21.15.16:0/4006209910 conn(0x7f65140b19c0 msgr2=0x7f65140b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:32:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.889+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 shutdown_connections 2023-11-24T06:32:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-11-24T06:32:30.889+0000 7f651359e640 1 -- 172.21.15.16:0/4006209910 wait complete. 2023-11-24T06:32:30.912 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-24T06:32:30.913 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T06:32:30.913 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:32:30.913 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:30.928 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:32:30.928 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:32:30.928 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:30.991 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:32:30.991 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:32:30.991 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.013 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:32:31.013 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:32:31.013 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.072 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:32:31.072 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T06:32:31.072 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T06:32:31.073 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.128 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T06:32:31.129 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T06:32:31.129 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.213 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T06:32:31.213 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T06:32:31.213 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.279 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T06:32:31.280 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T06:32:31.280 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.369 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T06:32:31.370 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T06:32:31.370 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.447 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T06:32:31.448 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T06:32:31.448 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.524 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T06:32:31.524 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T06:32:31.524 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.574 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T06:32:31.574 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T06:32:31.574 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.647 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T06:32:31.647 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T06:32:31.648 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T06:32:31.648 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.741 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T06:32:31.742 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T06:32:31.742 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.822 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T06:32:31.823 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T06:32:31.823 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T06:32:31.823 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.876 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T06:32:31.876 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T06:32:31.877 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.910 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T06:32:31.910 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T06:32:31.910 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:32:31.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-11-24T06:32:31.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-11-24T06:32:31.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-11-24T06:32:31.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-11-24T06:32:31.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-11-24T06:32:31.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-11-24T06:32:31.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-11-24T06:32:31.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-24T06:32:31.999 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T06:32:31.999 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T06:32:31.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T06:32:32.118 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:32:32.118 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T06:32:32.424 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:32:32.424 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T06:32:32.598 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:32:32.598 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T06:32:32.765 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:32:32.765 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T06:32:32.949 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:32:32.950 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T06:32:33.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:32:33.235 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T06:32:33.403 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:32:33.404 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T06:32:33.559 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:32:33.560 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T06:32:33.742 INFO:tasks.ceph:Archiving mon data... 2023-11-24T06:32:33.744 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/data/mon.a.tgz 2023-11-24T06:32:33.744 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-11-24T06:32:33.808 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.cK99stTdbd 2023-11-24T06:32:33.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cK99stTdbd 2023-11-24T06:32:34.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.cK99stTdbd 2023-11-24T06:32:34.265 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.cK99stTdbd is 745KB 2023-11-24T06:32:34.385 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.cK99stTdbd 2023-11-24T06:32:34.447 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/data/mon.b.tgz 2023-11-24T06:32:34.447 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2023-11-24T06:32:34.506 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.itU6W8mWUU 2023-11-24T06:32:34.506 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.itU6W8mWUU 2023-11-24T06:32:34.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.itU6W8mWUU 2023-11-24T06:32:34.982 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.itU6W8mWUU is 1011KB 2023-11-24T06:32:35.198 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.itU6W8mWUU 2023-11-24T06:32:35.257 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/data/mon.c.tgz 2023-11-24T06:32:35.258 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2023-11-24T06:32:35.313 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.EpfyduqgeG 2023-11-24T06:32:35.314 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.EpfyduqgeG 2023-11-24T06:32:35.610 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.EpfyduqgeG 2023-11-24T06:32:35.757 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.EpfyduqgeG is 1011KB 2023-11-24T06:32:35.874 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.EpfyduqgeG 2023-11-24T06:32:35.934 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T06:32:35.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:32:35.944 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:32:36.045 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T06:32:36.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:32:36.051 DEBUG:teuthology.orchestra.run.smithi173:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:32:36.154 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:32:36.157 INFO:teuthology.orchestra.run.smithi173.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:32:36.170 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T06:32:36.172 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/remote/smithi016/crash 2023-11-24T06:32:36.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:32:36.254 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/remote/smithi173/crash 2023-11-24T06:32:36.255 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:32:36.357 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T06:32:36.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:32:36.366 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:32:36.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:32:36.640 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:32:36.744 INFO:tasks.ceph:Compressing logs... 2023-11-24T06:32:36.745 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:32:36.754 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:32:36.844 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-24T06:32:36.844 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88782.log 2023-11-24T06:32:36.846 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.95554.log 2023-11-24T06:32:36.847 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.88782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.96883.log 2023-11-24T06:32:36.847 INFO:teuthology.orchestra.run.smithi173.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88782.log.gz 2023-11-24T06:32:36.848 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.1.95554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88942.log 2023-11-24T06:32:36.849 INFO:teuthology.orchestra.run.smithi173.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.1.95554.log.gz 2023-11-24T06:32:36.849 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105791.log 2023-11-24T06:32:36.850 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.1.96883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88862.log 2023-11-24T06:32:36.850 INFO:teuthology.orchestra.run.smithi173.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.1.96883.log.gz 2023-11-24T06:32:36.851 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.88942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88902.log 2023-11-24T06:32:36.851 INFO:teuthology.orchestra.run.smithi173.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88942.log.gz 2023-11-24T06:32:36.852 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106370.log 2023-11-24T06:32:36.852 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.88862.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88862.log.gz 2023-11-24T06:32:36.853 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:32:36.854 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.88902.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88902.log.gz 2023-11-24T06:32:36.854 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T06:32:36.854 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112229.log 2023-11-24T06:32:36.854 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105791.log.gz 2023-11-24T06:32:36.855 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117615.log 2023-11-24T06:32:36.855 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106370.log.gz 2023-11-24T06:32:36.856 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112229.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112229.log.gz 2023-11-24T06:32:36.856 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110753.log 2023-11-24T06:32:36.856 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T06:32:36.857 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94298.log 2023-11-24T06:32:36.857 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97548.log 2023-11-24T06:32:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117615.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117615.log.gz 2023-11-24T06:32:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106286.log 2023-11-24T06:32:36.859 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.1.94298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88822.log 2023-11-24T06:32:36.860 INFO:teuthology.orchestra.run.smithi173.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:32:36.860 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.1.97548.log: 90.8% -- replaced with /var/log/ceph/ceph-client.1.94298.log.gz 2023-11-24T06:32:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111749.log 2023-11-24T06:32:36.860 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110753.log.gz 2023-11-24T06:32:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106286.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106286.log.gz 2023-11-24T06:32:36.861 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93902.log 2023-11-24T06:32:36.863 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.94811.log 2023-11-24T06:32:36.863 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.88822.log: 89.2% -- replaced with /var/log/ceph/ceph-client.1.97548.log.gz 2023-11-24T06:32:36.863 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88982.log 2023-11-24T06:32:36.863 INFO:teuthology.orchestra.run.smithi173.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88822.log.gz 2023-11-24T06:32:36.864 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111749.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111749.log.gz 2023-11-24T06:32:36.864 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96953.log 2023-11-24T06:32:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106328.log 2023-11-24T06:32:36.865 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93902.log.gz 2023-11-24T06:32:36.865 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100476.log 2023-11-24T06:32:36.865 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96953.log.gz 2023-11-24T06:32:36.866 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106160.log 2023-11-24T06:32:36.866 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106328.log.gz 2023-11-24T06:32:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100476.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T06:32:36.869 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100476.log.gz 2023-11-24T06:32:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106160.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106160.log.gz 2023-11-24T06:32:36.869 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95757.log 2023-11-24T06:32:36.870 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93581.log 2023-11-24T06:32:36.870 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.3.94811.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T06:32:36.871 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.88982.log: 91.7% -- replaced with /var/log/ceph/ceph-client.3.94811.log.gz 2023-11-24T06:32:36.871 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-24T06:32:36.871 INFO:teuthology.orchestra.run.smithi173.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88982.log.gz 2023-11-24T06:32:36.871 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89022.log 2023-11-24T06:32:36.872 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T06:32:36.872 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.89022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89062.log 2023-11-24T06:32:36.872 INFO:teuthology.orchestra.run.smithi173.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89022.log.gz 2023-11-24T06:32:36.878 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:32:36.878 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.89062.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.89062.log.gz 2023-11-24T06:32:36.878 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log/var/log/ceph/ceph.log: 2023-11-24T06:32:36.878 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mon.b.log: gzip 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:32:36.879 INFO:teuthology.orchestra.run.smithi173.stderr: -5 --verbose -- /var/log/ceph/ceph-client.1.98231.log 2023-11-24T06:32:36.879 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95757.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95757.log.gz 2023-11-24T06:32:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99554.log 2023-11-24T06:32:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108696.log 2023-11-24T06:32:36.880 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93581.log.gz 2023-11-24T06:32:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99554.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.82304.log 2023-11-24T06:32:36.880 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99554.log.gz 2023-11-24T06:32:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108696.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108696.log.gz 2023-11-24T06:32:36.880 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111089.log 2023-11-24T06:32:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.tmp-client.admin.82304.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82304.log.gz 2023-11-24T06:32:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116391.log 2023-11-24T06:32:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111089.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111089.log.gz 2023-11-24T06:32:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112718.log 2023-11-24T06:32:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104255.log 2023-11-24T06:32:36.881 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116391.log.gz 2023-11-24T06:32:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112718.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112718.log.gz 2023-11-24T06:32:36.881 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102435.log 2023-11-24T06:32:36.882 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103102.log 2023-11-24T06:32:36.882 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T06:32:36.883 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.1.98231.log: 89.8% -- replaced with /var/log/ceph/ceph-client.1.98231.log.gz 2023-11-24T06:32:36.883 INFO:teuthology.orchestra.run.smithi016.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.104255.log.gz 2023-11-24T06:32:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102435.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102435.log.gz 2023-11-24T06:32:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101826.log 2023-11-24T06:32:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103102.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103102.log.gz 2023-11-24T06:32:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106725.log 2023-11-24T06:32:36.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116475.log 2023-11-24T06:32:36.884 INFO:teuthology.orchestra.run.smithi016.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.101826.log.gz 2023-11-24T06:32:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106725.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106725.log.gz 2023-11-24T06:32:36.885 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107236.log 2023-11-24T06:32:36.886 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106430.log 2023-11-24T06:32:36.886 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116475.log.gz 2023-11-24T06:32:36.887 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.96197.log 2023-11-24T06:32:36.887 INFO:teuthology.orchestra.run.smithi173.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T06:32:36.888 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115775.log 2023-11-24T06:32:36.888 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107236.log.gz 2023-11-24T06:32:36.888 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106430.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106430.log.gz 2023-11-24T06:32:36.889 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93231.log 2023-11-24T06:32:36.889 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T06:32:36.891 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102318.log 2023-11-24T06:32:36.891 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115775.log.gz 2023-11-24T06:32:36.891 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93231.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93231.log.gz 2023-11-24T06:32:36.892 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102855.log 2023-11-24T06:32:36.892 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.1.96197.log: /var/log/ceph/ceph-mon.c.log: 89.8% -- replaced with /var/log/ceph/ceph-client.1.96197.log.gz 2023-11-24T06:32:36.893 INFO:teuthology.orchestra.run.smithi173.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T06:32:36.893 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102318.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.92935.log 2023-11-24T06:32:36.893 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102318.log.gz 2023-11-24T06:32:36.894 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93189.log 2023-11-24T06:32:36.894 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102855.log.gz 2023-11-24T06:32:36.894 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92935.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92935.log.gz 2023-11-24T06:32:36.895 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109026.log 2023-11-24T06:32:36.896 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114454.log 2023-11-24T06:32:36.896 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93189.log.gz 2023-11-24T06:32:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109026.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109026.log.gz 2023-11-24T06:32:36.897 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117095.log 2023-11-24T06:32:36.898 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96869.log 2023-11-24T06:32:36.899 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114454.log.gz 2023-11-24T06:32:36.899 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117095.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117095.log.gz 2023-11-24T06:32:36.900 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114538.log 2023-11-24T06:32:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96869.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96869.log.gz 2023-11-24T06:32:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115282.log 2023-11-24T06:32:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108600.log 2023-11-24T06:32:36.903 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.114538.log.gz 2023-11-24T06:32:36.904 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115282.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115282.log.gz 2023-11-24T06:32:36.904 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102771.log 2023-11-24T06:32:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108600.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108600.log.gz 2023-11-24T06:32:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100136.log 2023-11-24T06:32:36.907 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103527.log 2023-11-24T06:32:36.907 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102771.log.gz 2023-11-24T06:32:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100136.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100136.log.gz 2023-11-24T06:32:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99806.log 2023-11-24T06:32:36.910 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111833.log 2023-11-24T06:32:36.910 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103527.log.gz 2023-11-24T06:32:36.911 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103569.log 2023-11-24T06:32:36.911 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99806.log.gz 2023-11-24T06:32:36.912 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99386.log 2023-11-24T06:32:36.912 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.111833.log.gz 2023-11-24T06:32:36.914 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107108.log 2023-11-24T06:32:36.914 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103569.log.gz 2023-11-24T06:32:36.915 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99386.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99386.log.gz 2023-11-24T06:32:36.915 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113918.log 2023-11-24T06:32:36.916 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101700.log 2023-11-24T06:32:36.916 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107108.log.gz 2023-11-24T06:32:36.917 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103186.log 2023-11-24T06:32:36.918 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113918.log.gz 2023-11-24T06:32:36.919 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97079.log 2023-11-24T06:32:36.919 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101700.log.gz 2023-11-24T06:32:36.920 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92891.log 2023-11-24T06:32:36.920 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103186.log.gz 2023-11-24T06:32:36.921 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97079.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97079.log.gz 2023-11-24T06:32:36.922 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107572.log 2023-11-24T06:32:36.923 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115156.log 2023-11-24T06:32:36.923 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92891.log.gz 2023-11-24T06:32:36.924 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95037.log 2023-11-24T06:32:36.924 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107572.log.gz 2023-11-24T06:32:36.925 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112886.log 2023-11-24T06:32:36.925 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115156.log.gz 2023-11-24T06:32:36.926 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95037.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.95037.log.gz 2023-11-24T06:32:36.927 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.94533.log 2023-11-24T06:32:36.928 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112886.log: gzip -5 87.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.112886.log.gz /var/log/ceph/ceph-client.admin.109792.log 2023-11-24T06:32:36.928 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-11-24T06:32:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94533.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.94533.log.gz -5 2023-11-24T06:32:36.929 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99134.log 2023-11-24T06:32:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114370.log 2023-11-24T06:32:36.931 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109792.log.gz 2023-11-24T06:32:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100840.log 2023-11-24T06:32:36.932 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99134.log.gz 2023-11-24T06:32:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114370.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114370.log.gz 2023-11-24T06:32:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103833.log 2023-11-24T06:32:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97468.log 2023-11-24T06:32:36.935 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100840.log.gz 2023-11-24T06:32:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104171.log 2023-11-24T06:32:36.936 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103833.log.gz 2023-11-24T06:32:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92797.log 2023-11-24T06:32:36.937 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97468.log.gz 2023-11-24T06:32:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109227.log 2023-11-24T06:32:36.938 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104171.log.gz 2023-11-24T06:32:36.940 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96827.log 2023-11-24T06:32:36.940 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92797.log.gz 2023-11-24T06:32:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109227.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.109227.log.gz 2023-11-24T06:32:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102072.log 2023-11-24T06:32:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107614.log 2023-11-24T06:32:36.942 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96827.log.gz 2023-11-24T06:32:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100924.log 2023-11-24T06:32:36.944 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102072.log.gz 2023-11-24T06:32:36.945 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102603.log 2023-11-24T06:32:36.945 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107614.log.gz 2023-11-24T06:32:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112475.log 2023-11-24T06:32:36.947 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100924.log.gz 2023-11-24T06:32:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102603.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102603.log.gz 2023-11-24T06:32:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108517.log 2023-11-24T06:32:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100178.log 2023-11-24T06:32:36.949 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112475.log.gz 2023-11-24T06:32:36.950 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105950.log 2023-11-24T06:32:36.950 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108517.log.gz 2023-11-24T06:32:36.951 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111935.log 2023-11-24T06:32:36.951 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100178.log.gz 2023-11-24T06:32:36.951 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-11-24T06:32:36.952 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-11-24T06:32:36.952 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-11-24T06:32:36.952 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-11-24T06:32:36.952 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-11-24T06:32:36.952 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-11-24T06:32:36.952 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-11-24T06:32:36.952 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-24T06:32:36.953 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105950.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.105950.log.gz --verbose 2023-11-24T06:32:36.953 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.107740.log 2023-11-24T06:32:36.954 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100882.log 2023-11-24T06:32:36.954 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111935.log.gz 2023-11-24T06:32:36.955 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108822.log 2023-11-24T06:32:36.955 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107740.log.gz 2023-11-24T06:32:36.956 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114244.log 2023-11-24T06:32:36.956 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100882.log.gz 2023-11-24T06:32:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98192.log 2023-11-24T06:32:36.958 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108822.log.gz 2023-11-24T06:32:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112601.log 2023-11-24T06:32:36.959 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114244.log.gz 2023-11-24T06:32:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99050.log 2023-11-24T06:32:36.961 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98192.log.gz 2023-11-24T06:32:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106473.log 2023-11-24T06:32:36.962 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112601.log.gz 2023-11-24T06:32:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96491.log 2023-11-24T06:32:36.963 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99050.log.gz 2023-11-24T06:32:36.964 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97163.log 2023-11-24T06:32:36.964 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106473.log.gz 2023-11-24T06:32:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112397.log 2023-11-24T06:32:36.966 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96491.log.gz 2023-11-24T06:32:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97163.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97163.log.gz 2023-11-24T06:32:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106767.log 2023-11-24T06:32:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107192.log 2023-11-24T06:32:36.968 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112397.log.gz 2023-11-24T06:32:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109068.log 2023-11-24T06:32:36.969 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106767.log.gz 2023-11-24T06:32:36.970 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103611.log 2023-11-24T06:32:36.971 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107192.log.gz 2023-11-24T06:32:36.972 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93103.log 2023-11-24T06:32:36.972 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109068.log.gz 2023-11-24T06:32:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103611.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103611.log.gz 2023-11-24T06:32:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101867.log 2023-11-24T06:32:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94911.log 89.0% 2023-11-24T06:32:36.975 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93103.log.gz 2023-11-24T06:32:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108559.log 2023-11-24T06:32:36.976 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101867.log.gz 2023-11-24T06:32:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107488.log 2023-11-24T06:32:36.978 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94911.log.gz 2023-11-24T06:32:36.978 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99722.log 2023-11-24T06:32:36.979 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108559.log.gz 2023-11-24T06:32:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97205.log 2023-11-24T06:32:36.980 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107488.log.gz 2023-11-24T06:32:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94701.log 2023-11-24T06:32:36.981 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99722.log.gz 2023-11-24T06:32:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114328.log 2023-11-24T06:32:36.983 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97205.log.gz 2023-11-24T06:32:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95079.log 2023-11-24T06:32:36.984 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94701.log.gz 2023-11-24T06:32:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102519.log 2023-11-24T06:32:36.985 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114328.log.gz 2023-11-24T06:32:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102477.log 2023-11-24T06:32:36.986 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.95079.log.gz 2023-11-24T06:32:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102519.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102519.log.gz 2023-11-24T06:32:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96911.log 2023-11-24T06:32:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110445.log 2023-11-24T06:32:36.989 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102477.log.gz 2023-11-24T06:32:36.990 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93569.log 2023-11-24T06:32:36.990 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96911.log.gz 2023-11-24T06:32:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97426.log 2023-11-24T06:32:36.992 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110445.log.gz 2023-11-24T06:32:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93569.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93569.log.gz 2023-11-24T06:32:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94491.log 2023-11-24T06:32:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97426.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97426.log.gz -5 2023-11-24T06:32:36.994 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.113493.log 2023-11-24T06:32:36.995 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.104605.log 2023-11-24T06:32:36.996 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94491.log.gz 2023-11-24T06:32:36.996 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97694.log 2023-11-24T06:32:36.997 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.113493.log.gz 2023-11-24T06:32:36.998 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.104605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110195.log 2023-11-24T06:32:37.000 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97694.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97694.log.gz 2023-11-24T06:32:37.000 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101742.log 2023-11-24T06:32:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110195.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110195.log.gz 2023-11-24T06:32:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107024.log 2023-11-24T06:32:37.003 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101742.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101742.log.gz 2023-11-24T06:32:37.003 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98720.log 2023-11-24T06:32:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107024.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107024.log.gz 2023-11-24T06:32:37.004 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113834.log 2023-11-24T06:32:37.006 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98720.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98720.log.gz 2023-11-24T06:32:37.006 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93900.log 2023-11-24T06:32:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113834.log: 92.4% -- replaced with /var/log/ceph/ceph-client.0.104605.log.gz 2023-11-24T06:32:37.007 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113834.log.gz 2023-11-24T06:32:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102360.log 2023-11-24T06:32:37.008 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113367.log 2023-11-24T06:32:37.009 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93900.log.gz 2023-11-24T06:32:37.009 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109563.log 2023-11-24T06:32:37.010 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102360.log.gz 2023-11-24T06:32:37.011 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112271.log 2023-11-24T06:32:37.011 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113367.log.gz 2023-11-24T06:32:37.012 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109563.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109563.log.gz 2023-11-24T06:32:37.013 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114286.log 2023-11-24T06:32:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112271.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112271.log.gz 2023-11-24T06:32:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97940.log 2023-11-24T06:32:37.015 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101134.log 2023-11-24T06:32:37.015 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114286.log.gz 2023-11-24T06:32:37.016 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98494.log 2023-11-24T06:32:37.016 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97940.log.gz 2023-11-24T06:32:37.017 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101616.log 2023-11-24T06:32:37.018 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.101134.log.gz 2023-11-24T06:32:37.019 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98494.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98494.log.gz 2023-11-24T06:32:37.019 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94995.log 2023-11-24T06:32:37.020 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117515.log 2023-11-24T06:32:37.020 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101616.log.gz 2023-11-24T06:32:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101092.log 2023-11-24T06:32:37.022 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94995.log.gz 2023-11-24T06:32:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117515.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117515.log.gz 2023-11-24T06:32:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101988.log 2023-11-24T06:32:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101092.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101092.log.gz 2023-11-24T06:32:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104213.log 2023-11-24T06:32:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115198.log 2023-11-24T06:32:37.027 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101988.log.gz 2023-11-24T06:32:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107530.log 2023-11-24T06:32:37.028 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104213.log.gz 2023-11-24T06:32:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115198.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115198.log.gz 2023-11-24T06:32:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110403.log 2023-11-24T06:32:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105503.log 2023-11-24T06:32:37.029 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107530.log.gz 2023-11-24T06:32:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104129.log 2023-11-24T06:32:37.030 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110403.log.gz 2023-11-24T06:32:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105503.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105503.log.gz 2023-11-24T06:32:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98410.log 2023-11-24T06:32:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96449.log 2023-11-24T06:32:37.032 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104129.log.gz 2023-11-24T06:32:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98840.log 2023-11-24T06:32:37.034 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98410.log.gz 2023-11-24T06:32:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96449.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96449.log.gz 2023-11-24T06:32:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114496.log 2023-11-24T06:32:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102915.log 2023-11-24T06:32:37.036 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98840.log.gz 2023-11-24T06:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110667.log 2023-11-24T06:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114496.log.gz 2023-11-24T06:32:37.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98762.log 2023-11-24T06:32:37.039 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102915.log.gz 2023-11-24T06:32:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110667.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110667.log.gz 2023-11-24T06:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99176.log 2023-11-24T06:32:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110489.log 2023-11-24T06:32:37.041 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98762.log.gz 2023-11-24T06:32:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99176.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99176.log.gz 2023-11-24T06:32:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100568.log 2023-11-24T06:32:37.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94248.log 2023-11-24T06:32:37.044 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110489.log.gz 2023-11-24T06:32:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113451.log 2023-11-24T06:32:37.045 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100568.log.gz 2023-11-24T06:32:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104087.log 2023-11-24T06:32:37.046 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94248.log.gz 2023-11-24T06:32:37.047 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113451.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113451.log.gz 2023-11-24T06:32:37.048 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115529.log 2023-11-24T06:32:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104831.log 2023-11-24T06:32:37.049 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104087.log.gz 2023-11-24T06:32:37.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94449.log 2023-11-24T06:32:37.050 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115529.log.gz 2023-11-24T06:32:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104831.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104831.log.gz 2023-11-24T06:32:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116843.log 2023-11-24T06:32:37.052 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110533.log 2023-11-24T06:32:37.053 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94449.log.gz 2023-11-24T06:32:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94251.log 2023-11-24T06:32:37.054 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116843.log.gz 2023-11-24T06:32:37.055 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110533.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110533.log.gz 2023-11-24T06:32:37.055 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:32:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93019.log 2023-11-24T06:32:37.056 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94251.log.gz 2023-11-24T06:32:37.058 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107443.log 2023-11-24T06:32:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93019.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93019.log.gz 2023-11-24T06:32:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99470.log 2023-11-24T06:32:37.060 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:32:37.060 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96294.log 2023-11-24T06:32:37.061 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107443.log.gz 2023-11-24T06:32:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116717.log 2023-11-24T06:32:37.062 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99470.log.gz 2023-11-24T06:32:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116759.log 2023-11-24T06:32:37.063 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96294.log.gz 2023-11-24T06:32:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116717.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116717.log.gz 2023-11-24T06:32:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92839.log 2023-11-24T06:32:37.065 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98108.log 2023-11-24T06:32:37.065 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116759.log.gz 2023-11-24T06:32:37.066 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108780.log 2023-11-24T06:32:37.067 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92839.log.gz 2023-11-24T06:32:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98108.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98108.log.gz 2023-11-24T06:32:37.068 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102813.log 2023-11-24T06:32:37.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93315.log 2023-11-24T06:32:37.070 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108780.log.gz 2023-11-24T06:32:37.070 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94785.log 2023-11-24T06:32:37.071 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102813.log.gz 2023-11-24T06:32:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93315.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93315.log.gz 2023-11-24T06:32:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110963.log 2023-11-24T06:32:37.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97982.log 2023-11-24T06:32:37.073 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94785.log.gz 2023-11-24T06:32:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110963.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110963.log.gz 2023-11-24T06:32:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100094.log 2023-11-24T06:32:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117053.log 2023-11-24T06:32:37.076 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97982.log.gz 2023-11-24T06:32:37.076 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109395.log 2023-11-24T06:32:37.077 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100094.log.gz 2023-11-24T06:32:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117053.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.117053.log.gz 2023-11-24T06:32:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116288.log 2023-11-24T06:32:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115403.log 2023-11-24T06:32:37.080 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109395.log.gz 2023-11-24T06:32:37.080 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107150.log 2023-11-24T06:32:37.081 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116288.log.gz 2023-11-24T06:32:37.081 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115403.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115403.log.gz 2023-11-24T06:32:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104003.log 2023-11-24T06:32:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114004.log 2023-11-24T06:32:37.083 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107150.log.gz 2023-11-24T06:32:37.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104045.log 2023-11-24T06:32:37.084 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104003.log.gz 2023-11-24T06:32:37.085 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114004.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114004.log.gz 2023-11-24T06:32:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92625.log 2023-11-24T06:32:37.087 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105749.log 2023-11-24T06:32:37.087 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104045.log.gz 2023-11-24T06:32:37.088 INFO:teuthology.orchestra.run.smithi173.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T06:32:37.088 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98536.log 2023-11-24T06:32:37.088 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92625.log.gz 2023-11-24T06:32:37.089 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105749.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105749.log.gz 2023-11-24T06:32:37.090 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96659.log 2023-11-24T06:32:37.091 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112760.log 2023-11-24T06:32:37.091 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98536.log.gz 2023-11-24T06:32:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100304.log 2023-11-24T06:32:37.092 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96659.log.gz 2023-11-24T06:32:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112760.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112760.log.gz 2023-11-24T06:32:37.093 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92754.log 2023-11-24T06:32:37.094 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108307.log 2023-11-24T06:32:37.096 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100304.log.gz 2023-11-24T06:32:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92754.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.92754.log.gz 2023-11-24T06:32:37.096 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106515.log 2023-11-24T06:32:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110583.log 2023-11-24T06:32:37.097 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108307.log.gz 2023-11-24T06:32:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102561.log 2023-11-24T06:32:37.098 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106515.log.gz 2023-11-24T06:32:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107330.log 2023-11-24T06:32:37.099 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110583.log.gz 2023-11-24T06:32:37.101 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102561.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102561.log.gz 2023-11-24T06:32:37.101 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93273.log 2023-11-24T06:32:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92399.log 2023-11-24T06:32:37.103 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107330.log.gz 2023-11-24T06:32:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105545.log 2023-11-24T06:32:37.104 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93273.log.gz 2023-11-24T06:32:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92399.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92399.log.gz 2023-11-24T06:32:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111893.log 2023-11-24T06:32:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102729.log 2023-11-24T06:32:37.106 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105545.log.gz 2023-11-24T06:32:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114098.log 2023-11-24T06:32:37.107 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111893.log.gz 2023-11-24T06:32:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102729.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102729.log.gz 2023-11-24T06:32:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103228.log 2023-11-24T06:32:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115850.log 2023-11-24T06:32:37.110 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114098.log.gz 2023-11-24T06:32:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98594.log 2023-11-24T06:32:37.111 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103228.log.gz 2023-11-24T06:32:37.112 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115850.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115850.log.gz 2023-11-24T06:32:37.112 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107698.log 2023-11-24T06:32:37.113 INFO:teuthology.orchestra.run.smithi173.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T06:32:37.113 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113012.log 2023-11-24T06:32:37.114 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98594.log.gz 2023-11-24T06:32:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109269.log 2023-11-24T06:32:37.115 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107698.log.gz 2023-11-24T06:32:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113012.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113012.log.gz 2023-11-24T06:32:37.117 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94869.log 2023-11-24T06:32:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103919.log 2023-11-24T06:32:37.118 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109269.log.gz 2023-11-24T06:32:37.119 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109437.log 2023-11-24T06:32:37.119 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94869.log.gz 2023-11-24T06:32:37.120 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109750.log 2023-11-24T06:32:37.122 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103919.log.gz 2023-11-24T06:32:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109437.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109437.log.gz 2023-11-24T06:32:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106202.log 2023-11-24T06:32:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97331.log 2023-11-24T06:32:37.123 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109750.log.gz 2023-11-24T06:32:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105587.log 2023-11-24T06:32:37.126 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106202.log.gz 2023-11-24T06:32:37.126 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97331.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97331.log.gz 2023-11-24T06:32:37.126 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93913.log 2023-11-24T06:32:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113876.log 2023-11-24T06:32:37.128 INFO:teuthology.orchestra.run.smithi173.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T06:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105587.log.gz 2023-11-24T06:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93579.log 2023-11-24T06:32:37.130 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93913.log.gz 2023-11-24T06:32:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113876.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113876.log.gz 2023-11-24T06:32:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108391.log 2023-11-24T06:32:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112928.log 2023-11-24T06:32:37.131 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93579.log.gz 2023-11-24T06:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108984.log 2023-11-24T06:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108391.log.gz 2023-11-24T06:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112928.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112928.log.gz 2023-11-24T06:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T06:32:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111529.log 2023-11-24T06:32:37.135 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108984.log.gz 2023-11-24T06:32:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111047.log 2023-11-24T06:32:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.111529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107849.log 2023-11-24T06:32:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109936.log 2023-11-24T06:32:37.138 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111047.log.gz 2023-11-24T06:32:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107849.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107849.log.gz 2023-11-24T06:32:37.139 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101658.log 2023-11-24T06:32:37.140 INFO:teuthology.orchestra.run.smithi016.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.111529.log.gz 2023-11-24T06:32:37.140 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105346.log 2023-11-24T06:32:37.141 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101658.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109936.log.gz 2023-11-24T06:32:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106244.log 2023-11-24T06:32:37.142 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101658.log.gz 2023-11-24T06:32:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105346.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105346.log.gz 2023-11-24T06:32:37.143 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99428.log 2023-11-24T06:32:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115072.log 2023-11-24T06:32:37.145 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106244.log.gz 2023-11-24T06:32:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99428.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99428.log.gz 2023-11-24T06:32:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109850.log 2023-11-24T06:32:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99344.log 2023-11-24T06:32:37.147 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115072.log.gz 2023-11-24T06:32:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109850.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109850.log.gz 2023-11-24T06:32:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94575.log 2023-11-24T06:32:37.149 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88762.log 2023-11-24T06:32:37.150 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99344.log.gz 2023-11-24T06:32:37.150 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94575.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94575.log.gz 2023-11-24T06:32:37.151 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94743.log 2023-11-24T06:32:37.152 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.88762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96339.log 2023-11-24T06:32:37.152 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88762.log.gz 2023-11-24T06:32:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94743.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94743.log.gz 2023-11-24T06:32:37.153 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106557.log 2023-11-24T06:32:37.155 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116144.log 2023-11-24T06:32:37.155 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96339.log.gz 2023-11-24T06:32:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106557.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106557.log.gz 2023-11-24T06:32:37.157 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100052.log 2023-11-24T06:32:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116144.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116144.log.gz -5 2023-11-24T06:32:37.159 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.109978.log 2023-11-24T06:32:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100052.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100052.log.gz 2023-11-24T06:32:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99092.log 2023-11-24T06:32:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109978.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.109978.log.gz --verbose 2023-11-24T06:32:37.161 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.110115.log 2023-11-24T06:32:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99092.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99092.log.gz 2023-11-24T06:32:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105083.log 2023-11-24T06:32:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112150.log 2023-11-24T06:32:37.163 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110115.log.gz 2023-11-24T06:32:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105083.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105083.log.gz 2023-11-24T06:32:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113792.log 2023-11-24T06:32:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108738.log 2023-11-24T06:32:37.165 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112150.log.gz 2023-11-24T06:32:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113792.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113792.log.gz 2023-11-24T06:32:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100262.log 2023-11-24T06:32:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108738.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108738.log.gz 2023-11-24T06:32:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110361.log 2023-11-24T06:32:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100262.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100262.log.gz 2023-11-24T06:32:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96575.log 2023-11-24T06:32:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99680.log 2023-11-24T06:32:37.172 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110361.log.gz 2023-11-24T06:32:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96575.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96575.log.gz 2023-11-24T06:32:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105992.log 2023-11-24T06:32:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99680.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.99680.log.gz -5 2023-11-24T06:32:37.175 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.103018.log 2023-11-24T06:32:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105992.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105992.log.gz 2023-11-24T06:32:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108265.log 2023-11-24T06:32:37.177 INFO:teuthology.orchestra.run.smithi173.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T06:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103653.log 2023-11-24T06:32:37.179 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103018.log.gz 2023-11-24T06:32:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108265.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.108265.log.gz 2023-11-24T06:32:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98276.log 2023-11-24T06:32:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103653.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103653.log.gz 2023-11-24T06:32:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94211.log 2023-11-24T06:32:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98276.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98276.log.gz 2023-11-24T06:32:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96785.log 2023-11-24T06:32:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97510.log 2023-11-24T06:32:37.183 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94211.log.gz 2023-11-24T06:32:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96785.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96785.log.gz 2023-11-24T06:32:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111707.log 2023-11-24T06:32:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97510.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97510.log.gz 2023-11-24T06:32:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100432.log 2023-11-24T06:32:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111707.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111707.log.gz 2023-11-24T06:32:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97594.log 2023-11-24T06:32:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97736.log 2023-11-24T06:32:37.190 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100432.log.gz 2023-11-24T06:32:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97594.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97594.log.gz 2023-11-24T06:32:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95601.log 2023-11-24T06:32:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107066.log 2023-11-24T06:32:37.192 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97736.log.gz 2023-11-24T06:32:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.95601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99302.log 2023-11-24T06:32:37.192 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.95601.log.gz 2023-11-24T06:32:37.193 INFO:teuthology.orchestra.run.smithi173.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T06:32:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101784.log 2023-11-24T06:32:37.197 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107066.log.gz 2023-11-24T06:32:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99302.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99302.log.gz 2023-11-24T06:32:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116517.log 2023-11-24T06:32:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101784.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101784.log.gz 2023-11-24T06:32:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101008.log 2023-11-24T06:32:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116517.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116517.log.gz 2023-11-24T06:32:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115240.log 2023-11-24T06:32:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97037.log 2023-11-24T06:32:37.203 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101008.log.gz 2023-11-24T06:32:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115240.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115240.log.gz 2023-11-24T06:32:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94246.log 2023-11-24T06:32:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97037.log: gzip -5 --verbose 89.4% -- -- replaced with /var/log/ceph/ceph-client.admin.97037.log.gz /var/log/ceph/ceph-client.admin.111239.log 2023-11-24T06:32:37.204 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-11-24T06:32:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94246.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94246.log.gz 2023-11-24T06:32:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117263.log 2023-11-24T06:32:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111239.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.111239.log.gz 2023-11-24T06:32:37.205 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109353.log 2023-11-24T06:32:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117263.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117263.log.gz 2023-11-24T06:32:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101477.log 2023-11-24T06:32:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109353.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109353.log.gz 2023-11-24T06:32:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110275.log 2023-11-24T06:32:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.101477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94953.log 2023-11-24T06:32:37.210 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.0.101477.log.gz 2023-11-24T06:32:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110275.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110275.log.gz 2023-11-24T06:32:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112355.log 2023-11-24T06:32:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94953.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94953.log.gz 2023-11-24T06:32:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111791.log 2023-11-24T06:32:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112355.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112355.log.gz 2023-11-24T06:32:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107782.log 2023-11-24T06:32:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111791.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111791.log.gz 2023-11-24T06:32:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93903.log 2023-11-24T06:32:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105833.log 2023-11-24T06:32:37.213 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.107782.log.gz 2023-11-24T06:32:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93903.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93903.log.gz 2023-11-24T06:32:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98678.log 2023-11-24T06:32:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105833.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105833.log.gz 2023-11-24T06:32:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100669.log 2023-11-24T06:32:37.217 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98678.log.gz 2023-11-24T06:32:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111173.log 2023-11-24T06:32:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100669.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100669.log.gz 2023-11-24T06:32:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92699.log 2023-11-24T06:32:37.221 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104321.log 2023-11-24T06:32:37.221 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.111173.log.gz 2023-11-24T06:32:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92699.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92699.log.gz 2023-11-24T06:32:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96995.log 2023-11-24T06:32:37.222 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104321.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104321.log.gz 2023-11-24T06:32:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T06:32:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96995.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96995.log.gz 2023-11-24T06:32:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112091.log 2023-11-24T06:32:37.225 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98452.log 2023-11-24T06:32:37.225 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112091.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112091.log.gz 2023-11-24T06:32:37.225 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113535.log 2023-11-24T06:32:37.232 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98452.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98452.log.gz 2023-11-24T06:32:37.232 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111005.log 2023-11-24T06:32:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116927.log 2023-11-24T06:32:37.233 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113535.log.gz 2023-11-24T06:32:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111005.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111005.log.gz 2023-11-24T06:32:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116102.log 2023-11-24T06:32:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116927.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116927.log.gz 2023-11-24T06:32:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117179.log 2023-11-24T06:32:37.233 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92576.log 2023-11-24T06:32:37.234 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116102.log.gz 2023-11-24T06:32:37.234 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117179.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117179.log.gz 2023-11-24T06:32:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113096.log 2023-11-24T06:32:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92576.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92576.log.gz 2023-11-24T06:32:37.235 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102687.log 2023-11-24T06:32:37.236 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94659.log 2023-11-24T06:32:37.236 INFO:teuthology.orchestra.run.smithi173.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-24T06:32:37.236 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113096.log.gz 2023-11-24T06:32:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102687.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102687.log.gz 2023-11-24T06:32:37.237 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97652.log 2023-11-24T06:32:37.238 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94659.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94659.log.gz 2023-11-24T06:32:37.239 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114202.log 2023-11-24T06:32:37.240 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110020.log 2023-11-24T06:32:37.241 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97652.log.gz 2023-11-24T06:32:37.242 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114202.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114202.log.gz 2023-11-24T06:32:37.242 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106862.log 2023-11-24T06:32:37.243 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110020.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110020.log.gz 2023-11-24T06:32:37.243 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113198.log 2023-11-24T06:32:37.245 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106862.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106862.log.gz 2023-11-24T06:32:37.245 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92495.log 2023-11-24T06:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98368.log 2023-11-24T06:32:37.247 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113198.log.gz 2023-11-24T06:32:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92495.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.92495.log.gz 2023-11-24T06:32:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109708.log 2023-11-24T06:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98368.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98368.log.gz 2023-11-24T06:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96617.log 2023-11-24T06:32:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93146.log 2023-11-24T06:32:37.251 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109708.log.gz 2023-11-24T06:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96617.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96617.log.gz 2023-11-24T06:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117575.log 2023-11-24T06:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93146.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93146.log.gz 2023-11-24T06:32:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93910.log 2023-11-24T06:32:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103144.log 2023-11-24T06:32:37.255 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117575.log.gz 2023-11-24T06:32:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93910.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93910.log.gz 2023-11-24T06:32:37.257 INFO:teuthology.orchestra.run.smithi173.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T06:32:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100756.log 2023-11-24T06:32:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103144.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103144.log.gz 2023-11-24T06:32:37.258 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98966.log 2023-11-24T06:32:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100756.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100756.log.gz 2023-11-24T06:32:37.260 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97778.log 2023-11-24T06:32:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98966.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98966.log.gz 2023-11-24T06:32:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117473.log 2023-11-24T06:32:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97289.log 2023-11-24T06:32:37.263 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97778.log.gz 2023-11-24T06:32:37.264 INFO:teuthology.orchestra.run.smithi173.stderr: 2023-11-24T06:32:37.264 INFO:teuthology.orchestra.run.smithi173.stderr:real 0m0.457s 2023-11-24T06:32:37.264 INFO:teuthology.orchestra.run.smithi173.stderr:user 0m2.015s 2023-11-24T06:32:37.265 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m0.305s 2023-11-24T06:32:37.265 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117473.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117473.log.gz 2023-11-24T06:32:37.265 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.96164.log 2023-11-24T06:32:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97289.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97289.log.gz 2023-11-24T06:32:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112970.log 2023-11-24T06:32:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.2.96164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105875.log 2023-11-24T06:32:37.270 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112970.log: 91.9% -- replaced with /var/log/ceph/ceph-client.2.96164.log.gz 2023-11-24T06:32:37.270 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100526.log 2023-11-24T06:32:37.270 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112970.log.gz 2023-11-24T06:32:37.271 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113241.log 2023-11-24T06:32:37.271 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105875.log.gz 2023-11-24T06:32:37.272 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100526.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100526.log.gz 2023-11-24T06:32:37.272 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95181.log 2023-11-24T06:32:37.274 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113241.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113241.log.gz 2023-11-24T06:32:37.274 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115976.log 2023-11-24T06:32:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.95181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116348.log 2023-11-24T06:32:37.276 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.95181.log.gz 2023-11-24T06:32:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115976.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115976.log.gz 2023-11-24T06:32:37.277 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100714.log 2023-11-24T06:32:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116801.log 2023-11-24T06:32:37.279 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116348.log.gz 2023-11-24T06:32:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100714.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100714.log.gz 2023-11-24T06:32:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117657.log 2023-11-24T06:32:37.281 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116801.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116801.log.gz 2023-11-24T06:32:37.281 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106599.log 2023-11-24T06:32:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105304.log 2023-11-24T06:32:37.283 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117657.log.gz 2023-11-24T06:32:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106599.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106599.log.gz 2023-11-24T06:32:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98924.log 2023-11-24T06:32:37.285 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105304.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105304.log.gz 2023-11-24T06:32:37.286 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98234.log 2023-11-24T06:32:37.287 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106641.log 2023-11-24T06:32:37.287 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98924.log.gz 2023-11-24T06:32:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98234.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98234.log.gz 2023-11-24T06:32:37.289 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99968.log 2023-11-24T06:32:37.290 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106641.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T06:32:37.291 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106641.log.gz 2023-11-24T06:32:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97820.log 2023-11-24T06:32:37.292 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99968.log.gz 2023-11-24T06:32:37.294 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108126.log 2023-11-24T06:32:37.296 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97820.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97820.log.gz 2023-11-24T06:32:37.296 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103445.log 2023-11-24T06:32:37.297 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.108126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115114.log 2023-11-24T06:32:37.299 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93565.log 2023-11-24T06:32:37.299 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103445.log.gz 2023-11-24T06:32:37.299 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.108126.log.gz 2023-11-24T06:32:37.300 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115114.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.115114.log.gz 2023-11-24T06:32:37.300 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.113710.log 2023-11-24T06:32:37.300 INFO:teuthology.orchestra.run.smithi016.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-24T06:32:37.301 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93567.log 2023-11-24T06:32:37.302 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93565.log.gz 2023-11-24T06:32:37.303 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113710.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113710.log.gz 2023-11-24T06:32:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102958.log 2023-11-24T06:32:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93567.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93567.log.gz 2023-11-24T06:32:37.304 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108433.log 2023-11-24T06:32:37.305 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92977.log 2023-11-24T06:32:37.306 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102958.log.gz 2023-11-24T06:32:37.307 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108433.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108433.log.gz 2023-11-24T06:32:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105041.log 2023-11-24T06:32:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.92977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117305.log 2023-11-24T06:32:37.309 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92977.log.gz 2023-11-24T06:32:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105041.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105041.log.gz 2023-11-24T06:32:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100220.log 2023-11-24T06:32:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105243.log 2023-11-24T06:32:37.312 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117305.log.gz 2023-11-24T06:32:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100220.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100220.log.gz 2023-11-24T06:32:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113054.log 2023-11-24T06:32:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105243.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105243.log.gz 2023-11-24T06:32:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116228.log 2023-11-24T06:32:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113054.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113054.log.gz 2023-11-24T06:32:37.317 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99596.log 2023-11-24T06:32:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116228.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116228.log.gz 2023-11-24T06:32:37.318 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109665.log 2023-11-24T06:32:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115892.log 2023-11-24T06:32:37.320 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99596.log.gz 2023-11-24T06:32:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109665.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109665.log.gz 2023-11-24T06:32:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106118.log 2023-11-24T06:32:37.323 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98150.log 2023-11-24T06:32:37.323 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115892.log.gz 2023-11-24T06:32:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106118.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106118.log.gz 2023-11-24T06:32:37.324 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112844.log 2023-11-24T06:32:37.325 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106076.log 2023-11-24T06:32:37.326 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98150.log.gz 2023-11-24T06:32:37.327 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112844.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112844.log.gz 2023-11-24T06:32:37.327 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99260.log 2023-11-24T06:32:37.329 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116018.log 2023-11-24T06:32:37.329 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106076.log.gz 2023-11-24T06:32:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99260.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99260.log.gz 2023-11-24T06:32:37.330 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88378.log 2023-11-24T06:32:37.332 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117137.log 2023-11-24T06:32:37.332 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116018.log.gz 2023-11-24T06:32:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.88378.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88378.log.gz 2023-11-24T06:32:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96743.log 2023-11-24T06:32:37.335 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105629.log 2023-11-24T06:32:37.335 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117137.log.gz 2023-11-24T06:32:37.336 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96743.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96743.log.gz 2023-11-24T06:32:37.337 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94253.log 2023-11-24T06:32:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93357.log 2023-11-24T06:32:37.338 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105629.log.gz 2023-11-24T06:32:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94253.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94253.log.gz 2023-11-24T06:32:37.340 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109185.log 2023-11-24T06:32:37.341 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93357.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.93357.log.gz 2023-11-24T06:32:37.341 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.110625.log 2023-11-24T06:32:37.343 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102030.log 2023-11-24T06:32:37.343 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109185.log.gz 2023-11-24T06:32:37.344 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102234.log 2023-11-24T06:32:37.345 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110625.log.gz 2023-11-24T06:32:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102030.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102030.log.gz 2023-11-24T06:32:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108475.log 2023-11-24T06:32:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93061.log 2023-11-24T06:32:37.347 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102234.log.gz 2023-11-24T06:32:37.349 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108475.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108475.log.gz 2023-11-24T06:32:37.349 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105707.log 2023-11-24T06:32:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93061.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93061.log.gz 2023-11-24T06:32:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115445.log 2023-11-24T06:32:37.352 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105707.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-24T06:32:37.352 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105707.log.gz 2023-11-24T06:32:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115445.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115445.log.gz 2023-11-24T06:32:37.353 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111131.log 2023-11-24T06:32:37.355 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104873.log 2023-11-24T06:32:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.111131.log: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T06:32:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97121.log 2023-11-24T06:32:37.357 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111131.log.gz 2023-11-24T06:32:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104873.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104873.log.gz 2023-11-24T06:32:37.358 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109311.log 2023-11-24T06:32:37.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102276.log 2023-11-24T06:32:37.360 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97121.log.gz 2023-11-24T06:32:37.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109311.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109311.log.gz 2023-11-24T06:32:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110921.log 2023-11-24T06:32:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102276.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102276.log.gz 2023-11-24T06:32:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98882.log 2023-11-24T06:32:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110921.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110921.log.gz 2023-11-24T06:32:37.364 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106034.log 2023-11-24T06:32:37.365 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112031.log 2023-11-24T06:32:37.366 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98882.log.gz 2023-11-24T06:32:37.367 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106034.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106034.log.gz 2023-11-24T06:32:37.367 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100346.log 2023-11-24T06:32:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109894.log 2023-11-24T06:32:37.369 INFO:teuthology.orchestra.run.smithi016.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.112031.log.gz 2023-11-24T06:32:37.370 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100346.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100346.log.gz 2023-11-24T06:32:37.370 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99886.log 2023-11-24T06:32:37.372 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113325.log 2023-11-24T06:32:37.372 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109894.log.gz 2023-11-24T06:32:37.373 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99886.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99886.log.gz 2023-11-24T06:32:37.373 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115487.log 2023-11-24T06:32:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113325.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113325.log.gz 2023-11-24T06:32:37.374 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102192.log 2023-11-24T06:32:37.376 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94237.log 2023-11-24T06:32:37.376 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115487.log.gz 2023-11-24T06:32:37.377 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102192.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102192.log.gz 2023-11-24T06:32:37.377 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115361.log 2023-11-24T06:32:37.379 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94237.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94237.log.gz 2023-11-24T06:32:37.379 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112559.log 2023-11-24T06:32:37.380 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115361.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115361.log.gz 2023-11-24T06:32:37.380 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110317.log 2023-11-24T06:32:37.381 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112802.log 2023-11-24T06:32:37.382 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112559.log.gz 2023-11-24T06:32:37.383 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110317.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110317.log.gz 2023-11-24T06:32:37.383 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117347.log 2023-11-24T06:32:37.385 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112802.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112802.log.gz 2023-11-24T06:32:37.385 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97898.log 2023-11-24T06:32:37.386 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98024.log 2023-11-24T06:32:37.386 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117347.log.gz 2023-11-24T06:32:37.387 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97898.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97898.log.gz 2023-11-24T06:32:37.388 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112643.log 2023-11-24T06:32:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100798.log 2023-11-24T06:32:37.389 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98024.log.gz 2023-11-24T06:32:37.390 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112643.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112643.log.gz 2023-11-24T06:32:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.114892.log 2023-11-24T06:32:37.391 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100798.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100798.log.gz 2023-11-24T06:32:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113630.log 2023-11-24T06:32:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.114892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116885.log 2023-11-24T06:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113630.log: 90.1% -- replaced with /var/log/ceph/ceph-client.0.114892.log.gz 2023-11-24T06:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113630.log.gz 2023-11-24T06:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117389.log 2023-11-24T06:32:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116885.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116885.log.gz 2023-11-24T06:32:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108864.log 2023-11-24T06:32:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93399.log 2023-11-24T06:32:37.398 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117389.log.gz 2023-11-24T06:32:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93914.log 2023-11-24T06:32:37.400 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108864.log.gz 2023-11-24T06:32:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93399.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93399.log.gz 2023-11-24T06:32:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:32:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114412.log 2023-11-24T06:32:37.402 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93914.log.gz 2023-11-24T06:32:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97552.log 2023-11-24T06:32:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114412.log: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:32:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113960.log 2023-11-24T06:32:37.406 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114412.log.gz 2023-11-24T06:32:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105461.log 2023-11-24T06:32:37.407 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97552.log.gz 2023-11-24T06:32:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97247.log 2023-11-24T06:32:37.408 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113960.log.gz 2023-11-24T06:32:37.410 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105461.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-24T06:32:37.410 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105461.log.gz 2023-11-24T06:32:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.97247.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97247.log.gz 2023-11-24T06:32:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103270.log 2023-11-24T06:32:37.412 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103060.log 2023-11-24T06:32:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103270.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103270.log.gz 2023-11-24T06:32:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116186.log 2023-11-24T06:32:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103060.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103060.log.gz 2023-11-24T06:32:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T06:32:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102645.log 2023-11-24T06:32:37.417 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116186.log.gz 2023-11-24T06:32:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103695.log 2023-11-24T06:32:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116675.log 2023-11-24T06:32:37.420 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102645.log.gz 2023-11-24T06:32:37.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93576.log 2023-11-24T06:32:37.422 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103695.log.gz 2023-11-24T06:32:37.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104957.log 2023-11-24T06:32:37.423 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116675.log.gz 2023-11-24T06:32:37.425 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110711.log 2023-11-24T06:32:37.425 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93576.log.gz 2023-11-24T06:32:37.426 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99512.log 2023-11-24T06:32:37.427 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104957.log.gz 2023-11-24T06:32:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99008.log 2023-11-24T06:32:37.429 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110711.log.gz 2023-11-24T06:32:37.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113409.log 2023-11-24T06:32:37.430 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99512.log.gz 2023-11-24T06:32:37.435 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116969.log 2023-11-24T06:32:37.436 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99008.log.gz 2023-11-24T06:32:37.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110837.log 2023-11-24T06:32:37.436 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113409.log.gz 2023-11-24T06:32:37.436 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100388.log 2023-11-24T06:32:37.436 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116969.log.gz 2023-11-24T06:32:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117431.log 2023-11-24T06:32:37.437 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110837.log.gz 2023-11-24T06:32:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99218.log 2023-11-24T06:32:37.437 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100388.log.gz 2023-11-24T06:32:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96533.log 2023-11-24T06:32:37.439 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117431.log.gz 2023-11-24T06:32:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108349.log 2023-11-24T06:32:37.440 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99218.log.gz 2023-11-24T06:32:37.442 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96533.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.96533.log.gz --verbose 2023-11-24T06:32:37.442 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.115733.log 2023-11-24T06:32:37.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108349.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.108349.log.gz 2023-11-24T06:32:37.443 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.113138.log 2023-11-24T06:32:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96701.log 2023-11-24T06:32:37.444 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115733.log.gz 2023-11-24T06:32:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113138.log: gzip 88.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.113138.log.gz -- 2023-11-24T06:32:37.447 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.109479.log 2023-11-24T06:32:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.96701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108942.log 2023-11-24T06:32:37.448 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96701.log.gz 2023-11-24T06:32:37.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110879.log 2023-11-24T06:32:37.450 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109479.log.gz 2023-11-24T06:32:37.450 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.108942.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108942.log.gz 2023-11-24T06:32:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94242.log 2023-11-24T06:32:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110879.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110879.log.gz 2023-11-24T06:32:37.453 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109605.log 2023-11-24T06:32:37.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93917.log 2023-11-24T06:32:37.455 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94242.log.gz 2023-11-24T06:32:37.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117221.log 2023-11-24T06:32:37.456 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109605.log.gz 2023-11-24T06:32:37.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93917.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93917.log.gz 2023-11-24T06:32:37.457 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103739.log 2023-11-24T06:32:37.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115934.log 2023-11-24T06:32:37.459 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117221.log.gz 2023-11-24T06:32:37.461 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98066.log 2023-11-24T06:32:37.461 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103739.log.gz 2023-11-24T06:32:37.461 INFO:teuthology.orchestra.run.smithi016.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T06:32:37.462 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107391.log 2023-11-24T06:32:37.463 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115934.log.gz 2023-11-24T06:32:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112313.log 2023-11-24T06:32:37.463 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98066.log.gz 2023-11-24T06:32:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107391.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107391.log.gz 2023-11-24T06:32:37.465 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94827.log 2023-11-24T06:32:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99638.log 2023-11-24T06:32:37.466 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112313.log.gz 2023-11-24T06:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94827.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94827.log.gz 2023-11-24T06:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106942.log 2023-11-24T06:32:37.468 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105163.log 2023-11-24T06:32:37.469 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99638.log.gz 2023-11-24T06:32:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106942.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106942.log.gz 2023-11-24T06:32:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114048.log 2023-11-24T06:32:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.105163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98636.log 2023-11-24T06:32:37.472 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105163.log.gz 2023-11-24T06:32:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114048.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114048.log.gz 2023-11-24T06:32:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93571.log 2023-11-24T06:32:37.474 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.98636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99764.log 2023-11-24T06:32:37.474 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98636.log.gz 2023-11-24T06:32:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93571.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93571.log.gz 2023-11-24T06:32:37.475 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104789.log 2023-11-24T06:32:37.476 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.99764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114604.log 2023-11-24T06:32:37.477 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99764.log.gz 2023-11-24T06:32:37.477 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104789.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104789.log.gz 2023-11-24T06:32:37.477 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106683.log 2023-11-24T06:32:37.479 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115607.log 2023-11-24T06:32:37.479 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.114604.log.gz 2023-11-24T06:32:37.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.106683.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106683.log.gz 2023-11-24T06:32:37.480 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107656.log 2023-11-24T06:32:37.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101200.log 2023-11-24T06:32:37.482 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115607.log.gz 2023-11-24T06:32:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107656.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107656.log.gz 2023-11-24T06:32:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88633.log 2023-11-24T06:32:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115649.log 2023-11-24T06:32:37.484 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101200.log.gz 2023-11-24T06:32:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.88633.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88633.log.gz 2023-11-24T06:32:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101946.log 2023-11-24T06:32:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115649.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115649.log.gz 2023-11-24T06:32:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94617.log 2023-11-24T06:32:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110795.log 2023-11-24T06:32:37.489 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101946.log.gz 2023-11-24T06:32:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94617.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.94617.log.gz 2023-11-24T06:32:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114140.log 2023-11-24T06:32:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.110795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113283.log 2023-11-24T06:32:37.491 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110795.log.gz 2023-11-24T06:32:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.114140.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.114140.log.gz 2023-11-24T06:32:37.493 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.116433.log 2023-11-24T06:32:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.113283.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113283.log.gz 2023-11-24T06:32:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104999.log 2023-11-24T06:32:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116433.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.116433.log.gz 2023-11-24T06:32:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100612.log 2023-11-24T06:32:37.497 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103365.log 2023-11-24T06:32:37.497 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104999.log.gz 2023-11-24T06:32:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100612.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100612.log.gz 2023-11-24T06:32:37.498 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109110.log 2023-11-24T06:32:37.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109521.log 2023-11-24T06:32:37.500 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103365.log.gz 2023-11-24T06:32:37.501 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109110.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104915.log 2023-11-24T06:32:37.501 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109110.log.gz 2023-11-24T06:32:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.109521.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109521.log.gz 2023-11-24T06:32:37.503 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107280.log 2023-11-24T06:32:37.504 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.104915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112517.log 2023-11-24T06:32:37.505 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104915.log.gz 2023-11-24T06:32:37.506 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.107280.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107280.log.gz 2023-11-24T06:32:37.507 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101050.log 2023-11-24T06:32:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.112517.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T06:32:37.508 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112517.log.gz 2023-11-24T06:32:37.508 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.101050.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101050.log.gz 2023-11-24T06:32:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103875.log 2023-11-24T06:32:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93916.log 2023-11-24T06:32:37.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103875.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103875.log.gz 2023-11-24T06:32:37.510 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103961.log 2023-11-24T06:32:37.512 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100010.log 2023-11-24T06:32:37.512 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93916.log.gz 2023-11-24T06:32:37.513 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102114.log 2023-11-24T06:32:37.514 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103961.log.gz 2023-11-24T06:32:37.515 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115691.log 2023-11-24T06:32:37.515 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100010.log.gz 2023-11-24T06:32:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.102114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116060.log 2023-11-24T06:32:37.517 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102114.log.gz 2023-11-24T06:32:37.518 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.115691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94252.log 2023-11-24T06:32:37.518 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115691.log.gz 2023-11-24T06:32:37.519 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.116060.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116060.log.gz 2023-11-24T06:32:37.519 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117011.log 2023-11-24T06:32:37.521 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.94252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103783.log 2023-11-24T06:32:37.521 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94252.log.gz 2023-11-24T06:32:37.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.117011.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117011.log.gz 2023-11-24T06:32:37.522 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93580.log 2023-11-24T06:32:37.524 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.103783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100966.log 2023-11-24T06:32:37.524 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103783.log.gz 2023-11-24T06:32:37.525 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.93580.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93580.log.gz 2023-11-24T06:32:37.526 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.100966.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100966.log.gz 2023-11-24T06:32:37.545 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T06:32:37.566 INFO:teuthology.orchestra.run.smithi016.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T06:32:37.723 INFO:teuthology.orchestra.run.smithi016.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T06:32:37.725 INFO:teuthology.orchestra.run.smithi016.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T06:32:37.731 INFO:teuthology.orchestra.run.smithi016.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T06:32:37.734 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-11-24T06:32:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m0.929s 2023-11-24T06:32:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m2.256s 2023-11-24T06:32:37.735 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m1.581s 2023-11-24T06:32:37.736 INFO:tasks.ceph:Archiving logs... 2023-11-24T06:32:37.736 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/remote/smithi016/log 2023-11-24T06:32:37.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:32:38.488 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/remote/smithi173/log 2023-11-24T06:32:38.488 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:32:38.830 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T06:32:38.840 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T06:32:38.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:32:38.850 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:32:39.016 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T06:32:39.026 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T06:32:39.026 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:32:39.033 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:32:39.085 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:32:39.090 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:32:39.096 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:32:39.096 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-11-24T06:32:39.097 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +1777us[+1773us] +/- 56ms 2023-11-24T06:32:39.097 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -2059us[-2063us] +/- 75ms 2023-11-24T06:32:39.097 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -442us[ -442us] +/- 74ms 2023-11-24T06:32:39.097 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:32:39.099 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:32:39.100 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2023-11-24T06:32:39.100 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 +1814us[+1826us] +/- 56ms 2023-11-24T06:32:39.100 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -2036us[-2036us] +/- 75ms 2023-11-24T06:32:39.101 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -419us[ -419us] +/- 74ms 2023-11-24T06:32:39.101 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:32:39.102 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T06:32:39.112 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T06:32:39.113 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T06:32:39.124 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:32:39.429 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:32:39.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:32:39.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:32:39.813 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:32:39.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:32:39.922 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T06:32:39.932 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T06:32:39.971 INFO:teuthology.task.internal:Duration was 1146.685798 seconds 2023-11-24T06:32:39.972 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T06:32:39.982 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T06:32:39.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:32:39.992 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:32:40.161 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:32:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:32:40.534 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T06:32:40.534 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2023-11-24T06:32:40.534 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:32:40.603 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2023-11-24T06:32:40.604 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:32:40.696 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T06:32:40.696 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:32:40.711 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:32:40.810 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T06:32:40.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:32:40.816 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:32:41.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T06:32:41.025 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T06:32:41.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:32:41.138 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:32:41.237 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T06:32:41.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:32:41.261 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:32:41.375 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2023-11-24T06:32:41.376 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2023-11-24T06:32:41.436 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:32:41.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:32:41.488 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:32:41.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:32:41.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T06:32:41.556 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T06:32:41.557 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/remote/smithi016 2023-11-24T06:32:41.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:32:41.665 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555/remote/smithi173 2023-11-24T06:32:41.665 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:32:41.771 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T06:32:41.772 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:32:41.784 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:32:41.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T06:32:41.858 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T06:32:41.858 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T06:32:41.870 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T06:32:41.870 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:32:41.880 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:32:41.933 INFO:teuthology.orchestra.run.smithi016.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:32 /home/ubuntu/cephtest 2023-11-24T06:32:41.942 INFO:teuthology.orchestra.run.smithi173.stdout: 393504 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:32 /home/ubuntu/cephtest 2023-11-24T06:32:41.945 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T06:32:41.958 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T06:32:42.070 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T06:32:42.098 DEBUG:teuthology.nuke:shortname: smithi016 2023-11-24T06:32:42.098 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:32:42.125 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:01:30.759339', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGD20YXGiNkytCHcjhlA6pBSfrL5G0UeUJxAQF/SQdBLaGQ6PMVANaB6OXial3GG4ffe7UR6oBNGhVUHZMKliLk='} 2023-11-24T06:32:42.129 DEBUG:teuthology.nuke:shortname: smithi173 2023-11-24T06:32:42.129 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:32:42.154 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465555', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 06:01:30.758117', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIt7kG84kgBObmDZ7+jmRlniMGqurplR3EL62tMOa+cq2ZU5Agt28ZQNwGdsUXFv+G1bQUPSmbQKlSBOCkcQT6M='} 2023-11-24T06:32:42.182 INFO:teuthology.orchestra.console.smithi016:Power off 2023-11-24T06:32:42.182 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:32:42.214 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-11-24T06:32:42.215 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:32:42.246 INFO:teuthology.orchestra.console.smithi173:Power off 2023-11-24T06:32:42.247 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:32:42.260 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-11-24T06:32:42.276 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2023-11-24T06:32:42.276 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:32:42.303 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2023-11-24T06:32:46.261 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:32:46.304 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:32:46.374 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-11-24T06:32:46.417 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2023-11-24T06:32:50.378 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:32:50.418 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:32:50.488 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-11-24T06:32:50.529 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-11-24T06:32:50.590 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-11-24T06:32:50.630 INFO:teuthology.orchestra.console.smithi173:Power off completed 2023-11-24T06:32:50.735 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} duration: 1146.685798406601 failure_reason: 'Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)' flavor: default mon.a-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 mon.b-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1f75226033be47bca1a68d69c0bad254 status: fail success: false 2023-11-24T06:32:50.736 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T06:32:50.833 INFO:teuthology.run:FAIL